Learn Before
Concept
Inductive Biases of Convolutional Layers
Convolutional layers drastically reduce the number of parameters required for image processing by imposing two key inductive biases: translation invariance and locality. Translation invariance enforces that weights are shared across spatial locations, while locality restricts the network to only incorporate information from a small local window when computing hidden activations. When these biases agree with the underlying reality of the data (e.g., natural images), they yield sample-efficient models that generalize well. Conversely, if the data does not exhibit these properties, the strong inductive biases may cause the model to struggle to fit the training data.
0
1
Updated 2026-05-13
Tags
D2L
Dive into Deep Learning @ D2L