Concept

Inductive Biases of Convolutional Layers

Convolutional layers drastically reduce the number of parameters required for image processing by imposing two key inductive biases: translation invariance and locality. Translation invariance enforces that weights are shared across spatial locations, while locality restricts the network to only incorporate information from a small local window when computing hidden activations. When these biases agree with the underlying reality of the data (e.g., natural images), they yield sample-efficient models that generalize well. Conversely, if the data does not exhibit these properties, the strong inductive biases may cause the model to struggle to fit the training data.

0

1

Updated 2026-05-13

Contributors are:

Who are from:

Tags

D2L

Dive into Deep Learning @ D2L