Deep learning frameworks allow us to programmatically apply padding to convolutional layers to manage feature map sizes. For example, applying a $$ 3 \times 3 $$ kernel with 1 pixel of padding on all sides to an $$ 8 \times 8 $$ input preserves the input's dimensionality, resulting in an $$ 8 \times 8 $$ output. Similarly, if the kernel is non-square, such as $$ 5 \times 3 $$, we can supply a tuple to set different padding values for the height and width (e.g., padding of 2 for height and 1 for width) to maintain the original spatial dimensions.

Claude

Google

In many convolutional neural network architectures, it is advantageous to keep the spatial height and width of the output identical to the input, making it easier to predict tensor shapes across deep networks. To preserve dimensionality when applying a kernel of size $$k_	extrm{h} 	imes k_	extrm{w}$$, we set the total number of padded rows to $$p_	extrm{h} = k_	extrm{h} - 1$$ and the total number of padded columns to $$p_	extrm{w} = k_	extrm{w} - 1$$. If the kernel dimension is odd, padding is distributed evenly (e.g., $$p_	extrm{h}/2$$ rows on both sides). If the kernel dimension is even, padding is distributed asymmetrically, such as padding $$\lceil p_	extrm{h}/2 ceil$$ rows on the top and $$\lfloor p_	extrm{h}/2 floor$$ rows on the bottom. Additionally, preserving dimensionality by using an odd-sized kernel and symmetric padding provides a useful clerical benefit: it guarantees that the output element $$Y[i, j]$$ is calculated with the convolution window centered exactly on the corresponding input element $$X[i, j]$$.

Preserving Dimensionality with Padding

Dive into Deep Learning

Convolutional neural networks frequently use kernels with odd spatial dimensions, such as $$1 	imes 1$$, $$3 	imes 3$$, $$5 	imes 5$$, or $$7 	imes 7$$. Choosing odd kernel sizes provides the advantage of preserving dimensionality while allowing for symmetric padding—adding the exact same number of rows on the top and bottom ($$p_	extrm{h}/2$$ rows on each side), and the same number of columns on the left and right. This symmetric padding also ensures a clerical benefit: for any input tensor, the output element at index $$(i, j)$$ is computed by a cross-correlation window centered exactly on the input element at $$(i, j)$$.

Learn Before

Related