Consider a two-dimensional cross-correlation operation using a vertical stride of $$3$$ and a horizontal stride of $$2$$. When computing the output, the convolution window slides down three rows for each vertical step and two columns to the right for each horizontal step. For example, the first two output elements of the first column and row might be computed as $$0	imes0+0	imes1+1	imes2+2	imes3=8$$ and $$0	imes0+6	imes1+0	imes2+0	imes3=6$$. If the window slides further and the remaining input elements cannot completely fill the convolution window, no output is generated for that position unless extra padding is added.

University of Michigan - Ann Arbor

Claude

In a standard cross-correlation operation, the convolution window slides over the input tensor one element at a time. However, to increase computational efficiency or to downsample the representation, we can traverse multiple elements per slide, skipping intermediate locations. The number of rows and columns traversed per slide is called the stride. Strided convolution is particularly useful when the convolution kernel is large, as it efficiently captures a broad area of the underlying image.

Strided Convolution

Dive into Deep Learning

i. Lesser Memory needed for output

ii. Processing of output is easier as the volume is smaller

iii. The output *spatial dimensions* are smaller, which when given as an input to the next layer, simplifies the model.

iv. It avoids **overfitting** especially in case of image processing having a large no. of attributes.

Advantages of Strided Convolution

When applying standard cross-correlation operations without padding, we encounter two main limitations. First, the spatial dimensions of the output shrink after each layer, which restricts the total number of successive convolutional layers that can be applied before the feature map is reduced to a $$1 	imes 1$$ size. Second, because the convolution kernel must fit entirely within the image boundaries, the pixels on the perimeter and corners participate in far fewer window calculations than the central pixels, leading to a significant loss of boundary information.

Limitations of Unpadded Convolution

Deep learning frameworks allow the use of strided convolutions to efficiently downsample input dimensions. By configuring the stride or strides parameter in a two-dimensional convolution layer, the cross-correlation window skips intermediate locations. For example, setting the strides to $$2$$ for both height and width will halve the input's spatial dimensions. Frameworks also support asymmetric strides by passing a tuple; for instance, specifying a stride of $$(3, 4)$$ will slide the convolution window down by $$3$$ rows and across by $$4$$ columns at each step.

Code Implementation of Stride in Convolution

Example of Two-Dimensional Cross-Correlation with Stride

When applying a convolution with kernel height $$k_\textrm{h}$$ and width $$k_\textrm{w}$$, total padding height $$p_\textrm{h}$$ and width $$p_\textrm{w}$$, and strides $$s_\textrm{h}$$ and $$s_\textrm{w}$$, the shape of the output generated from an input of height $$n_\textrm{h}$$ and width $$n_\textrm{w}$$ is:

$$\lfloor(n_\textrm{h}-k_\textrm{h}+p_\textrm{h}+s_\textrm{h})/s_\textrm{h}\rfloor \times \lfloor(n_\textrm{w}-k_\textrm{w}+p_\textrm{w}+s_\textrm{w})/s_\textrm{w}\rfloor$$

This formula simplifies under certain conditions. If padding is set to $$p_\textrm{h}=k_\textrm{h}-1$$ and $$p_\textrm{w}=k_\textrm{w}-1$$, the output shape reduces to $$\lfloor(n_\textrm{h}+s_\textrm{h}-1)/s_\textrm{h}\rfloor \times \lfloor(n_\textrm{w}+s_\textrm{w}-1)/s_\textrm{w}\rfloor$$. Furthermore, if the input dimensions are exactly divisible by the strides, the output shape simplifies to $$(n_\textrm{h}/s_\textrm{h}) \times (n_\textrm{w}/s_\textrm{w})$$.

Formula for Convolution Output Size with Stride and Padding

Similar to the shorthand notation used for symmetric padding, there is a standard convention for denoting strides in convolutional layers. When the horizontal stride ($$s_	extrm{h}$$) and the vertical stride ($$s_	extrm{w}$$) are identical, the notation is simplified by referring to the operation as having stride $$s$$.

Stride Convention

Using a stride greater than $$1$$ is an effective technique for downsampling an input. Specifically, applying a stride of $$n$$ (where $$n > 1$$) reduces the spatial resolution of the output, shrinking both the height and the width of the resulting feature map to approximately $$1/n$$ of the original input's dimensions.

Learn Before

Related