Input size $= n_h^{[l-1]} \times n_w^{[l-1]} \times n_c^{[l-1]}$
Kernel size $= f_h^{[l]} \times f_w^{[l]} \times n_c^{[l]}$
Where $n_c^{[l]} =$ number of filters in layer $l$ because it should match the number of outputs.
Stride size $= s_h^{[l]} \times s_w^{[l]}$
Padding size $= p_h^{[l]} \times p_w^{[l]}$

Output (Activations $a^{[l]}$) size $= n_h^{[l]} \times n_w^{[l]} \times n_c^{[l]}$
$$n_h^{[l]} = \left \lfloor{ \frac{n_h^{[l-1]} + 2p_h^{[l]} − k_h^{[l]}}{s_h^{[l]}}}\right \rfloor + 1$$
$$n_w^{[l]} = \left \lfloor{ \frac{n_w^{[l-1]} + 2p_w^{[l]} − k_w^{[l]}}{s_w^{[l]}}}\right \rfloor + 1$$

*# of Parameters:*
Weights: $f_h^{[l]} \times f_w^{[l]} \times n_c^{[l-1]} \times n_c^{[l]}$
Bias: $n_c^{[l]}$

A Convolution Layer Output Size and Parameters

Input size $= n_h^{[l-1]} \times n_w^{[l-1]}$
Kernel size $= k_h \times k_w$
Stride size $= s_h \times s_w$
Padding size $= p_h \times p_w$

Output size $= n_h^{[l]} \times n_w^{[l]}$
$$n_h^{[l]} = \left \lfloor{ \frac{n_h^{[l-1]} + 2p_h^{[l]} − k_h^{[l]}}{s_h^{[l]}}}\right \rfloor + 1$$
$$n_w^{[l]} = \left \lfloor{ \frac{n_w^{[l-1]} + 2p_w^{[l]} − k_w^{[l]}}{s_w^{[l]}}}\right \rfloor + 1$$

University of Michigan - Ann Arbor

When computing the cross-correlation, we start with the convolution window at the top-left corner of the input tensor, and then slide it over all locations both down and to the right. Usually, we default to slide one element at a time. However, sometimes, either for computational efficiency or because we wish to down-sample, we move our window more than one element at a time, skipping the intermediate locations.

We refer to the number of rows and columns traversed per slide as the stride.

Strided Convolution

The picture shows a two-dimensional cross-correlation operation with a stride of 3 vertically and 2 horizontally. 
The shaded portions are the output elements as well as the input and kernel tensor elements used for the output computation: 
 0×0+0×1+1×2+2×3=8 ,  0×0+6×1+0×2+0×3=6 . 

We can see that when the second element of the first column is outputted, the convolution window slides down three rows. The convolution window slides two columns to the right when the second element of the first row is outputted. When the convolution window continues to slide two columns to the right on the input, there is no output because the input element cannot fill the window (unless we add another column of padding).

Strided Convolution Example

i. Lesser Memory needed for output

ii. Processing of output is easier as the volume is smaller

iii. The output *spatial dimensions* are smaller, which when given as an input to the next layer, simplifies the model.

iv. It avoids **overfitting** especially in case of image processing having a large no. of attributes.

Advantages of Strided Convolution

Output Size of Strided and Padded Convolution

Note: Standard convolution, in this case, implies 0 padding and a stride of 1

1. Every convolution operation layer being applied would shrink the image further to a point where the final image size would then end up being 1x1. This essentially puts a hard limit on the number of convolution layers that a model can have.
2. Border pixels get used very little in the convolution operation as compared to other parts of the input and, as a result, the same effect gets propagated further down the layers. 



Learn Before

Related

Learn After