https://towardsdatascience.com/classifying-skin-lesions-with-convolutional-neural-networks-fc1302c60d54

University of Michigan - Ann Arbor

Pooling operators use a fixed-shape window that slides over all regions of the input tensor based on a specified stride. For each location traversed, this pooling window computes a single output. Unlike convolutional layers that compute cross-correlations using learnable parameters (kernels), the pooling layer contains no parameters. Instead, pooling operations are deterministic, typically calculating the maximum or average value within the pooling window.

Pooling Layer in Convolutional Deep Learning

Pooling in convolutional deep learning offers two primary advantages:

1. **Dimensionality Reduction**: It decreases the spatial dimensions of feature maps, which reduces the number of parameters and the computational power required for subsequent layers.
2. **Translation Invariance**: It extracts dominant features that are invariant to small positional shifts in the input, enhancing the network's robustness and maintaining effective model training.

Advantages of Pooling in Convolutional Deep Learning

Types of Pooling Layer in Convolutional Deep Learning

A pooling layer is configured entirely through hyperparameters rather than learned parameters. The key hyperparameters are the pooling window size (which can be specified as a single integer for a square window or as a tuple for a rectangular window with distinct height and width), the stride (the number of elements the window moves per step), the type of aggregation (max or average), and the amount of padding applied to the input borders. Because pooling computes a fixed, deterministic function—either the maximum or the average—there are no weight parameters to learn, and therefore no parameter initialization is needed. A widely adopted default configuration uses a $$2 	imes 2$$ pooling window, which quarters the spatial resolution of the output (halving both height and width), providing an effective balance between downsampling and information retention.

Pooling Parameters and Hyperparameters in Convolutional Deep Learning

1. Reduce parameters. By reducing the dimension of feature map, the parameters required for subsequent layers are effectively reduced

2. Translation variance. It means that for input, when the pixels in the neighborhood are slightly displaced, the output of the pooling layer is unchanged. This enhances the robustness of the network and has a certain anti disturbance effect (translation invariance)

The error of feature extraction mainly comes from two aspects:

(1) The variance of the estimated value increases due to the limited size of the neighborhood;

(2) The convolution layer parameter error causes the offset of the estimated mean.

Generally speaking, mean pooling can reduce the first error and retain more background information of the image. Max pooling can reduce the second error and retain more texture information.

Role of pooling

The forward propagation of a two-dimensional pooling layer can be implemented programmatically without using a kernel. A function `pool2d(X, pool_size, mode)` iterates over the input tensor `X` with a sliding window defined by `pool_size`. At each location, it extracts the sub-tensor and computes either its maximum or mean, depending on the `mode` parameter. Unlike cross-correlation, this computation relies purely on deterministic aggregation over the input regions rather than learnable parameters.

Two-Dimensional Pooling Layer Code Implementation

Pooling and convolutional layers handle multi-channel input data in fundamentally different ways. In a convolutional layer, a multi-channel cross-correlation is performed: the kernel computes a two-dimensional cross-correlation with each input channel separately, but the results from all $$c_	extrm{i}$$ channels are then summed together to produce a single two-dimensional output per output channel. In contrast, the pooling layer processes each input channel independently—it applies the pooling window to every channel in isolation without combining information across channels. As a direct consequence, the number of output channels of a pooling layer always equals the number of input channels, whereas a convolutional layer's output channel count is determined by the number of kernels ($$c_	extrm{o}$$), which is independent of $$c_	extrm{i}$$. This architectural difference means pooling preserves the per-channel feature representations established by preceding convolutional layers, while convolution mixes cross-channel information to create new feature combinations.

Pooling vs. Convolution in Multi-Channel Processing

For an input tensor of size $$n_H \times n_W \times n_C$$, the output size after applying a pooling layer with a square window of size $$f$$ and a stride of $$s$$ is:

$$\lfloor(n_H - f)/s + 1\rfloor \times \lfloor(n_W - f)/s + 1\rfloor \times n_C$$

Here, $$n_H$$, $$n_W$$, and $$n_C$$ are the height, width, and number of channels of the input layer. The number of output channels remains $$n_C$$ because pooling applies to each channel separately.

Learn Before

Related