Stochastic pooling is a pooling variant introduced by Zeiler and Fergus (2013) that incorporates randomization into the aggregation process. Unlike standard deterministic pooling methods—which compute a fixed maximum or average—stochastic pooling randomly selects an activation within the pooling window according to a probability distribution derived from the activations themselves. By combining aggregation with randomization, this approach acts as a form of regularization for deep convolutional neural networks, and can yield modest improvements in classification accuracy compared to conventional pooling strategies.

Claude

Types of Pooling Layer in Convolutional Deep Learning

Dive into Deep Learning

Average pooling is a deterministic operation that calculates the average value of the elements within a fixed-shape pooling window as it slides across an input tensor. The concept is akin to downsampling an image; rather than simply discarding pixels, average pooling combines information from adjacent pixels to obtain a lower-resolution output with an improved signal-to-noise ratio.

Average Pooling in Convolutional Deep Learning

Stochastic Pooling

Fractional max-pooling, proposed by Graham (2014), is a pooling technique that introduces randomization into the spatial reduction process. Rather than using a fixed, uniform grid of pooling regions as in standard max-pooling, fractional max-pooling uses pseudo-randomly varying pooling region sizes, allowing the spatial resolution to be reduced by non-integer (fractional) factors. This stochastic variation in the pooling regions can serve as a form of data augmentation at the feature-map level, potentially leading to slight improvements in model accuracy.

Fractional Max-Pooling

Max-pooling is a deterministic operation that selects the maximum value from the elements within a fixed-shape pooling window as it slides across an input tensor. Originally introduced in cognitive neuroscience research on hierarchical object recognition, max-pooling has become the dominant pooling method in modern convolutional neural networks. It is generally preferred over average pooling because it confers a degree of invariance to the output—small translations or distortions in the input are less likely to affect the maximum value within a window, making the learned features more robust to spatial shifts.

Learn Before

Related