Stacking multiple small convolutions provides the same receptive field as a single larger convolution while improving parameter efficiency and increasing depth. For example, applying two successive $$3 	imes 3$$ convolutions touches the same input pixels as a single $$5 	imes 5$$ convolution. However, while a single $$5 	imes 5$$ convolution requires $$25 \cdot c^2$$ parameters (where $$c$$ is the number of channels), three successive $$3 	imes 3$$ convolutions use a comparable number of parameters ($$3 \cdot 9 \cdot c^2 = 27 \cdot c^2$$) while providing an even larger receptive field. This analysis demonstrated that deep and narrow networks significantly outperform their shallow and wide counterparts, establishing stacked $$3 	imes 3$$ convolutions as a gold standard in deep learning architecture design.

Receptive Field and Parameter Efficiency of Stacked Convolutions

The VGG network architecture is often considered the first truly modern Convolutional Neural Network (CNN) because it introduced a systematic, modular approach to building deep models. While earlier models like AlexNet proved the effectiveness of large-scale CNNs, VGG established key design properties: a preference for deep and narrow networks, and the use of modular blocks containing multiple consecutive convolutional layers. By chaining these VGG blocks together, the architecture defines an entire family of similarly parameterized models—such as VGG-11 or VGG-16—that allow practitioners to easily trade off between computational complexity and execution speed.

VGG Network Architecture

A VGG block is a fundamental structural unit introduced by Simonyan and Zisserman that uses multiple convolutions between downsampling operations. Specifically, a VGG block consists of a sequence of convolutional layers using $$3 	imes 3$$ kernels with a padding of $$1$$ (which maintains the spatial height and width), followed by a $$2 	imes 2$$ max-pooling layer with a stride of $$2$$ (which halves the spatial dimensions after each block). By executing multiple convolutions within a single block before downsampling via max-pooling, the network can achieve greater depth without exhausting the spatial resolution too quickly.

Claude

The standard building block of early Convolutional Neural Networks consists of a convolutional layer with padding to maintain resolution, a nonlinearity (such as a ReLU), and a pooling layer (such as max-pooling) that reduces the spatial resolution. Because max-pooling typically halves the resolution in each block, this approach imposes a strict limit on network depth. Specifically, a network with an input dimension $$d$$ can accommodate at most $$\log_2 d$$ such convolutional layers before the spatial dimensions are entirely exhausted. For example, on ImageNet data, this architectural design restricts the network to a maximum of $$8$$ convolutional layers.

Learn Before

Related

Learn After