The architectural concept of receptive fields and convolutional kernels in deep learning draws heavy inspiration from neurophysiology, particularly studies on the mammalian visual cortex. Foundational biological research discovered that lower-level neurons naturally respond to simple visual stimuli like edges and specific shapes, a mechanism that is remarkably mirrored by how convolutional neural networks learn spatial representations. This biological link was further illustrated when convolutional kernels were shown to mimic these cortical responses on natural images, a phenomenon that persists even in the features extracted by the deeper layers of modern image classification networks.

Neurophysiological Inspiration of CNN Receptive Fields

When an element in a feature map requires a larger receptive field to detect input features over a broader area, a deeper network architecture can be utilized. For instance, applying an initial $$2 	imes 2$$ convolution kernel produces an output matrix $$\mathbf{Y}$$ where each element has a receptive field of four input elements. If an additional $$2 	imes 2$$ convolutional layer takes $$\mathbf{Y}$$ as its input to compute a single element $$z$$, the receptive field of $$z$$ on the intermediate layer $$\mathbf{Y}$$ consists of four elements, but its receptive field on the original input expands to encompass nine elements.

Effect of Network Depth on Receptive Field Size

For any given element $$x$$ within a layer of a convolutional neural network, its receptive field consists of all the elements from all preceding layers that can potentially influence the calculation of $$x$$ during forward propagation. Because a single output element aggregates information from a window of inputs, its receptive field may encompass an area significantly larger than its own dimensions.

Claude

In the context of convolutional neural networks, a feature map is the output produced by a convolutional layer. It acts as a learned representation, capturing specific spatial features—such as patterns across height and width dimensions—that are then passed forward as input to subsequent layers in the network.

Feature Map

Dive into Deep Learning

Receptive Field in Convolutional Neural Networks

Just as input images are represented as third-order tensors, hidden representations in convolutional neural networks are also formulated as third-order tensors, denoted as $$\mathsf{H}$$. Instead of a single value per spatial location, there is an entire vector of hidden representations for each location. These representations can be conceptualized as multiple two-dimensional grids stacked together, commonly referred to as channels or feature maps, where lower layers might recognize basic features like edges and deeper layers capture complex textures.

Hidden Representation as a 3rd-Order Tensor

To effectively detect objects of varying sizes, an object detection system can generate anchor boxes at multiple scales by utilizing feature maps of different spatial resolutions. Intuitively, smaller objects are more likely to appear on an image than larger ones; for example, a $$1 	imes 1$$ object can appear in more possible locations on a given image than a $$2 	imes 2$$ object. Therefore, a larger feature map provides more individual spatial units, allowing the system to uniformly sample a greater number of regions. This dense sampling is highly suitable for generating smaller anchor boxes to detect small objects. Conversely, by reducing the spatial dimensions of the feature map, the system samples fewer regions, which is ideal for placing larger anchor boxes intended to capture larger objects.

Learn Before

Related

Learn After