In deep learning, operations labeled as convolutions are technically cross-correlations. A strict two-dimensional discrete convolution uses index differences to sum elements, effectively flipping the kernel: $$(f * g)(i, j) = \sum_a \sum_b f(a, b) g(i-a, j-b)$$. In contrast, cross-correlation uses index additions: $$(i+a, j+b)$$. Because kernels are learned from data, a network using cross-correlation will simply learn a flipped version of the kernel matrix $$\mathbf{K}'$$ that a strict convolution would learn as $$\mathbf{K}$$, producing identical outputs. This cosmetic distinction justifies the standard terminology of calling cross-correlation a convolution.

Equivalence of Strict Convolution and Cross-Correlation

For two-dimensional tensors, the mathematical convolution operation is computed as a sum over two sets of indices. Extending the one-dimensional definition, the discrete two-dimensional convolution of functions $$f$$ and $$g$$ is defined as:
$$(f * g)(i, j) = \sum_a \sum_b f(a, b) g(i-a, j-b)$$
This calculates the overlap between the two functions when one is flipped across both dimensions (using index differences) and shifted by $$(i, j)$$.

Claude

In mathematics, a convolution measures the overlap between two functions when one is flipped and shifted. For continuous functions, such as $$f, g: \mathbb{R}^d 	o \mathbb{R}$$, it is defined by the integral:
$$(f * g)(\mathbf{x}) = \int f(\mathbf{z}) g(\mathbf{x}-\mathbf{z}) d\mathbf{z}$$
For discrete sequences over $$\mathbb{Z}$$, the integral becomes a sum. The one-dimensional discrete convolution of functions $$f$$ and $$g$$ is:
$$(f * g)(i) = \sum_a f(a) g(i-a)$$
In signal processing, this operation connects an input signal $$x[n]$$ and an impulse response $$h[n]$$ to produce an output $$(x * h)[n] = \sum_{k=-\infty}^{+\infty} x[k] h[n-k]$$, with bounded summation for finite-length signals.

Convolution

Dive into Deep Learning

Discrete Two-Dimensional Convolution

For an input signal $$x[n]$$ and an impulse response $$h[n]$$, their one-dimensional discrete convolution yields an output sequence $$y[n]$$ with initial terms calculated as: $$y[0] = x[0]h[0]$$, $$y[1] = x[0]h[1] + x[1]h[0]$$, and $$y[2] = x[0]h[2] + x[1]h[1] + x[2]h[0]$$. In Python, this 1-D convolution operation is implemented in the SciPy library using the convolve function.

Learn Before

Related

Learn After