We can construct a multi-output channel convolution kernel programmatically by concatenating existing kernel tensors. For example, if K is a multi-input channel kernel tensor for a single output channel, we can stack K, K + 1, and K + 2 along the $$0^	extrm{th}$$ dimension to create a kernel with three output channels. If the original K had a shape of $$2 	imes 2 	imes 2$$ (representing $$c_	extrm{i} 	imes k_	extrm{h} 	imes k_	extrm{w}$$), the new stacked kernel will have an overall shape of $$3 	imes 2 	imes 2 	imes 2$$.

Example of Multi-Output Channel Kernel Construction

In a multi-output channel cross-correlation, the result for each individual output channel is calculated independently from the other output channels, using its corresponding $$c_	extrm{i} 	imes k_	extrm{h} 	imes k_	extrm{w}$$ slice from the convolution kernel and the entire multi-channel input tensor. Specifically, the kernel slice performs a multi-channel cross-correlation with all $$c_	extrm{i}$$ channels of the input tensor to produce one channel of the output. While the computation for each channel is independent, the multiple output channels do not learn representations independently; rather, they are optimized jointly, meaning specific directions in the channel space may correspond to useful feature detections, such as edge detectors.

Multi-Output Channel Cross-Correlation Operation

In a convolutional layer, breaking up a standard convolution from $$c_\textrm{i}$$ input channels to $$c_\textrm{o}$$ output channels into $$g$$ independent groups is known as grouped convolution. In this operation, the input channels are divided into $$g$$ groups of size $$c_\textrm{i}/g$$, which independently generate $$g$$ outputs of size $$c_\textrm{o}/g$$. This approach proportionally reduces the computational cost from $$\mathcal{O}(c_\textrm{i} \cdot c_\textrm{o})$$ to $$\mathcal{O}(c_\textrm{i} \cdot c_\textrm{o} / g)$$, making it $$g$$ times faster. Additionally, the number of parameters required for the operation decreases from a single $$c_\textrm{i} \times c_\textrm{o}$$ matrix to $$g$$ smaller matrices of size $$(c_\textrm{i}/g) \times (c_\textrm{o}/g)$$, yielding a $$g$$ times reduction in parameters. It is assumed that both $$c_\textrm{i}$$ and $$c_\textrm{o}$$ are fully divisible by $$g$$.

Grouped Convolution

To produce an output with multiple channels ($$c_	extrm{o}$$), a kernel tensor of shape $$c_	extrm{i} 	imes k_	extrm{h} 	imes k_	extrm{w}$$ must be created for every output channel. These individual kernel tensors are then concatenated along the output channel dimension. This concatenation results in an overall convolution kernel shape of $$c_	extrm{o} 	imes c_	extrm{i} 	imes k_	extrm{h} 	imes k_	extrm{w}$$.

Claude

When the input data contains multiple channels, denoted as $$c_	extrm{i}$$, the convolution kernel must have the same number of input channels to perform cross-correlation. If the kernel's two-dimensional spatial window shape is $$k_	extrm{h} 	imes k_	extrm{w}$$, a kernel tensor of shape $$k_	extrm{h} 	imes k_	extrm{w}$$ is required for every input channel. Concatenating these $$c_	extrm{i}$$ tensors yields a convolution kernel with an overall shape of $$c_	extrm{i} 	imes k_	extrm{h} 	imes k_	extrm{w}$$.

Multi-Channel Convolution Kernel Structure

Dive into Deep Learning

To perform a cross-correlation operation on an input with $$c_	extrm{i}$$ channels, a two-dimensional cross-correlation is computed between the two-dimensional input tensor and the corresponding two-dimensional kernel tensor for each individual channel. The $$c_	extrm{i}$$ resulting two-dimensional tensors are then added together, summing over the channels, to yield a single two-dimensional output tensor.

Learn Before

Related

Learn After