Concept

CNN Sequence Processing Complexity

When processing a sequence of length nn, a one-dimensional Convolutional Neural Network (CNN) treats the text as a 'one-dimensional image', extracting local features such as nn-grams. For a convolutional layer with a kernel size of kk and dd input and output channels, the computational complexity is O(knd2)\mathcal{O}(knd^2). Because CNNs process local regions simultaneously and use a hierarchical structure, they require only O(1)\mathcal{O}(1) sequential operations, enabling parallel computation. Furthermore, the hierarchical architecture allows the receptive field to expand; for instance, a two-layer CNN with k=3k = 3 connects distant tokens like x1\mathbf{x}_1 and x5\mathbf{x}_5, resulting in a maximum path length of O(n/k)\mathcal{O}(n/k).

Image 0

0

1

Updated 2026-05-14

Contributors are:

Who are from:

Tags

D2L

Dive into Deep Learning @ D2L