CNN Sequence Processing Complexity
When processing a sequence of length , a one-dimensional Convolutional Neural Network (CNN) treats the text as a 'one-dimensional image', extracting local features such as -grams. For a convolutional layer with a kernel size of and input and output channels, the computational complexity is . Because CNNs process local regions simultaneously and use a hierarchical structure, they require only sequential operations, enabling parallel computation. Furthermore, the hierarchical architecture allows the receptive field to expand; for instance, a two-layer CNN with connects distant tokens like and , resulting in a maximum path length of .
0
1
Tags
D2L
Dive into Deep Learning @ D2L
Related
RNN Sequence Processing Complexity
Self-Attention Sequence Processing Complexity
Trade-offs in Sequence Architecture Selection
CNN Sequence Processing Complexity
Match each sequence-to-sequence evaluation parameter or component to its definition.
When comparing architectures for sequence mapping, requiring a smaller number of sequential operations is desirable because it enables ___ computation.
Which architectural evaluation property should the team focus on addressing to resolve this issue, and what specific characteristic should they seek in an alternative architecture?