CNN Sequence Processing Complexity
When processing a sequence of length , a one-dimensional Convolutional Neural Network (CNN) treats the text as a 'one-dimensional image', extracting local features such as -grams. For a convolutional layer with a kernel size of and input and output channels, the computational complexity is . Because CNNs process local regions simultaneously and use a hierarchical structure, they require only sequential operations, enabling parallel computation. Furthermore, the hierarchical architecture allows the receptive field to expand; for instance, a two-layer CNN with connects distant tokens like and , resulting in a maximum path length of .
0
1
References
Tags
D2L
Dive into Deep Learning @ D2L
Related
RNN Sequence Processing Complexity
Self-Attention Sequence Processing Complexity
Trade-offs in Sequence Architecture Selection
Match each sequence-to-sequence evaluation parameter or component to its definition.
When comparing architectures for sequence mapping, requiring a smaller number of sequential operations is desirable because it enables ___ computation.
Which architectural evaluation property should the team focus on addressing to resolve this issue, and what specific characteristic should they seek in an alternative architecture?
Related
RNN Sequence Processing Complexity
Self-Attention Sequence Processing Complexity
Trade-offs in Sequence Architecture Selection
Match each sequence-to-sequence evaluation parameter or component to its definition.
When comparing architectures for sequence mapping, requiring a smaller number of sequential operations is desirable because it enables ___ computation.
Which architectural evaluation property should the team focus on addressing to resolve this issue, and what specific characteristic should they seek in an alternative architecture?