Learn Before
Concept
Stop Words in Modern Neural Sequence Models
Stop words are extremely common but not particularly descriptive words, such as articles (e.g., 'the', 'a'), pronouns ('i', 'my'), and prepositions ('of', 'in'). While traditional text classifiers based on bag-of-words representations often filtered out these high-frequency words, modern RNN- and Transformer-based neural sequence models generally retain them. This retention is crucial because stop words still carry important syntactic meaning and contextual information that deep learning architectures are highly capable of utilizing.
0
1
Updated 2026-05-13
Tags
D2L
Dive into Deep Learning @ D2L