Low frame rate is the simplest algorithm for subsampling. For time $i$ the acoustic feature vector $f_i$ and the two prior vectors $f_{i-1}$  and $f_{i-2}$  are stacked (concatenated). This makes a new vector which is three times longer, following which $f_{i-1}$  and $f_{i-2}$ are deleted. Therefore, we end. up with a higher-dimensional vector with a shorter sequence length $n = \frac{t}{3}$. For example, instead of a 40-dimensional acoustic feature vector every 10 ms, we would get a longer vector (say 120-dimensional) every 30 ms.

Low Frame Rate

In the Listen, Attend, and Spell (LAS) architecture, subsampling achieves the goal of producing a shorter sequence $$X = x_1,..., x_n$$ that serves as the input to the encoder. The simplest algorithm for this compression is a method called low frame rate.

Columbia University

Google

The encoder-decoder architecture is very appropriate when input and output sequences have significant difference in length, and they do for speech, since long acoustic feature sequences tend to map to much shorter sequences of letters or words. For example, a single word might be 5 letters long, but if it is spoken for 2 seconds, it would take 200 acoustic frames of 10 milliseconds each to map to. Due to this extreme length difference in speech, the encoder-decoder architectures for speech require a special compression stage that shortens the acoustic feature sequence before the stage of encoding. An alternative to this is using a loss function that deals with compression, like the CTC or connectionist temporal classification loss function.

Compression in LAS

An on-going but a helpful book resource about NLP
https://web.stanford.edu/~jurafsky/slp3/

Learn Before

Related

Learn After