Training Data Construction for Sequence Models
To construct training data from a sequence for autoregressive models operating under a -order Markov assumption, one extracts input–output pairs where each label is and the corresponding feature vector is . Because sufficient history is unavailable for the first time steps, those examples are dropped, yielding total examples from a sequence of length , each with a fixed input dimensionality of . Rather than padding the missing early observations with zeros, this simple truncation approach is commonly used. This construction relies on the stationarity assumption—the belief that the dynamics generating the sequence do not change over time—which ensures that patterns extracted from any historical segment remain relevant for predicting future values.
0
1
Tags
D2L
Dive into Deep Learning @ D2L