step1: training set: large corpus of text

step2: tokenize text--> one-hot encoding --> add <EOS> token

step3: detecting unknown words

step4: start with the first word of the sentence and deed it as input to the RNN.

Language model with RNN

Language model alignment involves fine-tuning a pretrained language model to follow instructions and align with human intent. Systems like ChatGPT leverage this alignment to allow users to interact conversationally, enabling the model to serve as an assistant for solving complex problems, debugging code, and performing creative writing tasks.

Language Model Alignment

A sequence satisfies a Markov condition if the future is conditionally independent of the past, given the recent history. This means that when predicting the next element in a sequence, we can discard the history beyond the previous $$	au$$ time steps without any loss in predictive power, conditioning only on $$x_{t-1}, \ldots, x_{t-	au}$$ rather than the entire sequence history $$x_{t-1}, \ldots, x_1$$. While this is only approximately true for real text—where leftwards context continues to provide diminishing information gains—working with models that satisfy this condition obviates computational and statistical difficulties associated with processing arbitrarily long sequences.

Markov Condition

While the joint probability of a sequence $$P(x_1, \ldots, x_T)$$ can be mathematically factorized in reverse (right-to-left) or any random order, left-to-right (in-order) factorization is generally preferred for language modeling. First, it aligns with the natural human intuition of anticipating upcoming words while reading. Second, factorizing in order allows the same language model to easily assign probabilities to arbitrarily long sequences by continually multiplying the current probability by the conditional probability of the next token: $$P(x_{t+1}, \ldots, x_1) = P(x_{t}, \ldots, x_1) \cdot P(x_{t+1} \mid x_{t}, \ldots, x_1)$$. Third, for causally structured data where future events cannot influence the past, predicting forward ($$P(x_{t+1} \mid x_t)$$) is usually an easier predictive modeling problem than predicting backward ($$P(x_t \mid x_{t+1})$$).

Left-to-Right Factorization in Sequence Models

For natural language data composed of discrete tokens like words, sequence models are specifically called language models. They provide the capacity to evaluate the likelihood of sentences, sample new text sequences, and optimize for the most likely outputs. By applying the chain rule of probability, language modeling can be reduced to an autoregressive prediction problem, decomposing the joint density of a sequence $$P(x_1, \ldots, x_T)$$ into the product of conditional densities in a left-to-right fashion: $$P(x_1, \ldots, x_T) = P(x_1) \prod_{t=2}^T P(x_t \mid x_{t-1}, \ldots, x_1)$$. For discrete signals, the autoregressive model acts as a probabilistic classifier, outputting a full probability distribution over the vocabulary for the next word given the leftwards context.

Claude

University of Michigan - Ann Arbor

https://www.coursera.org/learn/nlp-sequence-models

Sequence Models

An autoregressive model predicts the next value in a sequence by regressing a signal on its own previous values. Specifically, given a time series of observations, the model estimates the conditional probability distribution $$P(x_t \mid x_{t-1}, \ldots, x_1)$$ or a key statistic thereof, such as the conditional expectation $$\mathbb{E}[(x_t \mid x_{t-1}, \ldots, x_1)]$$. A foundational challenge for autoregressive models is that the number of historical inputs $$x_{t-1}, \ldots, x_1$$ grows with $$t$$, causing each training example to have a different number of features. Two principal strategies address this: (1) conditioning on a fixed-length window of $$	au$$ recent observations, and (2) maintaining a latent summary state $$h_t$$ that compresses the entire history.

Autoregressive Model

Machine Learning Yearning, a free ebook from Andrew Ng, teaches you how to structure Machine Learning projects.
https://www.deeplearning.ai/machine-learning-yearning/

Note: The content of the book is aligned with the Coursera Deeplearning.ai specialization.  https://www.deeplearning.ai/deep-learning-specialization/ 

Machine Learning Yearning (Deeplearning.ai)

Dive into Deep Learning

- Represent input words are x<1>


- Output to match is y<1>


- Tx denotes length of X.


- Ty denotes length of y.


Sequence Models: Notation

Each word is represented as a vector, with value 1 for the word and 0 for all other words in the dictionary (similar to one-hot encoding)

Representing  Words

The task of predicting what word/letter comes next, unlike the FNN and CNN, in sequence modeling the current output is dependent on the previous a=input and the length of the input is not fixed.


Why sequence models? 

Language Model

Sequence labeling is a machine learning methodology used across many Natural Language Processing (NLP) applications. The core idea is to assign a specific class or tag to each token within a given input sequence. The resulting sequence of labels can then be interpreted to extract linguistic annotations. Common examples of this technique include Part-of-Speech (POS) tagging, where each word is labeled with its grammatical role, and Named Entity Recognition (NER), where tokens are tagged to identify entities like names or locations.

Sequence Labeling

To overcome the challenge of variable-length inputs in autoregressive modeling, a common strategy is to condition predictions only on a fixed-length window of recent history rather than the entire past. By using a window of length $$	au$$, the model only considers the observations $$x_{t-1}, \ldots, x_{t-	au}$$ to predict $$x_t$$. This ensures that for all time steps $$t > 	au$$, the number of input arguments remains constant, allowing the use of models that require fixed-length feature vectors.

Autoregressive Windowing Strategy

A latent autoregressive model addresses the challenge of variable-length histories by maintaining a hidden summary state, $$h_t$$, of past observations. Instead of conditioning directly on a growing sequence of past inputs, the model predicts the current observation $$\hat{x}_t$$ based on the current summary state ($$\hat{x}_t = P(x_t \mid h_t)$$). Simultaneously, it updates the summary state using the previous state and the previous observation ($$h_t = g(h_{t-1}, x_{t-1})$$). Because the state $$h_t$$ is never directly observed in the data, it is considered latent.

Latent Autoregressive Model

In autoregressive modeling, rather than estimating the entire probability distribution over continuously valued sequence data, it is often more practical to focus on key statistics such as the expected value. The autoregressive conditional expectation estimates the expected value of the next sequence element, $$x_t$$, given its entire history of past observations. This is formally denoted as $$\mathbb{E}[(x_t \mid x_{t-1}, \ldots, x_1)]$$, and can be estimated using strategies like linear regression.

Autoregressive Conditional Expectation

A major challenge in standard autoregressive modeling is that the number of historical inputs, $$x_{t-1}, \ldots, x_1$$, increases as the time step $$t$$ increases. This means that when treating historical sequence data as a training set, each example has a different number of features. This variable-length input problem necessitates the use of specialized techniques, such as fixed-length windowing or latent state models, to process the data effectively.

Learn Before

Related

Learn After