A standard network assumes that each input is unrelated to other inputs. You want to use sequence models (like in sentences) when it is useful to know the prior inputs (or words). In addition, neural networks are useful when the lengths of inputs and outputs are constant. The main point is if the output is dependent on the previous input → use a sequence model.


When are sequence models more useful than a standard neural network? 


Standard NN Problems:


1- Inputs, outputs can be different length in different examples.



2- Doesn't share features learned across different positions of a text.

Why not a standard network?

The task of predicting what word/letter comes next, unlike the FNN and CNN, in sequence modeling the current output is dependent on the previous a=input and the length of the input is not fixed.


University of Michigan - Ann Arbor

https://www.coursera.org/learn/nlp-sequence-models

Sequence Models

Machine Learning Yearning, a free ebook from Andrew Ng, teaches you how to structure Machine Learning projects.
https://www.deeplearning.ai/machine-learning-yearning/

Note: The content of the book is aligned with the Coursera Deeplearning.ai specialization.  https://www.deeplearning.ai/deep-learning-specialization/ 

Machine Learning Yearning (Deeplearning.ai)

- Represent input words are x<1>


- Output to match is y<1>


- Tx denotes length of X.


- Ty denotes length of y.


Sequence Models: Notation

Each word is represented as a vector, with value 1 for the word and 0 for all other words in the dictionary (similar to one-hot encoding)

Representing  Words

Why sequence models? 

Given a sequence of words, what is the most probable next word/sentence?

Language Model

Sequence labeling is a machine learning methodology used across many Natural Language Processing (NLP) applications. The core idea is to assign a specific class or tag to each token within a given input sequence. The resulting sequence of labels can then be interpreted to extract linguistic annotations. Common examples of this technique include Part-of-Speech (POS) tagging, where each word is labeled with its grammatical role, and Named Entity Recognition (NER), where tokens are tagged to identify entities like names or locations.

Learn Before

Related

Learn After