The assumption that the probability of a word depends only on the previous word is called a Markov assumption.

Markov used in NLP

They are a type of stochastic models that describe a sequence of events. The probability of each event only depends on the state of previous event.

University of Michigan - Ann Arbor

A Markov Process is a tuple (S,P), where
• S is a (finite) set of states
• P is a state transition probability matrix. $P_{ss'} = P[S_{t+1}=s'|S_t = s]$.

We make some constrains on states. 
A sequence of states is Markov if and only if the probability of moving to the next state $S_{t+1}$ depends only on the present state $S_t$ and not on the previous states $S_1, S_2, · · · , S_{t−1}$. That is, for all t,
$P[S_{t+1}|S_t] = P[S_{t+1}|S_1, S_2, · · · , S_t].$
In reinforcement learning, Markov Process is time-homogeneous. That is, the probability of the transition is independent of t:
$P[S_{t+1} = s'|S_t = s] = P[S_t = s'|S_{t−1} = s]$.


Markov Process

Machine Learning Yearning, a free ebook from Andrew Ng, teaches you how to structure Machine Learning projects.
https://www.deeplearning.ai/machine-learning-yearning/

Note: The content of the book is aligned with the Coursera Deeplearning.ai specialization.  https://www.deeplearning.ai/deep-learning-specialization/ 

Machine Learning Yearning (Deeplearning.ai)

Markov Chains

Recurrent models are a class of neural networks designed to process sequential data. They operate by maintaining a hidden state, denoted as 'h', which captures information from previous elements in the sequence. At each time step 'i', the model updates its state to 'h_i' by applying a function 'f' to the previous state 'h_{i-1}' and the current input 'input_i'. This relationship is formally expressed by the recurrence relation: $$h_i = f(h_{i-1}, \text{input}_i)$$ This mechanism allows the model to have a 'memory' of past events, making it suitable for tasks involving sequences like natural language processing.

Learn Before

Related

Learn After