The assumption that the probability of a word depends only on the previous word is called a Markov assumption.

Markov used in NLP

They are a type of stochastic models that describe a sequence of events. The probability of each event only depends on the state of previous event.

University of Michigan - Ann Arbor

Google

A Markov process is defined as a tuple $$(S, P)$$, where:
- $$S$$ is a finite set of states.
- $$P$$ is a state transition probability matrix, where $$P_{ss'} = P[S_{t+1}=s'|S_t = s]$$.

A sequence of states possesses the Markov property if and only if the probability of moving to the next state $$S_{t+1}$$ depends only on the present state $$S_t$$ and not on the previous states $$S_1, S_2, \dots, S_{t-1}$$. That is, for all $$t$$, $$P[S_{t+1}|S_t] = P[S_{t+1}|S_1, S_2, \dots, S_t]$$.

In reinforcement learning, a Markov process is typically time-homogeneous. This means the transition probability is independent of the time step $$t$$: $$P[S_{t+1} = s'|S_t = s] = P[S_t = s'|S_{t-1} = s]$$.

Markov Process

Machine Learning Yearning, a free ebook from Andrew Ng, teaches you how to structure Machine Learning projects.
https://www.deeplearning.ai/machine-learning-yearning/

Note: The content of the book is aligned with the Coursera Deeplearning.ai specialization.  https://www.deeplearning.ai/deep-learning-specialization/ 

Machine Learning Yearning (Deeplearning.ai)

Markov Chains

Recurrent models are a class of neural networks designed to process sequential data. They operate by maintaining a hidden state, denoted as 'h', which captures information from previous elements in the sequence. At each time step 'i', the model updates its state to 'h_i' by applying a function 'f' to the previous state 'h_{i-1}' and the current input 'input_i'. This relationship is formally expressed by the recurrence relation: $$h_i = f(h_{i-1}, \text{input}_i)$$ This mechanism allows the model to have a 'memory' of past events, making it suitable for tasks involving sequences like natural language processing.

Recurrent Models

A Markov decision process is a tuple (S, A,P, γ, R), where:
• S is a finite set of states.
• A is a finite set of actions.
• P is the state transition probability matrix, $P^a_{ss'} = P[S_{t+1} = s'|S_t = s, A_t = a]$.
• γ ∈ $[0, 1]$ is called the discount factor.
• R : S × A → R is a reward function.

The dynamic of Markov Decision Process proceeds as follows: 
We start in some state $s_0$, and choose some actions $a_0 ∈ A$ to take. As a result of our choice, the state of the Markov Decision Process randomly transits to some successor state $s_1$, drawn from $P^{a_0}_{s_0s_1}$. Then, from state $s_1$, we pick another action $a_1$. Again, we come to some state $s_2$, and so on.

Learn Before

Related

Learn After