The probability for a Katz backoff n-gram is given by
$$P_{BO}(w_n|w_{n-N+1:n-1}) = $$
$$
\begin{cases}
P^*(w_n|w_{n-N+1:n-1}), \text{ if } C(w_{n-N+1:n}) > 0 \\
\alpha(w_{n-N+1:n-1})P_{BO}(w_n|w_{n-N+2:n-1}), \text{ otherwise}
\end{cases}
$$
where $\alpha$ is a function that distributes higher-order n-grams to lower-order n-grams, to ensure that the backoff model gives a correct probability distribution.

University of Michigan - Ann Arbor

Katz Backoff is a kind of backoff where we rely on a discounted probability P* if we have seen this n-gram before. Otherwise, we recursively back off to the Katz probability for the (n-1)-gram.

Katz Backoff

An on-going but a helpful book resource about NLP
https://web.stanford.edu/~jurafsky/slp3/

Learn Before

Related