Learn Before
Concept

Add-One Smoothing Representation

Laplace smoothing applied to unigram probability of word wi with count cic_i, which is normalized by total number of work tokens NN and number of words in the vocabulary VV: PLaplace(wi)=ci+1N+VP_{Laplace}(w_i) = \frac{c_i+1}{N+V} It is more convenient to define an adjusted count cic_i^*, and turn it into a probability PiP_i^* by normalizing by NN: ci=(ci+1)NN+Vc_i^* = (c_i + 1)\frac{N}{N+V} The add-one smoothed bigram probability and adjust count are given by PLaplace(wnwn1)=C(wn1wn)+1w(C(wn1wn)+1)=C(wn1wn)+1C(wn1)+VP^*_{Laplace}(w_n|w_{n-1}) = \frac{C(w_{n-1}w_n) + 1}{\sum_w (C(w_{n-1}w_n) + 1)} = \frac{C(w_{n-1}w_n) + 1}{C(w_{n-1}) + V}

0

1

Updated 2022-06-29

Tags

Data Science