Some systems choose to completely ignore another class of words: stop words, very frequent words like the and a. But using a stop word list doesn’t improve performance, so in most cases, the entire vocabulary is used.

University of Michigan - Ann Arbor

While training, we first consider maximum likelihood estimate.
We first concatenate all documents with category c into one big “category c” text. Then we use the frequency of wi in this concatenated document to give a maximum likelihood estimate of the probability:
$P(w_i|c) = \frac{count(w_i,c)}{\sum_{w\in V}count(w,c)}$

Training the naive Bayes Classifier

An on-going but a helpful book resource about NLP
https://web.stanford.edu/~jurafsky/slp3/

Speech and Language Processing (3rd ed. draft) 

Whilie training, zero probabilities in the likelihood term for any class will cause the probability of the class to be zero, no matter the other evidence. A solution is the add-one (Laplace) smoothing:
$P(w_i|c) = \frac{count(w_i,c)+1}{\sum_{w\in V}{(count(w,c)+1)}}  = \frac{count(w_i,c)+1}{\sum_{w\in V}{count(w,c)}+|V|}$

Avoid zero posiblities

For words that occur in our test data but are not in our vocabulary (because they did not occur in any training documents),  ignore them—remove them from the test document and not include any probability for them at all.

Learn Before

Related