Three major modifications are made based on general Naive Bayes Classifier:
- Binary counts: The feature is defined as a binary count indicating whether a word is present or not, instead of the word counts.
- Negation: Negation is properly handled by adding a "NOT_" prefix to the words during text normalization.
- Sentiment Lexicons: When training dataset is insufficient, using Sentiment Lexicon can improve the performance.

Binary Multinomial Naive Bayes Classifier

Sentiment Analysis can be viewed as a classification task with two classes, that is, positive and negative. Naive Bayes Classifier can be modified as Binary Multinomial Naive Bayes Classifier to improve its performance on sentiment analysis.

New York University

Naive Bayes Classifier is a probabilistic model. It implements the idea of Bayes Inference and works on two assumptions: Bag-of-Word Assumption and Naive Bayes Assumption.

Navie Bayes Classifier

Sentiment analysis (also known as opinion mining or emotion AI) refers to the use of natural language processing, text analysis, computational linguistics, and biometrics to systematically identify, extract, quantify, and study affective states and subjective information. 


Sentiment analysis is widely applied to voice of the customer materials such as reviews and survey responses, online and social media, and healthcare materials for applications that range from marketing to customer service to clinical medicine.

Sentiment Analysis

An on-going but a helpful book resource about NLP
https://web.stanford.edu/~jurafsky/slp3/

Speech and Language Processing (3rd ed. draft) 

Bayes Inference derives the posterior probability based on a prior probability and a likelihood function according to Bayes' Theorem.
$P(C|D)=\frac{P(D|C)P(C)}{P(D)}$
Here, the probability $P(D|C)$ is the likelihood function, and $P(C)$ is the prior probability.

Bayesian Inference

The Naive Bayes classifier assigns the most probable class $$\hat{c}$$ by choosing the class which has the highest probability for a document $$d$$: $$\hat{c}=argmax_{c\in C}P(c|d)$$. Using Bayesian inference, this is equivalent to finding: $$\hat{c}=argmax_{c\in C}P(d|c)P(c)$$, where $$P(d|c)$$ is the likelihood function and $$P(c)$$ is the prior probability.

Bayesian Inference in Naive Bayes Classifier

Naive Bayes Classifier can be viewed as a generative model. Based on:
$\hat{c}=argmax_{c\in C}P(d|c)P(c)$, the generation of a document can be stated as: first, sample a class c from P(c); then, the words in the document can be generated by sampling from P(d|c).

Naive Bayes Classifier as a Generative Model

Bag-of-Word Assumption is used to simplify the computation of Naive Bayes Classifier. It assumes that the positions of words in a document does not affect its classification. Based on this assumption, the features of a document can be simplified as the identical word encoding, discarding their relative ordering. Hence, the likelihood function P(d|c) can be written as $P(f_1,...,f_n|c)$

Bag of Word Assumption

Naive Bayes Assumption is the conditional independence assumption that is used to simplify the computation of the likelihood function $P(f_1,...,f_n|c)=P(f_1|c)...P(f_n|c)$.

Naive Bayes Assumption

The general model of Naive Bayes Classifier can be written as:
$\hat{c}=argmax_{c\in C}\log P(c)+\sum_i \log P(w_i|c)$,
where $C$ is the set of all possible classes, $i$ is the position of word $w_i$ in the text.
This is a linear model, since the prediction is computed as a linear combination of input features, ie., the words.

The General Expression of Naive Bayes Classifier for Text Classification

Application of Naive Bayes Classifier in Sentiment Analysis

We use a sentiment analysis domain with the two classes positive
(+) and negative (-), and take the following miniature training and test documents simplified from actual movie reviews.

Example of Training and Testing Naive Bayes

Research papers, articles and examples about sentiment analysis in NLP

Sentiment Analysis References

In cases that we do not have sufficient labeled data using all words in training set for sentiment classification, we can derive the word features from sentiment lexicons, which are external sources of words with sentiment labels. A common way is to add the features to indicate whether the word instance has occurred in the positive/ negative lexicon N times.


Sentiment Lexicons

Clip the frequency of words in each document at 1, as binary occurrence of a word seems to matter more than its frequency in some text classification tasks, such as sentiment analysis. Duplicate words are then removed before concatenating the words into a document.


Binary Multinomial Naive Bayes (Binary NB)

Emotion can be defined as a relatively brief episode of response to the evaluation of an external or internal event. In machine learning, detecting emotions can improve many language processing tasks. For example, an online tutoring system can use emotion detection to see if a student is happy, sad, or bored. And detecting emotions in customer reviews can help businesses improve based on customer feedback.

Learn Before

Related

Learn After