One of the earliest linear embedding models, trained on 200,000 words using less than 1 million parameters.

University of California, Santa Cruz

The categorization of texts into organized groups, which is usually implemented by supervised learning.
Applications: sentiment analysis, spam detection, news categorization, topic analysis, and more.

Text Classification

Minaee, Shervin et al. Deep Learning Based Text Classification: A Comprehensive Review. Retrieved May 22, 2022, from https://arxiv.org/pdf/2004.03705.pdf

Deep Learning Based Text Classification: A Comprehensive Review

Latent Semantic Analysis (LSA)

Graph neural networks capture the internal graph structures of natural language, like syntactic and semantic parse trees.

Graph Neural Networks

A text classification category that views text as a "bag of words." This is the simplest model for text classification, but is still very successful. 

Steps:
1) Learn the vector representation for each word using an embedding model
2) Take vector sum or average of embeddings as the text representation
3) Pass through one or more feed-forward layers, known as Multi-Layer Perceptrons (MLPs)
4) Perform classification on final layer's representation using a classifier

Feed-Forward Neural Networks

Siamese Neural Networks are a special case of text classification designed for text matching.

Siamese Neural Networks

Linear classifiers use a linear combination of inputs to predict the category.

Linear Classifiers

Sentiment analysis is used to refer to the task of automatically determining the whether a piece of text is positive, negative, or neutral. It can also determine someone's attitude towards a particular target or topic.

Sentiment Analysis

Text-pair classification extends standard classification methods to process two distinct texts simultaneously. When provided with a pair of texts composed of tokens $$x_1 \dots x_m$$ and $$y_1 \dots y_n$$, the two sequences are concatenated into a single combined sequence. The total length of this unified sequence is $$len$$, calculated as $$len = n + m + 2$$ to account for necessary special tokens. A classification label is subsequently predicted for the entire sequence by utilizing the aggregated representation vector, specifically $$\mathbf{h}_{\mathrm{cls}}$$. This overarching framework addresses multiple NLP challenges, including semantic equivalence judgement (assessing if two texts share identical meanings), text entailment judgement (evaluating if a hypothesis logically stems from a premise), grounded commonsense inference (gauging the likelihood of an event given its context), and question-answering inference (verifying if an answer correctly matches a question).

Text-Pair Classification

In text classification, a specific category or label is assigned to a piece of text. For example, in the context of sentiment analysis, the sentence 'We had a delightful dinner together.' would be assigned the label 'Positive'.

Learn Before

Related