-bag-of-words
-TF-IDF
-LDA
-word2vec
-docs2vec
-GloVe
-fastText
-Tweet2vec
-Universal Sentence Encoder
-ELMo
-BERT
-ALBERT
-XLNet
-Sentence-BERT


Text Representation methods

Done with the use of large scale pre-trained language models that use Transformer-based architecture 

University of Michigan - Ann Arbor

Text corpora may have different features that are needed to process differently due to 1)language; 2)genre; 3)the demographic characteristics of the writer (or speaker); 4)time. Proper text normalization is required to make sure the models are fit to the text data. Text normalization make sure that the text is changed into a suitable and excellent form for further processing.

Text normalization

The process of text normalization includes:
- Tokenizing (segmenting) words 
- Normalizing word formats (case folding/lemmatization/stemming)
- Segmenting sentences

Process of text normalization

- A regular expression is "an algebraic notation for characterizing a set of strings". They are particularly useful for searching patterns in texts.
- Regular repressions helps to represent text patterns to be extracted from a document.

Regular Expression

Text Representation

Tokenization is a way of separating a piece of text into smaller units called tokens.

Tokenization

It's the task of determining that two words have the same root, despite their surface differences.

Lemmatization

An utterance is the spoken correlate of a sentence.

Utterance

The larger the corpora we look at, the more word types we find.
Assume $|V|$ is the number of types and $N$ is number of tokens. k and $\beta$ are positive numbers and $0< \beta <1$.  Then
$|V|= k N^{\beta}$

Learn Before

Related

Learn After