In order to keep track of the different characteristics of the texts, the corpus creator can make a datasheet or data statement that specifies factors such as the motivation, situation, language variety, speaker demographics, collection process, annotation process, and distribution of each text.

University of Michigan - Ann Arbor

Pieces of text used to train and/or develop NLP models can vary in several ways that may need to be accounted for, such as: 
- The language the text is written in
- The type/genre of text
- The author/speaker of the text
- How old the text is

Variation Factors of Input Texts

An on-going but a helpful book resource about NLP
https://web.stanford.edu/~jurafsky/slp3/

Speech and Language Processing (3rd ed. draft) 

It's crucial to test algorithms on more than one language, especially on languages with different properties. Unfortunately, many current algorithms tend to be developed and tested only for/in languages of large industrialized nations (e.g. English, Chinese, Spanish, Japanese, German).

Translation of the Text

The texts used for training can be any kind of written or spoken transcripts. That, combined with the unique demographic characteristics of the writer/speaker, as well as the lingual variations  from the time period the text is from, create a very unique style for every text used.

Learn Before

Related