Billions of parameters to be learnt.
Training data from raw text from a diverse set of sources (like Wikipedia, Reddit, and news sources


San Jose State University

Most of the latest TGM’s used Transformer model architecture ,it helps to learn long range dependency information and the generated texts are correct ,coherent and uses world knowledge.

Text Generative Model Architecture

Automatic Detection of Machine Generated Text: A Critical Survey
by Ganesh Jawahar, Muhammad Abdul-Mageed, Laks V.S. Lakshmanan
https://arxiv.org/pdf/2011.01314.pdf

Automatic Detection of Machine Generated Text: A Critical Survey


Training Data for TGM

Billions of parameters to be learned from millions of data which would need a humongous computational cost.

Learn Before

Related