Autocorrection is an automatic text validation tool commonly found in your phone's and computer's text editing interfaces. This tool can correct spelling errors and/or contextual errors depending on the algorithms.

Autocorrection

The technique (Shannon, 1951) generates a word in a sequence given the previous words, using cross-entropy as the loss function and perplexity as the evaluation method. The sequence ends once it reaches a predefined length. Each softmax layer is to decide the next word based on the current hidden state in the RNN component. The initial input token(s) would depend on the NLP task's nature (e.g. machine translation, question answering, summarization). 

RNN's Approach to Text Generation in NLP (Autoregressive Generation)

inform code-switching in multilingual contexts

A Survey of Code-switching: Linguistic and Social Perspectives for Language Technologie

Hämäläinen, M., & Alnajjar, K. (2021). Human Evaluation of Creative NLG Systems: An Interdisciplinary Survey on Recent Papers. arXiv preprint arXiv:2108.00308.

Human Evaluation of Creative NLG Systems: An Interdisciplinary Survey on Recent Papers

- To present a comprehensive overview of NLG developments since 2000.
- To provide researchers with synthesis and pointers to relevant research.
- To introduce the field to researchers who are less familiar.

Goals of the NLG survey

- Content determination:  Deciding which information to include in the text under construction.
- Text structuring: Determining in which order information will be presented in the text.
- Sentence aggregation: Deciding which information to present in individual sentences.
- Lexicalisation:  Finding the right words and phrases to express information.
- Referring expression generation:  Selecting the words and phrases to identify domain objects.

NLG tasks.

GRL Book

A survey of embedding models of entities and relationships for knowledge graph completion

In Natural Language Processing, text generation tasks that use existing text as input can be broadly divided into two main categories. The first is text completion, which involves continuing to write based on the provided text. The second is text transformation, which focuses on converting the input text into a new, different text.

Classification of Text Generation Problems in NLP

NLG, a subfield of artificial intelligence (AI), is a software process that automatically transforms data into plain-English content.

University of Michigan - Ann Arbor

- Information extraction
- Machine translation
- Discourse, dialogue and pragmatics
- Natural language generation
- Speech recognition
- Lexical semantics
- Phonology / morphology
- Language resources

Natural language processing in ACM Computing Classification

Book:
- Speech and Language Processing (3rd ed. draft) Written by Dan Jurafsky and James H. Martin
- Speech and Language Processing (2nd edition) Written by Dan Jurafsky and James H. Martin
Journal:
- A Survey on Dynamic Neural Networks for Natural Language Processing
- A Survey of Race, Racism, and Anti-Racism in NLP

NLP references

Natural language generation

Speech recognition

Phonology / Phonetics

Knowledge of the meaningful components of words; more specifically, the way words break down into components that carry meanings like singular vs. plural, I'm, can't, etc.

Morphology

Lexical semantics

Knowledge about linguistic units larger than a single utterance

Discourse

Knowledge of the relationship of meaning to the goals and intentions of the speaker.

Pragmatics

Information extraction

Machine translation is a sequence-to-sequence learning problem that involves ingesting a sentence in a source language and predicting its corresponding translation in a target language. A major challenge in this task is unaligned data, because the input and output sequences can have varying lengths and the corresponding words or regions may appear in completely different orders.

Learn Before

Related

Learn After