An NLG system should generate text that meet task-specific, controllable constraints. In dialogue response generation, one would control for politeness / offensive content. 

Controllability in Text Generation 


Early approaches were based on sequential methods and style embedding, which was followed by deep generative model-based methods such as Variational Autoencoders (VAEs), Generative Adversarial Nets (GANs), and Energy-based Models. 

Deep Learning-Based CTG Methods 


The CTG can be described as 
$P(Y|C) = p(y_1, y_2, …, y_n|C$ 
Where C varies according to the task. Additionally, the generated text Y should satisfy the constraint conditions while being reasonably human-readable. 

Formal Definition of CTG 

A CTG system has 3 components:

(1) The controlled element, which includes a controlled condition and source text 


(2) The generative model, as process P


(3) The generated text satisfying the input control condition, as output O. 

CTG Components 

An area within Natural Language Generation, Controllable Text Generation is integral in developing text generation technologies that are more natural and customizable in practical applications. CTG is the task of generating text given a controlled element. 

University of California, Berkeley

A Survey of Controllable Text Generation using Transformer-based Pre-trained Language Models

Zhang, H., Song, H., Li, S., Zhou, M., & Song, D. (2022). A Survey of Controllable Text Generation using Transformer-based Pre-trained Language Models. ArXiv, abs/2201.05337.

A Survey of Controllable Text Generation using Transformer-based Pre-trained Language Mode

Controllable Text Generation (CTG) 


The complement of Natural Language Understanding (NLU) which aims to produce a normalized representation of the idea expressed in input text (read/understand human language), NLG aims to transform potential representations into specific, self-consistent text, i.e. write in natural language. 

Learn Before

Related

Learn After