Each token in the input sequence is tagged with its grammatical categories like Noun,verb,adjective ,person ,location organisation etc which helps in understanding the input sequence.

POS tags and NER tags in NLG

Syntactic dependency graph is a directed acyclic graph which showcases the relationship between words.

Syntactic dependency graph in NLG

Semantic dependency graph represents the predicate-argument relationship between content words in a sentence ,it also portrays various semantic relationships based on different annotations. The stacked encoders consist of a sequence encoder and a graph encoder, in which the sentence encoder first reads the input text through stacked multi-head self-attention, and then the graph encoder captures semantic relationships and incorporates the semantic graph structure into the contextual-level representation. 

Semantic dependency graph in NLG

Encoder is fed with input sequences along with auxiliary hand crafted linguistic features like

- POS tags and NER tags
- Syntactic dependency graph
- Semantic dependency graph


San Jose State University

 - NLG Enhanced by Topic 
 - NLG Enhanced by Keywords
 - NLG Enhanced by Linguistic Features
 - NLG Enhanced by Open Knowledge Graphs

 NLG ENHANCED BY INTERNAL KNOWLEDGE

A Survey of Knowledge-Enhanced Text Generation

A Topic broadly represents the contents in a text and  is also the subject the text refers to that can be used to enhance natural text generation.

LDA( Latent Dirichlet allocation) is a classical topic modelling tool for inferring low dimensional representation that captures the underlying semantics of words and documents.
In LDA each topic is defined as a distribution over words and each document as a mixture distribution over topics. LDA gives a set of distributions of topics for each document and also a set of distributions of words for each topic

Topic enhanced NLG are categorized into 

- Leverage Topic Words from Generative Topic Models
- Jointly Optimize Generation Model and CNN Topic Model.
-  Enhance NLG by Neural Topic Models with Variational Inference


NLG Enhanced by Topic 

Keywords often represent compact representations of contents of a document. Keyword acquisition can be done in two ways
- keyword assignment
- keyword extraction

Keyword assignment means that keywords are chosen from a controlled vocabulary of terms or predefined taxonomy.

Keyword extraction selects the most representative words explicitly presented in the document, which is independent from any vocabulary.

Keyword enhanced NLG are categorized into

- Incorporate keyword assignment into text generation
- Incorporate keyword extraction into text generation


NLG Enhanced by Keywords(

NLG Enhanced by Linguistic Features

An internal knowledge graph is constructed from the input document and this information is fused into a transformer text generation model through attention.

Learn Before

Related

Learn After