An internal knowledge graph is constructed from the input document and this information is fused into a transformer text generation model through attention.

San Jose State University

 - NLG Enhanced by Topic 
 - NLG Enhanced by Keywords
 - NLG Enhanced by Linguistic Features
 - NLG Enhanced by Open Knowledge Graphs

 NLG ENHANCED BY INTERNAL KNOWLEDGE

A Survey of Knowledge-Enhanced Text Generation

A Topic broadly represents the contents in a text and  is also the subject the text refers to that can be used to enhance natural text generation.

LDA( Latent Dirichlet allocation) is a classical topic modelling tool for inferring low dimensional representation that captures the underlying semantics of words and documents.
In LDA each topic is defined as a distribution over words and each document as a mixture distribution over topics. LDA gives a set of distributions of topics for each document and also a set of distributions of words for each topic

Topic enhanced NLG are categorized into 

- Leverage Topic Words from Generative Topic Models
- Jointly Optimize Generation Model and CNN Topic Model.
-  Enhance NLG by Neural Topic Models with Variational Inference


NLG Enhanced by Topic 

Keywords often represent compact representations of contents of a document. Keyword acquisition can be done in two ways
- keyword assignment
- keyword extraction

Keyword assignment means that keywords are chosen from a controlled vocabulary of terms or predefined taxonomy.

Keyword extraction selects the most representative words explicitly presented in the document, which is independent from any vocabulary.

Keyword enhanced NLG are categorized into

- Incorporate keyword assignment into text generation
- Incorporate keyword extraction into text generation


NLG Enhanced by Keywords(

Encoder is fed with input sequences along with auxiliary hand crafted linguistic features like

- POS tags and NER tags
- Syntactic dependency graph
- Semantic dependency graph


Learn Before

Related