Neural topic models combine the advantages of neural networks and probabilistic topic models. They enable back propagation for joint optimization, contributing to more coherent topics, and can be scaled to large data sets.

San Jose State University

A Topic broadly represents the contents in a text and  is also the subject the text refers to that can be used to enhance natural text generation.

LDA( Latent Dirichlet allocation) is a classical topic modelling tool for inferring low dimensional representation that captures the underlying semantics of words and documents.
In LDA each topic is defined as a distribution over words and each document as a mixture distribution over topics. LDA gives a set of distributions of topics for each document and also a set of distributions of words for each topic

Topic enhanced NLG are categorized into 

- Leverage Topic Words from Generative Topic Models
- Jointly Optimize Generation Model and CNN Topic Model.
-  Enhance NLG by Neural Topic Models with Variational Inference


NLG Enhanced by Topic 

A Survey of Knowledge-Enhanced Text Generation

NLG applications that are enhanced by topic are 

- Dialogue system
 - Machine translation.
 - Paraphrase


NLG Applications Enhanced by Topic

 - One way is to discover topics using generative topic models (e.g., LDA), Incorporate the topics representations into neural generation models

 - Second way is to assign a topic to each sentence and then find out the top-k words falling into that topic and use word embeddings to represent the topic words.


Leverage Topic Words from Generative Topic Models

Convolutional neural networks (CNN) were used to learn latent topic representations through iterative convolution and pooling operations which replaced earlier  LDA models for topic generation.

Learn Before

Related