Each token in the input sequence is tagged with its grammatical categories like Noun,verb,adjective ,person ,location organisation etc which helps in understanding the input sequence.

San Jose State University

Encoder is fed with input sequences along with auxiliary hand crafted linguistic features like

- POS tags and NER tags
- Syntactic dependency graph
- Semantic dependency graph


NLG Enhanced by Linguistic Features

A Survey of Knowledge-Enhanced Text Generation

POS tags and NER tags in NLG

Syntactic dependency graph is a directed acyclic graph which showcases the relationship between words.

Syntactic dependency graph in NLG

Semantic dependency graph represents the predicate-argument relationship between content words in a sentence ,it also portrays various semantic relationships based on different annotations. The stacked encoders consist of a sequence encoder and a graph encoder, in which the sentence encoder first reads the input text through stacked multi-head self-attention, and then the graph encoder captures semantic relationships and incorporates the semantic graph structure into the contextual-level representation. 

Learn Before

Related