University of Michigan - Ann Arbor

1. Summarization: Common Crawl, WikiRef, QMDSCNN, QMDSIR
2. QA: XLDA, QANet
3. Sequence Tagging Tasks: DAGA, dependency tree morphing, SEQMIX
4. Parsing Tasks: Data Recombination, GRAPPA, compositionality
5. Grammatical Error Correction: Use annotated data or confusion sets
6. Neural Machine Translation: BackTranslation, SwitchOut, Soft Contextual, Data Diversification
7. Data-to-Text NLG: perturbing language game scores or inputs, or adding noise.
8. Open-Ended & Conditional Generation: GEAUNG
9. Dialogue: LIGHTWEIGHT AUGMENTATION, diversity rank, MADA, spoken language understanding
10. Multimodal Tasks: Augment parallel data seperately and reinforce the alignment.

Downstream NLP Tasks applying DA Techniques 

The popular BACKTRANSLATION method translates a sequence into another language and then back into the original language.

BackTranslation

WikiRef: Wikilinks as a route to recommending appropriate references for scientific Wikipedia pages

https://aclanthology.org/2020.emnlp-main.296/

Coarse-to-Fine Query Focused Multi-Document Summarization

The key of XLDA is to replace part of the input sentence using a sentence with similar or the same meaning in other languages. 

XLDA

XLDA: Cross-Lingual Data Augmentation for Natural Language Inference and Question Answering

QANet: Combining Local Convolution with Global Self-Attention for Reading Comprehension

The key concept is to use neural machine translation model to use multilingual QAs together.

QANet

https://aclanthology.org/2020.emnlp-main.488/

DAGA: Data Augmentation with a Generation Approach for Low-resource Tagging Tasks

DAGA linearizes the tagged data and uses it to train the language model. Then it uses the LM to generate the augmentation data.

DAGA

Generate dependency tree based on grammar or semantics. Then re-paraphrase the sentence based on the dependency tree and some related rules.

Dependency Tree Morphing

SeqMix: Augmenting Active Sequence Labeling via Sequence Mixup

GraPPa: Grammar-Augmented Pre-Training for Table Semantic Parsing

SwitchOut: an Efficient Data Augmentation Algorithm for Neural Machine Translation

The key strategy is to randomly replace words in both the source sentence and the target sentence with other random words from their corresponding vocabularies.

SwitchOut

Soft Contextual Data Augmentation for Neural Machine Translation

It replaces the word with a linear combination of words with similar meanings of the word in the sentence condition.

Soft Contextual

Data Diversification: A Simple Strategy For Neural Machine Translation

It diversifies the training data by using the predictions of multiple forward and backward models and then merging them with the original dataset on which the final NMT model is trained. 

Learn Before

Related