1. Pre-Trained Language Representations
2. Domain-Specific Pre-Training
3. Multilingual Language Models

Main Directions of Transfer Learning for Low-resource NLP

Transfer learning in NLP lies in the use of pre-trained language representations that are trained on unlabeled data like BERT. Some brand new finetuning and probing paradigms are also proposed in recent years for the direction.

University of Michigan - Ann Arbor

1. Data Augmentation
2. Transfer Learning
3.  Ideas From Low-Resource Machine Learning in Non-NLP Communities


Widely Used Tricks for Low-Resource NLP

https://aclanthology.org/2021.naacl-main.201.pdf

A Survey on Recent Approaches for Natural Language Processing in Low-Resource Scenarios


New instances can be obtained based on existing ones by modifying the features with transformations that do not change the label. To go beyond the token level and add more diversity to the augmented sentences, data augmentation can also be performed on sentence parts.

Data Augmentation for Low-Resource NLP

Transfer Learning for Low-resource NLP

1. Meta-learning.
2. Adversarial discriminators.
3. Adapters.

Learn Before

Related

Learn After