New instances can be obtained based on existing ones by modifying the features with transformations that do not change the label. To go beyond the token level and add more diversity to the augmented sentences, data augmentation can also be performed on sentence parts.

Data Augmentation for Low-Resource NLP

Transfer learning in NLP lies in the use of pre-trained language representations that are trained on unlabeled data like BERT. Some brand new finetuning and probing paradigms are also proposed in recent years for the direction.

Transfer Learning for Low-resource NLP

1. Meta-learning.
2. Adversarial discriminators.
3. Adapters.

Ideas From Low-Resource Machine Learning in Non-NLP Communities

1. Data Augmentation
2. Transfer Learning
3.  Ideas From Low-Resource Machine Learning in Non-NLP Communities


University of Michigan - Ann Arbor

While it is possible to build English NLP systems for many higher-level applications, low-resource languages lack the data foundation for this. Additionally, even if it is possible to create basic systems for tasks, such as tokenization and named entity recognition, for all tested low-resource languages, the training data is typical of lower quality compared to the English datasets, or very limited in size.

Low-Resource NLP

https://aclanthology.org/2021.naacl-main.201.pdf

A Survey on Recent Approaches for Natural Language Processing in Low-Resource Scenarios


1. The availability of task-specific labels.
2. The availability of unlabeled language- or domain-specific text.
3. The availability of auxiliary data.

Low-resource Dimensions in NLP

The threholds vary according to different tasks. 10-60 tokens apply for POS tasks. 350k tokens apply for NLG tasks. 

Learn Before

Related

Learn After