This class of techniques is also sometimes referred to as Mixed Sample Data Augmentation (MSDA). In the nlp area, related work often interpolates inner components, more general mixing schemes, and add adversaries.

Example Interpolation in NLP

In machine learning, interpolation of training data occurs when an over-parameterized model perfectly fits the training dataset, achieving zero training error. Because large neural networks possess vastly more parameters than necessary to fit the data, they have the capacity to memorize every training example. This tendency to interpolate is a key mechanism that allows deep neural networks to behave more like nonparametric models.

Interpolation of Training Data

Bilinear interpolation is a commonly used upsampling technique in image processing to scale up an image. To calculate a pixel at coordinate $$(x, y)$$ in the upsampled output image, the technique maps $$(x, y)$$ to a corresponding real-valued coordinate $$(x', y')$$ on the original input image based on the ratio of the input size to the output size. It then identifies the four pixels closest to $$(x', y')$$ on the input image and calculates the final output pixel value based on these four pixels and their relative distances from $$(x', y')$$. In deep learning, this technique is frequently used to initialize transposed convolutional layers.

Bilinear Interpolation Upsampling

Interpolation is the process of estimating unknown values that fall between known values.

University of Michigan - Ann Arbor

Data augmentation in data analysis are techniques used to increase the amount of data by adding slightly modified copies of already existing data or newly created synthetic data from existing data. 

Data Augmentation

https://arxiv-download.xixiaoyao.cn/pdf/2105.03075.pdf

A Survey of Data Augmentation Approaches for NLP

Data augmentation (DA) refers to strategies for increasing the diversity of training examples without explicitly collecting new data. Related methods are first explored in computer vision (CV), and DA’s adaptation for natural language processing (NLP) seems secondary and comparatively underexplored.

Data Augmentation in NLP

Rule-based data augmentation is to apply basic changes to the dataset which enlarges the divergence that the neural network sees.

Rule-based Data Augmentation

It mixes data and labels of real samples in the exact same way to create more available data.

Learn Before

Related

Learn After

Learn After

Related