Weakly supervised learning learns from E with limited supervised data, such as incomplete, inaccurate, inexact or noisy supervised information.

Weakly Supervised Learning

Imbalanced Learning learns from E that is imbalanced with the distribution of $y$.

Imbalanced Learning

Transfer Learning transfer the learned knowledge from one task/source with abundant training data to the current task/source with limited training data. 

Transfer Learning

Meta learning improves learning performance by first using meta-model to extract knowledge from meta data, and then aggregating meta models for the current task using task-specific information.

Meta Learning 

1)	Weakly Supervised Learning

2)	Imbalanced Learning

3)	Transfer Learning

4)	Meta Learning

University of Notre Dame

Real-world relation distributions are long-tail. This means only the common relations obtain sufficient training instances and most relations have very limited relational facts and corresponding sentences. Few-shot learning focuses on grasping tasks with only a few training examples and is a good fit for this need. The general idea of few-shot models is to train good representations of instances or learn ways of fast adaptation from existing large-scale data, and then transfer to new tasks.

Let T be some task, E be past experience related to this classes of task T, and P be performance measure of T. Few-Shot learning is a type of machine learning problem which is specified by E, T, and P, but only has limited E either quantitatively or qualitatively. E typically includes supervised information and prior knowledge.

Performing more efficient learning with few-shot learning for RE

Wang, Y., Yao, Q., Kwok, J. T., & Ni, L. M. (2020). Generalizing from a few examples: A survey on few-shot learning. ACM computing surveys (csur), 53(3), 1-34.

Generalizing from a Few Examples: A Survey on Few-Shot Learning

There are mainly two ways for handling few-shot learning for this purpose:
- metric learning
- meta learning

Ways for handling few-shot learning for RE

1.	Few-Shot Classification learns a classification function $\hat{h}$ and output a class label $y_i$ for each predictor $x_i$.
2.	Few-Shot Regression learns a regression function $\hat{h}$ and output a continuous value $y_i$ for each predictor $x_i$.
3.	Few-Shot Reinforcement Learning learns a policy from a trajectory of state-action pairs.


Types of Few-Shot Learning:

Learning Problem Relevant to Few-Shot Learning

Let $h$ be the hypothetical model, $x$ be predictors, $y$ be predicted variable, $p(x,y)$ be joint probability density of $x$ and $y$, then the Expected Risk with respect to $h$ can be defined as $R(h)=\mathbf{E}[l(h(x), y)]$.

Because $p(x,y)$ is unknown, expected risk can be estimated by Empirical Risk, which is defined as $R_I(h)=\frac{1}{I}\sum_{i=1}^{I} l(h(x_i), y_i)$.


Loss function of Few-Shot Learning

Few-Shot Learning can be divided by what part that prior knowledge has been added to:

1)	Data: The training data is augmented by prior knowledge

2)	Model: The possible model space is constrained according to prior knowledge

3)	Algorithm: The search strategy in hypothesis model space has been improved.

Learn Before

Related

Learn After