Let $\mathcal{H}$ be the hypothetical model space defined by Few-Shot Learning, $\hat{h}$ be the model that minimize expected risk (possibly not a Few-Shot earning model), $h^{*}$ be the model in $\mathcal{H}$ that minimizes expected risk, $h_I$ be the model in $\mathcal{H}$ that minimizes empirical risk, then the total error can be decomposed as $\mathcal{E}[R(h_I)-R(\hat{h})]=\mathcal{E}[R(h^*)-R(\hat{h})]+\mathcal{E}[R(h_I)-R(h^*)]$.

The first term is Approximation Error which is affected by $\mathcal{H}$, and the second term is Estimation Error which is affected by the quantity and quality of training data.

The problem of few-shot learning is that the training sample size is small, thus causing higher estimation error, and resulting in unreliable $h_I$.


Core Issue of Few-Shot Learning

Let $h$ be the hypothetical model, $x$ be predictors, $y$ be predicted variable, $p(x,y)$ be joint probability density of $x$ and $y$, then the Expected Risk with respect to $h$ can be defined as $R(h)=\mathbf{E}[l(h(x), y)]$.

Because $p(x,y)$ is unknown, expected risk can be estimated by Empirical Risk, which is defined as $R_I(h)=\frac{1}{I}\sum_{i=1}^{I} l(h(x_i), y_i)$.


University of Notre Dame

Real-world relation distributions are long-tail. This means only the common relations obtain sufficient training instances and most relations have very limited relational facts and corresponding sentences. Few-shot learning focuses on grasping tasks with only a few training examples and is a good fit for this need. The general idea of few-shot models is to train good representations of instances or learn ways of fast adaptation from existing large-scale data, and then transfer to new tasks.

Let T be some task, E be past experience related to this classes of task T, and P be performance measure of T. Few-Shot learning is a type of machine learning problem which is specified by E, T, and P, but only has limited E either quantitatively or qualitatively. E typically includes supervised information and prior knowledge.

Performing more efficient learning with few-shot learning for RE

Wang, Y., Yao, Q., Kwok, J. T., & Ni, L. M. (2020). Generalizing from a few examples: A survey on few-shot learning. ACM computing surveys (csur), 53(3), 1-34.

Generalizing from a Few Examples: A Survey on Few-Shot Learning

There are mainly two ways for handling few-shot learning for this purpose:
- metric learning
- meta learning

Ways for handling few-shot learning for RE

1.	Few-Shot Classification learns a classification function $\hat{h}$ and output a class label $y_i$ for each predictor $x_i$.
2.	Few-Shot Regression learns a regression function $\hat{h}$ and output a continuous value $y_i$ for each predictor $x_i$.
3.	Few-Shot Reinforcement Learning learns a policy from a trajectory of state-action pairs.


Types of Few-Shot Learning:

1)	Weakly Supervised Learning

2)	Imbalanced Learning

3)	Transfer Learning

4)	Meta Learning

Learning Problem Relevant to Few-Shot Learning

Loss function of Few-Shot Learning

Few-Shot Learning can be divided by what part that prior knowledge has been added to:

1)	Data: The training data is augmented by prior knowledge

2)	Model: The possible model space is constrained according to prior knowledge

3)	Algorithm: The search strategy in hypothesis model space has been improved.

Learn Before

Related

Learn After