Metric learning learns a semantic metric on existing data and classifies queries by comparing them with training examples. Most metric learning models perform distance measurement on sentence-level representation, but it is also possible to utilize token-level attention for finer-grained comparison.

Metric learning for few-shot learning

Meta learning aims to grasp the way of parameter initialization and optimization through the experience grain on the meta-train data. 


Meta-learning for few shot learning


There are mainly two ways for handling few-shot learning for this purpose:
- metric learning
- meta learning

Columbia University

Real-world relation distributions are long-tail. This means only the common relations obtain sufficient training instances and most relations have very limited relational facts and corresponding sentences. Few-shot learning focuses on grasping tasks with only a few training examples and is a good fit for this need. The general idea of few-shot models is to train good representations of instances or learn ways of fast adaptation from existing large-scale data, and then transfer to new tasks.

Let T be some task, E be past experience related to this classes of task T, and P be performance measure of T. Few-Shot learning is a type of machine learning problem which is specified by E, T, and P, but only has limited E either quantitatively or qualitatively. E typically includes supervised information and prior knowledge.

Performing more efficient learning with few-shot learning for RE

Ways for handling few-shot learning for RE

1.	Few-Shot Classification learns a classification function $\hat{h}$ and output a class label $y_i$ for each predictor $x_i$.
2.	Few-Shot Regression learns a regression function $\hat{h}$ and output a continuous value $y_i$ for each predictor $x_i$.
3.	Few-Shot Reinforcement Learning learns a policy from a trajectory of state-action pairs.


Types of Few-Shot Learning:

1)	Weakly Supervised Learning

2)	Imbalanced Learning

3)	Transfer Learning

4)	Meta Learning

Learning Problem Relevant to Few-Shot Learning

Let $h$ be the hypothetical model, $x$ be predictors, $y$ be predicted variable, $p(x,y)$ be joint probability density of $x$ and $y$, then the Expected Risk with respect to $h$ can be defined as $R(h)=\mathbf{E}[l(h(x), y)]$.

Because $p(x,y)$ is unknown, expected risk can be estimated by Empirical Risk, which is defined as $R_I(h)=\frac{1}{I}\sum_{i=1}^{I} l(h(x_i), y_i)$.


Loss function of Few-Shot Learning

Few-Shot Learning can be divided by what part that prior knowledge has been added to:

1)	Data: The training data is augmented by prior knowledge

2)	Model: The possible model space is constrained according to prior knowledge

3)	Algorithm: The search strategy in hypothesis model space has been improved.

Learn Before

Related

Learn After