Generative modeling assumes that there exists a distribution $$p(x)$$ of $$x$$, and aims to estimate this distribution. This typically involves estimating $$p(x|y)$$ and $$p(y)$$. Generative modeling uses prior knowledge by assuming that $$p(x;\theta)$$ is parameterized by $$\theta$$, and introduces a latent variable $$z$$ with distribution $$p(z;\gamma)$$. Thus, $$p(x;\theta)$$ can be estimated by integrating $$p(x|z;\theta)p(z;\gamma)$$ over $$z$$. The distribution $$p(z;\gamma)$$ is estimated using prior knowledge.

University of Notre Dame

Google

1)	Multitask Learning

2)	Embedding Learning

3)	Learning with External Memory

4)	Generative Modeling


Types of Few Shot Learning by Model

Wang, Y., Yao, Q., Kwok, J. T., & Ni, L. M. (2020). Generalizing from a few examples: A survey on few-shot learning. ACM computing surveys (csur), 53(3), 1-34.

Generalizing from a Few Examples: A Survey on Few-Shot Learning

Suppose there’s task $T_1,T_2,..,T_C$, each task has a training set $\{D_{train}^{c}, D_{testing}^{c}\}$, and some of them has larger training set, while some of them have smaller training set. The few shot tasks typically have less training set and are our target task. Multitask learning models are jointly trained on these tasks, i.e., parameters for one task are constrained by other tasks.

Multitask Learning

Embedding learning constrains the searching space $$\mathcal{H}$$ by mapping each input $$x_i \in \mathbb{R}^n$$ to a lower-dimensional space $$z_i \in \mathbb{R}^m$$, where $$m < n$$. The mapping process is achieved by an embedding function $$f(\cdot)$$. We can achieve better accuracy by using different embedding functions $$f_{\text{train}}$$ and $$f_{\text{test}}$$ for the training and testing sets. The prediction on the testing set is made by finding the closest training sample to a given testing sample in the embedded space and predicting the same label for the testing sample as the training sample.

Embedding Learning

Learning with external memory first maps the training sample to a lower dimension space using the an embedding function, and save the mapped value and class labels as key-value pair to computers. For a given testing sample, it’s first transformed to the same low dimension space, and then the most similar key-value pairs in computer memory are extracted and combined to form a representation of this testing sample. Lastly, a simple prediction function is used to predict the class of this testing sample.

Learn Before

Related