To reduce the high estimation error caused by small training sample sizes, few-shot learning methods utilize prior knowledge. These methods are typically categorized into three perspectives: the data perspective (augmenting training data), the model perspective (constraining the hypothesis space), and the algorithm perspective (guiding search strategies to find reliable parameters).

Google

Let $$\mathcal{H}$$ be the hypothesis space defined by Few-Shot Learning, $$\hat{h}$$ be the model that minimizes expected risk (which may not be in $$\mathcal{H}$$), $$h^{*}$$ be the model in $$\mathcal{H}$$ that minimizes expected risk, and $$h_I$$ be the model in $$\mathcal{H}$$ that minimizes empirical risk. The total error can be decomposed as:
$$\mathcal{E}[R(h_I) - R(\hat{h})] = \mathcal{E}[R(h^*) - R(\hat{h})] + \mathcal{E}[R(h_I) - R(h^*)]$$
The first term is the Approximation Error, which is determined by the hypothesis space $$\mathcal{H}$$. The second term is the Estimation Error, which is affected by the quantity and quality of the training data. The core issue of few-shot learning is that the small training sample size leads to a high estimation error, resulting in an unreliable empirical minimizer $$h_I$$.

Core Issue of Few-Shot Learning

Wang, Y., Yao, Q., Kwok, J. T., & Ni, L. M. (2020). Generalizing from a few examples: A survey on few-shot learning. ACM computing surveys (csur), 53(3), 1-34.

Learn Before

Related