When using a training dataset ($$D_{train}$$) to refine initial parameters ($$\theta_0$$) in few-shot learning, several techniques can be used to improve the results. These strategies include fine-tuning existing parameters by regularization, aggregating a set of parameters, and fine-tuning existing parameters with new parameters.

Strategies for Fine-Tuning Existing Parameters in Few-Shot Learning

In few-shot learning, the strategy of refining existing parameters uses initial parameters ($$\theta_0$$) from pre-trained models related to the current task as a starting point. It then uses the training dataset ($$D_{train}$$) to refine and find better parameters.

University of Notre Dame

Google

In terms of how searching strategy is affected by prior knowledge, searching algorithm motivated few-shot learning methods can be categorized as:

1)	Refining existing parameters

2)	Refining meta-learned parameters.

3)	Learning the optimizer


Types of few-shot learning by algorithms

Wang, Y., Yao, Q., Kwok, J. T., & Ni, L. M. (2020). Generalizing from a few examples: A survey on few-shot learning. ACM computing surveys (csur), 53(3), 1-34.

Generalizing from a Few Examples: A Survey on Few-Shot Learning

Modify Search Algorithm by Refining Existing Parameters

This strategy uses meta-knowledge to learn an optimizer, which can directly output a step size and step direction.

Modify Search Algorithm by Learning the Optimizer

In few-shot learning, the strategy of refining meta-learned parameters uses meta-learners to learn an initial parameter set ($$\theta_0$$) on tasks similar to the target task. The few-shot task then further refines this $$\theta_0$$. In contrast to refining existing parameters, which uses a fixed initial $$\theta_0$$, this approach dynamically learns $$\theta_0$$ using a meta-learner.

Learn Before

Related

Learn After