Parameter sharing directly shares parameters across different tasks and models to promote the training of few-shot tasks.

Parameter Sharing

Parameter tying encourages the similarities between parameters, and this is typically done by adding regularization. For example, any differences in parameters pairwise could be punished.

Parameter Tying

There are two strategies for constraining parameters of one task by other tasks:

1)	Parameter Sharing

2)	Parameter Tying



University of Notre Dame

Suppose there’s task $T_1,T_2,..,T_C$, each task has a training set $\{D_{train}^{c}, D_{testing}^{c}\}$, and some of them has larger training set, while some of them have smaller training set. The few shot tasks typically have less training set and are our target task. Multitask learning models are jointly trained on these tasks, i.e., parameters for one task are constrained by other tasks.

Multitask Learning

Wang, Y., Yao, Q., Kwok, J. T., & Ni, L. M. (2020). Generalizing from a few examples: A survey on few-shot learning. ACM computing surveys (csur), 53(3), 1-34.

Learn Before

Related

Learn After