In soft parameter sharing, each task has its own model with its own parameters. The distance between the parameters of the model is then regularized in order to encourage the parameters to be similar, as evidenced in figure below.

Hamilton College

In the context of Deep Learning, multi-task learning is typically done with either hard or soft parameter sharing of hidden layers.


Two MTL Methods for Deep Learning


Ruder, S. (n.d.). An Overview of Multi-Task Learning in Deep Neural Networks *. Retrieved May 22, 2022, from https://arxiv-download.xixiaoyao.cn/pdf/1706.05098.pdf

‌

Multi-Task Learning in Deep Neural Networks

Hard parameter sharing is the most commonly used approach to MTL in neural networks. It is generally applied by sharing the hidden layers between all tasks, while keeping several task-specific output layers as can be seen in the figure below.


Learn Before

Related