Hard parameter sharing is the sharing of the hidden layers while keeping specific layers for each task separate from one another. It reduces overfitting since it tries to find similarities across tasks rather than overfitting the original task.

Hard Parmeter Sharing


Soft parameter sharing is the regularization between parameter distance to make the parameters similar. However, in soft parameter sharing each task has its own parameters.

Soft Parameter Sharing


1) Hard parameter sharing
2) Soft parameter sharing


University of Michigan - Ann Arbor

- Start off trying to have one neural network do multiple things (can actually lead to better performance)
- differences and similarities amongst tasks is utilized to output a better result

Multi-task Learning

- When you are training on a set of tasks that could benefit from having shared lower-level features
- Usually: amount of data you have for each task is quite similar
- Can train big enough neural networks that are able  to do well on all tasks


When to use multi-task learning?

Multi-Task Learning

Two types of Multi task learning methods


- Samples sizes are increased as a general relationship to be found amongst all parameters is typically the goal
- Multi task learning helps focus data attention to relevant noise, ignoring less relevant data
- Multi task learning is a regularizer meaning it reduces overfitting


Learn Before

Related

Learn After