Learn Before
Concept
Continuation Methods
The underlying premise:
- In high dimensional feature spaces, there is a very large likelihood of encountering multiple local minima.
- This is compounded with the fact that the feature space isn’t well behaved i.e. an inevitable presence of plateaus, saddle points affecting the magnitude of the gradient along with the direction of the gradient not pointing towards the approximate region of lower cost.
Thus, being able to initialize parameters in a region smaller than the feature space itself and one that behaves well enough for the descent to lead to a reasonable solution is powerful. This is exactly what continuation methods try to achieve.
0
2
Updated 2021-06-24
Tags
Data Science