 Let z be a random variable representing an example for the learner (possibly an (x,y) pair for supervised learning). Let P(z) be the target training distribution from which the learner should ultimately learn a function of interest. Let $0 \leq W_{\lambda}(z) \leq 1$ be the weight applied to example z at step $\lambda$ in the curriculum sequence, with 
$0 \leq \lambda \leq 1$, and $W_{1}(z) = 1$. 
The corresponding training distribution at step $\lambda$ is 
$Q_{\lambda}(z)$ $\propto$ $W_{\lambda}(z) P(z)    \forall z$

Consider a monotonically increasing sequence of $\lambda$ values, starting from $\lambda = 0$ and ending at $\lambda = 1$.
The corresponding sequence of distributions $Q_{\lambda}$ a curriculum if the entropy of these distributions increases 
$H(Q_{\lambda}) < H(Q_{\lambda}+\epsilon) \forall \epsilon > 0$
and $W_{\lambda}(z)$ is monotonically increasing in $\lambda$, i.e., 
$W_{\lambda+\epsilon}(z) \geq W_{\lambda}(z) \forall z,\forall \epsilon > 0$. 

This builds up a sequential training sequence. Weights initially try and favor the simpler examples that can be learned relatively easily. The training undergoes adaptation in weighting to increase the probability of difficult examples entering training as a result of which the entropy increase.

Formalising Curriculum Learning

In the early iterations of this method, there is a strong reliance on defining the right curriculum and this would differ from one task to another and would need a good degree of effort.

A critical drawback of early curriculum based methods

https://qmro.qmul.ac.uk/xmlui/bitstream/handle/123456789/15972/Bengio%2C%202009%20Curriculum%20Learning.pdf?sequence=1&isAllowed=y

Curriculum Learning Reference

Curriculum learning is an approach in machine learning where tasks are sequentially structured so that their complexity gradually increases. This allows models to master simpler tasks first, which prepares them for more complex challenges as their skills develop. It is utilized as a method to mitigate the impact of sparse rewards in reinforcement learning by providing an organized path of learning.

University of Michigan - Ann Arbor

Google

The underlying premise:
- In high dimensional feature spaces, there is a very large likelihood of encountering multiple local minima. 
- This is compounded with the fact that the feature space isn’t well behaved i.e. an inevitable presence of plateaus, saddle points affecting the magnitude of the gradient along with the direction of the gradient not pointing towards the approximate region of lower cost.

Thus, being able to initialize parameters in a region smaller than the feature space itself and one that behaves well enough for the descent to lead to a reasonable solution is powerful. This is exactly what continuation methods try to achieve.

Continuation Methods

Reference of Foundations of Large Language Models Course

The goal to minimize the cost function $J(\Theta)$ is achieved with the following adaptation. A set of cost functions {$J^{(0)},...,J^{(n)}$} with increasing difficulty are constructed such that $J^{(0)}$ is the easiest to optimize and $J^{(n)}=J(\Theta)$ has the highest level of difficulty.

What does “easier to optimize” mean?
- It means that it is relatively more well-behaved i.e. smoother over the same region.

This gets us a good initial start point for $J^{(i+1)}$. Continuing this would result in getting us very close to solving the $J(\Theta)$ optimization problem.

How do we get cost functions to behave well in the same region?
- By “blurring” out the cost function $J(\Theta)$ i.e. by approximating $J^{(i)}(\Theta)=\mathop{\mathbb{E}}_{\Theta^{’}\sim\mathcal{N}(\Theta_{‘};\Theta,\sigma^{(i)2})}J(\Theta^{‘})$ via sampling. The intended effect is that the non-convex problem starts to look like a convex one. 

It is important to note that this method generally does not get us the global minima but does get us a superior local minima.


Goal of Continuation Methods

So is this the solution to optimizing all non-convex problems?
- Not really. In theory, the effect that a complex non-convex feature space has on the application of the continuation method would result in a very large number of objective functions which, as a result, would make the problem computationally intractable. (NP-hard optimization problems remain NP-hard even in this case).
- Another factor to consider is that non-convex problems might remain non-convex whatever the extent of blurring in some cases. E.g. $J(\Theta)=-\Theta^{T}\Theta$.
- The pitfall of blurring from the intuitive standpoint works here as well i.e. blurring out space could lead to a false convex and in turn force the solution path to inferior local minima.

The initial problem with large feature spaces was believed to be that of the local minima and issues like saddle points, etc that surround them. But, high-dimensional spaces behave very differently. The bigger problem that surrounds them is that of sparsity. 

Continuation methods as a solution to all non-convex problems?

So, can continuation methods still be of use?
Yes. they do really well at:
-  reducing estimation variance (The intuitive effect of blurring applies here too) in the gradients, 
- almost eliminate flat regions and 
- improve the conditioning of the Hessian matrix thereby reducing instability.

The real utility of continuation methods

Curriculum Learning

https://people.csail.mit.edu/hmobahi/pubs/aaai_2015.pdf

Learn Before

Related

Learn After