https://qmro.qmul.ac.uk/xmlui/bitstream/handle/123456789/15972/Bengio%2C%202009%20Curriculum%20Learning.pdf?sequence=1&isAllowed=y

University of Michigan - Ann Arbor

Curriculum learning is an approach in machine learning where tasks are sequentially structured so that their complexity gradually increases. This allows models to master simpler tasks first, which prepares them for more complex challenges as their skills develop. It is utilized as a method to mitigate the impact of sparse rewards in reinforcement learning by providing an organized path of learning.

Curriculum Learning

 Let z be a random variable representing an example for the learner (possibly an (x,y) pair for supervised learning). Let P(z) be the target training distribution from which the learner should ultimately learn a function of interest. Let $0 \leq W_{\lambda}(z) \leq 1$ be the weight applied to example z at step $\lambda$ in the curriculum sequence, with 
$0 \leq \lambda \leq 1$, and $W_{1}(z) = 1$. 
The corresponding training distribution at step $\lambda$ is 
$Q_{\lambda}(z)$ $\propto$ $W_{\lambda}(z) P(z)    \forall z$

Consider a monotonically increasing sequence of $\lambda$ values, starting from $\lambda = 0$ and ending at $\lambda = 1$.
The corresponding sequence of distributions $Q_{\lambda}$ a curriculum if the entropy of these distributions increases 
$H(Q_{\lambda}) < H(Q_{\lambda}+\epsilon) \forall \epsilon > 0$
and $W_{\lambda}(z)$ is monotonically increasing in $\lambda$, i.e., 
$W_{\lambda+\epsilon}(z) \geq W_{\lambda}(z) \forall z,\forall \epsilon > 0$. 

This builds up a sequential training sequence. Weights initially try and favor the simpler examples that can be learned relatively easily. The training undergoes adaptation in weighting to increase the probability of difficult examples entering training as a result of which the entropy increase.

Formalising Curriculum Learning

In the early iterations of this method, there is a strong reliance on defining the right curriculum and this would differ from one task to another and would need a good degree of effort.

Learn Before

Related