Learn Before
Concept

Goal of Continuation Methods

The goal to minimize the cost function J(Θ)J(\Theta) is achieved with the following adaptation. A set of cost functions \left\{J^{(0)},...,J^{(n)} ight\} with increasing difficulty are constructed such that J(0)J^{(0)} is the easiest to optimize and J(n)=J(Θ)J^{(n)}=J(\Theta) has the highest level of difficulty. What does “easier to optimize” mean? - It means that it is relatively more well-behaved i.e. smoother over the same region. This gets us a good initial start point for J(i+1)J^{(i+1)}. Continuing this would result in getting us very close to solving the J(Θ)J(\Theta) optimization problem. How do we get cost functions to behave well in the same region? - By “blurring” out the cost function J(Θ)J(\Theta) i.e. by approximating J(i)(Θ)=EΘN(Θ;Θ,σ(i)2)J(Θ)J^{(i)}(\Theta)=\mathop{\mathbb{E}}_{\Theta'\sim\mathcal{N}(\Theta';\Theta,\sigma^{(i)2})}J(\Theta') via sampling. The intended effect is that the non-convex problem starts to look like a convex one. It is important to note that this method generally does not get us the global minima but does get us a superior local minima.

Image 0

0

2

Updated 2026-05-17

Tags

Data Science