Learn Before
Concept

Cost Function Smoothing in Continuation Methods

Continuation methods aim to minimize a complex cost function J(Θ)J(\Theta) by constructing a sequence of increasingly difficult cost functions {J(0),,J(n)}\left\{J^{(0)}, \dots, J^{(n)}\right\}, where J(0)J^{(0)} is the most smoothed (easiest to optimize) and J(n)=J(Θ)J^{(n)}=J(\Theta) is the original function. Smoothing is often achieved by "blurring" the cost function via sampling, such as approximating it with J^{(i)}(Theta)=mathop{mathbb{E}}_{Theta'simmathcal{N}(Theta';Theta,sigma^{(i)2})}J(Theta'). Optimization begins at J(0)J^{(0)}, where the blurred non-convex space resembles a convex one, providing an effective initial starting point for the subsequent function J(1)J^{(1)}. Iterating this process through J(n)J^{(n)} typically yields a superior local minimum for the original optimization problem, though it does not guarantee finding the global minimum.

Image 0

0

2

Updated 2026-06-27

Tags

Data Science