Definition
Condition Number of an Optimization Problem
The condition number of an optimization problem, denoted as , is defined as the ratio between the largest and the smallest eigenvalue of the problem's defining matrix (such as the Hessian or in a quadratic objective). It is mathematically expressed as . When the condition number is large, indicating that some eigenvalues are significantly larger than others, the problem becomes exceedingly difficult to solve accurately because coordinates associated with small eigenvalues can experience dramatic, unstable changes, requiring optimization algorithms to carefully manage a large dynamic range of values.
0
1
Updated 2026-05-15
Tags
D2L
Dive into Deep Learning @ D2L