Definition

Condition Number of an Optimization Problem

The condition number of an optimization problem, denoted as κ\kappa, is defined as the ratio between the largest and the smallest eigenvalue of the problem's defining matrix (such as the Hessian or Q\mathbf{Q} in a quadratic objective). It is mathematically expressed as κ=Λ1Λd\kappa = \frac{\boldsymbol{\Lambda}_1}{\boldsymbol{\Lambda}_d}. When the condition number is large, indicating that some eigenvalues are significantly larger than others, the problem becomes exceedingly difficult to solve accurately because coordinates associated with small eigenvalues can experience dramatic, unstable changes, requiring optimization algorithms to carefully manage a large dynamic range of values.

0

1

Updated 2026-05-15

Contributors are:

Who are from:

Tags

D2L

Dive into Deep Learning @ D2L