Effect of Eigenvalues on Minimizer Sensitivity to Perturbation
When minimizing an eigendecomposed quadratic function, slight perturbations to the linear coefficient vector can cause disproportionate changes in the true minimizer . The sensitivity of the minimizer to these perturbations depends inversely on the magnitude of the eigenvalues of the quadratic matrix. Specifically, if the eigenvalues are large, perturbing results in only small changes to the minimizer's coordinates . Conversely, if the eigenvalues are small, even slight changes to can lead to dramatic, sensitive changes in the minimizer. This mathematical insight connects the eigendecomposition of a quadratic function to the definition of the condition number and the fundamental difficulty of ill-conditioned optimization.
0
1
Tags
D2L
Dive into Deep Learning @ D2L