Effect of Eigenvalues on Minimizer Sensitivity to Perturbation
When minimizing an eigendecomposed quadratic function, slight perturbations to the linear coefficient vector can cause disproportionate changes in the true minimizer . The sensitivity of the minimizer to these perturbations depends inversely on the magnitude of the eigenvalues boldsymbol{Lambda}_i of the quadratic matrix. Specifically, if the eigenvalues are large, perturbing results in only small changes to the minimizer's coordinates . Conversely, if the eigenvalues are small, even slight changes to can lead to dramatic, sensitive changes in the minimizer. This mathematical insight connects the eigendecomposition of a quadratic function to the definition of the condition number and the fundamental difficulty of ill-conditioned optimization.
0
1
Tags
D2L
Dive into Deep Learning @ D2L