Formula
Gradient Magnitude and Eigendecomposition
By analyzing a quadratic optimization problem in its eigendecomposed form , the gradient can be expressed as , where is the true minimizer. This relationship demonstrates that the magnitude of the gradient is directly proportional to both the eigenvalue matrix and the current distance from the optimal solution. When the distance from optimality remains relatively stable, the gradient's magnitude alone provides sufficient proportional information regarding the underlying scale of the Hessian matrix.
0
1
Updated 2026-05-15
Tags
D2L
Dive into Deep Learning @ D2L