Code

Scalar Quadratic Gradient Descent Convergence Visualization

The convergence dynamics of gradient descent on a scalar quadratic function f(x)=λ2x2f(x) = \frac{\lambda}{2} x^2 can be visualized programmatically by plotting the decay term (1ηλ)t(1 - \eta \lambda)^t over successive time steps tt. For instance, fixing the learning rate at η=0.1\eta = 0.1 and iterating over a set of curvature values (such as λ{0.1,1,10,19}\lambda \in \{0.1, 1, 10, 19\}) produces distinct trajectory curves. These plots practically reveal that values satisfying 0<ηλ10 < \eta \lambda \le 1 yield smooth exponential decay, values where 1<ηλ<21 < \eta \lambda < 2 exhibit oscillatory but converging behavior, and values approaching the theoretical limit of ηλ=2\eta \lambda = 2 show dangerously slow convergence before divergence occurs.

Image 0

0

1

Updated 2026-05-15

Contributors are:

Who are from:

Tags

D2L

Dive into Deep Learning @ D2L