Learn Before
Concept
Limitation of Gradient Clipping
While gradient clipping is an effective regularization technique to mitigate the issue of exploding gradients during neural network training, it is unable to address the problem of vanishing gradients. Clipping only bounds the maximum magnitude of the gradient to prevent excessively large updates, but it provides no mechanism to prevent the gradient from becoming infinitesimally small.
0
1
Updated 2026-05-14
Tags
D2L
Dive into Deep Learning @ D2L