Learn Before
Concept

Limitation of Gradient Clipping

While gradient clipping is an effective regularization technique to mitigate the issue of exploding gradients during neural network training, it is unable to address the problem of vanishing gradients. Clipping only bounds the maximum magnitude of the gradient to prevent excessively large updates, but it provides no mechanism to prevent the gradient from becoming infinitesimally small.

0

1

Updated 2026-05-14

Contributors are:

Who are from:

Tags

D2L

Dive into Deep Learning @ D2L