Learn Before
Formula
Common Learning Rate Decay Formula
, where is the learning rate in the current epoch, is the initial learning rate, is the current epoch, and is the selected decay rate. The decay rate is a tunable hyperparameter. Initializing and , we can graph an example with on the x-axis and on the y-axis to observe the decay of the learning rate.
0
2
Updated 2026-06-16
Contributors are:
Who are from:
Tags
Data Science
Related
Example Using Mini-Batch Gradient Descent (Learning Rate Decay)
Manual Implementation of Learning Rate Decay
Alternative Learning Rate Decay Formulas
Common Learning Rate Decay Formula
Which of these statements about mini-batch gradient descent do you agree with?
Mini-Batch Gradient Descent Algorithm
Common Learning Rate Decay Formula