If the gradient g is ill-conditioned, it will cost more time to train the model since in this situation the learning rate must shrink to cancel out the increase of gradient norm.

University of Minnesota - Twin Cities

Conditioning refers to how rapidly the output of a function changes in response to small changes in its input. This sensitivity is quantified by the $$\textbf{condition number}$$, which measures the rate of change of the function as the input varies.

Function Conditioning

Goodfellow, I., Bengio, Y., & Courville, A. (2016). $\mathit{Deep \ Learning.}$ MIT Press. Retrieved from [www.deeplearningbook.org](https://www.deeplearningbook.org) 

Learn Before

Related