Sometimes, higher order derivatives are needed for our models to learn. If we needed the second order derivatives, we could use the Hessian matrix. However, there are often millions or even billions of parameters in our models, so the Hessian is extremely difficult to calculate and represent.

For some function $f:\mathbb{R}^n\rightarrow \mathbb{R}$ with a Hessian $\bold{H}$, and an arbitrary vector $v$:

$\bold{H}v=\nabla_{\bold{x}}[(\nabla_{\bold{x}}f(x))^{\top}v]$