In mathematical optimization, an analytical solution represents an exact closed-form formula to find the minimum or maximum, whereas a numerical solution relies on iterative algorithms to approximate the optimal value. Because deep learning objective functions are typically highly complex and non-convex, they almost never have exact analytical solutions. Consequently, deep learning models must be trained using numerical optimization algorithms.