For Newton’s method, saddle points clearly constitute a problem. Gradient descent is designed to move “downhill” and is not explicitly designed to seek acritical point. Newton’s method, however, is designed to solve for a point where the gradient is zero. Without appropriate modiﬁcation, it can jump to a saddle point.

Newton's Method and Saddle Points

Some critical points are neither maxima nor minima. These are known as saddle points.

University of Michigan - Ann Arbor

Points where the derivative of a function is 0 are known as critical points, or stationary points.

The derivative also has to be zero according to every possible directional derivative at that point in higher dimensional functions.

$$\triangledown f = 0$$

Critical Points

Goodfellow, I., Bengio, Y., & Courville, A. (2016). $\mathit{Deep \ Learning.}$ MIT Press. Retrieved from [www.deeplearningbook.org](https://www.deeplearningbook.org) 

Deep Learning

A local maximum is a critical points which function value is higher than at all neighboringpoints.

Local Maximum

A local minimum is a critical point whose function value is lower than all neighboring points.

Learn Before

Related

Learn After