For a function whose input is a multi-dimensional vector and output is a scalar, a zero-gradient position represents a local maximum if all the eigenvalues of the function's Hessian matrix at that position are negative.

Local Maximum via Hessian Eigenvalues

A local maximum is a critical points which function value is higher than at all neighboringpoints.

University of Michigan - Ann Arbor

Claude

Points where the derivative of a function is 0 are known as critical points, or stationary points.

The derivative also has to be zero according to every possible directional derivative at that point in higher dimensional functions.

$$\triangledown f = 0$$

Critical Points

Goodfellow, I., Bengio, Y., & Courville, A. (2016). $\mathit{Deep \ Learning.}$ MIT Press. Retrieved from [www.deeplearningbook.org](https://www.deeplearningbook.org) 

Deep Learning

Dive into Deep Learning

Local Maximum

For a given objective function $$f(x)$$, a local minimum occurs at a point $$x$$ if the function's value $$f(x)$$ is smaller than its values at all other points within its immediate vicinity. In deep learning, objective functions usually possess many of these local optima. As an optimization algorithm nears a local minimum, the gradient approaches zero, which can cause the final numerical solution to only minimize the function locally rather than discovering the global minimum.

Local Minimum

A saddle point is a location where all gradients of a function vanish, but it is neither a global nor a local minimum. For example, the function $$f(x) = x^3$$ has a saddle point at $$x = 0$$, where its first and second derivatives are zero. Optimization algorithms might stall at these points because the gradient is zero, even though they do not represent true minima.

Learn Before

Related

Learn After