The mean absolute error (MAE) is one type of cost function that predicts the median value of $$y$$ for each $$x$$, as long as such a function may be described by the family of functions that are optimized over. Unfortunately, the mean absolute error may often end up with poor results when used with gradient-based optimization.

South Dakota State University

Google

Sometimes we only need a single conditional statistic of $$y$$ given $$x$$ (such as its mean or median) rather than the full conditional distribution $$p(y \mid x; \theta)$$. In that case we can minimize a cost functional — a mapping from functions to real numbers — instead of an ordinary cost function. This lets us treat learning as choosing a function rather than a set of parameters: the cost functional is designed so that its minimum is attained at the function that outputs the desired statistic. Optimizing over functions in this way relies on the calculus of variations.

Learning Conditional Statistics

A functional is a mapping from functions to real numbers.

Functional

The **calculus of variations** is a mathematical tool used to solve optimization problems where the goal is to find an optimal function rather than a set of parameters. For example, it can be used to choose a function that minimizes a given cost functional.

Learn Before

Related