A function of a function $$f$$ is called a **functional** $$J[f]$$.

Functionals

The **calculus of variations** is a mathematical tool used to solve optimization problems where the goal is to find an optimal function rather than a set of parameters. For example, it can be used to choose a function that minimizes a given cost functional.

University of Connecticut

South Dakota State University

Google

When we want to learn only one conditional statistic of y given x, instead of learning a full probability distribution p(y | x;θ), we can use a cost functional instead of just a cost function. In this way, we can view learning as choosing a function instead of choosing a set of parameters and our cost functional can be designed to have its minimum occur at a desired specific function. 

Learning Conditional Statistics

A functional is a mapping from functions to real numbers.

Functional

Calculus of variations

The mean absolute error (MAE) is one type of cost function that predicts the median value of $$y$$ for each $$x$$, as long as such a function may be described by the family of functions that are optimized over. Unfortunately, the mean absolute error may often end up with poor results when used with gradient-based optimization.

Learn Before

Related

Learn After