The mean absolute error is one type of cost function that predicts the median value of y for each x, as long as such a function may be described by the family of functions that are optimized over. Unfortunately, mean absolute error may often end up with poor results when used with gradient-based optimization.

South Dakota State University

When we want to learn only one conditional statistic of y given x, instead of learning a full probability distribution p(y | x;θ), we can use a cost functional instead of just a cost function. In this way, we can view learning as choosing a function instead of choosing a set of parameters and our cost functional can be designed to have its minimum occur at a desired specific function. 

Learning Conditional Statistics

A functional is a mapping from functions to real numbers.

Functional

The calculus of variations is a mathematical tool that helps with solving an optimization problem with respect to a function. 

Learn Before

Related