A computational graph is a directed dependency graph where nodes represent mathematical operations or variables, mapping the functional dependency from inputs to outputs. When evaluating a vector-valued function, this graph is traversed in a forward direction. Conversely, to compute gradients automatically, the graph is traced in a backwards direction from output to input, which involves multiplying matrices of partial derivatives along the paths.

Computational Graph

The partial derivative of a multivariate function $$y = f(x_1, x_2, \ldots, x_n)$$ with respect to its $$i^	extrm{th}$$ parameter $$x_i$$ measures how the function changes as $$x_i$$ increases while treating all other variables as constants. Formally, it is defined as the limit: $$\frac{\partial y}{\partial x_i} = \lim_{h ightarrow 0} \frac{f(x_1, \ldots, x_{i-1}, x_i+h, x_{i+1}, \ldots, x_n) - f(x_1, \ldots, x_i, \ldots, x_n)}{h}$$.

Partial Derivative

While single-variable calculus deals with functions of one variable, deep learning often requires working with functions of many variables, known as multivariate functions. A multivariate function takes an $$n$$-dimensional input, representing multiple parameters, and evaluates them to produce an output. This is typically written as $$y = f(x_1, x_2, \ldots, x_n)$$, where the variables $$x_1$$ through $$x_n$$ jointly determine the value of the function.

Claude

A mathematical function is a specific type of relation that assigns to each element in its domain exactly one element in its range. In terms of ordered pairs, a relation is a function if and only if each $$x$$-value (input) is matched with exactly one $$y$$-value (output).

Mathematical Function

Dive into Deep Learning

A relatable way to understand mathematical functions is through the analogy of birthdays. A relation mapping people to their birthdays represents a mathematical function because every person in the domain has exactly one birthday in the range. It is perfectly acceptable for two different people to share the same birthday, just as two different $$x$$-values can correspond to the same $$y$$-value in a function. However, if one person were to have two different birthdays, it would violate the definition of a mathematical function, illustrating how a single $$x$$-value cannot map to multiple $$y$$-values.

Birthday Analogy for Functions

To determine whether a relation given as a set of ordered pairs is a mathematical function, examine the first coordinates. If each $$x$$-value is paired with only one $$y$$-value, the relation is a function. However, if any single $$x$$-value corresponds to two or more different $$y$$-values, the relation is not a function.

Determining if a Set of Ordered Pairs is a Function

The vertical line test is a visual method used to determine whether a set of points graphed in a rectangular coordinate system represents a function. According to this test, if any vertical line intersects the graph at more than one point, the graph does not represent a mathematical function, because this indicates a single $$x$$-value (input) is mapped to multiple $$y$$-values (outputs). Conversely, if every possible vertical line drawn on the coordinate plane intersects the graph in at most one point, then every $$x$$-value corresponds to exactly one $$y$$-value, confirming that the relation is indeed a function.

Vertical Line Test

The cube function is formally defined by the equation $$f(x) = x^3$$. Since any real numerical value can be safely substituted for $$x$$, its domain consists of all real numbers, which is written in interval notation as $$(-\infty, \infty)$$. Furthermore, cubing a positive value results in a positive output, cubing a negative value results in a negative output, and cubing zero produces zero; thus, the range also encompasses all real numbers, or $$(-\infty, \infty)$$. Its graphical representation is a distinct, smooth curve that flows continuously through the origin $$(0, 0)$$ and stretches infinitely outward in both directions.

Cube Function

The square root function is algebraically defined by the equation $$f(x) = \sqrt{x}$$. Since the square root of a negative value does not produce a real number, the permissible inputs are strictly limited to non-negative real numbers, making the domain $$[0, \infty)$$. Additionally, because the principal square root is defined to be non-negative, the resulting outputs are also zero or positive, establishing the range as $$[0, \infty)$$. Visually, its graph is a steady curve that originates exactly at the origin $$(0, 0)$$ and arcs upward and to the right.

Square Root Function

A **polynomial function** is a function whose range values are defined by a polynomial expression. For example, the functions $$f(x) = x^2 + 5x + 6$$ and $$g(x) = 3x - 4$$ are considered polynomial functions because their defining expressions, $$x^2 + 5x + 6$$ and $$3x - 4$$, are polynomials.

Polynomial Function

For any mathematical function $$f$$, if $$f(x) = 0$$, then $$x$$ is a zero of the function. Because the function's output is zero at this input value, the corresponding point $$(x, 0)$$ represents an x-intercept on the graph of the function.

Zero of a Function

A limit describes the behavior of a mathematical expression as a specified variable approaches a particular value. In calculus, a common application is finding the value that a ratio, such as the change in function value over a perturbation size $$\frac{f(x+h) - f(x)}{h}$$, converges to as the perturbation $$h$$ shrinks to zero.

Limit of a Function

Multivariate Function

In mathematics, equations that define functions are often written using function notation. Functions are typically assigned single-letter names, most commonly $$f$$, $$g$$, $$h$$, $$F$$, $$G$$, or $$H$$. For a function named $$f$$, the notation specifies the relationship between an input value $$x$$ and its corresponding output value $$y$$ as $$y = f(x)$$. In this notation, $$x$$ represents the domain value, and $$f(x)$$ represents the corresponding range value. The expression $$f(x)$$ is read as '$$f$$ of $$x$$' or 'the value of $$f$$ at $$x$$'. The parentheses in this context indicate the input parameter mapping to the function and do not indicate multiplication.

Learn Before

Related

Learn After