In deep learning frameworks, higher-order tensors (those with an order greater than two) are constructed programmatically by expanding the number of components specified in their shape tuple. Similar to the creation of matrices, a generic sequential array of elements can be rearranged into a multidimensional structure by passing the desired axis lengths to a function like reshape, such as .reshape(2, 3, 4) for a $$3^{	extrm{rd}}$$-order tensor.

Programmatic Construction of Higher-Order Tensors

In deep learning frameworks, a mathematical matrix $$\mathbf{A} \in \mathbb{R}^{m \times n}$$ is programmatically implemented as a $$2^{\textrm{nd}}$$-order tensor possessing a shape of $$(m, n)$$. You can dynamically construct a matrix from an existing appropriately sized tensor by utilizing the `reshape` function (e.g., `.reshape(m, n)`). This operation reorganizes the sequential data into the specified row and column dimensions, allowing a generic one-dimensional tensor of $$m \times n$$ elements to be explicitly cast into a valid $$m \times n$$ matrix structure without altering the underlying numerical values.

Claude

A matrix is a rectangular array of numbers logically arranged in horizontal rows and vertical columns. In the context of multidimensional data arrays, a matrix is formally defined as a $$2^{\textrm{nd}}$$-order tensor, meaning it is a tensor with exactly two axes. It provides a structured mathematical format to compactly organize data and is frequently used to represent systems of linear equations.

Matrix

In mathematical notation, matrices are conventionally denoted by bold capital letters, such as $$\mathbf{X}$$, $$\mathbf{Y}$$, and $$\mathbf{Z}$$. To specify the dimensions and the set from which the elements are drawn, the notation $$\mathbf{A} \in \mathbb{R}^{m 	imes n}$$ is used. This expression formally indicates that the matrix $$\mathbf{A}$$ is composed of $$m 	imes n$$ real-valued scalars, logically arranged into exactly $$m$$ rows and $$n$$ columns.

Typographical Conventions for Matrices

Dive into Deep Learning

An augmented matrix provides a compact way to represent a system of linear equations. To create one, each equation must first be expressed in standard form. The variables' coefficients are organized into the left columns, while the constants are placed in the far-right column. A vertical line is inserted to replace the equal signs, visually separating the coefficients from the constants.

Augmented Matrix for a System of Equations

Every square matrix has a specific real number associated with it. This associated value is called the determinant of the square matrix.

Determinant of a Square Matrix

To find the determinant of a $$2 \times 2$$ square matrix $$\begin{bmatrix} a & b \\ c & d \end{bmatrix}$$, where $$a$$, $$b$$, $$c$$, and $$d$$ are real numbers, the matrix is first written with vertical bars instead of square brackets: $$\begin{vmatrix} a & b \\ c & d \end{vmatrix}$$. The real number value of this determinant is calculated by subtracting the product of the off-diagonal entries ($$bc$$) from the product of the main diagonal entries ($$ad$$):

$$\begin{vmatrix} a & b \\ c & d \end{vmatrix} = ad - bc$$

Determinant of a $$2 \times 2$$ Matrix

To evaluate the determinant of a $$2 \times 2$$ matrix, write the determinant with vertical lines, then subtract the products of its diagonals. For example, to evaluate the determinant of the matrix $$\begin{bmatrix} 4 & -2 \\ 3 & -1 \end{bmatrix}$$, first write it as $$\begin{vmatrix} 4 & -2 \\ 3 & -1 \end{vmatrix}$$. Next, subtract the product of the bottom-left and top-right entries from the product of the top-left and bottom-right entries:

$$(4)(-1) - (3)(-2)$$

Simplifying this expression yields $$-4 - (-6)$$, which equals $$-4 + 6 = 2$$. Thus, the determinant is $$2$$. Similarly, the determinant of $$\begin{bmatrix} -3 & -4 \\ -2 & 0 \end{bmatrix}$$ is evaluated as $$(-3)(0) - (-2)(-4)$$, which simplifies to $$0 - 8 = -8$$.

Example: Evaluating the Determinant of a $$2 \times 2$$ Matrix

Evaluate the determinant of the matrix $$\begin{bmatrix} 5 & -3 \\ 2 & -4 \end{bmatrix}$$.

**Solution:**

First, write the matrix in determinant form using vertical bars:

$$\begin{vmatrix} 5 & -3 \\ 2 & -4 \end{vmatrix}$$

Next, calculate the determinant by subtracting the product of the off-diagonal entries from the product of the main diagonal entries:

$$(5)(-4) - (2)(-3)$$

Now, simplify the expression by performing the multiplications:

$$-20 - (-6)$$

Finally, convert the subtraction to addition and solve:

$$-20 + 6 = -14$$

The determinant is $$-14$$.

Practice: Evaluating the Determinant of a $$2 \times 2$$ Matrix

The order of a matrix is defined by its dimensions, specifically its number of rows and columns. A matrix possessing $$m$$ rows and $$n$$ columns has an order of $$m 	imes n$$, which is often read as an '$$m$$ by $$n$$' matrix. For instance, a matrix containing $$2$$ rows and $$3$$ columns is designated as a $$2 	imes 3$$ matrix.

Order of a Matrix

A tensor is a generic mathematical framework for describing multidimensional arrays of numerical values, extending the concept of vectors and matrices to an arbitrary number of axes. The specific terminology depends on the number of axes: a one-dimensional tensor is a vector, a two-dimensional tensor is a matrix, and a tensor with $$n > 2$$ axes is referred to as an $$n^{	extrm{th}}$$-order tensor. Each individual scalar value within the tensor is called an element.

Tensor

When a multidimensional array requires more than two axes (where $$k > 2$$), the specialized names like vector or matrix are discarded. Instead, the data structure is formally referred to as a $$k^{	extrm{th}}$$-order tensor.

k-th Order Tensor (Axes/Dimensions)

Programmatic Representation of Matrices

In the context of machine learning, matrices serve as the standard mathematical structure for representing tabular datasets. Typically, this data is logically organized such that each horizontal row of the matrix corresponds to an individual data record or sample, while each vertical column corresponds to a distinct feature or attribute associated with those records. This structured format enables entire datasets to be processed efficiently using parallelized linear algebra operations.

Dataset Representation using Matrices

Every individual number or scalar located within a matrix is mathematically referred to as an element or an entry of that matrix.

Element of a Matrix

A matrix $$\mathbf{A} \in \mathbb{R}^{m 	imes n}$$ can be visualized as a vertical stack of $$m$$ row vectors. Formally, it is expressed as:

$$\mathbf{A}= \begin{bmatrix} \mathbf{a}^	op_{1} \ \mathbf{a}^	op_{2} \ \vdots \ \mathbf{a}^	op_m \ \end{bmatrix}$$

where each $$\mathbf{a}^	op_{i} \in \mathbb{R}^n$$ is a row vector representing the $$i^	extrm{th}$$ row of the matrix $$\mathbf{A}$$.

Matrix Representation via Row Vectors

Often, it is necessary to calculate the total sum of a tensor's elements mathematically. For a vector $$\mathbf{x}$$ of length $$n$$, the sum of its elements is expressed as $$\sum_{i=1}^n x_i$$. To express sums over tensors of arbitrary shape, the summation simply extends over all of its axes. For example, the sum of all elements in an $$m \times n$$ matrix $$\mathbf{A}$$ is written using multiple summations as $$\sum_{i=1}^{m} \sum_{j=1}^{n} a_{ij}$$. Programmatically, summing all the elements within a tensor yields a new tensor containing only a single element.

Tensor Element Summation

A matrix $$\mathbf{B} \in \mathbb{R}^{k \times m}$$ can be visualized as a horizontal collection of $$m$$ column vectors. Formally, it is expressed as $$\mathbf{B}=\begin{bmatrix} \mathbf{b}_{1} & \mathbf{b}_{2} & \cdots & \mathbf{b}_{m} \end{bmatrix}$$, where each $$\mathbf{b}_{j} \in \mathbb{R}^k$$ is a column vector representing the $$j^\textrm{th}$$ column of the matrix $$\mathbf{B}$$.

Matrix Representation via Column Vectors

In machine learning, matrices can be decomposed into factors that reveal underlying, low-dimensional structures within real-world datasets. These matrix decompositions are a foundational tool used by various machine learning subfields to analyze data, discover latent patterns, and solve complex prediction problems.

Matrix Decomposition

To refer to a specific element within a matrix, mathematical notation utilizes a lowercase letter corresponding to the matrix name, accompanied by subscripts denoting its exact position. For instance, $$a_{ij}$$ represents the specific scalar value located in the $$i^{	extrm{th}}$$ row and the $$j^{	extrm{th}}$$ column of the matrix $$\mathbf{A}$$. This notation allows for the precise algebraic definition of the entire matrix structure, which is visually illustrated as a tabular grid of these subscripted elements:

$$ \mathbf{A}=\begin{bmatrix} a_{11} & a_{12} & \cdots & a_{1n} \\ a_{21} & a_{22} & \cdots & a_{2n} \\ \vdots & \vdots & \ddots & \vdots \\ a_{m1} & a_{m2} & \cdots & a_{mn} \end{bmatrix} $$

Matrix Element Subscript Notation

A matrix is classified as a square matrix when its number of horizontal rows, $$m$$, is exactly equal to its number of vertical columns, $$n$$ (i.e., $$m = n$$).

Learn Before

Related

Learn After