A matrix whose eigenvalues are all positive or zero valued is called positive semidefi-
nite.

University of Michigan - Ann Arbor

Suppose that a matrix A has n linearly independent eigenvectors {$v^{(1)}$, . . . , $v^{(n)}$} with corresponding eigenvalues {$\lambda _1$, . . . , $\lambda _n$}. We may concatenate all the eigenvectors to form a matrix V with one eigenvector per column: V = [$v^{(1)}$ , . . . , $v^{(n)}$]. Similarly, we can get a vector containing eigenvalues, naming it as $\lambda$. The eigendecomposition of A is then given by
$A = Vdiag(\lambda)V^{-1}$

Linear Algebra - eigendecomposition

Goodfellow, I., Bengio, Y., & Courville, A. (2016). $\mathit{Deep \ Learning.}$ MIT Press. Retrieved from [www.deeplearningbook.org](https://www.deeplearningbook.org) 

Deep Learning

A matrix is singular if and only if any of the eigenvalues are zero.

Singular Matrix Judgement According to Eigenvalues

A matrix whose eigenvalues are all positive is called positive definite.

Positive Definite

Positive Semidefinite

If all eigenvalues are negative, the matrix is negative definite.

Negative Definite

If all eigenvalues are negative or zero valued, it is negative semidefinite.

Negative Semidefinite

When optimizing a convex quadratic objective function, the positive definite matrix $$\mathbf{Q}$$ can be decomposed into its eigensystem as $$\mathbf{Q} = \mathbf{O}^	op \boldsymbol{\Lambda} \mathbf{O}$$, where $$\mathbf{O}$$ is an orthogonal matrix and $$\boldsymbol{\Lambda}$$ is a diagonal matrix of strictly positive eigenvalues. By applying the change of variables $$\mathbf{z} = \mathbf{O} (\mathbf{x} - \mathbf{Q}^{-1} \mathbf{c})$$, the optimization problem is recast into a much simpler coordinate system that aligns with the eigenvectors of $$\mathbf{Q}$$. A key theorem demonstrates that when expressed in this eigensystem, both standard gradient descent and gradient descent with momentum do not mix different eigenspaces. Instead, the multi-dimensional optimization problem decomposes entirely into independent, coordinate-wise optimization processes along the directions of the eigenvectors.

Learn Before

Related