A practical formula for computing the Moore-Penrose pseudoinverse of a matrix $$A$$ uses its singular value decomposition (SVD). Given the SVD of $$A$$ as $$A = U D V^T$$, the pseudoinverse is $$A^\dagger = V D^\dagger U^T$$. The pseudoinverse $$D^\dagger$$ of the diagonal matrix $$D$$ is found by taking the reciprocal of its non-zero elements, and then taking the transpose of the resulting matrix.

Google

The Moore-Penrose pseudoinverse is used similarly to an inverse, but for nonsquare matrices. It can find approximate values for systems with no exact solution. Often denoted as $$A^\dagger$$, given an $$m \times n$$ matrix $$A$$ and a system $$A \vec{x} = \vec{y}$$, the pseudoinverse of $$A$$ is the matrix $$B$$ such that $$\vec{x} = B \vec{y}$$. Formal definition: $$A^\dagger = \lim_{\alpha \to 0} (A^T A + \alpha I )^{-1} A^T$$.

Moore-Penrose Pseudoinverse

Goodfellow, I., Bengio, Y., & Courville, A. (2016). $\mathit{Deep \ Learning.}$ MIT Press. Retrieved from [www.deeplearningbook.org](https://www.deeplearningbook.org) 

Deep Learning

The set of equations, $x_1 + 3x_2 = 17$, $5x_1 + 7x_2 = 19$, and $11x_1 + 13x_2 = 23$ has no real solution, but the pseudoinverse may be used to find close values. First, the matrix $A$ is set up: $A = \begin{bmatrix} 1 & 3\\ 5 & 7\\ 11 & 13 \end{bmatrix}$. We solve $A^\dagger = (A^TA)^{-1}A^T$ and get $A^\dagger = \begin{bmatrix} -0.5197 & -0.2171 & 0.2368\\ 0.4276 & 0.2039 & -0.1316 \end{bmatrix}$. Next, we find $\vec{x} = A^\dagger\vec{y}$ and multiply:
$\begin{bmatrix} -0.5197 & -0.2171 & 0.2368\\ 0.4276 & 0.2039 & -0.1316 \end{bmatrix}$ $\begin{bmatrix} 17\\ 19\\ 23 \end{bmatrix} =$ $\begin{bmatrix} -7.51\\ 8.12 \end{bmatrix}$
This means $x_1 \approx -7.51$ and $x_2 \approx 8.12$. 

Learn Before

Related