The set of equations, $x_1 + 3x_2 = 17$, $5x_1 + 7x_2 = 19$, and $11x_1 + 13x_2 = 23$ has no real solution, but the pseudoinverse may be used to find close values. First, the matrix $A$ is set up: $A = \begin{bmatrix} 1 & 3\\ 5 & 7\\ 11 & 13 \end{bmatrix}$. We solve $A^\dagger = (A^TA)^{-1}A^T$ and get $A^\dagger = \begin{bmatrix} -0.5197 & -0.2171 & 0.2368\\ 0.4276 & 0.2039 & -0.1316 \end{bmatrix}$. Next, we find $\vec{x} = A^\dagger\vec{y}$ and multiply:
$\begin{bmatrix} -0.5197 & -0.2171 & 0.2368\\ 0.4276 & 0.2039 & -0.1316 \end{bmatrix}$ $\begin{bmatrix} 17\\ 19\\ 23 \end{bmatrix} =$ $\begin{bmatrix} -7.51\\ 8.12 \end{bmatrix}$
This means $x_1 \approx -7.51$ and $x_2 \approx 8.12$. 

Moore-Penrose Pseudoinverse Example

A practical formula for computing the Moore-Penrose pseudoinverse of a matrix $$A$$ uses its singular value decomposition (SVD). Given the SVD of $$A$$ as $$A = U D V^T$$, the pseudoinverse is $$A^\dagger = V D^\dagger U^T$$. The pseudoinverse $$D^\dagger$$ of the diagonal matrix $$D$$ is found by taking the reciprocal of its non-zero elements, and then taking the transpose of the resulting matrix.

Moore-Penrose Pseudoinverse Practical Formula

The Moore-Penrose pseudoinverse is used similarly to an inverse, but for nonsquare matrices. It can find approximate values for systems with no exact solution. Often denoted as $$A^\dagger$$, given an $$m \times n$$ matrix $$A$$ and a system $$A \vec{x} = \vec{y}$$, the pseudoinverse of $$A$$ is the matrix $$B$$ such that $$\vec{x} = B \vec{y}$$. Formal definition: $$A^\dagger = \lim_{\alpha \to 0} (A^T A + \alpha I )^{-1} A^T$$.

Michigan State University

Google

An n-dimensional identity matrix $I_{n}$ is a matrix that has ones on the diagonal and zeros as the rest of the elements.
The inverse of a matrix A can be defined as $A^{-1}$ such that $A^{-1}$A=$I_{n}$
We can use matrix inverse to solve a $Ax=B$: $x=BA^{-1}$

Linear Algebra - Identity and Inverse Matrices

To get the transpose of a matrix, we should do a mirror symmetry operation on the matrix according to one of its specific diagonal line, called the main diagonal.  The gotten matrix should be denoted as $A^T$.

Transpose

Goodfellow, I., Bengio, Y., & Courville, A. (2016). $\mathit{Deep \ Learning.}$ MIT Press. Retrieved from [www.deeplearningbook.org](https://www.deeplearningbook.org) 

Deep Learning

Moore-Penrose Pseudoinverse

An important property of an orthogonal matrix $$A$$ is that $$A A^T = A^T A = I$$, which means that its inverse is simply its transpose: $$A^{-1} = A^T$$. This property makes the matrix inverse operation computationally cheap.

Orthogonal Matrix Inverse Property

Vectors $$x$$ and $$y$$ are orthogonal if their dot product is zero, meaning that $$x^Ty=0$$.

Learn Before

Related

Learn After