Use `scipy.linalg` for common dense-matrix decomposition routines:

```python
eigenvalues, eigenvectors = linalg.eig(A)
first_eigenvector = eigenvectors[:, 0]
eigenvalues_only = linalg.eigvals(A)

U, singular_values, Vh = linalg.svd(B)
m, n = B.shape
Sigma = linalg.diagsvd(singular_values, m, n)

P, L, U = linalg.lu(C)
```

`eig` computes eigenvalues and eigenvectors, while `eigvals` computes only eigenvalues. `svd` returns the singular value decomposition factors, `diagsvd` constructs the corresponding rectangular diagonal matrix, and `lu` returns the permutation, lower-triangular, and upper-triangular factors of an LU decomposition.

Michigan State University

Google

You’ll use the linalg and sparse modules. Note that scipy.linalg contains and expands on numpy.linalg. 
```
from scipy import linalg, sparse
```

Linear Algebra

In machine learning, matrices can be decomposed into factors that reveal underlying, low-dimensional structures within real-world datasets. These matrix decompositions are a foundational tool used by various machine learning subfields to analyze data, discover latent patterns, and solve complex prediction problems.

Matrix Decomposition

```
A = np.matrix(np.random.random((2,2)))

B = np.asmatrix(b)

C = np.mat(np.random.random((10,5)))

D = np.mat([[3,4], [5,6]])
```

Creating Matrix

•INVERSE
```
A.I #inverse

linalg.inv(A) #inverse
```
•TRANSPOSE and CONJUGATE
```
A.T #transpose matrix

A.H #conjugate
```
•TRACE
```
np.trace(A) #trace
```
•NORM
```
linalg.norm(A) #Frobenius norm

linalg.norm(A,1) #L1 Norm (max column sum)

linalg.norm(A,np.inf) #L inf Norm (max row sum)
```
•RANK
```
np.linalg.matrix_rank(C) #matrix rank
```
•DETERMINANT 
```
linalg.det(A) #determinant
```
•SOLVING LINEAR EQUATIONS
```
linalg.solve(A,b) #solver for dense matrices

E = np.mat(a).T #solver for dense matrices

linalg.lstsq(F,E) #least-squares solution to linear matrix equation
```

•GENERALIZED INVERSE
```
linalg.pinv(C) #Compute the pseudo-inverse of a matrix (least-squares solver)

linalg.pinv2(C) #Compute the pseudo-inverse of a matrix
(SVD)
```

Basic Matrix Routines

```
sparse.linalg.expm(I) #Sparse matrix exponential
```

Sparse Matrix Functions

```
la, v = sparse.linalg.eigs(F,1) #Eigenvalues and eigenvectors

sparse.linalg.svds(H, 2) #SVD```

Sparse Matrix Decomposition

The numpy (np) and scipy.linalg modules provide a variety of functions for matrix operations:

- Addition & Subtraction: np.add(A, D), np.subtract(A, D)
- Division: np.divide(A, D)
- Multiplication: A @ D (matrix multiplication operator), np.multiply(D, A) (element-wise), np.dot(A, D) (dot product), np.vdot(A, D) (vector dot product), np.inner(A, D) (inner product), np.outer(A, D) (outer product), np.tensordot(A, D) (tensor dot product), np.kron(A, D) (Kronecker product)
- Exponential: linalg.expm(A) (matrix exponential), linalg.expm2(A) (Taylor series), linalg.expm3(D) (eigenvalue decomposition)
- Logarithmic: linalg.logm(A)
- Trigonometric: linalg.sinm(D), linalg.cosm(D), linalg.tanm(A)
- Hyperbolic: linalg.sinhm(D), linalg.coshm(D), linalg.tanhm(A)
- Sign: np.signm(A)
- Square Root: linalg.sqrtm(A)
- Arbitrary: linalg.funm(A, lambda x: x*x)

NumPy and SciPy Matrix Functions

Vectorization is a computational technique used to replace explicit, element-by-element loops (such as Python for-loops) with optimized operations on entire data structures simultaneously. In machine learning, vectorization leverages highly optimized linear algebra libraries to perform mathematical operations—like calculating the elementwise sum of vectors or processing whole minibatches of examples—at once. This approach dramatically reduces execution time by yielding order-of-magnitude speedups, simplifies code, reduces the potential for bugs, and increases portability by offloading mathematics to underlying libraries.

Vectorization

A sparse matrix stores only its nonzero entries, saving memory when most elements are zero. Using NumPy together with SciPy's `scipy.sparse` module:

```python
from scipy import sparse
import numpy as np

F = np.eye(3, k=1)          # 3x3 matrix with ones on the first superdiagonal
G = np.mat(np.identity(2))  # 2x2 identity matrix
C[C > 0.5] = 0              # Zero out entries so the array becomes sparse

H = sparse.csr_matrix(C)    # Compressed Sparse Row (CSR) matrix
I = sparse.csc_matrix(D)    # Compressed Sparse Column (CSC) matrix
J = sparse.dok_matrix(A)    # Dictionary Of Keys (DOK) matrix

E.todense()                 # Convert a sparse matrix back to a dense (full) matrix
sparse.isspmatrix_csc(A)    # Check whether a matrix is stored in CSC format
```

Creating Sparse Matrices

SciPy Dense Matrix Decomposition Routines

The principles of matrix decomposition can be generalized to apply to high-order tensors. In machine learning, these tensor decompositions are utilized to discover underlying, low-dimensional structures in complex datasets that possess more than two axes, enabling the solution of advanced prediction problems.

Learn Before

Related