Singular value decomposition
In linear algebra, the singular-value decomposition (SVD) is a factorization of a real or complex matrix. It is the generalization of the eigendecomposition of a positive semidefinite normal matrix (for example, a symmetric matrix with positive eigenvalues) to any matrix via an extension of the polar decomposition. It has many useful applications in signal processing and statistics.
Formally, the singular value decomposition of an [math]\displaystyle{ m\times n }[/math] complex matrix [math]\displaystyle{ M }[/math] is a factorization of the form [math]\displaystyle{ \mathbf{U\Sigma V^*} }[/math], where [math]\displaystyle{ U }[/math] is an [math]\displaystyle{ m\times m }[/math] complex unitary matrix, [math]\displaystyle{ \mathbf\Sigma }[/math] is an [math]\displaystyle{ m\times n }[/math] rectangular diagonal matrix with non-negative real numbers on the diagonal, and [math]\displaystyle{ V }[/math] is an [math]\displaystyle{ n\times n }[/math] complex unitary matrix. The diagonal entries [math]\displaystyle{ \sigma_i=\Sigma_{ii} }[/math] of [math]\displaystyle{ \mathbf\Sigma }[/math] are known as the singular values of [math]\displaystyle{ M }[/math]. The columns of [math]\displaystyle{ U }[/math] and the columns of [math]\displaystyle{ V }[/math] are called the left-singular vectors and right-singular vectors of [math]\displaystyle{ M }[/math], respectively.
The singular-value decomposition can be computed using the following observations:
- The left-singular vectors of M are a set of orthonormal eigenvectors of MM∗.
- The right-singular vectors of M are a set of orthonormal eigenvectors of M∗M.
- The non-zero singular values of M (found on the diagonal entries of Σ) are the square roots of the non-zero eigenvalues of both M∗M and MM∗.
Singular Value Decomposition Media
Visualization of the matrix multiplications in singular value decomposition
Visualization of Reduced SVD variants. From top to bottom: 1: Full SVD,2: Thin SVD (remove columns of U not corresponding to rows of V*),3: Compact SVD (remove vanishing singular values and corresponding columns/rows in U and V*),4: Truncated SVD (keep only largest t singular values and corresponding columns/rows in U and V*)
