2
$\begingroup$

Could any give an intuitive understanding of SVD decomposition of a matrix? I know it can be used for image compress. But how to understand the decomposition within linear transform?

$\endgroup$
2
$\begingroup$

Generally, an $N\times N$ matrix can be seen as a point in a high dimensional $N^2$ space. For instance, if any image would have only three pixels, we could've put any image on a point in a 3D space. SVD gives the primary "directions" that this matrix represents. The center matrix gives the weight of each direction in descending order, ie if you want to get a single direction that your matrix "goes in" (maybe think of some high dimensional gaussian blob that has a distinct direction), you can take only this direction. The direction itself is found in the left-hand matrix.

You can try to play with simple matrices such as $\begin{bmatrix}0&1\\1&0\end{bmatrix}$, $\begin{bmatrix}0&1\\1&1\end{bmatrix}$, $\begin{bmatrix}2&2\\1&-1\end{bmatrix}$ etc., and look at the center matrix and left hand matrix.

$\endgroup$
2
$\begingroup$

So let's examine the SVD $A=U\Sigma V^*$ where everything is chosen to be square $n\times n$ matrices (by padding $\Sigma$ with zeros and completing $V$ to be a unitary matrix.)

In this case, it is exactly what it looks like: $A$ is expressed as a rotation, followed by a scaling, followed by another rotation.

Another useful interpretation is that if you view the effects of $A$ on the unit sphere in $\Bbb R^n$, it will in general be changed into some ellipsoid (often a lower dimensional ellipsoid.) The singular values are the measurements of the minor axes of the ellipsoid. This in effect displays the extent the sphere was stretched in different directions. The 0 singular values correspond to dimensions that were "crushed flat" by the transformation, rather than shrunk/stretched.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.