# how to interpret the SVD of a data matrix A

So I understand the proofs behind Singular Value Decomposition but I'm having trouble interpreting it in the context of a real world problem.

Specifically, If I'm given an $m\times n$ data matrix A, where we have m training examples and n features collected for each example, I'm having trouble understanding the meaning behind Av$_j$ = $\sigma$$_ju_j where L_A (left multiplication by A) is our linear transformation and \beta = {v_1, v_2, ... , v_n} is an orthonormal basis for F^n and \gamma = {u_1, u_2, ... , u_m} is an orthonormal basis for F^m. From reading various posts and articles, the idea seems to be a larger \sigma$$_j$ indicates more variation in the data along that vector u$_j$ while a smaller variation in a certain direction u$_j$ is captured with a smaller $\sigma$$_j. However, when we are looking at Av_j = \sigma$$_j$u$_j$ i'm not sure why we care about what L$_A$ is doing. After all, this relationship would be great if I wanted to see what L$_A$ does when it acts on a orthonormal basis $\beta$ but A is just a data matrix so i'm not sure how to interpret the range of a data matrix or what types of transformations A is making when presented a vector x to 'do' left multiplication on.

• If you use a matrix just to store some data and not multiply it to anything, then SVD would be a useless concept – polfosol Jan 6 '17 at 10:42

Your data sets are the rows of $A$. Thus you get the $k$th sample by computing $e_k^TA$. Using the SVD this is also $$e_k^TA=\sum_{j=1}^nσ_j(e_k^T{\bf u}_j)\,{\bf v}_j^T$$ You can reduce this sum to the leading $d$ terms with the largest $d$ singular values to get a good approximation of the data, which means that your data vectors are all close to the subspace spanned by ${\bf v}_1,…,{\bf v}_d$ for some suitably chosen $d<n$.
• The SVD is $A=U\Sigma V^T=\sum_jσ_j u_jv_j^T$. Now left-multiply with a canonical basis vector to extract the corresponding row... – Dr. Lutz Lehmann Jan 6 '17 at 18:41
• Yes, that is correct. You can also define the rank as the lowest number of dyadic products in a sum that equals the matrix. Thus the rank of the reduced SVD is $d$. - It is not guaranteed that the entries of the reduced matrix are smaller than the entries of the original matrix. – Dr. Lutz Lehmann Jan 8 '17 at 12:19