# SVD products and the approximation error using Frobenius norm

This is my first time posting a question here. The question is about quantifying the bound of error of SVD products using the Frobenius norm.

Suppose I have a rectangular & real matrix $A (m \times n, m < n)$. I perform the SVD on $A$ to get $A = USV^T$.

Applying the reduced rank approximation gives $A_k \approx U_kS_kV^T_k$

Ignoring $U, U_k$, I define two new matrices by

$T = SV^T (m \times n)$

$T_k = S_kV^T_k (k \times n)$

I want to quantify the error bound if I apply the the transpose of both matrices to some experiment model, denoting as a matrix ($K$) by

$\delta = \frac{KT^T_k}{KT^T}$

Applying the Frobenius norm gives

$\delta = \frac{||KT_k^T||_F}{||KT^T||_F}$

Using sub-multiplicative properties, the above equation can be written as

$\delta \leq \frac{||K||_F||T_k^T||_F}{||K||_F||T^T||_F}$

Then

$\delta \leq \frac{||K||_F||V_kS_k||_F}{||K||_F||VS||_F}$

As $V, V_k$ are orthogonal invariant, then

$\delta \leq \frac{||K||_F||S_k||_F}{||K||_F||S||_F}$

Finally,

$\delta \leq \frac{||S_k||_F}{||S||_F}$

which is reduced to

$\delta \leq \sqrt{\frac{\sigma_{k+1}^2+\dots+\sigma_{m}^2}{\sigma_{1}^2+\dots+\sigma_{m}^2}}$

Have I done anything wrong in any step? The result is completely the opposite of my linear algebra proof. I suspect that the sub-multiplicative thing is not right.

Shouldn't your last equation be $$\delta \leq k/n\sqrt{\frac{\sigma_{1}^2+\dots+\sigma_{k}^2}{\sigma_{1}^2+\dots+\sigma_{\min(m,n)}^2}}$$ $||V_k||_F=tr(V_k^TV_k)=tr(I_k)=k$, similarly $||V||_F=n$
And also I think you made typo for $||S_k||_F$ and $||S||_F$.
• I think $V_k$ and $V$ are invariant under Frobenius norm. – Ati Jun 16 '15 at 10:19