Take the 2-minute tour ×
Mathematics Stack Exchange is a question and answer site for people studying math at any level and professionals in related fields. It's 100% free, no registration required.

Suppose a singular value decomposition on matrix $P\in\mathbb{R}^{n\times m}$ is given,

$P=U\Sigma V^T$

with $U=[u_1,\dots, u_n]\in\mathbb{R}^{n\times n}$, $u\in\mathbb{R}^{n}$, containing the orthonormal left singular vectors of $P$; $V=[v_1,\dots, v_m]\in\mathbb{R}^{m\times m}$, $v\in\mathbb{R}^{m}$, containing the orthonormal right singular vectors of $B$, and $\Sigma\in\mathbb{R}^{n\times m}$ denoting a diagonal matrix of $m$ non-negative singular values.

Now suppose matrices $U^d\in\mathbb{R}^{n\times d}$, $V^d\in\mathbb{R}^{m\times d}$, and $\Sigma^d\in\mathbb{R}^{d\times d}$ are given. Can it be shown that


is minimal for matrices $U^d, V^d$ corresponding to singular vectors associated with the largest $d$ singular values of $P$ (with diagonal entries of $\Sigma^d$ corresponding to these largest singular values)? (It obviously holds in case of all singular vectors, but I'm not sure about the restriction to $d$ dominant ones). In case of a square matrix $P$, proof is possible.

share|improve this question
So you're asking about how to show that low-rank approximations are the best approximations to a rectangular matrix? –  J. M. Feb 2 '12 at 10:26
Yes; Eckart-Young theorem should suffice? Please correct me if I'm wrong here. –  user506901 Feb 2 '12 at 10:30
Supposedly; have you tried going through the proof of that and changing stuff as needed? –  J. M. Feb 2 '12 at 10:49
Yes; by simply considering the dominant singular values of $\Sigma$, and setting others to zero, the results should be equivalent (even without explicit statement on $U^d, V^d$; it is indirectly achieved by extracting only dominant singular values.) –  user506901 Feb 2 '12 at 11:13
It seems to me you can answer your own question... ;) –  J. M. Feb 2 '12 at 11:53

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Browse other questions tagged or ask your own question.