7
$\begingroup$

The classical proof of the existence of the SVD factorization by Trefethen and Bau reports

Set $\sigma_1 = \mid\mid A \mid\mid_2$. By a compactness argument, there must be a vector $v_1 \in \mathbb{C}^n$ with $\mid\mid v_1 \mid\mid = 1$ and $\mid\mid u_1 \mid\mid_2 = \sigma_1 $ where $u_1 = A v_1$.

where $A$ is a complex matrix of size $m \times n$.

Because it is presented in such a brisk fashion, I expect it to be something very elementary, but I cannot follow the reasoning at all. I guess that we are interested in the compactness of $\mathbb{C}^n$, but what are the implications of compactness which are relevant in this case?

Thanks!

$\endgroup$
  • $\begingroup$ the compactness argument is weird, since it is obvious that there is a $v$ such that $\frac{\|Av\|}{\|v\|} = \sigma_{max}$. see for example math.stackexchange.com/a/1737839/276986 and we are interested in the compactness of the sphere $\|v\| = 1$ in $\mathbb{C}^n$ $\endgroup$ – reuns May 27 '16 at 0:35
  • $\begingroup$ @reuns I wouldn't call it "obvious", but I agree with you that the use of "compactness" to do Weierstrass mini-max finite dimensional normed vector spaces (which can be proved by successive bissection, Cauchy criterion and sequential continuity of the target function without ever resorting to abstract compactness concepts) is a bit like explaining Schönberg's use of dissonance to a piano student practicing "Für Elise". But y'know mathies enjoy putting-off people with jargon. $\endgroup$ – Oskar Limka Apr 19 '18 at 10:48
4
$\begingroup$

It depends on how the induced matrix norm was defined. I don't have the book handy, but I expect some pages earlier the authors to have put $$\|A\|_2=\sup_{\|v\|_2=1}\|Av\|_2.$$ Note that a function's sup is not always achieved (think of $1-\exp x$ ($x$ real) and $1$). Compactness (for your purposes here) is a quick way of saying that the sup is achieved by a vector $v$, i.e., there is a specific vector $v_1$ which satisfies $$\|v_1\|_2=1\text{ and }\|Av_1\|=\|A\|_2.$$

$\endgroup$
4
$\begingroup$

We are interested in the compactness of the subset $S = \{v\mid ||v|| = 1\}\subseteq \Bbb C^n$. Compactness is relevant because among those vectors, $||Av||_2 \leq \sigma_1$, and compactness is used to guarantee the existence of a vector $v_1$ such that there is equality. In other words, the function $f:S \to \Bbb R$ given by $f(v) = ||Av||_2$ has sup $\sigma_1$ by definition of operator norm, and compactness guarantees that it is actually a max.

$\endgroup$
1
$\begingroup$

I have question about the same proof. In the book it mentioned

"Set $σ_{1}=\|A\|_{2}$.By a compactness argument, there must be vectors $v_{1} \in C^n$ and $u_{1} \in C^{m}$ with $\|v_{1}\|_{2}=\|u_{1}\|_{2}=1$ and $Av_{1}=σ_{1}$." Since it was defined earlier that $\|A\|_{2}=sup_{\|v\|_{2}=1} \|Av\|_{2}$ and $σ_{1}\geqslantσ_{2}\geqslant…$ , as semi-axis principle, I can follow why $σ_{1}=\|A\|_{2}$ but how it came to the conclusion that there is $u_{1}\in C^{m}$ with $\|u_{1}\|_{2}=1$ and $Av_{1}=σ_{1}u_1$.

$\endgroup$
  • $\begingroup$ the sphere $\|v\|=1$ in $\mathbb{C}^n$ is compact, and $f(v) = \|A v\|$ with $\|v\| = 1$ is a continuous function $\endgroup$ – reuns May 27 '16 at 0:37
  • $\begingroup$ My question is how it concludes that there is $u_1\in C_m with ∥u_{1}∥_{2}=1$ and $Av_{1}=σ_{1}u_{1}$. $\endgroup$ – Crimson May 27 '16 at 0:45
  • 1
    $\begingroup$ wiki/Extreme_value_theorem#Generalization_to_arbitrary_topological_spaces $\endgroup$ – reuns May 27 '16 at 0:51
  • $\begingroup$ Having "found" $v_1$, put $u_1:=Av_1/\sigma_1$ (and check that $\|u_1\|=1$). Again you don't need any compactness for $u_1$. That's badly written, change book. $\endgroup$ – Oskar Limka Apr 19 '18 at 10:51

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.