9
$\begingroup$

The classical proof of the existence of the SVD factorization by Trefethen and Bau reports

Set $\sigma_1 = \mid\mid A \mid\mid_2$. By a compactness argument, there must be a vector $v_1 \in \mathbb{C}^n$ with $\mid\mid v_1 \mid\mid = 1$ and $\mid\mid u_1 \mid\mid_2 = \sigma_1 $ where $u_1 = A v_1$.

where $A$ is a complex matrix of size $m \times n$.

Because it is presented in such a brisk fashion, I expect it to be something very elementary, but I cannot follow the reasoning at all. I guess that we are interested in the compactness of $\mathbb{C}^n$, but what are the implications of compactness which are relevant in this case?

Thanks!

$\endgroup$
2
  • $\begingroup$ the compactness argument is weird, since it is obvious that there is a $v$ such that $\frac{\|Av\|}{\|v\|} = \sigma_{max}$. see for example math.stackexchange.com/a/1737839/276986 and we are interested in the compactness of the sphere $\|v\| = 1$ in $\mathbb{C}^n$ $\endgroup$
    – reuns
    Commented May 27, 2016 at 0:35
  • $\begingroup$ @reuns I wouldn't call it "obvious", but I agree with you that the use of "compactness" to do Weierstrass mini-max finite dimensional normed vector spaces (which can be proved by successive bissection, Cauchy criterion and sequential continuity of the target function without ever resorting to abstract compactness concepts) is a bit like explaining Schönberg's use of dissonance to a piano student practicing "Für Elise". But y'know mathies enjoy putting-off people with jargon. $\endgroup$ Commented Apr 19, 2018 at 10:48

3 Answers 3

5
$\begingroup$

It depends on how the induced matrix norm was defined. I don't have the book handy, but I expect some pages earlier the authors to have put $$\|A\|_2=\sup_{\|v\|_2=1}\|Av\|_2.$$ Note that a function's sup is not always achieved (think of $1-\exp x$ ($x$ real) and $1$). Compactness (for your purposes here) is a quick way of saying that the sup is achieved by a vector $v$, i.e., there is a specific vector $v_1$ which satisfies $$\|v_1\|_2=1\text{ and }\|Av_1\|=\|A\|_2.$$

$\endgroup$
5
$\begingroup$

We are interested in the compactness of the subset $S = \{v\mid \lVert v \rVert = 1\}\subseteq \Bbb C^n$. Compactness is relevant because among those vectors, $||Av||_2 \leq \sigma_1$, and compactness is used to guarantee the existence of a vector $v_1$ such that there is equality. In other words, the function $f:S \to \Bbb R$ given by $f(v) = \lVert Av\rVert_2$ has sup $\sigma_1$ by definition of operator norm, and compactness guarantees that it is actually a max.

$\endgroup$
1
$\begingroup$

I have question about the same proof. In the book it mentioned

"Set $σ_{1}=\|A\|_{2}$.By a compactness argument, there must be vectors $v_{1} \in C^n$ and $u_{1} \in C^{m}$ with $\|v_{1}\|_{2}=\|u_{1}\|_{2}=1$ and $Av_{1}=σ_{1}$." Since it was defined earlier that $\|A\|_{2}=sup_{\|v\|_{2}=1} \|Av\|_{2}$ and $σ_{1}\geqslantσ_{2}\geqslant…$ , as semi-axis principle, I can follow why $σ_{1}=\|A\|_{2}$ but how it came to the conclusion that there is $u_{1}\in C^{m}$ with $\|u_{1}\|_{2}=1$ and $Av_{1}=σ_{1}u_1$.

$\endgroup$
4
  • $\begingroup$ the sphere $\|v\|=1$ in $\mathbb{C}^n$ is compact, and $f(v) = \|A v\|$ with $\|v\| = 1$ is a continuous function $\endgroup$
    – reuns
    Commented May 27, 2016 at 0:37
  • $\begingroup$ My question is how it concludes that there is $u_1\in C_m with ∥u_{1}∥_{2}=1$ and $Av_{1}=σ_{1}u_{1}$. $\endgroup$
    – Crimson
    Commented May 27, 2016 at 0:45
  • 1
    $\begingroup$ wiki/Extreme_value_theorem#Generalization_to_arbitrary_topological_spaces $\endgroup$
    – reuns
    Commented May 27, 2016 at 0:51
  • $\begingroup$ Having "found" $v_1$, put $u_1:=Av_1/\sigma_1$ (and check that $\|u_1\|=1$). Again you don't need any compactness for $u_1$. That's badly written, change book. $\endgroup$ Commented Apr 19, 2018 at 10:51

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .