14
$\begingroup$

I was reading Gilbert Strang's book and he says that if $A=USV'$ be the SVD of A ( assume square for the moment) then the nullspace of A is given by the last $n-r$ columns of V and the column space by the first $r$ columns of $U$.

I understand the null space part. If \begin{align} A&=USV'\\ AV&=US\\ [Av_1 \;Av_2\cdots Av_r\cdots Av_n]&=[Us_1 \;Us_2\cdots Us_r\cdots Us_n] \end{align} The columns $Us_{r+1}\cdots Us_n$ are $0$ vectors and correspond to (applicatoin of A on) linearly independent vectors $v_{r+1}\cdots v_n$. Thus, vectors $v_{r+1}\cdots v_n$ form the null space of A.

How do I prove the statement on column space? I am not even able to start. This is not homework by the way. Just wanted to prove everything in Strang myself for fun.

I feel it has to be inclusion proof where I show that any vector in the column space of A is in the column space of $\hat{U}$ and vice versa. ($\hat{U}$ is first r columns of $U$)

$\endgroup$
10
$\begingroup$

Since $A = USV'$, then its column space must be the same as the column space of $US$, since $V$ is invertible. And $S$ is a diagonal matrix and only the first $r$ diagonal entries of $S$ are nonzero, so check that only the first $r$ columns of $U$ "survive" being multiplied by $S$.

$\endgroup$
2
  • $\begingroup$ I'm not getting the part: "US, since V is invertible" Can you elaborate a little on why that holds? $\endgroup$ – user2820379 Mar 13 '15 at 23:39
  • 5
    $\begingroup$ If $A = BC$, then the column space of $A$ must be a subset of the column space of $B$, since the matrix $BC$ is literally a matrix whose columns are some linear combinations of the columns of $B$. If $C$ is also invertible, then also $AC^{-1} = B$, which implies the column space of $B$ is a subset of the column space of $A$. Thus both inclusions hold so they have equal column space. Now set $B = US$ and $C = V'$. $\endgroup$ – Christopher A. Wong Mar 14 '15 at 2:25
6
$\begingroup$

in fact you can write as the sum of $r$ rank one matrices $$A = s_1u_1v_1^T + s_2u_2v_2^T + \cdots+s_ru_rv_r^T$$ where $u_1,u_2,\ldots, u_r$ and $v_1,v_2,\ldots, v_r$ make up the columns of the orthogonal matrices $U,V$ and $r$ is the rank of $A.$ to show that $u_1,\ldots, u_r$ spans the column space of $A,$ look at $Ax.$ $$Ax = s_1(v_1^T x) u_1 + s_2(v_2^Tx)u_2 + \cdots+s_r(v_r^Tx)u_r$$

you can look at the row space of $A$ by computing $y^TA$ in the same way.

$\endgroup$
1
  • $\begingroup$ If we see the matrix $\mathbf{A}_{n\times m}(\mathbb{R})$ as a linear operator $T(\mathbf{x})=\mathbf{Ax}$, then $C(\mathbf{A})= \{ T(\mathbf{x})|\mathbf{x}\in \mathbb{R}^n \}$. Is it your point that for each $\mathbf{x}\in\mathbb{R}^n$ $T(\mathbf{x})$ can be written a combination of the $r$ columns of $\mathbf{U}$ and thus belong to $C(\mathbf{U})$? Thanks. $\endgroup$ – darkmoor Nov 13 '17 at 20:04
2
$\begingroup$

Note that $Us_i = \sigma_i u_i$ because $S$ is diagonal where $\sigma_i$ is the $i$th singular value and $u_i$ is the $i$th column of $U$. So $A v_i = \sigma_i u_i$, $i=1,2,\dots,r$ spans the column space of $A$.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.