1
$\begingroup$

Every (real) $m\times n$ matrix $A$ of rank $r$ has an SVD $$ A = U\Sigma V^T $$

Now, I have read about the following properties:

  • $\text{Image}(A) = \text{span}\{u_1,\dots,u_r\}$
  • $\text{Null space}(A) = \text{span}\{v_{r+1},\dots,v_n\}$

Maybe I am lacking some knowledge from my linear algebra courses, but how can these properties be proven?

$\endgroup$
1
$\begingroup$

Image(A) means column space of A. I will assume $m\geq n$ and rank(A) = r without loss of generality. To see these identity, distinguish compact and full svd decomposition: \begin{align} A = U_1\Sigma_1 V_1^T = \underbrace{[U_1,\ U_2]}_U\underbrace{\begin{bmatrix} \Sigma_1 & 0\\0 & 0\end{bmatrix}}_\Sigma\underbrace{\begin{bmatrix} V_1^T \\ V_2^T\end{bmatrix} }_V \end{align} where $U \in R^{m\times m}$, $\Sigma = diag(\sigma_1,\ldots,\sigma_r)\in R^{r\times r}$, and $V \in R^{n\times n}$. Matrix U constitutes an orthonormal basis for $R^{m\times m}$ with $$U^TU=UU^T = I_m.$$ In the same way, matrix V constitutes an orthonormal basis for $R^{n\times n}$ with $$V^TV=VV^T=I_n.$$ On the other hand, by compact SVD, you get only orthonormal basis for the range and domain space of matrix A. So, you do not have the property $$U_iU^T_i = I_m \ or \ V_iV^T_i = I_n, \ i = 1,2. $$ Knowing this distinction, come back to the your question. Consider following matrix transformation for $\forall x\in R^n$ \begin{align} Ax &= \sum_i^r \sigma_i(v_i^Tx)u_i \\&= U_1\Sigma_1V_1^Tx = U_1z\\ &=\sum_i^rz_iu_i \end{align} where $z_i = \sigma_i(v_i^Tx)$. So, Ax is just weighted linear combination of vectors $\{u_1,\ldots,u_r\}$ which equivalent to say $\mathcal{R}(A) = span\{u_1,\ldots,u_r\}$. On the other hand, null space is the vectors that are mapped to zero, i.e. if $x\in \mathcal{N}(A)$, then $Ax = 0$. Since the set $\{u_1,\ldots,u_r\}$ is orthonormal, \begin{equation} Ax = U_1z = 0 \end{equation} can be possible only if z is zero vector. \begin{align} z = \Sigma_1V_1Tx = 0 \leftrightarrow V_1^Tx \end{align} $\Sigma$ is just nonzero diagonal matrix, we can ignore it. So, vector z is $0$ if and only if vector x is perpendicular to $span\{v_1,\ldots,v_r\}$ which means that $$x\in span\{v_{r+1},\ldots,v_n\} = \mathcal{R}(V_2)\rightarrow Ax = 0$$

$\endgroup$
0
$\begingroup$

One simple possibility is to use this form of SV decomposition of $A$:

$$A = \sum_{i=1}^{r}{\lambda_i u_i v_i^T}$$

Then, for an input $$x = \sum_{i=1}^{n} x_iv_i$$ It follows $$Ax = \sum_{i=1}^{r}{\lambda_i x_i u_i} $$

The properties you are looking for are a direct consequence of the last relation.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.