Dimensions of matrices of singular vectors I have seen the singular value decomposition (SVD) represented in two different ways and wanted to know a) if they're both correct and b) what their relationship is.

*

*In the first version a matrix $A \in \mathbb{R}^{m \times n}$ such that $\mbox{rank}(A) = s$, can be decomposed as $A = V D U^t$ where $V \in \mathbb{R}^{m \times s}$, $D \in \mathbb{R}^{s\times s}$ and $U \in \mathbb{R}^{n \times s}$.


*In the second version a matrix $A \in \mathbb{R}^{m \times n}$, can be decomposed as $A = V D U^t$ where $V \in \mathbb{R}^{m \times m}$, $D \in \mathbb{R}^{m \times n}$ and $U \in \mathbb{R}^{n \times n}$.
 A: Both representations are correct, and they coincide as far as the smaller matrices $U,V$.
The first one is called "thin SVD". The second one has the same elements of $D$ as the first one, but the rest of the matrix is filled out with zeros.
The difference is that matrices $U$ and $V$ in the "larger" representation contain extra columns, which annihilate after being multiplied by zero elements of $D$. 
A: Start with the matrix
$$
\mathbf{A} \in \mathbb{C}^{m\times n}_{\rho}
$$
and consider the most general case of rank deficiency where $\rho < \min \left( m, n\right)$.
The singular value decomposition is
$$
\begin{align}
  \mathbf{A} &=
  \mathbf{U} \, \Sigma \, \mathbf{V}^{*} \\
%
 &=
% U 
  \left[ \begin{array}{cc}
     \color{blue}{\mathbf{U}_{\mathcal{R}}} & \color{red}{\mathbf{U}_{\mathcal{N}}}
  \end{array} \right]  
% Sigma
  \left[ \begin{array}{cccc|cc}
     \sigma_{1} & 0 & \dots &  &   & \dots &  0 \\
     0 & \sigma_{2}  \\
     \vdots && \ddots \\
       & & & \sigma_{\rho} \\\hline
       & & & & 0 & \\
     \vdots &&&&&\ddots \\
     0 & & &   &   &  & 0 \\
  \end{array} \right]
% V 
  \left[ \begin{array}{c}
     \color{blue}{\mathbf{V}_{\mathcal{R}}}^{*} \\ 
     \color{red}{\mathbf{V}_{\mathcal{N}}}^{*}
  \end{array} \right]  \\
%
  & =
% U
   \left[ \begin{array}{cccccccc}
    \color{blue}{u_{1}} & \dots & \color{blue}{u_{\rho}} & \color{red}{u_{\rho+1}} & \dots & \color{red}{u_{m}}
  \end{array} \right]
% Sigma
  \left[ \begin{array}{cc}
     \mathbf{S}_{\rho\times \rho} & \mathbf{0} \\
     \mathbf{0} & \mathbf{0} 
  \end{array} \right]
% V
   \left[ \begin{array}{c}
    \color{blue}{v_{1}^{*}} \\ 
    \vdots \\
    \color{blue}{v_{\rho}^{*}} \\
    \color{red}{v_{\rho+1}^{*}} \\
    \vdots \\ 
    \color{red}{v_{n}^{*}}
  \end{array} \right]
%
\end{align}
$$
This presents orthonormal bases for both the domain and the codomain. The Fundamental Theorem of Linear Algebra prescribed four fundamental subspaces:
$$
\begin{align}
%
  \mathbf{C}^{m} = 
    \color{blue}{\mathcal{R} \left( \mathbf{A} \right)} \oplus
    \color{red} {\mathcal{N} \left( \mathbf{A}^{*} \right)} \\
%
  \mathbf{C}^{n} = 
    \color{blue}{\mathcal{R} \left( \mathbf{A}^{*} \right)} \oplus
    \color{red} {\mathcal{N} \left( \mathbf{A} \right)}
%
\end{align}
$$
$$
\begin{array}{ll}
%
     column \ vectors & span \\\hline
%
     \color{blue}{u_{1}}  \dots  \color{blue}{u_{\rho}} & 
     \color{blue}{\mathcal{R} \left( \mathbf{A} \right)} \\
%
     \color{blue}{v_{1}}  \dots  \color{blue}{v_{\rho}} & 
     \color{blue}{\mathcal{R} \left( \mathbf{A}^{*} \right)} \\
%
     \color{red}{u_{\rho+1}}  \dots  \color{red}{u_{m}} & 
     \color{red}{\mathcal{N} \left( \mathbf{A}^{*} \right)} \\
%
     \color{red}{v_{\rho+1}}  \dots  \color{red}{v_{n}} & 
     \color{red}{\mathcal{N} \left( \mathbf{A} \right)} \\
%
  \end{array}
$$
The SVD can be expressed solely in terms of range space quantities. This is the thin or economical SVD
$$
 \mathbf{A} = 
\color{blue}{\mathbf{U}_{\mathcal{R}}} \,
\mathbf{S} \,
\color{blue}{\mathbf{V}_{\mathcal{R}}}^{*}
$$
Write the dimensions explicitly
$$
\begin{array}{ccccc}
%
     \mathbf{A} & = 
& \color{blue}{\mathbf{U}_{\mathcal{R}}}
& \mathbf{S} 
& \color{blue}{\mathbf{V}_{\mathcal{R}}}^{*} \\
%
  m \times n & &  m \times \rho &   \rho \times \rho & \rho \times n \\[5pt]
%
     \mathbf{A} & = 
%
& \mathbf{U}
& \Sigma
& \mathbf{V}^{*} \\
%
  m \times n & &  m \times m &   m \times n & n \times n \\
%
\end{array}
$$
