SVD - Decomposed Matrix Sizes I had a question about SVD. Specifically about the size of matrices $U$, $\Sigma$ and $V$ decomposed from the $m\times n$ matrix $X$ using the formula
$$X = U \Sigma V^T $$
Most of the the tutorial literature says that the resulting sizes are


*

*$U$ is $m \times m$

*$\Sigma$ is $m \times n$

*$V$ is $n \times n$


However, there have been quite few times when the sizes given are


*

*$U$ is $m \times n$

*$\Sigma$ is $n \times n$

*$V$ is $n \times n$


In other words instead of $\Sigma$ being the matrix with possibly different number of rows and columns, its $U$ with the different number of rows and columns.
The math works out, so why (and in what cases) is this less frequent version used?
 A: Assume that $A$ is a "tall" matrix; that is $m \ge n$. Otherwise, we work with the transpose of $A$. The number of non-zero singular values, at most, is $n$. Consider your first version of the SVD which I call the full version. $A = U \Sigma V^t$. Often $m$ is large and $m >> n$ (such as in signal processing, statistics and data sciences). In that case, $U$ is a very large matrix.
The SVD can be written as
$$
A = [U_1 \; U_2] \left[ \begin{array}{c} 
\Sigma_1 \\
O \end{array} \right] V^t \;\;\;,\;\;\; U=[U_1 \; U_2]
$$
where $U_1$ is a $m \times n$ matrix, $U_2$ is a $m \times (m-n)$ and $\Sigma_1$ is a square diagonal matrix with entries in the non-increasing order. 
Expanding the above, we get
$$
A = U_1 \Sigma_1 V^t
$$
which is sometimes called the "economical version". This is your second answer. Most of the time, $U_2$ is not required and hence the computation time and memory are saved by using the second version. However, note that $U_2^t A = 0$ (since $U_2^tU_1=0$) and if you require the left null space then you should compute $U_2$.  However, in general, $U_2$ is not unique.
