# L2 Norm of Pseudo-Inverse Relation with Minimum Singular Value

Consider a matrix $A \in\mathbb R^{n\times m}$ with $n>m$. It has full column rank, i.e. $\operatorname{rank}(A)=m$. Its left pseudo-inverse is given by; $$A^{-1}_\text{left}=(A^TA)^{-1}A^T$$

From two different results during my studies, I have realized the following: $$\|A^{-1}_\text{left}\|_2 = \frac{1}{\sigma_{\min}(A)}$$ just like the case as if $A$ is square invertible matrix.

I have seen a similar question, however I couldn't relate the answer with the equality given above.

My question is: How can we show that the L2 norm of left pseudo-inverse of $A$ is related to its minimum singular value?

• If $U,V$ are orthogonal (and square), then $\|AV\| = \|UA\| = \|A\|$
• $(U\Sigma V^T)^\dagger = V\Sigma^{\dagger}U^T$ (where $\dagger$ denotes the pseudo-inverse)
• If $\Sigma$ is a diagonal matrix of singular values, then $\|\Sigma^{\dagger}\| = 1/\sigma_{min}(\Sigma)$
Perhaps you can put the pieces together from here. Happily, this approach still works when $A$ does not have full column-rank.
For brevity, denote this left inverse $$A^{-1}_{\text{left}}$$ by $$B$$. It is well-known that $$AB$$ is the orthogonal projection onto the column space of $$A$$. Therefore, $$ABy\perp(I-AB)y$$ for every vector $$y$$. It follows that \begin{aligned} \|B\|_2&=\max_{y\ne0}\frac{\|By\|}{\|y\|}\\ &=\max_{y\ne0}\frac{\|By\|}{\|ABy+(I-AB)y\|}\\ &=\max_{By\ne0}\frac{\|By\|}{\|ABy+(I-AB)y\|}\\ &\le\max_{By\ne0}\frac{\|By\|}{\|ABy\|}\\ &\le\max_{x\ne0}\frac{\|x\|}{\|Ax\|}\\ &=\left(\min_{x\ne0}\frac{\|Ax\|}{\|x\|}\right)^{-1}\\ &=\frac{1}{\sigma_\min(A)}. \end{aligned}