Moore-Penrose Pseudoinverse identity

Question

I came across the following:

Let $n \leq m$, $L \in \mathbb{R}^{m \times n}$, with $\operatorname{Ker} L=\{0\}$ and $A \in \mathbb{R}^{n \times m}$.

Denote by $L^{\dagger}$ the Moore-Penrose pseudoinverse of $L$, and $\Pi_F$ is the orthogonal projector onto $F$. We know that $L^{\dagger}=\left(L^{\top} L\right)^{-1} L^{\top}$.

I read $D L^{\dagger}(A)=-L^{\dagger} A L^{\dagger}+\left(L^{\top} L\right)^{-1} A^{\top} \Pi_{(\mathrm{Im} L)^{\perp}}$, but there is no mention of $D$. I suspect that $D$ might be a diagonal matrix.

Does this identity (or a corrected version of it) ring a bell for anyone?

Answer 1

The symbol $D$ signifies derivative. Let $M$ be an invertible matrix. Since $M^{-1}=\operatorname{adj}(M)/\det(M)$ is a rational function in the entries of $M$, $DM^{-1}$ exists when $M^{-1}$ exists. As \begin{aligned} (M+H)(M+H)^{-1}-I &=(M+H)\left(M^{-1}+DM^{-1}(H)+O(\|H\|^2)\right)-I\\ &=MDM^{-1}(H) + HM^{-1} + O(\|H\|^2) \end{aligned} we get $DM^{-1}(H)=-M^{-1}HM^{-1}$ and hence $$ (M+H)^{-1}=M^{-1}-M^{-1}HM^{-1}+O(\|H\|^2). $$

Similarly, $L^+=(L^TL)^{-1}L^T$ is a rational function in the entries of $L$. So, it is differentiable whenever it exists. Now, suppose $A$ is a small perturbation to $L$. Then $$ (L+A)^T(L+A)=\underbrace{L^TL}_M+\underbrace{L^TA+A^TL+O(\|A\|^2)}_H=M+H. $$ It follows that \begin{aligned} &\left[(L+A)^T(L+A)\right]^{-1}(L+A)^T-(L^TL)^{-1}L^T\\ =&(M+H)^{-1}(L+A)^T-M^{-1}L^T\\ =&\left(M^{-1}-M^{-1}HM^{-1}+O(\|H\|^2)\right)(L+A)^T-M^{-1}L^T\\ =&\left(M^{-1}-M^{-1}HM^{-1}+O(\|A\|^2)\right)(L+A)^T-M^{-1}L^T\\ =&M^{-1}A^T-M^{-1}HM^{-1}L^T+O(\|A\|^2)\\ =&M^{-1}A^T-M^{-1}\left(L^TA+A^TL+O(\|A\|^2)\right)M^{-1}L^T+O(\|A\|^2)\\ =&-M^{-1}L^TAM^{-1}L^T+M^{-1}A^T(I-LM^{-1}L^T)+O(\|A\|^2)\\ =&-L^+AL^+ + (L^TL)^{-1}A^T(I-LL^+)+O(\|A\|^2). \end{aligned} Therefore $DL^+(A)=-L^+AL^+ + (L^TL)^{-1}A^T\Pi_{(\operatorname{Im} L)^{\perp}}$.