Derivative of Frobenius norm of pseudo inverse with respect to original matrix Given a matrix  $\text{A} \in \mathbb{R}^{m \times n}$, what are the derivatives of the squared Frobenius norm of its left and right Moore-Penrose pseudo-inverse with respect to the $\text{A}$?


*

*$\frac{\partial}{\partial \text{A}} \left \Vert \left(\text{A}^T \text{A} \right)^{-1}\text{A}^T \right\Vert_F^2$

*$\frac{\partial}{\partial \text{A}} \left \Vert \text{A}^T \left(\text{A} \text{A}^T \right)^{-1} \right\Vert_F^2$

 A: Let's use a colon to denote the trace/Frobenius product, i.e.
$$\eqalign{
 X:Y = {\rm tr}(X^TY) \cr
}$$
Let's also define the symmetrization function
$$\eqalign{
 {\rm sym}(X) = \frac{1}{2}(X+X^T) \cr
}$$
and some new matrix variables
$$\eqalign{
 S &= A^TA &\implies S^T = S \cr
 B &= S^{-1}A^T &\implies BA = I \cr
}$$
Write the first function in terms of these new variables, then find its differential and gradient
$$\eqalign{
 \beta &= \|B\|_F^2 = B:B \cr\cr
 d\beta &= 2B:dB \cr
 &= 2B:\Big(S^{-1}\,dA^T - (S^{-1}\,dS\,S^{-1})A^T\Big) \cr
 &= 2S^{-1}B:dA^T - 2S^{-1}BAS^{-1}:dS \cr
 &= 2B^TS^{-1}:dA - 2S^{-2}:2\,{\rm sym}(A^T\,dA) \cr
 &= 2AS^{-2}:dA - 4\,{\rm sym}(S^{-2}):A^T\,dA \cr
 &= -2AS^{-2}:dA \cr
 &= -2A(A^TA)^{-1}(A^TA)^{-1}:dA \cr\cr
\frac{\partial\beta}{\partial A} &= -2A(A^TA)^{-1}(A^TA)^{-1} \cr\cr
}$$
The second function can be handled similarly
$$\eqalign{
 R &= AA^T \cr
 C &= A^TR^{-1} \cr
 \gamma &= C:C \cr
\frac{\partial\gamma}{\partial A} &= -2(AA^T)^{-1}(AA^T)^{-1}A \cr
}$$
