2
$\begingroup$

Let $A$ be a matrix, supposedly $k\times k$ matrix. I know that $$\frac{\partial A^{-1}}{\partial A} = -A^{-2} $$ I do not know how I am supposed to obtain the following results using this fact. I want to know the step of $$\frac{\partial a^\top A^{-1} b}{\partial A} = -(A^\top)^{-1}ab^\top (A^\top)^{-1} $$ Also, I want to know the solution to $$\frac{\partial (A^\top)^{-1}ab^\top (A^\top)^{-1} }{\partial A} = ? $$

$\endgroup$
  • 2
    $\begingroup$ First formula; is not it $-A^ {-2}$ ? $\endgroup$ – Damien Dec 3 '18 at 6:36
  • $\begingroup$ @Damien Yes, i editted $\endgroup$ – user1292919 Dec 3 '18 at 15:26
  • $\begingroup$ Do you just know the first identity you stated or also how to obtain it? Because if so, you should be able to figure out the other identities too. However, please let us know what your attempts are. $\endgroup$ – user526015 Dec 3 '18 at 15:41
4
$\begingroup$

Start with the defining equation for the matrix inverse and find its differential. $$\eqalign{ I &= A^{-1}A \cr 0 &= dA^{-1}\,A + A^{-1}\,dA \cr 0 &= dA^{-1}\,A + A^{-1}\,dA\,A^{-1} \cr dA^{-1} &= -A^{-1}\,dA\,A^{-1} \cr }$$ Next note the gradient of a matrix with respect to itself. $$ {\mathcal H}_{ijkl} = \frac{\partial A_{ij}}{\partial A_{kl}} = \delta_{ik}\delta_{jl} $$ Note that ${\mathcal H}$ is a 4th order tensor with some interesting symmetry properties (isotropic). It is also the identity element for the Frobenius product, i.e. for any matrix $B$ $${\mathcal H}:B=B:{\mathcal H}=B$$ Now we can answer your first question. The function of interest is scalar-valued. Let's find its differential and gradient $$\eqalign{ \phi &= a^TA^{-1}b \cr &= ab^T:A^{-1} \cr d\phi &= ab^T:dA^{-1} \cr &= -ab^T:A^{-1}\,dA\,A^{-1} \cr &= -A^{-T}ab^TA^{-T}:dA \cr \frac{\partial\phi}{\partial A} &= -A^{-T}ab^TA^{-T} \cr }$$ Now let's try the second question. This time the function of interest is matrix-valued. $$\eqalign{ F &= A^{-1}ab^TA^{-1} \cr dF &= dA^{-1}ab^TA^{-1} + A^{-1}ab^TdA^{-1} \cr &= -A^{-1}\,dA\,A^{-1}ab^TA^{-1} - A^{-1}ab^TA^{-1}\,dA\,A^{-1} \cr &= -A^{-1}\,dA\,F - F\,dA\,A^{-1} \cr &= -\Big(A^{-1}{\mathcal H}F^T + F{\mathcal H}A^{-T}\Big):dA \cr \frac{\partial F}{\partial A} &= -\Big(A^{-1}{\mathcal H}F^T+F{\mathcal H}A^{-T}\Big) \cr }$$ This gradient is a 4th order tensor.

If you prefer, you can vectorize the matrices to flatten the result. $$\eqalign{ {\rm vec}(dF) &= -{\rm vec}(-A^{-1}\,dA\,F + F\,dA\,A^{-1}) \cr &= -(F^T\otimes A^{-1} + A^{-T}\otimes F)\,{\rm vec}(dA) \cr df &= -(F^T\otimes A^{-1} + A^{-T}\otimes F)\,da \cr \frac{\partial f}{\partial a} &= -\Big(F^T\otimes A^{-1} + A^{-T}\otimes F\Big) \cr\cr }$$ In some step above, a colon was used to denote the Frobenius (double-contraction) product $$\eqalign{ A &= {\mathcal H}:B &\implies &A_{ij} &= \sum_{kl}{\mathcal H}_{ijkl} B_{kl} \cr \alpha &= H:B &\implies &\alpha &= \sum_{ij}H_{ij} B_{ij} = {\rm Tr}(H^TB) \cr }$$

$\endgroup$
  • 1
    $\begingroup$ it might be obvious to you and others, but can you explain how to reach from this step $-A^{-1}\,dA\,F - F\,dA\,A^{-1}$ to this step $-\Big(A^{-1}{\mathcal H}F^T + F{\mathcal H}A^{-T}\Big):dA$? $\endgroup$ – user550103 Dec 4 '18 at 6:04
  • 1
    $\begingroup$ It's not obvious, it's just one of the properties of ${\mathcal H}$. Work out a simple example like $(A{\mathcal H}B:X)$ in index notation and, utilizing the properties of those Kronecker deltas, you'll find that it equals $(AXB^T)$ $\endgroup$ – greg Dec 4 '18 at 13:14
  • 1
    $\begingroup$ It's also closely related to the equally non-obvious formula $${\rm vec}(AXB^T) = (B\otimes A)\,{\rm vec}(X)$$ which uses a Kronecker product instead of Kronecker deltas. $\endgroup$ – greg Dec 4 '18 at 13:22
  • $\begingroup$ Thank you for the explanation, greg! $\endgroup$ – user550103 Dec 5 '18 at 9:58

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.