I am struggling on the derivative of a function with respect to a matrix. Let$f(B)=a^\prime_{1\times n} B_{n\times k}\left( B^{T}A_{n\times n}B\right) ^{-1}B^{T}a_{n\times 1}$, and $a$ is a $n\times 1$ vector of constant, $A_{n\times n}$ is a $n\times n$ full rank constant matrix, and $B_{n\times k}$ is a $n\times k$ matrix with rank $k$ (i.e., I assume $k<n$). What's partial derivative of $f(B)$ with respetive to $B$, i.e., $\frac{\partial f(B)}{\partial B}$? Thanks a lot!

  • $\begingroup$ You need some rank conditions on $B$ to make $f$ well-defined. $\endgroup$ – user251257 Jan 29 '18 at 18:49
  • $\begingroup$ Hint: if $F(t)$ is an invertible Matrix and $G(t) = F^{-1}(t)$, then $G'(t) = G(t) F'(t) G(t)$, where $t$ is a scalar. Now apply the product and chain rules. $\endgroup$ – user251257 Jan 29 '18 at 18:57
  • $\begingroup$ Thanks for the note, I have revised it. I am not a math major, so could you please be more specific? Thanks a lot! $\endgroup$ – Charles Chou Jan 29 '18 at 19:29

Define a new matrix variable and calculate the various differentials of it $$\eqalign{ M &= B^TAB \cr dM &= dB^T\,AB+B^TA\,dB \cr dM^{-1} &= -M^{-1}\,dM\,M^{-1} \cr }$$ Now write down the function and find the differential of it $$\eqalign{ f &= BM^{-1}B^T:aa^T \cr\cr df &= (dB\,M^{-1}B^T+B\,dM^{-1}\,B^T+BM^{-1}\,dB^T):aa^T \cr &= dB:aa^TBM^{-T} -BM^{-1}\,dM\,M^{-1}B^T:aa^T+aa^TBM^{-1}:dB \cr &= \Big(aa^TB(M^{-1}+M^{-T})-ABM^{-1}B^Taa^TBM^{-1}-A^TBM^{-T}B^Taa^TBM^{-T}\Big):dB \cr\cr \frac{\partial f}{\partial B} &= aa^TB(M^{-1}+M^{-T})-ABM^{-1}B^Taa^TBM^{-1}-A^TBM^{-T}B^Taa^TBM^{-T} \cr\cr }$$ In the above, colons are used to denote the trace/Frobenius product $$\eqalign{ A:B = {\rm tr}(A^TB) }$$

| cite | improve this answer | |
  • $\begingroup$ Thanks a lot! You are indeed very knowledgeable. There might be a typo on the final results. Since the matrix is symmetric, then the derivative should also be symmetric, so I guess the last term might be $ABM^{-1}B^Taa^TBM^{-T}$. Anyway, thanks a lot! $\endgroup$ – Charles Chou Jan 30 '18 at 16:57
  • $\begingroup$ @CharlesChou You're right, the last term was incorrect. I have updated my answer. I don't see anything in your question that indicates that $A$ is a symmetric matrix. But if it is then so is $M$, and this symmetry can be used to simplify the result further. $\endgroup$ – greg Feb 2 '18 at 23:32
  • $\begingroup$ Thanks a lot! It is very kind of you! $\endgroup$ – Charles Chou Feb 4 '18 at 1:11

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.