2
$\begingroup$

Based on the answer to this question, I wonder how, using the differential notation, one finds $\frac{\partial}{\partial X} f(X\otimes A)$?

Assume that $X,A$ are positive definite matrices, and we know what is $\frac{\partial}{\partial U} f(U)$.

Edit: I have in mind this specific $f(U)=d^\intercal U d$

My try: I'll rewrite f as $f(X)=d^\intercal (X\otimes A)d$. We know that if $d=vec(D)$, then $(X\otimes A)d=vec(ADX^\intercal)$. Therefore, $d^\intercal(X\otimes A)d=d^\intercal vec(ADX^\intercal)=Tr(D^\intercal ADX^\intercal)$

so, we have $f=D:ADX^\intercal$, then $df=D:AD \ dX^\intercal=(AD)^\intercal D : (dX)^\intercal$.

Now since f is scalar, we have $df=D^\intercal AD \ dX$ which produces

$$\frac{\partial }{\partial X} f =D^\intercal AD$$

$\endgroup$
  • $\begingroup$ What $\otimes$ means? $\endgroup$ – Alex Silva Nov 8 '18 at 13:26
  • $\begingroup$ @AlexSilva kronecker product $\endgroup$ – An old man in the sea. Nov 8 '18 at 13:27
  • $\begingroup$ See theorem 11 (page 488) of this paper. $\endgroup$ – Alex Silva Nov 8 '18 at 13:42
  • $\begingroup$ @AlexSilva it's a bit different from what I could see... The theorem in the paper treats the case "$f\circ g$" and I want "$g\circ f$". $\endgroup$ – An old man in the sea. Nov 8 '18 at 13:56
  • $\begingroup$ What you've done makes perfect sense. $\endgroup$ – greg Nov 8 '18 at 15:28
0
$\begingroup$

Thanks to Greg, and the answers in the linked questions to this one (see section above, on the right)

I'll rewrite f as $f(X)=d^\intercal (X\otimes A)d$. We know that if $d=vec(D)$, then $(X\otimes A)d=vec(ADX^\intercal)$. Therefore, $d^\intercal(X\otimes A)d=d^\intercal vec(ADX^\intercal)=Tr(D^\intercal ADX^\intercal)$

so, we have $f=D:ADX^\intercal$, then $df=D:AD \ dX^\intercal=(AD)^\intercal D : (dX)^\intercal$.

Now since f is scalar, we have $df=D^\intercal AD \ dX$ which produces

$$\frac{\partial }{\partial X} f =D^\intercal AD$$

$\endgroup$
  • $\begingroup$ Since the vector $d$ can be reshaped into several differently shaped matrices, the matrix you've denoted as $D$ might actually be two matrices with the same elements but having different shapes. So you might want to write the result as $$\frac{\partial f}{\partial X} = D_1^TAD_2$$ As a concrete example consider matrices of the following dimensions $$\eqalign{ m,n &= {\rm size}(X) \cr p,q &= {\rm size}(A) \cr p,m &= {\rm size}(D_1) \cr q,n &= {\rm size}(D_2) \cr qn,1 &= {\rm size}(d) \cr }$$ with $\,\,p*m = q*n$ $\endgroup$ – greg Nov 8 '18 at 17:17
  • $\begingroup$ Nevermind, I just noticed that you specified $(A,X)$ as being positive definite, and therefore square. So in my comment above $m=n=p=q$ and the two $D$ matrices have the same shape. $\endgroup$ – greg Nov 8 '18 at 17:27

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.