# What's $\frac{\partial}{\partial X} f(X\otimes A)$?

Based on the answer to this question, I wonder how, using the differential notation, one finds $$\frac{\partial}{\partial X} f(X\otimes A)$$?

Assume that $$X,A$$ are positive definite matrices, and we know what is $$\frac{\partial}{\partial U} f(U)$$.

Edit: I have in mind this specific $$f(U)=d^\intercal U d$$

My try: I'll rewrite f as $$f(X)=d^\intercal (X\otimes A)d$$. We know that if $$d=vec(D)$$, then $$(X\otimes A)d=vec(ADX^\intercal)$$. Therefore, $$d^\intercal(X\otimes A)d=d^\intercal vec(ADX^\intercal)=Tr(D^\intercal ADX^\intercal)$$

so, we have $$f=D:ADX^\intercal$$, then $$df=D:AD \ dX^\intercal=(AD)^\intercal D : (dX)^\intercal$$.

Now since f is scalar, we have $$df=D^\intercal AD \ dX$$ which produces

$$\frac{\partial }{\partial X} f =D^\intercal AD$$

• What $\otimes$ means? – Alex Silva Nov 8 '18 at 13:26
• @AlexSilva kronecker product – An old man in the sea. Nov 8 '18 at 13:27
• See theorem 11 (page 488) of this paper. – Alex Silva Nov 8 '18 at 13:42
• @AlexSilva it's a bit different from what I could see... The theorem in the paper treats the case "$f\circ g$" and I want "$g\circ f$". – An old man in the sea. Nov 8 '18 at 13:56
• What you've done makes perfect sense. – greg Nov 8 '18 at 15:28

I'll rewrite f as $$f(X)=d^\intercal (X\otimes A)d$$. We know that if $$d=vec(D)$$, then $$(X\otimes A)d=vec(ADX^\intercal)$$. Therefore, $$d^\intercal(X\otimes A)d=d^\intercal vec(ADX^\intercal)=Tr(D^\intercal ADX^\intercal)$$
so, we have $$f=D:ADX^\intercal$$, then $$df=D:AD \ dX^\intercal=(AD)^\intercal D : (dX)^\intercal$$.
Now since f is scalar, we have $$df=D^\intercal AD \ dX$$ which produces
$$\frac{\partial }{\partial X} f =D^\intercal AD$$
• Since the vector $d$ can be reshaped into several differently shaped matrices, the matrix you've denoted as $D$ might actually be two matrices with the same elements but having different shapes. So you might want to write the result as $$\frac{\partial f}{\partial X} = D_1^TAD_2$$ As a concrete example consider matrices of the following dimensions \eqalign{ m,n &= {\rm size}(X) \cr p,q &= {\rm size}(A) \cr p,m &= {\rm size}(D_1) \cr q,n &= {\rm size}(D_2) \cr qn,1 &= {\rm size}(d) \cr } with $\,\,p*m = q*n$ – greg Nov 8 '18 at 17:17
• Nevermind, I just noticed that you specified $(A,X)$ as being positive definite, and therefore square. So in my comment above $m=n=p=q$ and the two $D$ matrices have the same shape. – greg Nov 8 '18 at 17:27