13
$\begingroup$

I googled around and searched inside the forum but I'm still confused about a problem.

I have 2 matrix functions $f,g : \mathbb{R}^{n \times n} \times \mathbb{R}^{a \times b} \rightarrow \mathbb{R}^{n \times n}$. Starting from this, I have the following expression:

$$ t(Q, X, Y) = \text{tr}(f(g(Q, X),Y))$$

where $\text{tr}$ is the trace operator and $X, Y \in \mathbb{R}^{a \times b}$ and $Q \in \mathbb{R}^{n \times n}$.

How do I evaluate $\frac{\partial t(Q,X, Y)}{\partial X}$ and $\frac{\partial t(Q,X, Y)}{\partial Y}$?

I mean, I would like to know how to correctly apply the chain rule.

* Addition *

I will try to give more information about my problem. Suppose that $a = b = n$ and that $f(A,B) = AB$ and $g(A,B) = BA + AB$ (actually this is only an example of possible functions $f$ and $g$). Then I have that:

$$f(g(Q,X),Y) = f(XQ + QX, Y) = XQY + QXY$$

Then, using matrix calculus (hoping there are no error!), I have that:

$$ \frac{\partial t(Q,X,Y)}{\partial X} = QY + YQ\\ \frac{\partial t(Q,X,Y)}{\partial Y} = XQ + QX$$

I can easily compute the result if I know the form of $f$ and $g$. Notice that the derivatives I obtained are in a matrix form. But actually I need to deal with generic functions. And for this reason I need to use the chain rule. The problem is that the chain rule formulas I know are helpful to derive the derivative with respect to a certain element of the matrix $X$ (or $Y$). In this case, I'm not able to have a matrix form of the derivatives.

So, my question is... there is a chain rule formula I'm missing which let me describe these derivatives in a matrix form?

* Addition 2 *

The chain rule formulas that I know are reported here http://en.wikipedia.org/wiki/Matrix_calculus#Scalar-by-matrix_identities (see the 7th row of the table)

$\endgroup$
4
  • 1
    $\begingroup$ It might help to consider the function $h(Q,X,Y) = (g(Q,X),Y)$, whose differential is easily calculated. Then $t(Q,X,Y) = l \circ f \circ h (Q,X,Y)$, where $l$ is the trace $\endgroup$ Dec 17, 2012 at 20:49
  • $\begingroup$ I know this formula (en.wikipedia.org/wiki/… - it is the 7th formula into the table). This is performed on each $X_{i,j}$ separately! I would like to know the formula with respect to all $X$. $\endgroup$ Dec 18, 2012 at 14:34
  • $\begingroup$ I'm going to try to give more details in my question $\endgroup$ Dec 19, 2012 at 20:11
  • 1
    $\begingroup$ nice question .... +1 $\endgroup$
    – TShiong
    Mar 9, 2023 at 18:51

1 Answer 1

4
$\begingroup$

$ \def\LR#1{\left(#1\right)} $Let's use uppercase letters for the matrix variables, so they're easy to distinguish from the lowercase scalars, and $I$ for the identity matrix $$\eqalign{ G &= G(Q,X) \cr F &= F(G,Y) \cr t &= {\rm tr}(F) = I:F \cr dt &= I:dF \cr }$$ First, let's calculate the differential and gradient wrt $Y$ $$\eqalign{ dt &= I:\LR{\frac{\partial F}{\partial Y}:dY} \\ \frac{\partial t}{\partial Y} &= I:\frac{\partial F}{\partial Y} \\ }$$ And now wrt $X$ $$\eqalign{ dt &= I:\LR{\frac{\partial F}{\partial G}:\frac{\partial G}{\partial X}:dX} \\ \frac{\partial t}{\partial X} &= I:\frac{\partial F}{\partial G}:\frac{\partial G}{\partial X} \\\\ }$$ Note that the matrix-by-matrix gradients are $4^{th}$ order tensors. For example, here is one of the gradients in component form $$\eqalign{ \LR{\frac{\partial G}{\partial X}}_{ijkl} = \frac{\partial G_{ij}}{\partial X_{kl}} \\\\ }$$

Also note that colons are used to denote the double-contraction product, e.g. $$\LR{\frac{\partial F}{\partial G}:\frac{\partial G}{\partial X}}_{ijkl} = \sum_p\sum_q\;\frac{\partial F_{ij}\;}{\partial G_{pq}}\;\frac{\partial G_{pq}}{\partial X_{kl}\;}$$

$\endgroup$

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .