I have looked throughout the matrix cookbook and other sources, but am a bit confused by this problem. If I have a function $F = ABC$, what is the partial derivative of $F$ with respect to $B$? When it comes to matrix functions I could not find a nice answer to this problem. I did find something akin to $dF = CAdX$, but that doesn't seem to make sense as the dimensions of $C$ and $A$ would not match up for multiplication. A could be a $3\times3$, $B$ a $3\times 3$, and $ C$ a $3\times 4$. I'd appreciate help on this. Thanks.

Edit- Well I had changed the problem to be more in line with what I was originally working with below in the comments. Here is the problem: $f = tr((ABC)(ABC)^T)$ and I want the partial derivative $\frac{\partial}{\partial B} tr((ABC)(ABC)^T)$. I ended up using a modified version of copper.hat's answer. I combined $A$ and $B$ to get something like this $2(ABC)C^T$ for the gradient.

Thank you both for the help. Both answers below are technically correct, I just accepted the shorter and more convenient form.

  • $\begingroup$ The function $G:B\longmapsto ABC$ is linear and bounded. So it is differentiable, with constant derivative equal to $G$. $\endgroup$ – Julien Apr 9 '13 at 16:01

When dealing with functions of this sort, I find it easier to deal with the derivative applied to some perturbation.

Let $F(B) = ABC$. Since $F$ is linear, we have $F(B+\Delta) = F(B) + A\Delta C$, so it follows that $DF(B)(\Delta) = A\Delta C$. However, since $DF(B)$ is a map $\mathbb{R}^{n \times n} \to \mathbb{R}^{n \times n}$ (or $\mathbb{C}$, as the case may be), there is no particularly convenient matrix representation.

Note: Each coordinate of $DF(B)$ has a convenient representation. To see, let $\phi_{ij}(B) = e_i^T F(B) e_j = e_i^T ABC e_j$. Then, as above, we have $D\phi_{ij}(B)(\Delta) = e_i^T A\Delta C e_j$. This gives: \begin{eqnarray} D\phi_{ij}(B)(\Delta) &=& e_i^T A \Delta C e_j \\ &=& \operatorname{tr} (e_i^T A \Delta C e_j) \\ &=& \operatorname{tr} (e_j e_i^T A \Delta C ) \\ &=& \operatorname{tr} (C e_j e_i^T A \Delta ) \\ &=& \operatorname{tr} ((A^T e_i e_j^TC^T)^T \Delta ) \\ &=& \langle A^T e_i e_j^TC^T, \Delta \rangle_F \end{eqnarray} Where $\langle X, Y \rangle_F = \operatorname{tr}(X^TY)$ is the inner product induced by the Frobenius norm. With this inner product, the gradient is the given by $\nabla \phi_{ij}(B) = A^T e_i e_j^TC^T $.

Another note: If $G(B) = \operatorname{tr} ((ABC)(ABC)^T) $, then you can write $G(B) = \langle ABC, ABC \rangle_F$. Then $G(B+\Delta ) = G(B) + \langle A \Delta C, ABC \rangle_F + \langle ABC, A \Delta C \rangle_F + \langle A \Delta C, A \Delta C \rangle_F$, from which you can see that \begin{eqnarray} DG(B) (\Delta) &=& \langle A \Delta C, ABC \rangle_F + \langle ABC, A \Delta C \rangle_F \\ &=& 2 \langle ABC, A \Delta C \rangle_F \\ &=& 2 \langle A^TABC C^T, \Delta \rangle_F \end{eqnarray} From which it follows that $\nabla G(B) = 2A^TABC C^T$.

| cite | improve this answer | |
  • $\begingroup$ Can you look at the comment I made to the answer below this? If there is no convenient solution I likely messed up elsewhere. $\endgroup$ – Robert N. Apr 9 '13 at 16:08
  • $\begingroup$ Well, there is a solution, it doesn't have a convenient representation. The point is that it is easier to consider the application of the derivative to a particular perturbation ($\Delta$ above). I don't know what you mean by a gradient for a function that is not $\mathbb{R}$-valued. $\endgroup$ – copper.hat Apr 9 '13 at 16:14
  • $\begingroup$ Well the trace function is a real valued function. The function is the trace of ABC squared. $\endgroup$ – Robert N. Apr 9 '13 at 16:20
  • $\begingroup$ @RobertN.: I don't see trace in the question? Anyway, I added a note that shows how each coordinate of $F(B)$ can be represented as a gradient. $\endgroup$ – copper.hat Apr 9 '13 at 16:36
  • $\begingroup$ @copper.hat Nice solution! $\endgroup$ – Elias Costa Apr 9 '13 at 16:38

The derivative is just the best linear approximation and so $ \frac{dF}{dB}(X) = A X C $

In a different language, $f = L_A \circ R_C \implies df = dL_A \circ dR_C$

| cite | improve this answer | |
  • $\begingroup$ A linear approximation? Why is there no direct solution. Maybe I screwed up elsewhere. I got to this problem from here: dF/dB = d/dB TR((ABC)transpose(ABC)). After doing one half of the derivative, I got to a point where I had to do d/dB ABC. $\endgroup$ – Robert N. Apr 9 '13 at 16:05
  • $\begingroup$ No, the derivative I gave is the direct solution to the derivative. I was trying to suggest that a derivative is a linear approximation to usually non-linear functions, but in your case, the function is linear in $B$. The best linear approximation to a linear function is the function itself. $\endgroup$ – muzzlator Apr 9 '13 at 16:07
  • $\begingroup$ What would be the course of action when working on an optimization problem where the derivative would be used as a gradient? $\endgroup$ – Robert N. Apr 9 '13 at 16:10
  • $\begingroup$ $\frac{dF}{dB}(X)$ where $F = \operatorname{Tr}(ABC) (ABC)^T$ is given by a product rule: $$\operatorname{Tr}(AXC) (ABC)^T + \operatorname{Tr}(ABC) (AXC)^T$$ This follows from the linearity of trace and bilinearity of multiplication $\endgroup$ – muzzlator Apr 9 '13 at 16:10
  • $\begingroup$ After the product rule should you not continue with the chain rule on ABC? I'll try working with this for now though. Thanks a lot! If I have any additions I'll add them here in about an hour or two, and accept the answer then. $\endgroup$ – Robert N. Apr 9 '13 at 16:17

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.