If $X$ is a square matrix, obtain the differential and the derivative of the functions:

  1. $f(X) = \operatorname{tr}(X)$,
  2. $f(X) = \operatorname{tr}(X^2)$,
  3. $f(X) = \operatorname{tr}(X^p)$ ($p$ is a natural number).

To find the differential I thought I could just find the differential of the compostion function first and then take the trace of that differential. Am I right in saying so? As for the derivative, I have no idea how I should do it for traces. Could anyone please help me out?

wj32's answer makes sense to me, however, I wonder if it is also possible to solve this question by using the ordinary way of finding differentials and derivatives, namely f(x+dx)-f(x). Is there someone who could maybe show me how this would be done (if possible)?

  • $\begingroup$ $\operatorname{tr}$ is linear $\endgroup$ – wj32 Feb 24 '13 at 10:31
  • $\begingroup$ @wj32 Ok, so what does that tell me? $\endgroup$ – dreamer Feb 24 '13 at 10:33
  • $\begingroup$ If $f:E\to F$ is linear, then $Df(x)=f$ for every $x\in E$. I'm only familiar with the Fréchet derivative, so I'm not sure what "differential" means. I'll try to post an answer. $\endgroup$ – wj32 Feb 24 '13 at 10:36
  • $\begingroup$ @wj32 thanks, would be highly appreciated :) $\endgroup$ – dreamer Feb 24 '13 at 10:43
  • 1
    $\begingroup$ Well, sort of. I highly recommend studying the Frechét derivative rigorously if you want to really understand what's going on. I have a series of articles on this at my website which you can find by visiting my profile. $\endgroup$ – wj32 Feb 24 '13 at 13:14

1) The trace is linear and bounded (which is automatic in finite dimension) so its derivative is equal to itself everywhere $$ df_X(H)=\mbox{tr}(H). $$

2) This is the composition of the trace with the bounded bilinear map $g:(X,Y)\longmapsto XY$ whose derivative is $$ dg_{(X,Y)}(H,K)=g(X,K)+g(H,Y)=XK+HY. $$ and the bounded linear map $h:X\longmapsto (X,X)$ whose derivative is itself at every point.

So by the chain rule $$ df_X(H)=d\mbox{tr}_{X^2}\circ dg_{(X,X)}\circ dh_{X}(H)=\mbox{tr}(XH+HX). $$ And by commutativity of the trace, this yields $$ 2\mbox{tr}(XH). $$

3) This is a composition again, of the trace with the $p$-linear map $$ k:(X_1,\ldots,X_p)\longmapsto X_1\cdots X_p $$ and the linear map $l:X\longmapsto (X,\ldots,X)$.

The derivative of $k$ is $$ dk_{(X_1,\ldots,X_p)}(H_1,\ldots,H_p)=H_1X_2\cdots X_p+X_1H_2X_3\cdots X_p+\ldots+X_1\cdots X_{p-1}H_p. $$

And the derivative of $l$ is itself. So by the chain rule $$ df_X(H)=d\mbox{tr}_{X^p}\circ dk_{(X,\ldots,X)}\circ dl_X(H). $$ Using the commutativity of the trace, we find $$ p\mbox{tr} (X^{p-1}H). $$

  • $\begingroup$ Thanks a lot. That's exactly what I was looking for. I very much appreciate your help :) $\endgroup$ – dreamer Feb 24 '13 at 11:33
  • $\begingroup$ @user48288 Oh, great! You're welcome. $\endgroup$ – Julien Feb 24 '13 at 11:34

$\newcommand{\tr}{\operatorname{tr}}$I'm not familiar with matrix-ey notation, so I'll just write down what I know.

Let $V$ be a finite-dimensional real vector space (or more generally a Banach space) and let $L(V)$ be the space of continuous linear operators on $V$. The trace is a linear map $\tr:L(V)\to\mathbb{R}$, so $D\tr(x)=\tr$ for every $x\in L(V)$. That answers your first question.

For the third question, we just use the chain rule. Define $p_n:L(V)\to L(V)$ by $p_n(x)=x^n$. You want \begin{align} D(\tr\circ p_n)(x)u &= (D\tr(x^n)\circ Dp_n(x))u \\ &= \tr\left(\sum_{k=0}^{n-1} x^kux^{n-k-1}\right) \\ &= \sum_{k=0}^{n-1}\tr\left(x^kux^{n-k-1}\right) \\ &= n\tr(x^{n-1}u). \end{align}

Lemma (Power rule). Let $E$ be a Banach algebra and let $p_n:E \to E$ be the map defined by $p_n(x)=x^n$. Then $$Dp_n(x)u=\sum_{k=0}^{n-1} x^kux^{n-k-1}.$$ In particular, if $E$ is commutative then $$Dp_n(x)u=nx^{n-1}u,$$ which is just the plain old power rule.

Proof. We use induction on $n$. The case $n=0$ is clear, so suppose the result holds for $n-1$. Since $p_n(x)=xp_{n-1}(x)$, the product rule shows that \begin{align} Dp_n(x)u &= up_{n-1}(x)+xDp_{n-1}(x)u \\ &= ux^{n-1}+x\sum_{k=0}^{n-2}x^kux^{n-k-2} \\ &= \sum_{k=0}^{n-1}x^kux^{n-k-1}. \end{align}

  • $\begingroup$ Thank you for your help :). Maybe a stupid question, but what does the u denote? Is that a unit vector? And why is it needed here? $\endgroup$ – dreamer Feb 24 '13 at 11:00
  • 1
    $\begingroup$ @user48288: Well as I said, I only know the "proper" Fréchet derivative notation. I'm guessing that your "differential" is the $u$, so maybe you want to say $$df=n\operatorname{tr}(X^{n-1}\,dX).$$ $\endgroup$ – wj32 Feb 24 '13 at 11:05
  • $\begingroup$ O ok, I get it now, thank you :). $\endgroup$ – dreamer Feb 24 '13 at 11:09

I found a really good and well understandable explanation here:

Matrix Calculus - Notes on the Derivative of a Trace, Johannes Traa

It writes matrix calculations as sums, where you can find the derivative with the known rules for scalars. In the end, you can convert the result back to matrix notation.

  • $\begingroup$ You should add brief explanation about what the link says. $\endgroup$ – choco_addicted Oct 10 '17 at 14:36
  • $\begingroup$ OK thanks for the hint $\endgroup$ – Daniel Frisch Oct 11 '17 at 15:05

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.