Derivative of trace of inverse of a matrix function

I am trying to derive the derivative of the trace of inverse of a matrix function (of X), i.e. $$f(X)=Tr\left((HXH^{H}+I)^{-1}\right)$$ where $H\in R^{n\times m}, X\in R^{m\times m}$.

So $HXH^{H}+I$ is invertible while $HXH^{H}$ is not. Any one know how to find $\frac{\partial f}{\partial X}$ ?

• Presumably, $H^H$ means the conjugate transpose of $H$. Is this so? – Omnomnomnom Sep 1 '15 at 17:18
• Yes, you are right. – Mafen Sep 2 '15 at 3:48
• I am used to the notation $H^*$, so this is what I used in my answer. – Omnomnomnom Sep 2 '15 at 3:50

Note: In the below, I used $X$ instead of $H$ by accident. Also, $X^*$ is my notation for $X^H$.

We can find this inverse by the chain rule. Note that $f = f_1 \circ f_2 \circ f_3$, where $$f_1(A) = Tr(A)\\ f_2(A) = A^{-1}\\ f_3(A) = XAX^* + I$$ The derivatives of these functions are given by $$[f_1'(A)](B) = Tr(B)\\ [f_2'(A)](B) = -A^{-1}BA^{-1}\\ [f_3'(A)](B) = XBX^*\\$$ The chain rule tells us that $$[(f_1 \circ f_2 \circ f_3)'(A)] = \\ [f_1'(f_2(f_3(A)))]\circ [f_2'(f_3(A))] \circ f_3'(A)$$ So, all together, that gives us $$[f'(A)] = \\ (B \mapsto Tr[(XBX^* + I)^{-1}])\circ (B \mapsto -A^{-1}(XBX^* + I)A^{-1})\circ (B \mapsto XBX^*) =\\ (B \mapsto Tr[(XBX^* + I)^{-1}])\circ (B \mapsto -A^{-1}(X[XBX^*]X^* + I)A^{-1}) =\\ (B \mapsto Tr[(XBX^* + I)^{-1}])\circ (B \mapsto -A^{-1}(X^2BX^{*2} + I)A^{-1}) =\\ (B \mapsto Tr[(X[-A^{-1}(X^2BX^{*2} + I)A^{-1}]X^* + I)^{-1}])$$ In other words, if $A$ is parameterized as a function of $t \in \Bbb R$, then we can write $$\frac{d}{dt}f(A(t)) = Tr[(X[-A^{-1}(X^2\frac{dA}{dt}X^{*2} + I)A^{-1}]X^* + I)^{-1}]$$

Clarification of my derivatives:

By $[f'(A)](B)$, I mean that for every $A$, $f'(A)$ is a function on $B$. You are right to be confused. Let's look at an example.

Take $f(X) = Tr(X)$, and find its derivative my way. In fact, $f$ is linear, so this will be easy: we have $$[f'(A)](B) = Tr(B)$$ Note that, in this case, $A$ doesn't pop up in the definition of $f'$. In other words, $f$ has a constant derivative. This makes sense because $f$ is linear. Another way of writing the above is to say that if $A(t)$ is a matrix-valued function on $t \in \Bbb R$, then $$\frac{d}{dt}f(A(t)) = Tr \frac{dA}{dt}$$ Now, my understanding of your notation is that $$\frac{\partial f}{\partial X}[i,j] = \frac{\partial f}{x_{ji}}$$ equivalently, define $A_{ij}(t) = A + t E_{ji}$ (where $E_{ji}$ is the matrix with $0$s everywhere except at the $j,i$ entry, where it has a $1$). Then we should have $$\frac{\partial f}{\partial X}(A)[i,j] = \frac {d}{dt} f(A_{ij}(t))$$ By my above statement, that means we can write $$\frac{\partial f}{\partial X}(A)[i,j] = Tr(E_{ji}) = \begin{cases} 1 & i=j\\ 0 & i \neq j \end{cases}$$ In other words, $\frac {\partial f}{\partial X}$ is $I$, which is exactly what we expected.

• Thanks for your quick and detailed answer! But I haven't quite got it. What do you mean by $[f^{\prime}_1 (A)](B)$? – Mafen Sep 2 '15 at 4:03
• Well, this is the notion of derivatives that I'm used to: if $f:X \to Y$, then at each $A \in X$, $f'(A)$ is a linear function from $X$ to $Y$ (this outlook is used in Rudin, for example). Unfortunately, I am not familiar enough with your $\frac{\partial f}{\partial X}$ notation to work well with it. I do attempt to clarify things with my equation at the end, though, if that's any help. – Omnomnomnom Sep 2 '15 at 4:08
• If you play around with the last equation, you can get the partial derivatives that you need for the matrix $\frac{\partial f}{\partial X}$, whatever that happens to look like. – Omnomnomnom Sep 2 '15 at 4:13
• Do you mean $[f^{\prime}(A)](B)$ by substituting $B$ into $A$ in $f^{\prime}(A)$? According to what I know, $\frac{\partial Tr{A}}{\partial A}=I$ and $\frac{\partial Tr{AB}}{\partial A}=B$. (Sorry for my weakness in math.) – Mafen Sep 2 '15 at 4:14
• Not quite. I'll add something to the end of my answer to explain better. – Omnomnomnom Sep 2 '15 at 4:19

I got an answer myself in a relative simple way: $$\frac{\partial f}{\partial X}=H^H((HXH^H+I)^{-2})^{H}H$$ through the chain rule. That is, let $U=HXH^H$ and $g(U)=Tr((U+I)^{-1})$, then $$\frac{\partial f}{\partial X_{i,j}}=Tr[(\frac{\partial g(U)}{\partial U})^H \frac{\partial U}{\partial X_{i,j}}]$$.Since we can easily get $\frac{\partial g(U)}{\partial U}=(U+I)^{-2}$ and $$U=\sum_i \sum_j X_{i,j} h_i h_j^H$$, we know $$\frac{\partial U}{\partial X_{i,j}}=h_i h_j^H$$.We thus have $$\frac{\partial f}{\partial X_{i,j}}=Tr[((HXH^H+I)^{-2})^{H}h_ih_j^{H}]=Tr[h_j^{H}((HXH^H+I)^{-2})^{H}h_i]=h_j^{H}((HXH^H+I)^{-2})^{H}h_i$$. Reorganizing them in a compact form we have the above result (hopefully there is no error). Actually the $X$ I am dealing with is diagonal, so the derivation is simpler than above. Thank you all for your attention and especially for @Omnomnomnom!

• This looks like it could be correct. Of course, it helps to know the derivative of $g(U)$ in this form, and to have a chain rule that works with your notation. – Omnomnomnom Sep 2 '15 at 11:47