Mathematics Stack Exchange is a question and answer site for people studying math at any level and professionals in related fields. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

Short question: How do I calculate the gradient of the $MSE(a, b)$ equation below?

Longer explanation: This problem arises, while I'm following a derivation of a term for an optimal beamvector $a$ in a data transmission. The mean square error (MSE) of this data transmission is calculated as follows:

$$MSE(a, b) = a^H(Hbb^HH^H+R_n)a + 1 - a^HHb - b^HH^Ha$$


  • $a$, $b$: vectors, which can be chosen
  • $H$, $R_n$: matrices, which are fixed
  • $a^H$: denotes the Hermitian adjoint of $a$

The vector $a$ can be optimized (in dependece of $b$) by setting the gradient of the MSE to zero.

The problem is that I don't know how to calculate the gradient when the equation has the above form. The $a^H$ at the beginning and the $a$ at the end of the first summand irritates me...

The answer shall be:

$$ a^* = (Hbb^HH^H+R_n)^{-1}Hb = R_n^{-1}Hb\frac{1}{1+b^HH^HR_n^{-1}Hb}$$

But how to calculate this?


Using equations from The Matrix Cookbook I got this far:

$$\frac{\partial MSE(a, b)}{\partial a} = \frac{\partial}{\partial a} \left[ a^H\left(Hbb^HH^H+R_n\right)a\right] + \frac{\partial}{\partial a} 1 - \frac{\partial}{\partial a} \left[a^HHb\right] - \frac{\partial}{\partial a} \left[b^HH^Ha\right]$$


  • $\frac{\partial}{\partial a} 1 = 0$
  • $\frac{\partial b^TX^TDXx}{\partial X} = D^TXbc^T + DXcb^T$ (Cookbook (74))

I get:

$$\frac{\partial MSE(a, b)}{\partial a} = (Hbb^HH^H+R_n)^Ha + (Hbb^HH^H+R_n)a - \frac{\partial}{\partial a} \left[a^HHb\right] - \frac{\partial}{\partial a} \left[b^HH^Ha\right]$$

And that's it. I don't even know if I used equation (74) from the cookbook right, but it was the closed equation for the first summand. I'm sorry, I just don't get it...

share|cite|improve this question
$b$ is a constant vector for your purposes (of differentiation)? – J. M. Apr 19 '11 at 9:00
@mrothe, matrix cookbook has chapters dedicated to differentiation of matrices in general. I think it has enough information to get you started. – mpiktas Apr 19 '11 at 9:56
@J.M.: Yes, assume $b$ is a constant, i.e. unknown, but fixed for the differentiation regarding $a$. – user1058 Apr 19 '11 at 9:59
@mpiktas: Thank you. I have already tried that. I'll update my question with the steps, where I got stuck. – user1058 Apr 19 '11 at 10:12
Is the matrix $R_n$ real valued? or does it possibly have complex valued entries? (Not that it matters in the "general" setup, but in the "specific" case you are considering if the answer to my question is yes, the notation can be vastly simplified.) – Willie Wong Apr 19 '11 at 12:47
up vote 0 down vote accepted

I'm not sure whether the following results hold for complex cases.

Let all the vectors and matrices be real valued. Then $$A=a^TBa+1-a^THb-b^TH^Ta$$ where $B=Hbb^TH^T+R_n$. $B$ is symmetric if $R_n$ is symmetric. Then $$dA=da^TBa+a^TBda-da^THb-b^TH^Tda$$ Let the gradient be zero. $$a^T(B^T+B)-2b^TH^T=0$$ If $B$ is symmetric, we have $2Ba=2Hb$ which implies $$a^*=B^{-1}Hb=(Hbb^TH^T+R_n)^{-1}Hb$$

But for the rest of your expected answer, I'm not sure. Because $$(cc^T+R_n)^{-1}c=\frac{R_n^{-1}c}{c^TR_n^{-1}c+1}, c=Hb$$ implies $$cc^TR_n^{-1}=c^TR_n^{-1}cI$$ Take trace on both sides of the above equation. The equation holds only when dimension is one.

share|cite|improve this answer

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.