2
$\begingroup$

Short question: How do I calculate the gradient of the $MSE(a, b)$ equation below?


Longer explanation: This problem arises, while I'm following a derivation of a term for an optimal beamvector $a$ in a data transmission. The mean square error (MSE) of this data transmission is calculated as follows:

$$MSE(a, b) = a^H(Hbb^HH^H+R_n)a + 1 - a^HHb - b^HH^Ha$$

where:

  • $a$, $b$: vectors, which can be chosen
  • $H$, $R_n$: matrices, which are fixed
  • $a^H$: denotes the Hermitian adjoint of $a$

The vector $a$ can be optimized (in dependece of $b$) by setting the gradient of the MSE to zero.

The problem is that I don't know how to calculate the gradient when the equation has the above form. The $a^H$ at the beginning and the $a$ at the end of the first summand irritates me...

The answer shall be:

$$ a^* = (Hbb^HH^H+R_n)^{-1}Hb = R_n^{-1}Hb\frac{1}{1+b^HH^HR_n^{-1}Hb}$$

But how to calculate this?


Update:

Using equations from The Matrix Cookbook I got this far:

$$\frac{\partial MSE(a, b)}{\partial a} = \frac{\partial}{\partial a} \left[ a^H\left(Hbb^HH^H+R_n\right)a\right] + \frac{\partial}{\partial a} 1 - \frac{\partial}{\partial a} \left[a^HHb\right] - \frac{\partial}{\partial a} \left[b^HH^Ha\right]$$

With

  • $\frac{\partial}{\partial a} 1 = 0$
  • $\frac{\partial b^TX^TDXx}{\partial X} = D^TXbc^T + DXcb^T$ (Cookbook (74))

I get:

$$\frac{\partial MSE(a, b)}{\partial a} = (Hbb^HH^H+R_n)^Ha + (Hbb^HH^H+R_n)a - \frac{\partial}{\partial a} \left[a^HHb\right] - \frac{\partial}{\partial a} \left[b^HH^Ha\right]$$

And that's it. I don't even know if I used equation (74) from the cookbook right, but it was the closed equation for the first summand. I'm sorry, I just don't get it...

$\endgroup$
  • $\begingroup$ $b$ is a constant vector for your purposes (of differentiation)? $\endgroup$ – J. M. is a poor mathematician Apr 19 '11 at 9:00
  • $\begingroup$ @mrothe, matrix cookbook has chapters dedicated to differentiation of matrices in general. I think it has enough information to get you started. $\endgroup$ – mpiktas Apr 19 '11 at 9:56
  • $\begingroup$ @J.M.: Yes, assume $b$ is a constant, i.e. unknown, but fixed for the differentiation regarding $a$. $\endgroup$ – user1058 Apr 19 '11 at 9:59
  • $\begingroup$ @mpiktas: Thank you. I have already tried that. I'll update my question with the steps, where I got stuck. $\endgroup$ – user1058 Apr 19 '11 at 10:12
  • 1
    $\begingroup$ Is the matrix $R_n$ real valued? or does it possibly have complex valued entries? (Not that it matters in the "general" setup, but in the "specific" case you are considering if the answer to my question is yes, the notation can be vastly simplified.) $\endgroup$ – Willie Wong Apr 19 '11 at 12:47
0
$\begingroup$

I'm not sure whether the following results hold for complex cases.

Let all the vectors and matrices be real valued. Then $$A=a^TBa+1-a^THb-b^TH^Ta$$ where $B=Hbb^TH^T+R_n$. $B$ is symmetric if $R_n$ is symmetric. Then $$dA=da^TBa+a^TBda-da^THb-b^TH^Tda$$ Let the gradient be zero. $$a^T(B^T+B)-2b^TH^T=0$$ If $B$ is symmetric, we have $2Ba=2Hb$ which implies $$a^*=B^{-1}Hb=(Hbb^TH^T+R_n)^{-1}Hb$$

But for the rest of your expected answer, I'm not sure. Because $$(cc^T+R_n)^{-1}c=\frac{R_n^{-1}c}{c^TR_n^{-1}c+1}, c=Hb$$ implies $$cc^TR_n^{-1}=c^TR_n^{-1}cI$$ Take trace on both sides of the above equation. The equation holds only when dimension is one.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy