I want to compute the gradient of a function $f$ w.r.t to $\mathbf{d}$ which is a real-valued vector but there are also complex matrices and vectors involved. I would normally compute the gradient w.r.t. $\mathbf{d^{*}}$ to go towards the maximum change but since the vector is real it seems that the conjugate function does not play a role here. Anyway, something in my derivation of the gradient seems to be wrong because my matlab program is not converging to a solution. My program is ok, I have checked it several times. Hope you can help guys!

$ f =\mathbf{\|\Phi\|}_2^2$ with $f \in \mathbb{R}$ and $\Phi \in \mathbb{R}^{Nx1}$

$\mathbf{\Phi = y \odot y^{*}}$ where $\mathbf{y} \in \mathbb{C}^{Nx1}$

$\mathbf{y = AXw}$ with $\mathbf{A}$ and $\mathbf{X}$ are Hermitian of dimensions NxN

$\mathbf{w} \in \mathbb{C}^{Nx1}$ is formed by some function of the elements in vector $\mathbf{d} \in \mathbb{R}^{Mx1}$

The gradient $\nabla f$ w.r.t to $\mathbf{d}$ is

$\nabla f = \frac{\partial \mathbf{w}}{\partial \mathbf{d}} \frac{\partial \mathbf{y}}{\partial \mathbf{w}} \frac{\partial \mathbf{\Phi}}{\partial \mathbf{y}} \frac{\partial f}{\partial \mathbf{\Phi}}$

$\frac{\partial \mathbf{w}}{\partial \mathbf{d}} = \mathbf{Z}$, where $\mathbf{Z} \in \mathbb{C}^{MxN}$

$\frac{\partial \mathbf{y}}{\partial \mathbf{w}} = \mathbf{AX}$

$\frac{\partial \mathbf{\Phi}}{\partial \mathbf{y}} =diag(\mathbf{y^{*}})$

$\frac{\partial f}{\partial \mathbf{\Phi}} = 2\mathbf{\Phi}$

My update equation is $\mathbf{d = d} - u Real\{\nabla f\}$ in order to enforce the vector $\mathbf{d}$ remain real but the algorithm is not converging. Any mistakes that you could have detected here? Is my logic wrong? How could the function $f$ be minimized then?

  • $\begingroup$ What does $\odot$ mean? $\endgroup$ – littleO Dec 2 '16 at 21:15
  • $\begingroup$ It is really hard to parse what you are looking for. $\endgroup$ – copper.hat Dec 2 '16 at 22:23
  • $\begingroup$ $\frac{\partial y}{\partial w}=X^TA^T=X^*A^*\,\,\,$ -- this term had the opposite layout convention compared to your other gradients. $\endgroup$ – greg Dec 3 '16 at 0:25
  • $\begingroup$ LittleO, it means point-wise product. $\endgroup$ – Duns Dec 3 '16 at 6:48
  • $\begingroup$ What do you mean greg? $\endgroup$ – Duns Dec 3 '16 at 6:49

For the vector $d$ let's use $e,\,$because $d$ is too easily confused with differential/derivative operators.
For the vector $\Phi$ let's use $p,\,$because it's easier to type.
I'll use $B$ in place of $(AX)$ for the same reason.

And instead of the chain rule, let's attack the problem with successive substitutions in differential expressions.

We're given the relationships $$\eqalign{ w &= Z^Te \cr y &= Bw \cr p &= y\odot y^* \cr P &= \operatorname{Diag}(p) \cr f &= p:p \cr }$$ where $\odot$ represents the Hadamard element-wise product, and $:$ the Frobenius inner product.

Now let's find the differential and gradient of $f$ $$\eqalign{ df &= 2p:dp \cr &= 2p:(y^*\odot dy + y\odot dy^*) \cr &= 2p\odot y^*:dy + 2p\odot y:dy^* \cr &= 2Py^*:dy + 2Py:dy^* \cr &= 2Py^*:Bdw + 2Py:B^*dw^* \cr &= 2B^TPy^*:dw + 2B^HPy:dw^* \cr &= 2B^TPy^*:Z^Tde + 2B^HPy:Z^Hde^* \cr &= 2(ZB^TPy^* + Z^*B^HPy):de \cr\cr \frac{\partial f}{\partial e} &= 2(ZB^TPy^* + Z^*B^HPy) \cr &= 4\,\operatorname{Re}(ZB^TPy^*) \cr }$$ Translating this back into your preferred notation $$\eqalign{ \frac{\partial f}{\partial d} &= 4\,\operatorname{Re}(ZX^TA^T(\Phi\odot y^*)) \cr\cr }$$ In the above, I made use of the fact that $(de^*=de),\,$since it's a real vector.

I also used the commutivity (and mutual commutivity) of the Hadamard and Frobenius products, i.e. $$\eqalign{ A:B &= B:A \cr A\odot B &= B\odot A \cr A:B\odot C &= A\odot B:C \cr }$$ and the mixed-product rules for Frobenius-Matrix products $$\eqalign{ A:BC &= B^TA:C \cr A:BC &= AC^T:B \cr }$$ which follow from the Frobenius-Trace equivalence $$A:B = \operatorname{tr}(A^TB)$$ and the cyclic properties of the trace.

| cite | improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.