# Minimization with complex gradient descent

I want to compute the gradient of a function $f$ w.r.t to $\mathbf{d}$ which is a real-valued vector but there are also complex matrices and vectors involved. I would normally compute the gradient w.r.t. $\mathbf{d^{*}}$ to go towards the maximum change but since the vector is real it seems that the conjugate function does not play a role here. Anyway, something in my derivation of the gradient seems to be wrong because my matlab program is not converging to a solution. My program is ok, I have checked it several times. Hope you can help guys!

$f =\mathbf{\|\Phi\|}_2^2$ with $f \in \mathbb{R}$ and $\Phi \in \mathbb{R}^{Nx1}$

$\mathbf{\Phi = y \odot y^{*}}$ where $\mathbf{y} \in \mathbb{C}^{Nx1}$

$\mathbf{y = AXw}$ with $\mathbf{A}$ and $\mathbf{X}$ are Hermitian of dimensions NxN

$\mathbf{w} \in \mathbb{C}^{Nx1}$ is formed by some function of the elements in vector $\mathbf{d} \in \mathbb{R}^{Mx1}$

The gradient $\nabla f$ w.r.t to $\mathbf{d}$ is

$\nabla f = \frac{\partial \mathbf{w}}{\partial \mathbf{d}} \frac{\partial \mathbf{y}}{\partial \mathbf{w}} \frac{\partial \mathbf{\Phi}}{\partial \mathbf{y}} \frac{\partial f}{\partial \mathbf{\Phi}}$

$\frac{\partial \mathbf{w}}{\partial \mathbf{d}} = \mathbf{Z}$, where $\mathbf{Z} \in \mathbb{C}^{MxN}$

$\frac{\partial \mathbf{y}}{\partial \mathbf{w}} = \mathbf{AX}$

$\frac{\partial \mathbf{\Phi}}{\partial \mathbf{y}} =diag(\mathbf{y^{*}})$

$\frac{\partial f}{\partial \mathbf{\Phi}} = 2\mathbf{\Phi}$

My update equation is $\mathbf{d = d} - u Real\{\nabla f\}$ in order to enforce the vector $\mathbf{d}$ remain real but the algorithm is not converging. Any mistakes that you could have detected here? Is my logic wrong? How could the function $f$ be minimized then?

• What does $\odot$ mean? – littleO Dec 2 '16 at 21:15
• It is really hard to parse what you are looking for. – copper.hat Dec 2 '16 at 22:23
• $\frac{\partial y}{\partial w}=X^TA^T=X^*A^*\,\,\,$ -- this term had the opposite layout convention compared to your other gradients. – greg Dec 3 '16 at 0:25
• LittleO, it means point-wise product. – Duns Dec 3 '16 at 6:48
• What do you mean greg? – Duns Dec 3 '16 at 6:49

For the vector $d$ let's use $e,\,$because $d$ is too easily confused with differential/derivative operators.
For the vector $\Phi$ let's use $p,\,$because it's easier to type.
I'll use $B$ in place of $(AX)$ for the same reason.
We're given the relationships \eqalign{ w &= Z^Te \cr y &= Bw \cr p &= y\odot y^* \cr P &= \operatorname{Diag}(p) \cr f &= p:p \cr } where $\odot$ represents the Hadamard element-wise product, and $:$ the Frobenius inner product.
Now let's find the differential and gradient of $f$ \eqalign{ df &= 2p:dp \cr &= 2p:(y^*\odot dy + y\odot dy^*) \cr &= 2p\odot y^*:dy + 2p\odot y:dy^* \cr &= 2Py^*:dy + 2Py:dy^* \cr &= 2Py^*:Bdw + 2Py:B^*dw^* \cr &= 2B^TPy^*:dw + 2B^HPy:dw^* \cr &= 2B^TPy^*:Z^Tde + 2B^HPy:Z^Hde^* \cr &= 2(ZB^TPy^* + Z^*B^HPy):de \cr\cr \frac{\partial f}{\partial e} &= 2(ZB^TPy^* + Z^*B^HPy) \cr &= 4\,\operatorname{Re}(ZB^TPy^*) \cr } Translating this back into your preferred notation \eqalign{ \frac{\partial f}{\partial d} &= 4\,\operatorname{Re}(ZX^TA^T(\Phi\odot y^*)) \cr\cr } In the above, I made use of the fact that $(de^*=de),\,$since it's a real vector.
I also used the commutivity (and mutual commutivity) of the Hadamard and Frobenius products, i.e. \eqalign{ A:B &= B:A \cr A\odot B &= B\odot A \cr A:B\odot C &= A\odot B:C \cr } and the mixed-product rules for Frobenius-Matrix products \eqalign{ A:BC &= B^TA:C \cr A:BC &= AC^T:B \cr } which follow from the Frobenius-Trace equivalence $$A:B = \operatorname{tr}(A^TB)$$ and the cyclic properties of the trace.