Gradient of $L=\|Y - X\|_F^2 + \sum_i^I u_i ( \| A_i X_i \|_2^2 - \alpha_i ) + \sum_k^K v_k ( {\rm tr}( (X^* B_k X^T)) - \beta_k)$ w.r.t. $X$

I am not sure how to start with computing the gradient $$\frac{\partial L}{\partial X}$$ of the following function:

\begin{align} L = \| Y - X \|_F^2 + \sum_i^I u_i \left( \| A_i X_i \|_2^2 - \alpha_i \right) + \sum_k^K v_k \left( {\rm tr} \left( (X^* B_k X^T) \right) - \beta_k \right) \end{align} where

• $$Y \in \mathbb{C}^{m \times n}$$, i.e., complex-valued matrix,
• $$X \in \mathbb{C}^{m \times n}$$,
• $$X^*$$ denotes complex conjugate only, $$X^T$$ corresponds to transpose of the matrix $$X$$,
• $$X_i \in \mathbb{C}^{m \times 1}$$ denotes $$i$$th column vector of $$X$$ matrix,
• $$A_i \in \mathbb{C}^{p \times m}$$ is given,
• $$B_k \in \mathbb{C}^{n \times n}$$ is given,
• $$u_i, \alpha_i, v_k, \beta_k \in \mathbb{R}$$ are given.

I thought if I could write the second part in matrix form, then probably I can move forward and try to compute the gradient. But I fail to do that. Your suggestions and help will be highly appreciated.

• What is the difficult part? Is the term with $X_i$ or the second sum? – Duns Jun 14 '19 at 12:50
• Yes, the second sum including the $X_i$ vectors of a matrix $X$ – user550103 Jun 14 '19 at 15:53

The first piece \eqalign{K &= \|X-Y\|_F^2 = (X-Y)^*:(X-Y) \cr dK &= (X-Y)^*:dX} The second piece. \eqalign{M &=\|AXe\|_F^2 =(AXe)^*:(AXe)\cr dM &=(AXe)^*:A\,dX\,e =A^TA^*X^*ee^T:dX} And the third. \eqalign{ N &={\rm Tr}(X^*BX^T) =X^*B:X\cr dN &=X^*B:dX} Now put it all together, with various summation coefficients (omit the constant terms). \eqalign{ L &= K + \sum_iu_iM_i + \sum_kv_kN_k \cr dL &= \Big((X-Y)^* + \sum_iu_iA_i^TA_i^*X^*e_ie_i^T + \sum_kv_kX^*B_k\Big):dX \cr \frac{\partial L}{\partial X} &= X^*-Y^* + \sum_iu_iA_i^TA_i^*X^*e_ie_i^T + \sum_kv_kX^*B_k \cr\cr } In the above derivation, a colon denotes the double-dot product $$A:B = {\rm Tr}(A^TB)$$ Also $$X^*$$ is treated as being independent of $$X$$ under differentiation, also known as Wirtinger derivatives or the $${\mathbb {CR}-}$$calculus.
And $$e_i$$ denotes the $$i^{th}$$ standard basis vector for $${\mathbb R}^{n}$$
• Thank you greg. You might be correct. But I am so sorry and I am still confused with the second piece, particularly. $X_i$ is an $i$th column vector of matrix $X$. I intend to take the gradient with respect to $X$ matrix. So, how can $dM_i = A_i^T A_i^* X^* : dX$? Should it not be $dM_i = A_i^T A_i^* X_i^* : dX_i$? If yes, then how to arrange $dX_i$ as $dX$. – user550103 Jun 14 '19 at 15:58
• Sorry, I didn't see the subscript on $X_i$. The answer has been updated. – greg Jun 14 '19 at 16:57