Derivative of quadratic with Hadamard product

Question

I proposed a similar question involving logarithms, but the problem is about scalar.

I am trying to solve the more generalized form:

$$ \min_{\mathbf{x} \in \mathbb{R}^N_+} \left( \sum_i \left( h_i^T(\mathbf{x}\circ\mathbf{x}) - \mathbf{c_1}\log h_i^T(\mathbf{x}\circ\mathbf{x})\right) + r_1\parallel \mathbf{x} - \mathbf{c_2}\parallel_2^2 \right)$$

where $\mathbf{c_1} \in \mathbb{R}^+$, $\mathbf{c_2} \in \mathbb{R}^N_+$, $h_i \in \mathbb{R}^{N}_+$ is each column of a known matrix, $r_1 \in \mathbb{R}^+$. All are constants.

So, for scalar quadratic eqation, we can find the square root via dividing $(1+r_1)$, but when $1$ is a matrix $H$, how to deal with this form?

But I don't know whether directly extension of scalar quadratic equation is right. Can anyone help me? Thanks in advance!

---

Edit: formulate the problem to make it clearly.

Answer 1

Using a convention where uppercase latin letters are matrices, lowercase latin are vectors, and greek letters are scalars, define the following variables. $$\eqalign{ \rho &= r_1,\quad \lambda = c_1,\quad a = c_2,\quad H = [\,h_1\;h_2\;h_3\ldots] \\ w &= H(x\circ x),\quad W = {\rm Diag}(w),\quad X = {\rm Diag}(x) \\ dw &= 2HX\,dx,\quad Q = I-\lambda W^{-1} \\ y &= w - \lambda\log(w) \;\implies\; dy = (I-\lambda W^{-1})\,dw = 2QHX\,dx \\ \phi &= \rho(x-a):(x-a) + {\tt\large 1}:y \\ }$$ where the scalar on the final line is the objective function written using a colon to represent the trace/Frobenius product, i.e. $\;A:B = {\rm Tr}(A^TB)$.

Calculate the differential and gradient of the final scalar. $$\eqalign{ d\phi &= 2\rho(x-a):dx + {\tt\large 1}:dy \\ &= 2\rho(x-a):dx + {\tt\large 1}:2QHX\,dx \\ &= \big(2\rho(x-a) + 2XH^TQ{\tt\large 1}\big):dx \\ \frac{\partial\phi}{\partial x} &= 2\rho(x-a) + 2XH^TQ{\tt\large 1} \\ }$$ Set the gradient to zero, substitute $Q$, and multiply by $\frac{1}{2}X^{-1}$ $$\eqalign{ 0 &= \rho{\tt\large 1} - \rho\bigg(\frac{a}{x}\bigg) + H^T{\tt\large 1} - \lambda H^T\bigg(\frac{{\tt\large 1}}{w}\bigg) \\ \bigg(\frac{\rho a}{x}\bigg) &= \rho{\tt\large 1}+H^T{\tt\large 1}-\lambda H^T\bigg(\frac{{\tt\large 1}}{w}\bigg)\\ x &= \frac{\rho a}{\rho{\tt\large 1} + H^T\Big({\tt\large 1}-\frac{\lambda{\tt\large 1}}{w}\Big)} \\ }$$ This is not an explicit solution because the vector $w$ in the denominator on the RHS is a function of $x$, but it can form the basis of an iterative numerical method.

NB: The notation $\big(\frac{a}{b}\big)$ represents element-wise division of the vector $a$ by the vector $b$.