Gradient of scalar function defined using the $\mbox{diag}$ operator Let $f : \mathbb R^n \to \mathbb R$ be defined by
$$f(x) := \left( c - M \mbox{diag} (x) x \right)^\top \left( c - M \mbox{diag} (x) x \right)$$
where $k \times n$ matrix $M$ and $k$-vector $c$ are given. What is the derivative of $f$ with respect to $x$?
If I wanted to to find the $x$ that makes $\nabla f (x) = 0$, what would it be? Please note that $M$ is not square, and therefore, its inverse doesn't exist. 
 A: For typing convenience, define the variables
$$\eqalign{
X &= {\rm Diag}(x) \\
y &= (MXx-c) \;&=\; M(x\odot x)-c \\
w &= M^Ty \\
}$$
where $\odot$ is the elementwise/Hadamard product.
Write the function in terms of these variables. Then find its differential and gradient.
$$\eqalign{
f &= y:y \\
df &= 2y:dy \\
 &= 2y:M(2x\odot dx) \\
 &= 4M^Ty:(x\odot dx) \\
 &= (4x\odot w):dx \\
\frac{\partial f}{\partial x}
 &= (4x\odot w) \\
 &= 4XM^Ty \\
 &= 4XM^T(MXx-c) \\
}$$
In some of the steps above, a colon is used to denote the trace/Frobenius product, i.e.
$$\eqalign{
A:B &= {\rm Tr}(A^TB)
}$$
The properties of the trace allow terms such products to be rearranged in many different ways.
$$\eqalign{
A:B &= B:A \;=\; B^T:A^T \\
A:BC &= B^TA:C \;=\; AC^T:B \\
}$$
Update
Set the gradient to zero and solve for $x$.
$$\eqalign{
M^Ty &= 0 \\
M^TM(x\odot x) &= M^Tc \\
x &= \Big(M^+c + (I-M^+M)\,a\Big)^{1/2} \\
}$$
where $M^+$ is the pseudoinverse and the square root function is applied element-wise.
The $a$-vector is arbitrary and can be used to eliminate negative elements before taking the square root.
A second (trivial) solution is $\,x=0$.
