Derivative of conjugate transpose of matrix Building off of my previous question, I am trying to derive the normal equations for the least squares problem:
$$
\min_W \|WX - Y\|_2 \\
W \in \mathbb{C}^{N \times N} \quad X, Y \in \mathbb{C}^{N \times M}
$$
The intuitive way of viewing this problem is that I am trying to predict a vector $y$ (of length $N$) from a corresponding $x$ vector using a matrix $W$, and to estimate $W$ I have multiple ($M$, to be precise) realizations of $x$ and $y$ packed into matrices $X$ and $Y$.
I am trying to define this in terms of a least-squares problem and derive the normal equations myself, but I'm running into issues in taking the derivative.  To spell it out explicitly, I can re-state the above equation as:
$$
\min_W (WX - Y)^H (WX - Y) \\
= \min_W (X^H W^H - Y^H) (WX - Y) \\
= \min_W (X^H W^H W X - Y^H W X - X^H W^H Y + Y^H Y)
$$
Now typically I would take the derivative with respect to $W$, set it equal to zero, and solve for $W$.  However, my matrix calculus is rusty and everything I know is basically summed up on this webpage, where it explicitly states:

Note that the Hermitian transpose is not used because complex conjugates are not analytic.

Now, because of Michael C. Grant's answer I feel there must be some way of doing this, but I am at a loss as to how. Thank you all in advance!
 A: In the first version of my answer I overlooked the fact that you defined the $L_2$ error in a way that is not suitable for matrices. For vectors it would have been OK, but if your complex error is $E=WX-Y$, then $E^HE$ is not the error that you want to minimize, because $E^HE$ is a matrix (if $E$ were a vector, then $E^HE$ would be a scalar and you could minimize it). The appropriate definition of the (scalar) error measure is given by the Frobenius norm
$$\epsilon=\|E\|_F^2=\|WX - Y\|_F^2=\\
=\text{trace}\left ( E^HE\right)=\text{trace}\left ( (WX-Y)^H(WX-Y)\right)=\\
=\text{trace}\left ( X^HW^HWX - X^HW^HY - Y^HWX +Y^HY\right)$$
Now we can apply the trick of taking the conjugate complex derivative (reference) w.r.t. $W^H$:
$$\frac{\partial\epsilon}{\partial W^H} = WXX^H - YX^H$$
Setting the derivative to zero gives the solution
$$W=YX^H(XX^H)^{-1}$$
assuming $(XX^H)^{-1}$ exists.
EDIT: just an additional simple example to show how the trick with the conjugate derivative works when solving for an unknown complex vector (instead of a matrix). In this case we get a well-known result:
Assume we have an overdetermined system of complex linear equations
$$Ax=b$$
where $A$ is $m\times n$, $m>n$, $x$ is $n\times 1$, and $b$ is $m\times 1$. Minimizing the squared error
$$\epsilon=(Ax-b)^H(Ax-b)=x^HA^HAx - x^HA^Hb-b^HAx+b^Hb$$
by taking the derivative w.r.t. $x^H$ and equating it with zero gives
$$A^HAx-A^Hb=0$$
which leads to the well-known Moore-Penrose pseudoinverse:
$$x=(A^HA)^{-1}A^Hb$$
EDIT 2: Let's have a look at two little examples showing that it is correct to regard a complex variable $z=z_R+iz_I$ as constant when taking the derivative w.r.t. $z^*$. The conjugate derivative is defined as (ref.)
$$\frac{\partial f}{\partial z^*}=\frac{1}{2}\left [\frac{\partial f}{\partial z_R} +i\frac{\partial f}{\partial z_I} \right]$$
For $f=z$ we get
$$\frac{\partial f}{\partial z^*}=
\frac{1}{2}\left [\frac{\partial (z_R+iz_I)}{\partial z_R} +i\frac{\partial (z_R+iz_I)}{\partial z_I} \right]=\frac{1}{2}(1+i^2)=0$$
For $f=zz^*$ we have
$$\frac{\partial f}{\partial z^*}=
\frac{1}{2}\left [\frac{\partial (z_R^2+z_I^2)}{\partial z_R} +i\frac{\partial (z_R^2+z_I^2)}{\partial z_I} \right]=\frac{1}{2}(2z_R+2iz_I)=z$$
