Complex ($\mathbb C$) least squares derivation

Question

I know how to derive the least squares in the real domain.

If a tall matrix $A$ and a column vector $b$ are real, then the solution of the least squares problem $Ax = b$ can be derived as:

$$\begin{align} \{E(x)\}^2 &= ||Ax - b||^2 \\ &= (Ax-b)^T (Ax-b) \\ &= x^T A^T Ax - x^T A^T b - b^T Ax + b^T b \\ &= x^T A^T Ax - 2 x^T A^T b + b^T b \qquad (\because (Ax)^T b = b^T (Ax)) \end{align}$$

Differentiating both sides with respect to $x$,

$$\begin{align} \frac{d \{E(x)\}^2}{dx} &= 2A^T Ax - 2 A^T b \end{align}$$

Setting $\frac{d \{E(x)\}^2}{dx} = 0$ to find when we get the minimum $E(x)$,

$$ 2A^T Ax - 2 A^T b = 0 \\ A^T Ax = A^T b \\ x = (A^T A)^{-1} A^T b $$

Now, we turn to the complex-valued situation.
Assume $A$ and $b$ are complex,

$$\begin{align} \{E(x)\}^2 &= ||Ax - b||^2 \\ &= (Ax-b)^H (Ax-b) \\ &= x^H A^H Ax - x^H A^H b - b^H Ax + b^H b \\ \end{align}$$

Here, I have some problems.
First, $x^H A^H b \neq b^H Ax$ unless $(Ax)^H b$ is real.
Most of all, I don't know how to differentiate the complex matrices above.

How to proceed the derivation?
There are plenty of derivations in the real domain in Google, but I couldn't find detailed explanation of the general complex case.

Answer 1

Denote the complex conjugate, transpose, and conjugate transpose of the matrix $A$ as $(A^*, A^T, A^H)$ respectively.

Use the Frobenius (:) Inner Product to write the function and take its differential $$\eqalign{ f &= (Ax-b)^*:(Ax-b) \cr\cr df &= (Ax-b)^*:A\,dx \cr &= A^T(Ax-b)^*:dx \cr }$$ Since $df=\Big(\frac{\partial f}{\partial x}:dx\Big),\,$ the gradient must be $$\eqalign{ \frac{\partial f}{\partial x} &= A^T(Ax-b)^* \cr }$$ Set the gradient to zero, take the complex conjugate, and solve for $x$ $$\eqalign{ A^T(Ax)^* &= A^Tb^* \cr A^HAx &= A^Hb \cr x &= (A^HA)^{-1}A^Hb \cr &= A^{+}b \cr }$$ Notice that $x$ and $x^*$ are treated as independent variables for the purpose of differentiation.