We need to solve the following least square problem \begin{align} Y=X\theta+W, \end{align} with quadratic constraint \begin{align} \theta^TA\theta = 0, \end{align} where the complex matrix $$X \in \mathbb{C}^{m\times n}$$, the complex vector $$Y \in \mathbb{C}^{m\times 1}$$, and the real matrix $$A \in \mathbb{R}^{n\times n}$$ are given, and $$W \in \mathbb{C}^{m\times1}$$ is the complex additive white Gaussian noise (zero mean and unit variance). $$\theta$$ is the parameter to estimate, so basically we need to solve
$$$$\begin{array}{rrclcl} \displaystyle \min_{\theta} & {\|Y-X\theta\|^2} \\ \textrm{s.t.} &\theta^TA\theta& = & 0 \\ \end{array}$$$$ in a closed form.
Indeed we try to have some intuition by starting from working on a special case: $$m=n=4$$, $$A =\begin{pmatrix}0_{2\times2}& a \\ 0_{2\times 2} & 0_{2\times 2}\end{pmatrix}$$, where $$a = \begin{pmatrix}0& 1 \\ -1 & 0\end{pmatrix}$$. By employing Lagrange multipliers we have \begin{align} -2X^HY+2X^HX\theta+\lambda \begin{pmatrix}0_{2\times2}& a \\ -a & 0_{2\times 2}\end{pmatrix}\theta &= 0\nonumber \\ \theta^T\begin{pmatrix}0_{2\times2}& a \\ 0_{2\times 2} & 0_{2\times 2}\end{pmatrix}\theta &= 0 \end{align}