Find $\sup_{a a^* + b b^* = 1} | a^2x + aby |$ for fixed $x,y \in \mathbb{C}$, $|x| = 1$, $y \ne 0$. While trying to come up with an answer to this question without using the theorem used in the existing answer, whose proof is non-trivial, I tried to find
$$ \tag{1}
\sup_{a a^* + b b^* = 1} | a^2x + aby |
$$
for some fixed complex numbers $x,  y \in \mathbb{C}$
In the question the conditions $|x| = 1$ and $y \ne 0$ are added but I am interested if there's also a more general solution without those constrains.
The task of the question is to prove that $$(1)> |x|.$$
My progress
Aiming to use Lagrange multipliers, I expanded the term $a^2x + aby$ in terms of $n_1 := \Re(n)$ and $n_2 := \Im(n)$ for $n \in \{a,b,x,y\}$ to then use that
$$\tag{2}
|z|
= \sqrt{z_1^2 + z_2^2}
$$ for all $z \in \mathbb{C}$. I ended up with
$$
\Re(a^2x + aby)
= a_1 b_1 y_1 + a_1 b_2 y_2 + a_2 b_1 y_2 - a_2 b_1 y_2 + a_2^1 x_1 + a_2^2 x_1
$$
and
$$
\Im(a^2 x + aby)
= a_1 b_1 y_2 - a_1 b_2 y_1 + a_2 b_1 y_1 + a_2 b_2 y_2 + a_1^2 x_2 + a_2^2 x_2.
$$
Plugging this into (2) yields the square root of more than 40 terms. Is there a nice way to factor this or to approach this?
If this were only a real problem, we'd have
$$
\sup_{a^2 + b^2 = 1} ax^2 + a b y
= \sup_{|a| \le 1} a + a \sqrt{1 - a^2} y
$$
Setting the derivative w.r.t to $a$ equal to zero we obtain
$$
0 = \sqrt{1 - a^2} y + 1 - \frac{a^2 y}{\sqrt{1 - a^2}},
$$
giving
$$
a^2 = \frac{\pm \sqrt{8 y^2 + 1} - 1}{8y^2} + \frac{1}{2}.
$$
Plugging the positive root in gives
$$
a + a \sqrt{1 - a^2} y
= \left(\sqrt{\frac{\pm \sqrt{8 y^2 + 1} - 1}{8y^2} + \frac{1}{2}}\right)\left(1 + \sqrt{\frac{1}{2} - \frac{\pm \sqrt{8 y^2 + 1} - 1}{8y^2}} \cdot y\right),
$$
which looks approximately linear in $y$ when plotted with WolframAlpha.
Unfortunately, $\Re\left(\sqrt{\frac{1}{2} - \frac{\pm \sqrt{8 y^2 + 1} - 1}{8y^2}}\right) = 0$ for all $y \in \mathbb{R}$.
 A: Write
$$a=e^{i\alpha}\cos(\psi),\qquad b=e^{i\beta}\sin(\psi),\qquad 0\leq\psi\leq{\pi\over2}\ .$$
Then $a\bar a+b\bar b=1$. We have to find the sup of the quantity
$$Q:=\left|e^{2i\alpha}\cos^2(\psi) \>x+e^{i(\alpha+\beta)}\cos(\psi)\sin(\psi)\>y\right|\ ,$$
whereby $\alpha$, $\beta$, and $\psi$ are variables. As $|x|=1$ and $y\ne0$   we can write
$$x=e^{i\xi},\qquad y=|y|e^{i\eta}\ ,$$
so that
$$
Q
=\left|\cos^2(\psi) \>e^{i\xi}+\cos(\psi)\sin(\psi)\>|y|\>e^{i(\beta-\alpha+\eta)}\right|\ .
$$
Since $\xi$ and $\eta$ are given, and $0\leq\psi\leq{\pi\over2}$, this $Q$ is maximal when $\beta-\alpha=\xi-\eta$ mod $2\pi$. It follows that
$$\max_{\alpha, \beta,\psi}Q
=\max_{0\leq\psi\leq\pi/2}\left(\cos^2(\psi)+\cos(\psi)\sin(\psi)\>|y|\right)\ .$$
In this way we are left with a single variable ($\psi$) extremal problem:
We have
$$
\frac{\textrm{d}}{\textrm{d} \phi} (\cos^2(\psi)+\cos(\psi)\sin(\psi) \cdot |y|)
= | y | \cos(2\psi) - \sin(2\psi)
\overset{!}{=} 0
\implies x = \frac{1}{2} \tan^{-1}(|y|)
$$
(and $\frac{\textrm{d}^2}{\textrm{d} \psi^2} \big(\cos^2(\psi)+\cos(\psi)\sin(\psi)|y|\big) \big|_{x = \frac{1}{2} \tan^{-1}(|y|)} < 0$).
Plugging in gives
$$
\max_{\alpha, \beta, \phi} Q
= \frac{\sqrt{|y|^2 + 1} + 1}{2},
$$
which is always greater as 1 and only equal to one if $y = 0$, which we have excluded.
Plotted as a function of its real $(=x)$ and imaginary part $(=y)$, it looks like this:

A: Here is the solution to  Christian Blatter's maximization problem, writing $p$ for $\psi$:
We have
$$f(p)
=\cos^2(p)+\cos(p)\sin(p)y
=\frac12(\cos(2p)+1)+\frac12\sin(2p) y
$$
so maximizing this is the same as
$g(q)
:=\cos(q)+\sin(q)y
$
with $p = q/2$.
Let
$\tan(r) = y$,
so
$\sin(r)
=\frac{y}{\sqrt{1+y^2}}
$
and
$\cos(r)
=\frac{1}{\sqrt{1+y^2}}
$.
Then
$\begin{array}\\
g(q)
&=\cos(q)+\sin(q)y\\
&=\sqrt{1+y^2}(\frac1{\sqrt{1+y^2}}\cos(q)+\frac{y}{\sqrt{1+y^2}}\sin(q))\\
&=\sqrt{1+y^2}(\cos(r)\cos(q)+\sin(r)\sin(q))\\
&=\sqrt{1+y^2}\cos(r-q)\\
\end{array}
$
This is maximized when
$r-q=0$
so that
$q = r
=\arctan(y)
$
or
$2p = r
$
or
$p = \frac12\arctan(y)
$.
