$x \perp y$ if and only if $\Vert x + \alpha y \Vert \ge \Vert x \Vert$ for all scalars $\alpha$ Here's Prob. 8 in the Problems after Sec. 3.2 in Introductory Functional Analysis With Applications by Erwine Kreyszig: 

Show that in an inner product space, $x \perp y$ if and only if $\Vert x + \alpha y \Vert \ge \Vert x \Vert$ for all scalars $\alpha$. 

If $x \perp y$, then $\langle x, y \rangle = 0$; so for any scalar $\alpha$, we have 
$$
\begin{align*}
\Vert x + \alpha y \Vert^2 &= \langle x + \alpha y, x + \alpha y \rangle \\
&= \Vert x \Vert^2 + 2 \Re \bar{\alpha} \langle x, y \rangle + \vert \alpha \vert^2 \ \Vert y \Vert^2 \\ 
&= \Vert x \Vert^2 +  \vert \alpha \vert^2 \ \Vert y \Vert^2 \\ 
&\ge \Vert x \Vert^2. 
\end{align*}
$$
So 
$$
\Vert x + \alpha y \Vert \geq \Vert x \Vert.
$$
Am I right?
Now how to prove the converse? It is my guess that we will have to put in a particular value for $\alpha$. 
 A: $\boldsymbol{\langle x,y\rangle=0\implies\|x+\alpha y\|^2\ge\|x\|^2}$
$$
\begin{align}
\|x+\alpha y\|^2
&=\|x\|^2+2\mathrm{Re}\left(\langle x,\alpha y\rangle\right)+|\alpha|^2\|y\|^2\\
&=\|x\|^2+2\mathrm{Re}\left(\overline{\alpha}\langle x,y\rangle\right)+|\alpha|^2\|y\|^2\\
&=\|x\|^2+|\alpha|^2\|y\|^2\\
&\ge\|x\|^2
\end{align}
$$

$\boldsymbol{\|x+\alpha y\|^2\ge\|x\|^2\implies\langle x,y\rangle=0}$
Let $\alpha=t\langle x,y\rangle$ for $t\in\mathbb{R}$, so that $\langle x,\alpha y\rangle=t|\langle x,y\rangle|^2\in\mathbb{R}$.
Suppose $\langle x,y\rangle\ne0$. For $-\frac2{\|y\|^2}\lt t\lt0$, we have
$$
\begin{align}
\|x+\alpha y\|^2
&=\|x\|^2+2\mathrm{Re}\left(\langle x,\alpha y\rangle\right)+|\alpha|^2\|y\|^2\\
&=\|x\|^2+2t|\langle x,y\rangle|^2+t^2|\langle x,y\rangle|^2\|y\|^2\\
&=\|x\|^2+t|\langle x,y\rangle|^2\left(2+t\|y\|^2\right)\\
&\lt\|x\|^2
\end{align}
$$
Therefore, $\langle x,y\rangle=0$.
A: If $(x,y) \ne 0$ then $\|y\|\ne 0$, and the following is an orthogonal decomposition:
$$
                      x = \left[x-\frac{(x,y)}{(y,y)}y\right]+\frac{(x,y)}{(y,y)}y.
$$
Hence,
$$
              \|x\|^{2} = \left\|x-\frac{(x,y)}{(y,y)}y\right\|^{2}+\frac{|(x,y)|^{2}}{\|y\|^{2}} > \left\|x-\frac{(x,y)}{(y,y)}y\right\|^{2}.
$$
A: Assumig
\begin{align*}
 \Vert x \Vert^2 + 2 \Re \bar{\alpha} \langle x, y \rangle + \vert \alpha \vert^2 \ \Vert y \Vert^2 \ge
\Vert x \Vert^2,
\end{align*}
we get
\begin{align*}
2 \Re \bar{\alpha} \langle x, y \rangle + \vert \alpha \vert^2 \ \Vert y \Vert^2 \ge 0
\end{align*}
and
\begin{align}
\Re \bar{\alpha} \langle x, y \rangle \ge -\frac{1}{2} \vert \alpha \vert^2 \ \Vert y \Vert^2.
\end{align}
Now take $\alpha = 2\varepsilon / \Vert y \Vert^2 > 0$, and you can cancel $\alpha$ from both sides to get
\begin{align*}
\Re \langle x, y \rangle \ge -\frac{1}{2} \alpha  \Vert y \Vert^2 =-\varepsilon.
\end{align*}
Since this holds for all $\varepsilon$, we get $\Re \langle x, y \rangle \ge 0$.
Similarly, taking $\alpha = - 2\varepsilon / \Vert y \Vert^2$, we get $\Re \langle x, y \rangle \le 0$, and hence $\Re \langle x, y \rangle = 0$.
In the same manner, taking $\alpha = \pm i 2\varepsilon / \Vert y \Vert^2$, one can show that $\Im \langle x, y \rangle =0$.
A slightly shorter argument along the same lines, is as follows. Assume $\langle x, y \rangle = |\langle x, y \rangle| e^{i\theta}$. Taking $\alpha :=e^{i\theta} 2\varepsilon / \Vert y \Vert^2$, gives
\begin{align*}
\Re \bar{\alpha} \langle x, y \rangle  = \Re \big(e^{-i\theta} |\alpha| |\langle x, y \rangle| e^{i\theta}\big) = |\alpha| |\langle x, y \rangle|
\end{align*}
from which we get $|\langle x, y \rangle| \le 0$
and similarly taking $\alpha :=e^{i(\theta-\pi)} 2\varepsilon / \Vert y \Vert^2$, gives $|\langle x, y \rangle| \ge 0$.
A: For the converse, if $\langle x,y \rangle \ne 0$ then in line 2 of your argument you can choose $\alpha \in \mathbb{R}$ to make the r.h.s. smaller than $||x||^2$, since for any constants $a \ne 0,b$ (in this case $a=2\langle x,y\rangle$ and $b=||y||$) you can choose $\alpha$ such that $a\alpha + b\alpha^2 < 0$.  (the parabola $y=ax+bx^2$ intersects but isn't tangent to the x-axis.)
