# Intuition on proof of Cauchy Schwarz inequality

To prove Cauchy Schwarz inequality for two vectors $x$ and $y$ we take the inner product of $w$ and $w$ where $w=y-kx$ where $k=\frac{(x,y)}{|x|^2}$ ($(x,y)$ is the inner product of $x$ and $y$) and use the fact that $(w,w) \ge0$ . I want to know the intuition behind this selection. I know that if we assume this we will be able to prove the theorem, but the intuition is not clear to me.

-
Is there a "proof without words" for the C.S. inequality in 2-dimensions? – Jon Bannon Aug 15 '12 at 15:52

Pick $k$ so as to minimize the distance from $kx$ to $y$, or rather its square, noting that $$\def\Re{\operatorname{Re}} 0\le(y-kx,y-kx)=(x,x)k^2-2\Re((x,y)k)+(y,y)$$ for all $k$. Now pick the $k$ that minimizes the right hand side (in order to get the most out of the inequality), and find that it is the very same $k$ used in the standard proof.

You can determine this $k$ by standard calculus methods, or by completing the square. But first, pick the phase of $k$ to make $(x,y)k$ positive, thus avoiding the difficulty of working with the real part.

-
Regarding my final comment: Do it in a real Hilbert space first to get the intuition. Then return to the problem of phase in the complex case. Or even better: Reduce the complex case to the real case, by noting that any complex Hilbert space is also a real one (take the real part of the inner product, and forget about non-real scalars), and replacing $y$ by $cy$ where $\lvert c\rvert=1$ and $(x,cy)$ is real. – Harald Hanche-Olsen Aug 15 '12 at 13:54
confused. I don't understand why I should try to minimize $|y-kx|$ to prove the inequality – aaaaaa Aug 17 '12 at 14:13
what do you mean by "in order to get the most out of the inequality" ? – aaaaaa Aug 17 '12 at 14:21
I have an inequality saying that something is always nonnegative. By adjusting one or more of the variable [$k$ in this case] in that expression to make the expression the smallest possible, it stands to reason that I gain the most information about the other variable(s) in the expression [$x$ and $y$ in this case, or rather their various inner products]. – Harald Hanche-Olsen Aug 18 '12 at 7:15
And to your question before the one I answered: No, if all you know is that you wish to prove Cauchy–Schwarz, it is not at all obvious that minimizing the given expression leads to that goal. It is something that emerges in fairly short order if you play around a bit, however. It is not uncommon in mathematics that just trying things out suddenly reveals a non-obvious fact with a simple proof. It's one of those things that makes mathematics exciting, but also frustrating at times to beginners. – Harald Hanche-Olsen Aug 18 '12 at 7:22

This is similar to Harald's answer but there is no explicit choice of $k$. For simplicity let us take the real case. Fix vectors $x$ and $y$, and consider the quadratic polynomial $$p(k)=(x+ky,x+ky)=(x,x)+2k(x,y)+k^2(y,y)\geq0.$$ The polynomial must be nonnegative for any value of $k\in\mathbb{R}$, meaning that the equation $p(k)=0$ has at most one solution. This can be expressed in terms of the discriminant as $$D = (x,y)^2-(x,x)(y,y)\leq0,$$ giving the Cauchy-Bunyakowsky-Scwarz inequality.

The moral of the story is that the choice of $k$ is mainly due to the fact that there is a quadratic polynomial behind the proof.

-

My favorite proof is inspired by Axler and uses the Pythagorean theorem (that $\|v+w\|^2 =\|v\|^2+\|w\|^2$ when $(v,w)=0$). It motivates the choice of $k$ as the component of $y$ in an orthogonal decomposition (i.e., $kx$ is the projection of $y$ onto the space spanned by $x$ using the decomposition $\langle x\rangle\oplus \langle x\rangle^\perp$). For simplicity we will assume a real inner product space (a very small tweak makes it work in both cases).

The idea is that we want to show $$\left|\left(\frac{x}{\|x\|},y\right)\right| = |(\hat{x},y)| \leq \|y\|,$$ where I have divided both sides by $\|x\|$ and let $\hat{x}=x/\|x\|$. If we interpret taking an inner product with a unit vector as computing a component, the above says the length of $y$ is at least the component of $y$ in the $x$ direction (quite plausible).

Following the above comments we will prove this statement by decomposing $y$ into two components: one in the direction of $x$ and the other orthogonal to $x$. Let $y=k\hat{x}+(y-k\hat{x})$ where $k = (\hat{x},y)$. We see that $$(\hat{x},y-k\hat{x}) = (\hat{x},y) - (\hat{x},k\hat{x}) = k - k(\hat{x},\hat{x})=0,$$ showing $\hat{x}$ and $y-k\hat{x}$ are orthogonal. This allows us to apply the Pythagorean theorem: $$\|y\|^2 = \|k\hat{x}\|^2+\|y-k\hat{x}\|^2 = |k|^2 + \|y-k\hat{x}\|^2 \geq |k|^2,$$ since norms are non-negative. Taking square roots gives the result.

As a final comment, note that $$k\hat{x} = (\hat{x},y)\hat{x} = \left(\frac{x}{\|x\|},y\right)\frac{x}{\|x\|} = \frac{(x,y)}{\|x\|^2}x$$ matches your formulation.

-
This is an elegant proof. But there is a little conceptual problem here. The meaning of vectors being orthogonal if their inner product is $0$ relies on us defining angles using the inner product. But, to do that we need the Cauchy-Scwarz inequality.... – Ittay Weiss Mar 1 '14 at 11:28
@IttayWeiss No, you only need the "Pythagorean Theorem" as stated above. This follows simply from the properties of the inner product (just take the inner product of v + w with itself and see what happens when (v,w) = 0 ). – Chan-Ho Suh Mar 4 at 16:02
@Chan-HoSuh my comment was about a conceptual problem. There is no problem in the proof (in fact its my favourite proof). All I said was that before you interpret orthogonality in the geometric sense as "inner product is $0$" you need angles. And before you have CS, you don't have angles. Of course, algebraically it all works out, and the intuition is vindicated after the fact. And that's fine. That's all I said. – Ittay Weiss Mar 4 at 23:13

Basically, the geometric intuition behind this proof is that the area of a parallelogram $x\wedge y$ is positive unless $x$ and $y$ are co-linear. In fact, it is precisely $||x||\cdot||y||$ when the vectors are perpendicular.

Otherwise, we can observe that the area of $x\wedge y$ is the same as of $x\wedge(y-kx)$, thus, by choosing $k$ to minimize the length of the second edge (as in the proof), we can make both edges perpendicular. The area of the resulting parallelogram must be non-negative, giving the desired inequality.

-
I understand that if we minimize $|y-kx|$ then area of $x\wedge y$ is the same as of $x\wedge(y-kx)$. But why is the quantity "area" important here ?. I can reach the cauchy schwarz inequality from the fact that inner product of $y-kx$ with itself is >=0 – aaaaaa Aug 17 '12 at 14:18
Because it is an invariant, explaining the very reason why you are trying to minimise $||y-kx||$. – Norbert Pintye Aug 19 '12 at 1:20