Cauchy-Schwarz inequality proof (but not the usual one)

Question

Before you downvote/vote-to-close, I am not asking for a proof of: $$\sum^n_{i=1}a_ib_i\le\sqrt{\sum_{i=1}^na_i^2}\sqrt{\sum^n_{i=1}b_i^2 } $$ Which is what EVERY link I've found assumes is the inequality (for a proof of that see: http://www.maths.kisogo.com/index.php?title=Cauchy-Schwarz_inequality&oldid=692 )

---

I am reading a book that claims the Cauchy-Schwarz inequality is actually: $$\vert\langle x,y\rangle\vert\le\Vert x\Vert\Vert y\Vert$$ where $\Vert x\Vert :=\sqrt{\langle x,x\rangle}$

with the additional claim: equality holds $\iff\ x,y$ are linearly dependent

I cannot find a proof of this claim (only proofs for the dot product inner product). My question is: what is the simplest way to prove this (I define simplest to mean "using as few definitions outside of the inner product itself as possible" - so that's not including Gramm-Schmitt orthonormalisation, for example.)

I've just found some possible answers in the links on the right (annoyingly) but fortunately I have a second, albeit softer, question:

What inequalities that are common for say $\mathbb{R}^n$ are actually based on the inner product?

(In future I will check this site first, I was targeting lecture notes and such in my search)

Addendum:
I want to use this on not-finite dimensional vector spaces. I've found one proof that relies on finite-ness.

Answer 1

If I remember correctly, Cauchy proved the inequality that you don't want proven. The generalisation that you do want proven was proven later by Schwarz. So, I think that the former inequality ought to be called the Cauchy inequality. Oh well.

The proof of the (general) Cauchy-Schwarz inequality essentially comes down to orthogonally decomposing $x$ into a component parallel to $y$ and a component perpendicular to $y$, and using the fact that the perpendicular component has a non-negative square norm. That is, we start from the statement: $$\left\|x - \frac{\langle x, y \rangle}{\langle y, y \rangle} y\right\|^2 \ge 0.$$ The rest is just expanding the inner product. We have, \begin{align*} 0 &\le \left\langle x - \frac{\langle x, y \rangle}{\langle y, y \rangle} y, x - \frac{\langle x, y \rangle}{\langle y, y \rangle} y\right\rangle \\ &= \langle x, x \rangle - \frac{\overline{\langle x, y \rangle}}{\langle y, y \rangle} \langle x, y \rangle - \frac{\langle x, y \rangle}{\langle y, y \rangle} \langle y, x \rangle + \frac{\langle x, y \rangle}{\langle y, y \rangle} \frac{\overline{\langle x, y \rangle}}{\langle y, y \rangle} \langle y, y\rangle \\ &= \langle x, x \rangle - 2\frac{|\langle x, y \rangle|^2}{\langle y, y \rangle} + \frac{|\langle x, y \rangle|^2}{\langle y, y \rangle} \\ &= \langle x, x \rangle - \frac{|\langle x, y \rangle|^2}{\langle y, y \rangle}. \end{align*} Rearranging yields the inequality.

Answer 2

Since $\langle tx+y,tx+y\rangle\ge0$ for all $t$, $\;\;\langle x,x\rangle t^2+2\langle x,y\rangle t+\langle y, y\rangle\ge0$ for all $t$, so

$(2\langle x,y\rangle)^2-4\langle x,x\rangle\langle y,y\rangle\le0\implies \langle x,y\rangle^2\le\langle x,x\rangle\langle y,y\rangle=\lvert\lvert x\rvert\rvert^2 \lvert\lvert y\rvert\rvert^2$.

Answer 3

Hint: $\|x-y\|^2=\langle x-y,x-y\rangle=\langle x,x\rangle-2\langle x,y\rangle+\langle y,y\rangle=\|x\|^2-2\langle x,y\rangle+\|y\|^2$.

Answer 4

A proof of the inequality mimicks the proof used for $\mathbf R^n$: for $\lambda\in \mathbf R$ consider the inner product: $$\langle\lambda x-y,\lambda x-y\rangle=\lambda^2\langle x,x\rangle-2\lambda\langle x,y\rangle+\langle\mkern1mu y,y\rangle $$ This is a quadratic polynomial in $\lambda$, which is nonnegative for every $\lambda$, hence its reduced discriminant is $\le 0$. Now $\langle x,x\rangle=\lVert x\rVert^2$ and similarly for $y$, so: $$\Delta'=\langle x,y\rangle^2- \langle x,x\rangle\langle\mkern1mu y,y\rangle\le0\iff\langle x,y\rangle^2\le\lVert x\rVert^2\lVert y\rVert^2\iff\lvert\langle x,y\rangle\rvert\le \lVert x\rVert \lVert y\rVert$$ This proves the inequality.

Furthermore, if the inequality is an equality, i.e. if $\Delta'=0$, the quadratic polynomial has a double root, which is equal to $\lambda=\dfrac{\langle x,y\rangle}{\langle x,x\rangle}$; in other words $$\Bigl\langle\frac{\langle x,y\rangle}{\langle x,x\rangle}x-y,\langle\frac{\langle x,y\rangle}{\langle x,x\rangle}x-y\Bigr\rangle=0$$

and as an inner product is a definite positive bilinear form, this implies $$y=\frac{\langle x,y\rangle}{\langle x,x\rangle}x. $$ The converse (if $x$ and $y$ are linearly dependent, the inequality is an equality) is trivial.

Answer 5

We have

$$||x-\beta y||^2=\langle x-\beta y,x-\beta y\rangle \ge 0 \tag 1$$

for any $\beta \in \mathscr{R}$ and any two vectors $x$ and $y$. We find the minimum of the square of the norm in $(1)$ by setting the derivative with respect to $\beta$ to zero. Therefore,

$$\begin{align} \frac{d||x-\beta y||^2}{d\beta}&=2\beta ||y||^2-2\langle x,y \rangle\\\\ &=0\\\\ &\implies \beta =\frac{\langle x,y \rangle}{||y||^2} \end{align}$$

whereupon we immediately see that the minimum of the norm squared is

$$\min_{\beta}\left(||x-\beta y||^2\right)=||x||^2-\frac{\langle x,y \rangle^2}{||y||^2} \tag 2$$

Inasmuch as the right-hand side of $(2)$ must be non-negative, we have the desired result

$$\bbox[5px,border:2px solid #C0A000]{\langle x,y \rangle \le ||x||\,||y||}$$

with equality if $x=\beta y$.

Answer 6

The inequality means that the determinant $$\left | \begin{array}{cc} \langle x, x \rangle & \langle x, y \rangle \\ \langle y, x \rangle & \langle y, y \rangle \end{array} \right | \ge 0$$

with equality if and only if the vectors $x$, $y$ are linearly dependent (proportional in this case).

In general, for several vectors $x_1$, $\ldots$, $x_m$ we have $$\left | \begin{array}{ccc} \langle x_1, x_1 \rangle & \ldots& \langle x_1, x_m \rangle \\ \ldots & & \ldots\\ \langle x_m, x_1 \rangle & \ldots& \langle x_m, x_m \rangle \end{array} \right | \ge 0$$ with equality if and only if the vectors $x_1$, $\ldots$, $x_m$ are linearly dependent.

Answer 7

Here is the proof (over the field of complex numbers) of the additional claim that you mentioned. Cauchy-Schwarz inequality is also proven.

Problem: Show that for an inner product space $V$ over $\mathbb{C}$ $$|\langle x,y \rangle|=\|x\| \cdot \|y\| \quad \text{if and only if} \quad x=\lambda y, \, \text{where} \, \lambda \in \mathbb{C}.$$

Solution: Notice that substituting $\lambda y$ for $x$ gives us \begin{equation} \begin{split} |\langle x,y \rangle | & = |\langle \lambda y, y \rangle |\\ & = |\lambda \cdot \langle y,y \rangle |\\ & = | \lambda | \cdot \|y\| \cdot \|y\|\\ & = \|\lambda y\| \cdot \|y\|\\ & = \|x\| \cdot \|y\|\\ \end{split} \end{equation}

Conversely, assume that $|\langle x,y \rangle|=\|x\| \cdot \|y\|$ for some $x,y \in V$. Then notice that by the property of an inner product, $\forall \, \lambda \in \mathbb{C}$ $$\langle x-\lambda y, x-\lambda y \rangle \geq 0$$ Also, notice that the left side can be rewritten as \begin{equation} \begin{split} \langle x-\lambda y, x-\lambda y \rangle & = \langle x,x \rangle - \lambda \langle y,x \rangle - \bar{\lambda} \langle x,y \rangle + \lambda \bar{\lambda} \langle y,y \rangle\\ & = \langle x,x \rangle - \lambda \overline{\langle {x,y} \rangle} - \bar{\lambda} \langle x,y \rangle + \lambda \bar{\lambda} \langle y,y \rangle\\ & = \langle x,x \rangle - 2\text{Re} \left(\lambda \overline{\langle {x,y} \rangle}\right) +|\lambda|^2 \langle y,y \rangle\\ \end{split} \end{equation}

We know that $\langle x,y \rangle = re^{i\theta}$, where $r=|\langle x,y \rangle|$ and $\theta$ is some angle. Then $\lambda \overline{\langle {x,y} \rangle}=\lambda r e^{-i\theta}$. Since this holds for all $\lambda$, Let $\lambda=te^{i \theta}$, where $t$ is any real number. Then $$\lambda \overline{\langle {x,y} \rangle}=te^{i \theta} re^{-i \theta}=tr \in \mathbb{R}$$ Thus, for this $\lambda$ (where $t$ can be any real number), the above equation and equality can be combined to give us \begin{equation} \begin{split} \langle x-\lambda y, x-\lambda y \rangle & = \langle x,x \rangle - \lambda \langle y,x \rangle - \bar{\lambda} \langle x,y \rangle + \lambda \bar{\lambda} \langle y,y \rangle\\ & = \langle x,x \rangle - \lambda \overline{\langle {x,y} \rangle} - \bar{\lambda} \langle x,y \rangle + \lambda \bar{\lambda} \langle y,y \rangle\\ & = \langle x,x \rangle - 2\text{Re} \left(\lambda \overline{\langle {x,y} \rangle}\right) +|\lambda|^2 \langle y,y \rangle\\ & = \langle x,x \rangle -2tr +t^2 \langle y,y \rangle\\ & \geq 0 \end{split} \end{equation} Since this expression is positive for all $t$ and since the above inequality is a quadratic function of $t$, we know that its discriminant has to be smaller or equal $0$. Thus, $$4r^2-4 \langle x,x \rangle \cdot \langle y,y \rangle \leq 0$$ Plugging back in $|\langle x,y \rangle|$ for $r$ gives us $$|\langle x,y \rangle|^2\leq \langle x,x \rangle \cdot \langle y,y \rangle$$ which is equivalent to $$|\langle x,y \rangle| \leq \|x\| \cdot \|y\|$$ which is known as the Cauchy-Schwarz inequality. However, we know that $|\langle x,y \rangle|=\|x\| \cdot \|y\|$. Thus, it has to be that $$4r^2-4 \langle x,x \rangle \cdot \langle y,y \rangle = 0$$ But discriminant being $0$ implies that there exists $t$ such that $$\langle x,x \rangle -2tr +|\lambda|^2 \langle y,y \rangle=0$$ Choose that $t$. Then for $\lambda=te^{i \theta}$, where $t$ is the specific value that we just chose, we get \begin{equation} \begin{split} \langle x-\lambda y, x-\lambda y \rangle & = \langle x,x \rangle - \lambda \langle y,x \rangle - \bar{\lambda} \langle x,y \rangle + \lambda \bar{\lambda} \langle y,y \rangle\\ & = \langle x,x \rangle - \lambda \overline{\langle {x,y} \rangle} - \bar{\lambda} \langle x,y \rangle + \lambda \bar{\lambda} \langle y,y \rangle\\ & = \langle x,x \rangle - 2\text{Re} \left(\lambda \overline{\langle {x,y} \rangle}\right) +|\lambda|^2 \langle y,y \rangle\\ & = \langle x,x \rangle -2tr +t^2 \langle y,y \rangle\\ & = 0 \end{split} \end{equation} But $\langle x-\lambda y, x-\lambda y \rangle=0$ implies that $x-\lambda y=0$, which in turns gives us the desired result, namely $x=\lambda y$.

Answer 8

It think the usual formulations and proofs of the Cauchy-Schwarz inequality do include statement and proof of the additional claim that equality holds if and only if the vectors are linearly dependent. For the most obvious online example consider the proof from Wikipedia: the before-last sentence says "Moreover, if the relation '$\geq$'... is actually an equality, then... establishes a relation of linear dependence" (this assumes without mentioning the easy converse that in case of linear dependence, the inequality will be an equality; this is just a direct application of basic properties of the scalar product). Note that the mentioned proof does not assume finite dimensionality.