5
$\begingroup$

Before you downvote/vote-to-close, I am not asking for a proof of: $$\sum^n_{i=1}a_ib_i\le\sqrt{\sum_{i=1}^na_i^2}\sqrt{\sum^n_{i=1}b_i^2 } $$ Which is what EVERY link I've found assumes is the inequality (for a proof of that see: http://www.maths.kisogo.com/index.php?title=Cauchy-Schwarz_inequality&oldid=692 )


I am reading a book that claims the Cauchy-Schwarz inequality is actually: $$\vert\langle x,y\rangle\vert\le\Vert x\Vert\Vert y\Vert$$ where $\Vert x\Vert :=\sqrt{\langle x,x\rangle}$

with the additional claim: equality holds $\iff\ x,y$ are linearly dependent

I cannot find a proof of this claim (only proofs for the dot product inner product). My question is: what is the simplest way to prove this (I define simplest to mean "using as few definitions outside of the inner product itself as possible" - so that's not including Gramm-Schmitt orthonormalisation, for example.)

I've just found some possible answers in the links on the right (annoyingly) but fortunately I have a second, albeit softer, question:

What inequalities that are common for say $\mathbb{R}^n$ are actually based on the inner product?

(In future I will check this site first, I was targeting lecture notes and such in my search)

Addendum:
I want to use this on not-finite dimensional vector spaces. I've found one proof that relies on finite-ness.

$\endgroup$
18
$\begingroup$

If I remember correctly, Cauchy proved the inequality that you don't want proven. The generalisation that you do want proven was proven later by Schwarz. So, I think that the former inequality ought to be called the Cauchy inequality. Oh well.

The proof of the (general) Cauchy-Schwarz inequality essentially comes down to orthogonally decomposing $x$ into a component parallel to $y$ and a component perpendicular to $y$, and using the fact that the perpendicular component has a non-negative square norm. That is, we start from the statement: $$\left\|x - \frac{\langle x, y \rangle}{\langle y, y \rangle} y\right\|^2 \ge 0.$$ The rest is just expanding the inner product. We have, \begin{align*} 0 &\le \left\langle x - \frac{\langle x, y \rangle}{\langle y, y \rangle} y, x - \frac{\langle x, y \rangle}{\langle y, y \rangle} y\right\rangle \\ &= \langle x, x \rangle - \frac{\overline{\langle x, y \rangle}}{\langle y, y \rangle} \langle x, y \rangle - \frac{\langle x, y \rangle}{\langle y, y \rangle} \langle y, x \rangle + \frac{\langle x, y \rangle}{\langle y, y \rangle} \frac{\overline{\langle x, y \rangle}}{\langle y, y \rangle} \langle y, y\rangle \\ &= \langle x, x \rangle - 2\frac{|\langle x, y \rangle|^2}{\langle y, y \rangle} + \frac{|\langle x, y \rangle|^2}{\langle y, y \rangle} \\ &= \langle x, x \rangle - \frac{|\langle x, y \rangle|^2}{\langle y, y \rangle}. \end{align*} Rearranging yields the inequality.

$\endgroup$
  • $\begingroup$ Thank you for answering my question. You've already got my +1, you'll get the tick when I've confirmed the answer. But really thanks! $\endgroup$ – Alec Teal Jul 12 '15 at 0:07
  • 1
    $\begingroup$ You're welcome. :-) And I really don't want to sound patronising or anything like that, but please give Chuks and the others a break. They only want to help, I promise. $\endgroup$ – Theo Bendit Jul 12 '15 at 0:16
12
$\begingroup$

Since $\langle tx+y,tx+y\rangle\ge0$ for all $t$, $\;\;\langle x,x\rangle t^2+2\langle x,y\rangle t+\langle y, y\rangle\ge0$ for all $t$, so

$(2\langle x,y\rangle)^2-4\langle x,x\rangle\langle y,y\rangle\le0\implies \langle x,y\rangle^2\le\langle x,x\rangle\langle y,y\rangle=\lvert\lvert x\rvert\rvert^2 \lvert\lvert y\rvert\rvert^2$.

$\endgroup$
  • 3
    $\begingroup$ And if $x$ and $y$ are dependent, say $ax + by = 0$, then wlog we have $b\ne 0$, so $(a/b) x + y = 0$, and setting $t = a/b$, we get an equality in this derivation; on the other hand, if they're independent, then the first $\ge$ can be replaced by $>$, so the "equality if and only if dependent" comes from a closer reading of this proof. $\endgroup$ – John Hughes Jul 11 '15 at 23:38
  • $\begingroup$ @JohnHughes Thanks for your comment; I was trying to add the justification of this claim to my answer, but you already did. $\endgroup$ – user84413 Jul 11 '15 at 23:42
  • $\begingroup$ It's old question but I didn't understand that point why $\Delta \le 0$? Namely why the quadratic equation has at most one real root? $\endgroup$ – user519955 Jun 11 '18 at 10:33
  • $\begingroup$ @user519955 This is using the fact that there are two real roots when $\Delta>0$. $\endgroup$ – user84413 Jul 7 '18 at 20:52
5
$\begingroup$

Hint: $\|x-y\|^2=\langle x-y,x-y\rangle=\langle x,x\rangle-2\langle x,y\rangle+\langle y,y\rangle=\|x\|^2-2\langle x,y\rangle+\|y\|^2$.

$\endgroup$
  • $\begingroup$ I've removed all the comments on this answer, as they were all off-topic. If you want to discuss the general philosophies for or against certain styles of answer, go to meta or chat. $\endgroup$ – davidlowryduda Jul 12 '15 at 20:36
5
$\begingroup$

A proof of the inequality mimicks the proof used for $\mathbf R^n$: for $\lambda\in \mathbf R$ consider the inner product: $$\langle\lambda x-y,\lambda x-y\rangle=\lambda^2\langle x,x\rangle-2\lambda\langle x,y\rangle+\langle\mkern1mu y,y\rangle $$ This is a quadratic polynomial in $\lambda$, which is nonnegative for every $\lambda$, hence its reduced discriminant is $\le 0$. Now $\langle x,x\rangle=\lVert x\rVert^2$ and similarly for $y$, so: $$\Delta'=\langle x,y\rangle^2- \langle x,x\rangle\langle\mkern1mu y,y\rangle\le0\iff\langle x,y\rangle^2\le\lVert x\rVert^2\lVert y\rVert^2\iff\lvert\langle x,y\rangle\rvert\le \lVert x\rVert \lVert y\rVert$$ This proves the inequality.

Furthermore, if the inequality is an equality, i.e. if $\Delta'=0$, the quadratic polynomial has a double root, which is equal to $\lambda=\dfrac{\langle x,y\rangle}{\langle x,x\rangle}$; in other words $$\Bigl\langle\frac{\langle x,y\rangle}{\langle x,x\rangle}x-y,\langle\frac{\langle x,y\rangle}{\langle x,x\rangle}x-y\Bigr\rangle=0$$

and as an inner product is a definite positive bilinear form, this implies $$y=\frac{\langle x,y\rangle}{\langle x,x\rangle}x. $$ The converse (if $x$ and $y$ are linearly dependent, the inequality is an equality) is trivial.

$\endgroup$
3
$\begingroup$

We have

$$||x-\beta y||^2=\langle x-\beta y,x-\beta y\rangle \ge 0 \tag 1$$

for any $\beta \in \mathscr{R}$ and any two vectors $x$ and $y$. We find the minimum of the square of the norm in $(1)$ by setting the derivative with respect to $\beta$ to zero. Therefore,

$$\begin{align} \frac{d||x-\beta y||^2}{d\beta}&=2\beta ||y||^2-2\langle x,y \rangle\\\\ &=0\\\\ &\implies \beta =\frac{\langle x,y \rangle}{||y||^2} \end{align}$$

whereupon we immediately see that the minimum of the norm squared is

$$\min_{\beta}\left(||x-\beta y||^2\right)=||x||^2-\frac{\langle x,y \rangle^2}{||y||^2} \tag 2$$

Inasmuch as the right-hand side of $(2)$ must be non-negative, we have the desired result

$$\bbox[5px,border:2px solid #C0A000]{\langle x,y \rangle \le ||x||\,||y||}$$

with equality if $x=\beta y$.

$\endgroup$
  • 1
    $\begingroup$ Normally, you're expected to receive a message on the editing. However, the editing was on the last 2-3 lines of the proof. $\endgroup$ – Chuks Jul 12 '15 at 15:04
  • $\begingroup$ @Chuks Yes, I did see it. Just a couple of typos. Thanks much! $\endgroup$ – Mark Viola Jul 12 '15 at 15:21
2
$\begingroup$

The inequality means that the determinant $$\left | \begin{array}{cc} \langle x, x \rangle & \langle x, y \rangle \\ \langle y, x \rangle & \langle y, y \rangle \end{array} \right | \ge 0$$

with equality if and only if the vectors $x$, $y$ are linearly dependent (proportional in this case).

In general, for several vectors $x_1$, $\ldots$, $x_m$ we have $$\left | \begin{array}{ccc} \langle x_1, x_1 \rangle & \ldots& \langle x_1, x_m \rangle \\ \ldots & & \ldots\\ \langle x_m, x_1 \rangle & \ldots& \langle x_m, x_m \rangle \end{array} \right | \ge 0$$ with equality if and only if the vectors $x_1$, $\ldots$, $x_m$ are linearly dependent.

$\endgroup$
1
$\begingroup$

It think the usual formulations and proofs of the Cauchy-Schwarz inequality do include statement and proof of the additional claim that equality holds if and only if the vectors are linearly dependent. For the most obvious online example consider the proof from Wikipedia: the before-last sentence says "Moreover, if the relation '$\geq$'... is actually an equality, then... establishes a relation of linear dependence" (this assumes without mentioning the easy converse that in case of linear dependence, the inequality will be an equality; this is just a direct application of basic properties of the scalar product). Note that the mentioned proof does not assume finite dimensionality.

$\endgroup$
0
$\begingroup$

Here is the proof (over the field of complex numbers) of the additional claim that you mentioned. Cauchy-Schwarz inequality is also proven. You can find the original proof here:

http://www.polishedproofs.com/equality-in-the-cauchy-schwarz-inequality/

Show that for an inner product space $V$ over $\mathbb{C}$ $$|\langle x,y \rangle|=\|x\| \cdot \|y\| \quad \text{if and only if} \quad x=\lambda y, \, \text{where} \, \lambda \in \mathbb{C}$$

Notice that substituting $\lambda y$ for $x$ gives us \begin{equation} \begin{split} |\langle x,y \rangle | & = |\langle \lambda y, y \rangle |\\ & = |\lambda \cdot \langle y,y \rangle |\\ & = | \lambda | \cdot \|y\| \cdot \|y\|\\ & = \|\lambda y\| \cdot \|y\|\\ & = \|x\| \cdot \|y\|\\ \end{split} \end{equation}

Conversely, assume that $|\langle x,y \rangle|=\|x\| \cdot \|y\|$ for some $x,y \in V$. Then notice that by the property of an inner product, $\forall \, \lambda \in \mathbb{C}$ $$\langle x-\lambda y, x-\lambda y \rangle \geq 0$$ Also, notice that the left side can be rewritten as \begin{equation} \begin{split} \langle x-\lambda y, x-\lambda y \rangle & = \langle x,x \rangle - \lambda \langle y,x \rangle - \bar{\lambda} \langle x,y \rangle + \lambda \bar{\lambda} \langle y,y \rangle\\ & = \langle x,x \rangle - \lambda \overline{\langle {x,y} \rangle} - \bar{\lambda} \langle x,y \rangle + \lambda \bar{\lambda} \langle y,y \rangle\\ & = \langle x,x \rangle - 2\text{Re} \left(\lambda \overline{\langle {x,y} \rangle}\right) +|\lambda|^2 \langle y,y \rangle\\ \end{split} \end{equation}

We know that $\langle x,y \rangle = re^{i\theta}$, where $r=|\langle x,y \rangle|$ and $\theta$ is some angle. Then $\lambda \overline{\langle {x,y} \rangle}=\lambda r e^{-i\theta}$. Since this holds for all $\lambda$, Let $\lambda=te^{i \theta}$, where $t$ is any real number. Then $$\lambda \overline{\langle {x,y} \rangle}=te^{i \theta} e^{-i \theta}=tr \in \mathbb{R}$$ Thus, for this $\lambda$ (where $t$ can be any real number), the above equation and equality can be combined to give us \begin{equation} \begin{split} \langle x-\lambda y, x-\lambda y \rangle & = \langle x,x \rangle - \lambda \langle y,x \rangle - \bar{\lambda} \langle x,y \rangle + \lambda \bar{\lambda} \langle y,y \rangle\\ & = \langle x,x \rangle - \lambda \overline{\langle {x,y} \rangle} - \bar{\lambda} \langle x,y \rangle + \lambda \bar{\lambda} \langle y,y \rangle\\ & = \langle x,x \rangle - 2\text{Re} \left(\lambda \overline{\langle {x,y} \rangle}\right) +|\lambda|^2 \langle y,y \rangle\\ & = \langle x,x \rangle -2tr +t^2 \langle y,y \rangle\\ & \geq 0 \end{split} \end{equation} Since this expression is positive for all $t$ and since the above inequality is a quadratic function of $t$, we know that its discriminant has to be smaller or equal $0$. Thus, $$4r^2-4 \langle x,x \rangle \cdot \langle y,y \rangle \leq 0$$ Plugging back in $|\langle x,y \rangle|$ for $r$ gives us $$|\langle x,y \rangle|^2\leq \langle x,x \rangle \cdot \langle y,y \rangle$$ which is equivalent to $$|\langle x,y \rangle| \leq \|x\| \cdot \|y\|$$ which is known as the Cauchy-Schwarz inequality. However, we know that $|\langle x,y \rangle|=\|x\| \cdot \|y\|$. Thus, it has to be that $$4r^2-4 \langle x,x \rangle \cdot \langle y,y \rangle = 0$$ But discriminant being $0$ implies that there exists $t$ such that $$\langle x,x \rangle -2tr +\| \lambda \|^2 \langle y,y \rangle=0$$ Choose that $t$. Then for $\lambda=te^{i \theta}$, where $t$ is the specific value that we just chose, we get \begin{equation} \begin{split} \langle x-\lambda y, x-\lambda y \rangle & = \langle x,x \rangle - \lambda \langle y,x \rangle - \bar{\lambda} \langle x,y \rangle + \lambda \bar{\lambda} \langle y,y \rangle\\ & = \langle x,x \rangle - \lambda \overline{\langle {x,y} \rangle} - \bar{\lambda} \langle x,y \rangle + \lambda \bar{\lambda} \langle y,y \rangle\\ & = \langle x,x \rangle - 2\text{Re} \left(\lambda \overline{\langle {x,y} \rangle}\right) +|\lambda|^2 \langle y,y \rangle\\ & = \langle x,x \rangle -2tr +t^2 \langle y,y \rangle\\ & = 0 \end{split} \end{equation} But $\langle x-\lambda y, x-\lambda y \rangle=0$ implies that $x-\lambda y=0$, which in turns gives us the desired result, namely $x=\lambda y$.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.