22
$\begingroup$

The Cauchy-Schwarz inequality tells us that for two vectors $u$ and $v$ in an inner product space, $$\lvert (u,v)\rvert \leq \lVert u\rVert \lVert v \rVert$$
with the equality holding iff one vector is a constant multiplier of the other.

Prove the analogue of the Cauchy-Schwarz inequality for random variables:

$$\lvert E[XY]\rvert \leq \sqrt{E[X^2]} \sqrt{E[Y^2]}$$

(Hint: Use the fact that $E[ (\alpha X + Y)^2 ] \geq 0$ for all real (constants) $\alpha$)

$\endgroup$
2
17
$\begingroup$

Look at the following two expectations:

$$E[(aX+bY)^2]=a^2E[X^2] + b^2E[Y^2] + 2abE[XY] \ge 0$$ $$E[(aX-bY)^2]=a^2E[X^2] + b^2E[Y^2] - 2abE[XY] \ge 0$$

Now let $a^2=E[Y^2]$ and $b^2=E[X^2]$. This gives

$$ 2abE[XY] \ge -2a^2b^2$$ $$ 2abE[XY] \le 2a^2b^2$$

Dividing by $2ab$ results in

$$ -\sqrt{E[X^2]} \sqrt{E[Y^2]} \le E[XY] \le \sqrt{E[X^2]} \sqrt{E[Y^2]}$$ which is equivalent to

$$ |E[XY]| \le \sqrt{E[X^2]} \sqrt{E[Y^2]}$$

$\endgroup$
3
$\begingroup$

There is some debate about whether or not the mapping $(X, Y) \mapsto E[XY]$ defines an inner product. If it does, then your result follows from the standard Cauchy-Schwarz inequality in general inner product spaces.

But, even if it's correct, this approach is rather a cop-out, and could easily be regarded as circular reasoning.

To avoid the circular reasoning, just adapt any proof of the Cauchy-Schwarz inequality. A few of them are given on this wikipedia page.

Here is one such proof: As your hint says, we know that $E[(kX+Y)^2] \ge 0$, for any real $k$. Expanding this, we get:

$$k^2E[X^2] + 2kE[XY] + E[Y^2] \ge 0$$

Let's regard this as a quadratic in $k$. It is non-negative for all $k$, so it has at most one real root, so its discriminant (the $b^2 - 4ac$ thing) must be $\le 0$. In other words $$ (2E[XY])^2 - 4E[X^2]E[Y^2] \le 0$$ and the result follows immediately.

Note that I didn't prove the statement about when equality occurs. In fact, I don't think this statement is true, as WimC's comment pointed out.

$\endgroup$
3
  • 1
    $\begingroup$ Although this is a nice way to look at it, this does not always define an inner product. It need not be defined on all pairs of random variables and it can be positive semi-definite (have a non-trivial kernel). $\endgroup$
    – WimC
    Apr 21 '13 at 12:07
  • $\begingroup$ @WimC -- Hi. Yes, my assumed definitions of "random variable" and "expectation" were a bit naive. I think the proof that I added is valid, even though some of the initial comments are only correct in finite-dimensional cases. $\endgroup$
    – bubba
    Apr 21 '13 at 12:25
  • $\begingroup$ This page is interesting: en.wikipedia.org/wiki/Inner_product_space. It says that expectation defines an inner product, and then immediately gives an example showing that it does not ?? $\endgroup$
    – bubba
    Apr 21 '13 at 12:37
0
$\begingroup$

When $\mathbb EY^2\not=0$, one has $$ \mathbb EX^2\mathbb EY^2-(\mathbb E XY)^2=\frac{\mathbb E\bigl((\mathbb EY^2) X - (\mathbb E XY) Y\bigr)^2}{\mathbb EY^2}. $$

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.