# proving $(cov(X,Y))^2 \leq var(X)var(Y)$ with cauchy schwarz

I've been trying to use the cauchy-schwarz inequality to prove that $(cov(X,Y))^2 \leq var(x)var(Y)$. The cauchy-schwarz inequality can be expressed as follows: if $u$ and $v$ are vectors in an inner product space then $<u,v>^2 \leq (\parallel u\parallel)^2 (\parallel v\parallel)^2$. How do you define the vectors and the inner product so to prove the result in the first sentence. I've seen something like $(cov(X,Y))^2 = < (x-E(X)),(y - E(Y)) > \leq (\parallel (x-E(X) \parallel )^2(\parallel (y-E(Y) \parallel )^2 = var(X)var(Y)$ but how is $< (x-E(X)),(y - E(Y)) >$ expressed as a sum? Also how can you let $(x-E(X))$ and $(y - E(Y))$ be vectors?

Any insight would be great.

-

The inner product space you might have in mind is $L^2(\Omega, \mathscr A, P)$, the space of all square integrable random variables $X \colon \Omega \to \mathbb R$. Square integrable means that we have $\|X\|^2 := E(|X|^2) = \int_{\Omega} |X|^2\, dP < \infty$ The inner product is given by $\left<X,Y\right> := \int_\Omega XY\, dP.$ So it isn't a sum, but an integral and the vectors are the elements of the vector space $L^2(\Omega)$, that is, random variables.

Cauchy-Schwarz reads $\int_\Omega XY\, dP = \left<X,Y\right> \le \|X\|^2 \|Y\|^2 = \int_\Omega X^2\, dP \cdot \int_\Omega Y^2\, dP$ and applying this to $X-E(X)$ and $Y - E(Y)$ gives the desired inequality.

In cases where $P$ is discrete, that is there is a countable subset $\Omega'$ of $\Omega$ with $P(\Omega') = 1$, the integrals are sums, in this case we have $\left<X,Y\right> = \int_\Omega XY\, dP = \sum_{\omega \in \Omega'} X(\omega)Y(\omega)$ and $\left\|X\right\|^2 = \sum_{\omega \in \Omega'} X(\omega)^2.$

$\def\cov{\operatorname{cov}}$ Addendum after comment: As you write, for discrete random variables we have $\cov(X,Y) = \sum_{\omega\in \Omega} \bigl( X(\omega) - E(X)\bigr)\bigl(Y(\omega) - E(Y)\bigr)$ If we want to write this as a sum over the values of $X$ and $Y$, just note that the joint probability mass (or as you write density) function $p_{X,Y}$ is given by $p_{X,Y}(x,y) = P({\omega \mid X(\omega) = x, Y(\omega) = y})$ Grouping these terms in the above sum, we obtain $\cov(X,Y) = \sum_{x\in X[\Omega]}\sum_{y\in Y[\Omega]} \bigl(x- E(X)\bigr)\bigl(y-E(Y)\bigr)p_{X,Y}(x,y).$

-
Sorry about late comment. Thanks for your answer. For discrete RVs you have $cov(X,Y) = /sum_{\omega \in \Omega'} (X(\omega)-E(X(\omega)))(Y(\omega)-E(Y(\omega)))$. Is this equivalent to – Sam Forbes Oct 10 '12 at 17:45
IGNORE ABOVE! Sorry about late comment. Thanks for your answer. For discrete RVs you have $cov(X,Y) = \sum_{\omega \in \Omega'} (X(\omega)-E(X(\omega)))(Y(\omega)-E(Y(\omega)))$. Is this equivalent to $\sum_{x \in R}\sum_{y \in M} (x-E(X)(y-E(Y) p_{X,Y}(x,y)$ where the RV $X$ takes values $x$ in a set $R$ and the RV $Y$ takes values $y$ in a set $M$ and $p_{X,Y}(x,y)$ is the joint prob. density function of $X$ and $Y$? – Sam Forbes Oct 10 '12 at 17:53
Sorry for late comment. You haven't used prob. density functions in your notation for variance and covariance but I think these are equivalent. I think you would need a double sum if there was a different no. of $y_{i}$s say with $x_{j}$s. – Sam Forbes Oct 10 '12 at 17:59