# proving $(cov(X,Y))^2 \leq var(X)var(Y)$ with cauchy schwarz

I've been trying to use the cauchy-schwarz inequality to prove that $(cov(X,Y))^2 \leq var(x)var(Y)$. The cauchy-schwarz inequality can be expressed as follows: if $u$ and $v$ are vectors in an inner product space then $<u,v>^2 \leq (\parallel u\parallel)^2 (\parallel v\parallel)^2$. How do you define the vectors and the inner product so to prove the result in the first sentence. I've seen something like $(cov(X,Y))^2 = < (x-E(X)),(y - E(Y)) > \leq (\parallel (x-E(X) \parallel )^2(\parallel (y-E(Y) \parallel )^2 = var(X)var(Y)$ but how is $< (x-E(X)),(y - E(Y)) >$ expressed as a sum? Also how can you let $(x-E(X))$ and $(y - E(Y))$ be vectors?

Any insight would be great.

-

The inner product space you might have in mind is $L^2(\Omega, \mathscr A, P)$, the space of all square integrable random variables $X \colon \Omega \to \mathbb R$. Square integrable means that we have $\|X\|^2 := E(|X|^2) = \int_{\Omega} |X|^2\, dP < \infty$ The inner product is given by $\left<X,Y\right> := \int_\Omega XY\, dP.$ So it isn't a sum, but an integral and the vectors are the elements of the vector space $L^2(\Omega)$, that is, random variables.

Cauchy-Schwarz reads $\int_\Omega XY\, dP = \left<X,Y\right> \le \|X\|^2 \|Y\|^2 = \int_\Omega X^2\, dP \cdot \int_\Omega Y^2\, dP$ and applying this to $X-E(X)$ and $Y - E(Y)$ gives the desired inequality.

In cases where $P$ is discrete, that is there is a countable subset $\Omega'$ of $\Omega$ with $P(\Omega') = 1$, the integrals are sums, in this case we have $\left<X,Y\right> = \int_\Omega XY\, dP = \sum_{\omega \in \Omega'} X(\omega)Y(\omega)$ and $\left\|X\right\|^2 = \sum_{\omega \in \Omega'} X(\omega)^2.$

$\def\cov{\operatorname{cov}}$ Addendum after comment: As you write, for discrete random variables we have $\cov(X,Y) = \sum_{\omega\in \Omega} \bigl( X(\omega) - E(X)\bigr)\bigl(Y(\omega) - E(Y)\bigr)$ If we want to write this as a sum over the values of $X$ and $Y$, just note that the joint probability mass (or as you write density) function $p_{X,Y}$ is given by $p_{X,Y}(x,y) = P({\omega \mid X(\omega) = x, Y(\omega) = y})$ Grouping these terms in the above sum, we obtain $\cov(X,Y) = \sum_{x\in X[\Omega]}\sum_{y\in Y[\Omega]} \bigl(x- E(X)\bigr)\bigl(y-E(Y)\bigr)p_{X,Y}(x,y).$

-
 Sorry about late comment. Thanks for your answer. For discrete RVs you have $cov(X,Y) = /sum_{\omega \in \Omega'} (X(\omega)-E(X(\omega)))(Y(\omega)-E(Y(\omega)))$. Is this equivalent to – Sam Forbes Oct 10 '12 at 17:45 IGNORE ABOVE! Sorry about late comment. Thanks for your answer. For discrete RVs you have $cov(X,Y) = \sum_{\omega \in \Omega'} (X(\omega)-E(X(\omega)))(Y(\omega)-E(Y(\omega)))$. Is this equivalent to $\sum_{x \in R}\sum_{y \in M} (x-E(X)(y-E(Y) p_{X,Y}(x,y)$ where the RV $X$ takes values $x$ in a set $R$ and the RV $Y$ takes values $y$ in a set $M$ and $p_{X,Y}(x,y)$ is the joint prob. density function of $X$ and $Y$? – Sam Forbes Oct 10 '12 at 17:53 @SamForbes Yes, added something about this in my answer above. – martini Oct 10 '12 at 21:48 great, thanks again – Sam Forbes Oct 11 '12 at 9:47

The cauchy-schwarz inequality is for discrete case, (∑ (a_i)^2 ) (∑ (b_i)^2 ) ≥ ( ∑ (a_i)(b_i) ) ^2 , where the sum is over the suffix i from 1 to n ( n is the number of values under consideration). So, denoting the mean of the values of x by u and the mean of the values of y by t , we get by writing a_i = x_i - u and b_i = y_i - t , ( ∑ ( x_i - u)^2 )( ∑ ( y_i - t)^2 ) ≥ ( ∑ ( x_i - u)(y_i - t) )^2 , i.e. , by definition, { n.var(x) } { n.var(y)} ≥ {n.cov(x,y)}^2 , so that var(x) var(y) ≥ {cov(x,y)}^2 ; this is the proof of the inequality in case of Bivariate Analysis you may also extend it for probability distribution.

-
 Sorry for late comment. You haven't used prob. density functions in your notation for variance and covariance but I think these are equivalent. I think you would need a double sum if there was a different no. of $y_{i}$s say with $x_{j}$s. – Sam Forbes Oct 10 '12 at 17:59