Take the 2-minute tour ×
Mathematics Stack Exchange is a question and answer site for people studying math at any level and professionals in related fields. It's 100% free, no registration required.

I've been trying to use the cauchy-schwarz inequality to prove that $(cov(X,Y))^2 \leq var(x)var(Y)$. The cauchy-schwarz inequality can be expressed as follows: if $u$ and $v$ are vectors in an inner product space then $<u,v>^2 \leq (\parallel u\parallel)^2 (\parallel v\parallel)^2$. How do you define the vectors and the inner product so to prove the result in the first sentence. I've seen something like $(cov(X,Y))^2 = < (x-E(X)),(y - E(Y)) > \leq (\parallel (x-E(X) \parallel )^2(\parallel (y-E(Y) \parallel )^2 = var(X)var(Y) $ but how is $< (x-E(X)),(y - E(Y)) >$ expressed as a sum? Also how can you let $(x-E(X))$ and $(y - E(Y))$ be vectors?

Any insight would be great.

share|improve this question

2 Answers 2

The inner product space you might have in mind is $L^2(\Omega, \mathscr A, P)$, the space of all square integrable random variables $X \colon \Omega \to \mathbb R$. Square integrable means that we have \[ \|X\|^2 := E(|X|^2) = \int_{\Omega} |X|^2\, dP < \infty \] The inner product is given by \[ \left<X,Y\right> := \int_\Omega XY\, dP. \] So it isn't a sum, but an integral and the vectors are the elements of the vector space $L^2(\Omega)$, that is, random variables.

Cauchy-Schwarz reads \[ \int_\Omega XY\, dP = \left<X,Y\right> \le \|X\|^2 \|Y\|^2 = \int_\Omega X^2\, dP \cdot \int_\Omega Y^2\, dP \] and applying this to $X-E(X)$ and $Y - E(Y)$ gives the desired inequality.


In cases where $P$ is discrete, that is there is a countable subset $\Omega'$ of $\Omega$ with $P(\Omega') = 1$, the integrals are sums, in this case we have \[ \left<X,Y\right> = \int_\Omega XY\, dP = \sum_{\omega \in \Omega'} X(\omega)Y(\omega) \] and \[ \left\|X\right\|^2 = \sum_{\omega \in \Omega'} X(\omega)^2. \]


$\def\cov{\operatorname{cov}}$ Addendum after comment: As you write, for discrete random variables we have \[ \cov(X,Y) = \sum_{\omega\in \Omega} \bigl( X(\omega) - E(X)\bigr)\bigl(Y(\omega) - E(Y)\bigr) \] If we want to write this as a sum over the values of $X$ and $Y$, just note that the joint probability mass (or as you write density) function $p_{X,Y}$ is given by \[ p_{X,Y}(x,y) = P({\omega \mid X(\omega) = x, Y(\omega) = y}) \] Grouping these terms in the above sum, we obtain \[ \cov(X,Y) = \sum_{x\in X[\Omega]}\sum_{y\in Y[\Omega]} \bigl(x- E(X)\bigr)\bigl(y-E(Y)\bigr)p_{X,Y}(x,y). \]

share|improve this answer
    
Sorry about late comment. Thanks for your answer. For discrete RVs you have $cov(X,Y) = /sum_{\omega \in \Omega'} (X(\omega)-E(X(\omega)))(Y(\omega)-E(Y(\omega)))$. Is this equivalent to –  Sam Forbes Oct 10 '12 at 17:45
    
IGNORE ABOVE! Sorry about late comment. Thanks for your answer. For discrete RVs you have $cov(X,Y) = \sum_{\omega \in \Omega'} (X(\omega)-E(X(\omega)))(Y(\omega)-E(Y(\omega)))$. Is this equivalent to $\sum_{x \in R}\sum_{y \in M} (x-E(X)(y-E(Y) p_{X,Y}(x,y)$ where the RV $X$ takes values $x$ in a set $R$ and the RV $Y$ takes values $y$ in a set $M$ and $p_{X,Y}(x,y)$ is the joint prob. density function of $X$ and $Y$? –  Sam Forbes Oct 10 '12 at 17:53
    
@SamForbes Yes, added something about this in my answer above. –  martini Oct 10 '12 at 21:48
    
great, thanks again –  Sam Forbes Oct 11 '12 at 9:47

The cauchy-schwarz inequality is for discrete case, (∑ (a_i)^2 ) (∑ (b_i)^2 ) ≥ ( ∑ (a_i)(b_i) ) ^2 , where the sum is over the suffix i from 1 to n ( n is the number of values under consideration). So, denoting the mean of the values of x by u and the mean of the values of y by t , we get by writing a_i = x_i - u and b_i = y_i - t , ( ∑ ( x_i - u)^2 )( ∑ ( y_i - t)^2 ) ≥ ( ∑ ( x_i - u)(y_i - t) )^2 , i.e. , by definition, { n.var(x) } { n.var(y)} ≥ {n.cov(x,y)}^2 , so that var(x) var(y) ≥ {cov(x,y)}^2 ; this is the proof of the inequality in case of Bivariate Analysis you may also extend it for probability distribution.

share|improve this answer
    
Sorry for late comment. You haven't used prob. density functions in your notation for variance and covariance but I think these are equivalent. I think you would need a double sum if there was a different no. of $y_{i}$s say with $x_{j}$s. –  Sam Forbes Oct 10 '12 at 17:59

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.