11
$\begingroup$

Can someone define independence of two random variables with this "product rule", or are there any counterexamples?

$\endgroup$
5
  • 3
    $\begingroup$ Sure there are, try X standard normal and Y=UX with U symmetric Bernoulli. $\endgroup$
    – Did
    Oct 29, 2015 at 19:13
  • $\begingroup$ @Did, what do you mean by symmetric Bernoulli? $P(U=0) = P(U=1)$? $\endgroup$ Oct 29, 2015 at 19:16
  • $\begingroup$ P(U=1)=P(U=-1)=1/2 $\endgroup$
    – Did
    Oct 29, 2015 at 19:29
  • $\begingroup$ @Did Is that a Bernoulli variable? I don't think so. $\endgroup$ Oct 29, 2015 at 19:32
  • 2
    $\begingroup$ @CarlosMendoza Many people call Bernoulli what you probably call a two-points distribution. $\endgroup$
    – Did
    Oct 29, 2015 at 19:33

1 Answer 1

12
$\begingroup$

Independence implies uncorrelation

We say that two random variables are uncorrelated if $E[XY] = E[X]E[Y]$.

If $X$ and $Y$ are independent and discrete (this can also be extended to continuous random variables), then

$$p_{X,Y}(x,y) = p_X(x)p_Y(y)\qquad \forall (x,y)$$

Using this in the definition of $E[XY]$ and doing some rearrangements of terms, we have

\begin{align} E[XY] &= \sum_x\sum_y xyp_{X,Y}(x,y)\\ &= \sum_x\sum_y xyp_X(x)p_Y(y)\\ &= \sum_x\sum_y [xp_X(x)][yp_Y(y)]\\ &= \sum_xxp_X(x)\sum_yyp_Y(y)\\ &= E[X]E[Y] \end{align}

Therefore, if $X$ and $Y$ are independent, then they are uncorrelated.

$\\$

Uncorrelation does not imply independence (in general)

In general$^1$, it is false to say that if $X$ and $Y$ are uncorrelated, then they are independent. Let's see a counterexample. Let $X$ and $Y$ be random variables taking values in $\{(0,1), (1,0), (-1,0), (0,-1)\}$ with equal probability $0.25$. Then, $XY = 0$ with probability $1$, and therefore $E[XY] = 0$.

By other hand,

$\displaystyle p_X(x) = \left\{ \begin{array}{ll} \frac{1}{4} & x=-1\\ \frac{1}{2} & x=0\\ \frac{1}{4} & x=1\\ \end{array} \right. $ $\hspace{1cm}$ and $\hspace{1cm}$ $p_Y(y) = \left\{ \begin{array}{ll} \frac{1}{4} & y=-1\\ \frac{1}{2} & y=0\\ \frac{1}{4} & y=1\\ \end{array} \right. $

which implies that $E[X]=E[Y]=0$. Then, $E[XY] = E[X]E[Y]$ and the random variables are uncorrelated.

However,

$$p_X(1)p_Y(1) = \frac{1}{4}\frac{1}{4} \neq p_{X,Y}(1,1) = 0$$

which shows that $X$ and $Y$ are not independent.

We can conclude then that independence is a sufficient but not a necessary condition for uncorrelation.


$^1$ The only two exceptions I am aware of are the jointly Gaussian variables and two Bernoulli random variables...any other?

$\endgroup$
6
  • $\begingroup$ So you group terms into 2 sums because y's terms are not dependent on x's sum and vice versa please? $\endgroup$
    – Avv
    Apr 6, 2022 at 22:33
  • 1
    $\begingroup$ @Avv. I edited my answer to add two additional steps and hopefully make it clearer. In the third line, you move the factor $xp_X(x)$ out of the sum $\sum_y$ because it doesn't depend on $y$, only on $x$. $\endgroup$ Apr 7, 2022 at 13:40
  • $\begingroup$ I upvoted the answer. It's very clear now. Thank you. $\endgroup$
    – Avv
    Apr 7, 2022 at 18:57
  • $\begingroup$ One more thing please. Why if $g(y) = E[X|Y=y]$ then $E[E[X|y=y]]$ is not expectation of a number but expectation of a random variable? Should not $E[X|y=y]$ be treated as a number not a random variable please so we have $E[E[X|y=y]] = E[X|Y=y]$? $\endgroup$
    – Avv
    Apr 7, 2022 at 19:47
  • $\begingroup$ @Avv, I am not sure how your question relates to the original question and my answer. Please elaborate. Your question seems to be a new question, and you should probably ask it in a new post, but I will try to answer in the next comment.... $\endgroup$ Apr 8, 2022 at 17:34

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .