62
$\begingroup$

It's a really simple question. However I didn't see it in books and I tried to find the answer on the web but failed.

If I have two independent random variables, $X_1$ and $X_2$, then I define two other random variables $Y_1$ and $Y_2$, where $Y_1$ = $f_1(X_1)$ and $Y_2$ = $f_2(X_2)$.

Intuitively, $Y_1$ and $Y_2$ should be independent, and I can't find a counter example, but I am not sure. Could anyone tell me whether they are independent? Does it depend on some properties of $f_1$ and $f_2$?

Thank you.

$\endgroup$
0

5 Answers 5

42
$\begingroup$

Yes, they are independent.

If you are studying rigorous probability course with sigma-algebras then you may prove it by noticing that the sigma-algebra generated by $f_{1}(X_{1})$ is smaller than the sigma-algebra generated by $X_{1}$, where $f_{1}$ is borel-measurable function.

If you are studying an introductory course - then just remark that this theorem is consistent with our intuition: if $X_{1}$ does not contain info about $X_{2}$ then $f_{1}(X_{1})$ does not contain info about $f_{2}(X_{2})$.

$\endgroup$
9
  • 2
    $\begingroup$ Thank you very much. I am studying an introductory course and it seems to be a little hard for me to get things too serious. $\endgroup$
    – LLS
    Nov 5, 2010 at 11:57
  • $\begingroup$ What about counterexample, say $f(x)=x^2$? $\endgroup$ Dec 14, 2017 at 11:04
  • 3
    $\begingroup$ @Mr.Newman It is not a counterexample. If $X_1$ and $X_2$ are independent then $X_1^2$ and $X_2^2$ are also independent. If you mean that $X$ may be independent of $X^2$ then you are right, but the question is about two random variables $X_1$ and $X_2$ and their functions :) $\endgroup$
    – Roah
    Dec 15, 2017 at 12:24
  • $\begingroup$ @Roah No no, I mean $X^2$ and $Y^2$ are independent, and I want $X$ and $Y$ to be dependent $\endgroup$ Dec 16, 2017 at 13:05
  • 4
    $\begingroup$ That's not a problem: $X$ and $Y$ are independent and take values $-1$ and $1$ with equal probability. In such a case $X^2$ and $Y^2$ are constant and hence independent :) $\endgroup$
    – Roah
    Dec 17, 2017 at 14:14
33
$\begingroup$

For any two (measurable) sets $A_i$, $i=1,2$, $Y_i \in A_i$ if and only if $X_i \in B_i$, where $B_i$ are the sets { $s : f_i (s) \in A_i$ }. Hence, since the $X_i$ are independent, ${\rm P}(Y_1 \in A_1 , Y_2 \in A_2) = {\rm P}(Y_1 \in A_1) {\rm P}(Y_2 \in A_2)$. Thus, the $Y_i$ are independent (which is intuitively clear anyway). [We have used here that random variables $Z_i$, $i=1,2$, are independent if and only if ${\rm P}(Z_1 \in C_1 , Z_2 \in C_2) = {\rm P}(Z_1 \in C_1) {\rm P}(Z_2 \in C_2)$ for any two measurable sets $C_i$.]

$\endgroup$
3
  • 2
    $\begingroup$ I had no idea about the theorem of measurable sets and independence. Anyway, it seems to be a valid proof. (But I have no idea what the measurable sets are) $\endgroup$
    – LLS
    Nov 5, 2010 at 11:54
  • 1
    $\begingroup$ On the one hand, my answer also assumes that the functions $f_i$ are measurable. On the other hand, the use of the prefix "measurable" (for sets/functions) may be omitted in an introductory setting. $\endgroup$
    – Shai Covo
    Nov 5, 2010 at 12:30
  • 1
    $\begingroup$ What if Y_1 = sort(X_1) and Y_2 = sort(X_2)? In that case Y_1 and Y_2 are clearly highly correlated even though the two vectors X_1 and X_2 are independent. And if in this case Y_1 and Y_2 are not independent, then what does it mean for sample quantiles of X_1 and X_2 which are estimates as k-th elements of Y_1 and Y_2? $\endgroup$
    – Confounded
    May 4, 2018 at 8:44
12
$\begingroup$

Yes, they are independent.

The previous answers are sufficient and rigorous. On the other hand, it can be restated as followed. Assume they are discrete random variable.

$\Pr[Y_1 = f_1(X_1) \wedge Y_2 = f_2(X_2)] = \Pr[X_1 \in f_1^{-1}(Y_1)\wedge X_2\in f_2^{-1}(Y_2)] = \Pr[X_1 \in A_1 \wedge X_2 \in A_2]$

and we expand it by probability mass function derived

$ = \sum_{x_1 \in A_1\wedge x_2 \in A_2}\Pr(x_1, x_2) = \sum_{x_1 \in A_1\wedge x_2 \in A_2}\Pr(x_1)\Pr(x_2) $

Here we use the independency of $X_1$ and $X_2$, and we shuffle the order of summation

$= \sum_{x_1 \in A_1}\Pr(x_1)\cdot \sum_{x_2 \in A_2} \Pr(x_2) = \Pr[X_1\in f_1^{-1}(Y_1)]\cdot \Pr[X_2 \in f_2^{-1}(Y_2)] = \Pr[Y_1 = f_1(X_1)]\Pr[Y_2 = f_2(X_2)] $

Here we show the function of independent random variable is still independent

$\endgroup$
6
$\begingroup$

I'll add another proof here, the continuous analog of Fang-Yi Yu's proof:

Assume $Y_1$ and $Y_2$ are continuous. For real numbers $y_1$ and $y_2$, we can define:

$S_{y_1} = \{{x_1: g(x_1)\le y_1} \}$ and

$S_{y_2} = \{{x_2: h(x_2)\le y_2} \}$.

We can then write the joint cumulative distribution function of $Y_1$ and $Y_2$ as:

\begin{eqnarray*} F_{Y_{1},Y_{2}}(y_{1},y_{2}) & = & P(Y_{1}\le y_{1},Y_{2}\le y_{2})\\ & = & P(X_{1}\in S_{y_{1}},X_{2}\in S_{y_{2}})\\ & = & P(X_{1}\in S_{y_{1}})P(X_{2}\in S_{y_{2}}) \end{eqnarray*}

Then the joint probability density function of $Y_{1}$ and $Y_{2}$ is given by:

\begin{eqnarray*} f_{Y_{1},Y_{2}}(y_{1},y_{2}) & = & \frac{\partial^{2}}{\partial y_{1}\partial y_{2}}F_{Y_{1},Y_{2}}(y_{1},y_{2})\\ & = & \frac{d}{dy_{1}}P(X_{1}\in S_{y_{1}})\frac{d}{dy_{2}}P(X_{2}\in S_{y_{2}}) \end{eqnarray*}

Since the first factor is a function only of $y_{1}$ and the second is a function only of $y_{2}$, then we know $Y_{1}$ and $Y_{2}$ are independent (recall that random variables $U$ and $V$ are independent random variables if and only if there exists functions $g_{U}(u)$ and $h_{V}(v)$ such that for every real $u$ and $v$, $f_{U,V}(u,v)=g_{U}(u)h_{V}(v)$).

$\endgroup$
0
$\begingroup$

Yes. For two sets $A_1$ and $A_2$, you find

$$P(Y_1\in A_1,Y_2\in A_2)=P(f_1(X_1)\in A_1,f_2(X_2)\in A_2)=P(X_1\in f_1^{-1}(A_1),X_2\in f_2^{-1}(A_2))$$ and $$P(Y_1\in A_1)P(Y_2\in A_2)=P(f_1(X_1)\in A_1)P(f_2(X_2)\in A_2)=P(X_1\in f_1^{-1}(A_1))P(X_2\in f_2^{-1}(A_2))$$

Since $X_1$ and $X_2$ are independent, it holds

$$P(X_1\in f_1^{-1}(A_1),X_2\in f_2^{-1}(A_2))=P(X_1\in f_1^{-1}(A_1))P(X_2\in f_2^{-1}(A_2))$$

and therefore

$$P(Y_1\in A_1,Y_2\in A_2)=P(Y_1\in A_1)P(Y_2\in A_2)$$

$\endgroup$

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .