1
$\begingroup$

How do we define independence for random vectors?

Eg, let $\mathbf{e}$ be a random n by 1 vector and $\mathbf{b}$ be a random k by 1 vector (where n does not necessarily have to equal to k), then how do we define independence between $\mathbf{e}$ and $\mathbf{b}$?

For example, if we take just 2 random variables, X, Y then X and Y are independent iff f(x,y) = f(x)f(y) where f(x,y) characterizes their joint pdf and f(x), f(y) are their marginal pdfs. If we adapt this onto the random vector case, we have $f(\mathbf{e}, \mathbf{b}) = f(\mathbf{e}) f(\mathbf{b})$, examining the RHS, we see that $f(\mathbf{e})$ is simply the joint distribution of $e_1, e_2, ..., e_n$ and likewise, $f(\mathbf{b})$ is the joint distribution of $b_1, b_2, ..., b_k$, but then what is $f(\mathbf{e}, \mathbf{b}) $? How is it defined?

Continuing on, for a more special case, consider when $\mathbf{e}$ is jointly multivariate normal, ie, $\mathbf{e} \sim N(\mathbf{\mu_e}, \mathbf{\Sigma_e})$ and $\mathbf{b}$ is also jointly multivariate normal, ie, $\mathbf{b} \sim N(\mathbf{\mu_b}, \mathbf{\Sigma_b})$. Now in the special case of when two jointly normally distributed variables X and Y, a sufficient condition for independence is when cov(X,Y) = 0 where cov(.) denotes the covariance between X and Y. How then, do we generalise that into the random vector case? Ie, can we say that $\mathbf{e}$ and $\mathbf{b}$ are independent iff $cov(\mathbf{e}, \mathbf{b}) = 0$? But then we would have to show $\mathbf{e}$ and $\mathbf{b}$ are jointly normally distributed... but this doesn't really make sense since $\mathbf{e}$ and $\mathbf{b}$ are already itself jointly normally distributed, so then wouldn't we be talking about the 'joint' distribution of two joint distributions? (How would I make sense of that?)

Thanks

$\endgroup$
  • $\begingroup$ What level of probability are you doing? Are you comfortable with measure theory? $\endgroup$ – Nick Peterson Jul 13 '13 at 15:29
4
$\begingroup$

The best definition for independence I can give you, without sweating measure theory all that much, is this:

Two random variables $X$ and $Y$ (which share the same probability space, but we don't assume have the same codomain) are independent if the following holds:

For any event $A$ for $X$ and any event $B$ for $Y$, we have $$ P(X\in A,\ Y\in B)=P(X\in A)\cdot P(Y\in B). $$ If you want to start talking about probability densities and things of that nature for the case when you have two random vectors of different dimensions, you're going to need some more serious theory; such things can be defined fairly easily in terms of measure theory, but not as easily otherwise.

Of course, if you have an $n$-long vector and a $k$-long vector, you can think of their joint distribution as being an $(n+k)$-long vector; in this case, assuming you are talking about continuous distributions, you could probably think of it as the condition that $$ f_{\vec{x},\vec{y}}(x_1,\ldots,x_n,y_1,\ldots,y_k)=f_{\vec {x}}(x_1,\ldots,x_n)\cdot f_{\vec{y}}(y_1,\ldots,y_k), $$ where $f_{\vec{x},\vec{y}}$ is the joint density function and $f_{\vec{x}}$ and $f_{\vec{y}}$ are their respective marginal densities.

$\endgroup$
  • $\begingroup$ Thanks nrpeterson, that makes much more sense. Speaking of measure theory, do you have any recommendations for introductory texts on the area? Possibly a self-contained text? Thanks. $\endgroup$ – Trts Jul 13 '13 at 16:10
  • $\begingroup$ My pleasure! As for references... there's some material at the end of baby-Rudin, but I'd say you're better off going with Folland. The material gets more and more intense very quickly, but the chapters on measure theory are pretty well-done. $\endgroup$ – Nick Peterson Jul 13 '13 at 17:06

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.