We have a random vector $X=(X_j)_{j=1,...,n}$, whose components $X_j$ are mutually independent. We build a new random vector $Y=A X+b$, with $Y=(Y_k)_{k=1,...,m}$, $A=(a_{k,j})_{k=1,...,m;j=1,...,n}$, $b=(b_k)_{k=1,...,m}$.

When the components of $Y$ are still mutually independent random variables? How to prove the result? I suppose this can be related with the rank of $A$ but I'm not sure and I cannot find a way to prove if I don't make some special assumption about the law of $X$.

I know $U$ and $V$ are independent iff $f(U)$ and $g(V)$ are independent for each and every pair of measurable function $f$ and $g$. This can help, I think, when, for example, $Y_1=X_1+X_2$ and $Y_2=X_3+X_4$.

But when does $Y_1 = X_1 + X_2$ and $Y_2 = X_1 - X_2$? Or when does $Y_1=X_1 + X_2$ and $Y_2=X_5$?

None of the texts I searched discuss seriously this topic, but leave it in a certain way to "intuition"?


  • $\begingroup$ I don't think there's a general result here unless the $X_i$ are Gaussian. $\endgroup$ – Qiaochu Yuan May 30 '11 at 13:14
  • $\begingroup$ As you've noted, the general result is true if $A = a_1 \oplus a_2 \oplus \cdots \oplus a_p$ for some $p$ where the $a_i$ are row vectors. $\endgroup$ – cardinal May 30 '11 at 13:48

If $A = a_1 \oplus a_2 \oplus \cdots \oplus a_m$, for $m \leq n$, where $a_i$ are row vectors of dimension $n_i$ such that $\sum_{i=1}^m n_i = n$ and $\oplus$ denotes the direct sum, then the random vector $Y$ has independent coordinates.

This is not hard to see since $Y_1$ is measurable with respect to $\sigma(X_1, \ldots X_{n_1})$, $Y_2$ is measurable with respect to $\sigma(X_{n_1+1}, \ldots, X_{n_1+n_2})$, etc., and these $\sigma$-algebras are independent since the $X_i$ are independent (essentially, by definition).

Obviously, this result still holds if we consider matrices that are column permutations of the matrix $A$ described above. Indeed, as we see below, in the case where the distribution of each $X_i$ is non-normal (though perhaps depending on the index $i$), this is essentially the only form that $A$ can take for the desired result to hold.

In the normal-distribution case, as long as $A A^T = D$ for some diagonal matrix $D$, then the coordinates of $Y$ are independent. This is easily checked with the moment-generating function.

Suppose $X_1$ and $X_2$ are iid with finite variance. If $X_1 + X_2$ is independent of $X_1 - X_2$, then $X_1$ and $X_2$ are normal distributed random variables. See here. This result is known as Bernstein's theorem and can be generalized (see below). A proof can be found in Feller or here (Chapter 5).

In the case where $A$ cannot be written as a direct sum of row vectors, you can always cook up a distribution for $X$ such that $Y$ does not have independent coordinates. Indeed, we have

Theorem (Lukacs and King, 1954): Let $X_1, X_2, \cdots, X_n$ be $n$ independently (but not necessarily identically) distributed random variables with variances $\sigma_i^2$, and assume that the $n$th moment of each $X_i(i = 1, 2, \cdots, n)$ exists. The necessary and sufficient conditions for the existence of two statistically independent linear forms $Y_1 = \sum^n_{i=1} a_i X_i$ and $Y_2 = \sum^n_{i=1} b_i X_i$ are

  1. Each random variable which has a nonzero coefficient in both forms is normally distributed, and
  2. $\sum^n_{i=1} a_i b_i \sigma^2_i = 0$.
  • $\begingroup$ Great answer. (And thanks for the Lukacs and King result.) $\endgroup$ – Did Jun 5 '11 at 8:21
  • $\begingroup$ @cardinal Nice answer. But wouldn't it be more satisfying if there was some result about matrices that are not $2 \times n$? Do you know anything about when $A*X$ has independent components when $X$ does and $A$ has deterministic entries? $\endgroup$ – Jeff Jan 6 '14 at 6:40

I think little can be said in general, apart from the rather trivial case in which each of the resulting $y_i$ depends on non-intersecting subssets of $X$ (what would correspond to each column of A having no more than one non-null value).

Some weaker result can be obtained by considering non-correlation instead of independence (or, what would equivalent, restricting to gaussian variables), because the covariances matrices are simply related

$C_Y = A \; C_X A^t $

By assumption, $C_X$ is diagonal, and we want fo find out for which $A$ matrices $C_Y$ is also diagonal. Again, little can be said in general, more than just that. If we restrict further the assumptions, and assume that $x_i$ are iid (or just equal variances), then we get that orthogonality of $A$ in sufficient (but not necessary).

In particular, for the case $Y_1 = X_1 + X_2$, $Y_2 = X_1 - X_2$, $Y$ is not guaranteed to be uncorrelated, unless we assume $x_i$ are iid (or have same variance).

  • $\begingroup$ Something relatively strong can be said. See my updated answer. $\endgroup$ – cardinal May 30 '11 at 14:46
  • $\begingroup$ @cardinal: At first sight, that seems practically equivalent to what I stated. The normal assumption is equivalent to use correlation in place of independence. And the summation seems as the non-diagonal terms in my $C_y$ to me. $\endgroup$ – leonbloy May 30 '11 at 16:38
  • $\begingroup$ It appears much stronger to me. $\endgroup$ – cardinal May 30 '11 at 16:42
  • $\begingroup$ Ah, yes, I see it now. $\endgroup$ – leonbloy May 30 '11 at 17:23

The Skitovich-Darmois theorem A (Skitovich(1953), Darmois(1953), see also A. Kagan, Yu. Linnik, and C.R. Rao (1973, Ch.3)). Let $\xi_j$, where $j=1, 2,\dots, n,$ and $n\geq 2$, be independent random variables. Let $\alpha_j, \beta_j$ be nonzero constants. If the linear statistics $L_1=\alpha_1\xi_1+\cdots+\alpha_n\xi_n$ and $L_2=\beta_1\xi_1+\cdots+\beta_n\xi_n$ are independent, then all random variables $\xi_j$ are Gaussian.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.