5
$\begingroup$

It is easy to show that if we have n independent normally distributed random variables, then a linear combination fo them ar normally distributed.

It is also said that if (x1,x2,..,xn) is multivariate normally distributed, but not nececarrily independent, then any linear combination is also normally distributed. This is stated here: http://en.wikipedia.org/wiki/Multivariate_normal_distribution#Definition

But does this mean that any linear combination of normally distributed random variables are normally distributed, even if they are not independent? This will follow from the definition if the joint distribution of set of normally distributed random variables(not nececarrily independent) are jointly multivarite distributed?

$\endgroup$
7
$\begingroup$

This will follow from the definition if the joint distribution of set of normally distributed random variables(not nececarrily independent) are jointly multivarite distributed?

Yes.

But does this mean that any linear combination of normally distributed random variables are normally distributed, even if they are not independent?

No. A frequently mentioned counterexample is based on $X$ standard normal and $Y=SX$ with $S=\pm1$ symmetric and independent of $X$. Then $X$ and $Y$ are normal but $X+Y$ is not since $P(X+Y=0)=\frac12$, a property that no normal random variable satisfies.

$\endgroup$
  • $\begingroup$ But if Y = S*X then it is not a linear combination, because S is not a scalar, but a random variable?EDIT: Sorry I see your point now. $\endgroup$ – user126421 Feb 12 '14 at 19:24
  • $\begingroup$ Would you say then that we can partion it in 3 cases?, let X={x1,x2,..xn} be normally random varaibles, either 1. all variables in X are independent, or 2. they are jointly normally distributed, or 3. they are not independent and not jointly normally distributed. In cases 1 and 2 any linear combination of the x's are normally distributed? And in case number 3 we do not know, it may be that they are normally distributed, but it may also be that they are not? $\endgroup$ – user126421 Feb 12 '14 at 19:31
  • $\begingroup$ Yes, except that case 1. is a subcase of case 2. $\endgroup$ – Did Feb 12 '14 at 20:28
  • $\begingroup$ Thank you very much for your time! $\endgroup$ – user126421 Feb 12 '14 at 20:32
2
$\begingroup$

You need independence. Otherwise the simplest counterexample is $X$ and $-X$, which are both normally distributed, but their sum $X+(-X) = 0$ is not.

$\endgroup$
  • $\begingroup$ But Wikipedia says that jointly multivariate distributed is sufficient and not independence. So does that mean that in your example, X and -X can not not have a jointly bivariate distribution? $\endgroup$ – user126421 Feb 12 '14 at 19:34
  • $\begingroup$ Think of it this way: a multivariate normal distribution needs a covariance matrix that is strictly positive definite - that's a necessary condition and kills the 'any linear combination, regardless of independence' part. In particular, if one variable is a linear combination of several of the others, that makes the covariance matrix singular. $\endgroup$ – huh Feb 12 '14 at 19:48
  • $\begingroup$ The zero random variable IS normally distributed. $\endgroup$ – Did Feb 12 '14 at 20:26
  • $\begingroup$ only if you consider Dirac's $\delta(x)$ as being a normal distribution, not just a functional limit of normal distributions (which one normally doesn't, as it's not a regular function) $\endgroup$ – huh Feb 12 '14 at 20:43
  • 1
    $\begingroup$ One "normally" very much considers Dirac measures as members of the normal family. Not doing so quickly leads to a mess, especially in higher dimension. (Unrelated: please use @.) $\endgroup$ – Did Feb 13 '14 at 12:15
-1
$\begingroup$

Just thinking in the univariate case, note that if two RVs are iid, distributed as $N(\mu, \sigma^2)$, then their sum is distributed as $N(2\mu, 2\sigma^2)$. If they are correlated with $\rho=1$, it is the same as $2X=X+X$, which is distributed as $N(2\mu, 4\sigma^2)$ - in other words, it is 'stretched out' twice as far in variance. This makes sense, because uncorrelated would have a standard deviation that grows by $\sqrt 2$, whereas adding to itself must double the standard deviation (which is equivalent to $4\sigma^2$).

All other cases of correlation fall in-between. Though, as people have pointed out, $pho=-1$ gives you zero as a degenerate point.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.