What can we say about the probability distribution function of $n$ independent random variables with different means and variances but the same PDF?

For example, lets say $X_1,X_2,\dots,X_n$ are independent random variables with the following PDFs:

$$ \begin{align} X_1 & \sim N(μ_1,σ_1) \\ X_2 & \sim N(μ_2,σ_2) \\ & {}\,\vdots \\ X_n & \sim N(μ_n,σ_n) \end{align} $$

So they are all Normal random variables with different means and variances and their summation would also have Normal distribution. Now my question is what about other distribution functions?

  • 2
    $\begingroup$ If they have the same pdf they have the same mean and variance. $\endgroup$ – André Nicolas Aug 30 '13 at 22:46
  • 1
    $\begingroup$ @AndréNicolas : I'd guess that what is meant it that they belong to the same location-scale family, so that the distribution is $f\left(\dfrac{x-\mu}{\sigma}\right)\cdot\dfrac{dx}{\sigma}$, but the value of $(\mu,\sigma)$ is different for each one. $\endgroup$ – Michael Hardy Aug 30 '13 at 23:53
  • $\begingroup$ @Michael Hardy: Yes That is exactly what I meant. $\endgroup$ – May Aug 30 '13 at 23:58
  • $\begingroup$ @May: We are then basically looking for information about a linear combination of iid random variables. In the case of the normal, things are very nice, but I do not know of general results. $\endgroup$ – André Nicolas Aug 31 '13 at 0:01

If $f(x)\,dx$ is a probability distribution with expected value $0$ and variance $1$, and the distribution of $X_i$ is

$$f\left(\dfrac{x-\mu_i}{\sigma_i}\right)\cdot\dfrac{dx}{\sigma_i},$$ and $X_i$ are independent, then certainly the distribution of $X_1+\cdots+X_n$ has expected value $\mu_1+\cdots+\mu_n$ and variance $\sigma_1^2+\cdots+\sigma_n^2$. Also, the higher cumulants would add together in the same way. (The fourth cumulant, for example, is $\mathbb E((X-\mu)^4) - 3(\mathbb E((X-\mu)^2))^2$, and the coefficient $3$ is the only number that makes this functional additive in the sense that the fourth cumulant of a sum of independent random variables is the sum of their fourth cumulants.)

We've tacitly assumed $\sigma_i<\infty$. I think if $\sum_{i=1}^\infty\sigma_i^2=\infty$, then as $n$ grows, the distribution would approach a normal distribution (I'm not recalling the appropriate generalization of the central limit theorem clearly enough to state it precisely.) But what happens for small $n$ is another question, and the answer would depend on what function $f$ is.

I said above that $f(x)\,dx$ has expectation $0$ and variance $1$. But one can also have perfectly good location-scale families in which the expectation, and a fortiori, the variance, do not exist. The most well-known case is the Cauchy distribution. The simplest result there is that $(X_1+\cdots+X_n)/n$ actually has the same Cauchy distribution as $X_1$ if these $n$ variables are i.i.d. It doesn't get narrower. So a lot depends on which function $f$ is.

  • $\begingroup$ About CLT normalization: remember nothing except that one should put the mean to 0 and the variance to 1. $\endgroup$ – Did Aug 31 '13 at 7:16

It seems the question is to find families $F$ of distributions, other than the gaussian family, such that the distribution of every linear combination of independent random variables with distributions in $F$ is in $F$. The problem is not really well defined hence let us specify things somewhat:

Let $F_h$ denote the family of distributions of random variables $X$ such that $E[\mathrm e^{\mathrm itX}]=\mathrm e^{\mathrm it\mu-h(\sigma t)}$ for some $(\mu,\sigma)$ with $\sigma\geqslant0$. Does it satisfy the property above?

Note that the gaussian case is when $h(t)=\frac12t^2$. The family $F_h$ satisfies the property above if and only if, for every $\sigma_1$ and $\sigma_2$, there exists some $\mu$ and $\sigma$ such that $$ h(\sigma_1 t)+h(\sigma_2 t)=h(\sigma t)+\mu t. $$ In particular, stable distributions (for some fixed exponent $\alpha$) spring to mind.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.