I was trying to show that the sum of i.i.d. normals is normal. Since I had no idea how to do it I started looking online and found out about moment generating functions (MGF). Most proofs about that sum concluded the argument saying that because the MGF of the sum equals the MGF of a normal, then the sum must be a normal (example: look at last phrase of the document).

Question: Why is it true? If two random variables have the same MGF, then they must have the same distribution? Is there a one-to-one correspondence between distributions and MGFs? Can someone shed some light on this?

Thanks for helping!

  • Yes, same mgf implies same distribution. – André Nicolas Sep 20 '15 at 2:54
  • Thanks! Do you know where I could find the proof for this? – Guilherme Salomé Sep 20 '15 at 2:55
  • Most serious books do it. The one I have (oldie) is Feller, second volume. – André Nicolas Sep 20 '15 at 3:00
  • 2
    To use characteristic functions rather than MGF is safer (no problem of convergence of integrals). For the fact that the MGF (when it exists in a neighborhood of zero) or the CF (always) indeed characterize the distribution, see the statements and references at the obvious places: en.wikipedia.org/wiki/Moment-generating_function en.wikipedia.org/wiki/… – Did Sep 20 '15 at 8:49
up vote 1 down vote accepted

It is possible and instructive to prove the theorem directly, because the path through characteristic functions (or moment generating functions) is actually a very long one that requires far more than one might think, including complex analysis and dominated convergence theorem from measure theory. $\def\pp{\mathbb{P}}$

Notation

Given random variable $X$, let "$f_X$" denote the density function for $X$.

Convolution

Given continuous real-valued independent random variables $X,Y$, and $Z = X+Y$:

  For any $v \in \mathbb{R}$:

    $\pp( Z \le v ) = \pp( Y \le v-X ) = \int_{-\infty}^\infty \int_{-\infty}^{v-x} f_X(x) f_Y(y)\ dy\ dx$

    $= \int_{-\infty}^\infty \int_{-\infty}^{v} f_X(x) f_Y(y-x)\ dy\ dx$

    $= \int_{-\infty}^{v} \int_{-\infty}^\infty f_X(x) f_Y(y-x)\ dx\ dy$ [The swap is valid since the integrand is non-negative].

    Thus $f_{Z}(v) = \frac{d}{dv} \pp( Z \le v ) = \int_{-\infty}^\infty f_X(x) f_Y(v-x)\ dx$ [by FTC].

Sum of normal random variables

Given independent random variables $X \sim N(a,s^2)$ and $Y \sim N(b,t^2)$, and $Z = X+Y$:

  Let $W = Z - (a+b) = (X-a) + (Y-b)$.

  For any $v \in \mathbb{R}$:

    $f_W(v) = \int_{-\infty}^\infty \frac{1}{\sqrt{2πs^2}} \exp(-\frac{x^2}{2s^2}) \frac{1}{\sqrt{2πt^2}} \exp(-\frac{(v-x)^2}{2t^2})\ dx$

    $= \int_{-\infty}^\infty \frac{1}{\sqrt{2πs^2}} \frac{1}{\sqrt{2πt^2}} \exp(-\frac{ x^2 t^2 + (v-x)^2 s^2 }{ 2 s^2 t^2 })\ dx$

    $= \int_{-\infty}^\infty \frac{1}{\sqrt{2πs^2}} \frac{1}{\sqrt{2πt^2}} \exp(-\frac{ (s^2+t^2) x^2 - 2vx s^2 + v^2 s^2 }{ 2 s^2 t^2 })\ dx$

    $= \int_{-\infty}^\infty \frac{1}{\sqrt{2πs^2}} \frac{1}{\sqrt{2πt^2}} \exp(-\frac{ ( (s^2+t^2) x - v s^2 )^2 + v^2 s^2 t^2 }{ 2 s^2 t^2 (s^2+t^2) })\ dx$. [by completing the square]

    $= \frac{1}{\sqrt{2π(s^2+t^2)}} \exp(-\frac{ v^2 }{ 2 (s^2+t^2) } ) \int_{-\infty}^\infty \frac{1}{\sqrt{2πc}} \exp(-\frac{ ( x - v \frac{s^2}{s^2+t^2} )^2 }{ 2 c })\ dx$ where $c = \frac{s^2t^2}{s^2+t^2}$

    $= \frac{1}{\sqrt{2π(s^2+t^2)}} \exp(-\frac{ v^2 }{ 2 (s^2+t^2) } )$ [the above known integral motivates the previous steps].

  Therefore $W \sim N(0,s^2+t^2)$ and hence $Z \sim N(a+b,s^2+t^2)$.

  • Hey! Thanks for your answer, I still haven't accepted it as a the answer because I haven't had the time to read it and understand it properly, I'll do that asap. Thanks again! :D – Guilherme Salomé Sep 22 '15 at 14:01
  • @GuilhermeSalomé: Sure. Feel free to ask if any part is not clear! This technique is the standard one to compute the density function of a sum of random variables, and you can try it on other usual ones like uniform or binomial or exponential. – user21820 Sep 22 '15 at 14:03

Your Answer

 

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Not the answer you're looking for? Browse other questions tagged or ask your own question.