10
$\begingroup$

Many results are based on the fact of the Moment Generating Function (MGF) Uniqueness Theorem, that says:

If $X$ and $Y$ are two random variables and equality holds for their MGF's: $m_X(t) = m_Y(t)$ then $X$ and $Y$ have the same probability distribution: $F_X(x) = F_Y(y)$.

The proof of this theorem is never shown in textbooks, and I cannot seem to find it online or in any book I have access to.

Can someone show me the proof or tell me where to look it up?

Thanks for your time.

$\endgroup$
  • $\begingroup$ Got something from an answer below? $\endgroup$ – Did Apr 11 '14 at 10:27
  • 2
    $\begingroup$ Indeed, the formulas are very handy, but I wish there were more "meat" on them (details). $\endgroup$ – Shuzheng Apr 11 '14 at 12:27
  • $\begingroup$ Which part(s) are you unable to complete after 30 seconds of thinking? $\endgroup$ – Did Apr 11 '14 at 13:43
  • 2
    $\begingroup$ Both, I don't see why the equalities are true - there need some intermediary calculations. $\endgroup$ – Shuzheng Apr 12 '14 at 13:20
  • $\begingroup$ Why have you written two equations ? One for natural numbers and one for real numbers ? What is $s$ in this connection - a real number ? $\endgroup$ – Shuzheng Apr 12 '14 at 18:04
6
$\begingroup$

$$(\forall n\geqslant0)\qquad \left.\frac{\mathrm d^n}{\mathrm ds^n}\mathbb E[s^X]\right|_{s=0}=n!\cdot\mathbb P[X=n] $$ $$(\forall x\in\mathbb R)\qquad \int_0^{2\pi}\mathbb E[\mathrm e^{\mathrm itX}]\,\mathrm e^{-\mathrm itx}\,\mathrm dt=2\pi\cdot\mathbb P[X=x] $$

$\endgroup$
  • 1
    $\begingroup$ I guess Did is giving a formula for recovering a discrete probability distribution from its MGF. $\endgroup$ – Amritanshu Prasad Aug 4 '13 at 4:31
  • 1
    $\begingroup$ Then $E[s^X]=ps^j+(1-p)s^k$. Alternatively, in the general case, $E[s^X]=M_X(\log s)$. $\endgroup$ – Did Aug 5 '13 at 10:21
  • 1
    $\begingroup$ Does your formula works for negative values of X ? What is X is -16 ? $\endgroup$ – Shuzheng Aug 6 '13 at 19:22
  • 3
    $\begingroup$ Let me add that it is a tad unsettling to see that, when it is suggested that you perform yourself a trivial computation which would allow you to understand the general case, you do not even acknowledge the suggestion but fall back on demands already answered a long time ago. If you think maths can be learned as a spectator, you are wrong (but hey, this is your call). $\endgroup$ – Did Apr 12 '14 at 18:41
  • 1
    $\begingroup$ "If you think maths can be learned as a spectator, you are wrong " well put $\endgroup$ – Math1000 Nov 1 '15 at 11:54
10
$\begingroup$

Let us first clarify the assumption. Denote the moment generating function of $X$ by $M_X(t)=Ee^{tX}$.

Uniqueness Theorem. If there exists $\delta>0$ such that $M_X(t) = M_Y(t) < \infty$ for all $t \in (-\delta,\delta)$, then $F_X(t) = F_Y(t)$ for all $t \in \mathbb{R}$.

To prove that the moment generating function determines the distribution, there are at least two approaches:

  • To show that finiteness of $M_X$ on $(-\delta,\delta)$ implies that the moments $X$ do not increase too fast, so that $F_X$ is determined by $(EX^k)_{k\in\mathbb{N}}$, which are in turn determined by $M_X$. This proof can be found in Section 30 of Billingsley, P. Probability and Measure.

  • To show that $M_X$ is analytic and can be extended to $(-\delta,\delta)\times i\mathbb{R} \subseteq \mathbb{C}$, so that $M_X(z)=Ee^{zX}$, so in particular $M_X(it)=\varphi_X(t)$ for all $t\in\mathbb{R}$, and then use the fact that $\varphi_X$ determines $F_X$. For this approach, see Curtiss, J. H. Ann. Math. Statistics 13:430-433 and references therein.

At undergraduate level, it is interesting to work with the moment generating function and state the above theorem without proving it. The proof requires far more advanced mathematics than undergraduate level.

In fact, the proof is so advanced that, at such a point it usually makes more sense to accept working with complex numbers, forget about moment generating function and work with the charachteristic function $\varphi_X(t)=Ee^{itX}$ instead. Almost every graduate textbook takes this path and proves that the characteristic function determines the distribution as a corollary of the inversion formula.

This proof of the inversion formula is bit long, but it only requires Fubini Theorem to switch an expectation with an integral and Dominated Convergence Theorem to switch an integral with a limit. A direct proof of uniqueness without inversion formula is shorter and simpler, and it only requires Weierstrass Theorem to approximate a continuous function by a trigonometric polynomial.

Side remark. If you only admit random variables whose support are contained in $\mathbb{Z}_+$, then the probability generating function $G_X(z)=Ez^X$ determines $p_X$ (and thus $F_X$). This elementary result is proved in most undergraduate textbooks and is mentioned in Did's answer. If you only admit random variables whose support are contained in $\mathbb{Z}$, then it is simpler to show that $\varphi_X$ determines $p_X$, as also mentioned in Did's answer, and the proof uses Fubini.

$\endgroup$
  • 2
    $\begingroup$ This is a super helpful answer. thanks. $\endgroup$ – Tim kinsella Apr 4 '18 at 22:19
5
$\begingroup$

In the case where $X$ has density function $\phi(x)$, $$ M_X(it) = E(e^{itX}) = \int_{-\infty}^\infty e^{itx}\phi(x)dx, $$ which is the Fourier transform of $\phi(x)$. Therefore $\phi(x)$ can be recovered from its MGF using the Fourier inversion formula.

The function $M_X(it)$ is called the characteristic function of $X$. See Chapter 6 of Kai Lai Chung's book A Course in Probability Theory for more details.

$\endgroup$
  • $\begingroup$ Thanks alot. So in order to understand the proof I must look into Fourier transformation theory ? $\endgroup$ – Shuzheng Aug 3 '13 at 12:19
  • 1
    $\begingroup$ I guess Did's answer above works nicely for random variables on a finite probability space. $\endgroup$ – Amritanshu Prasad Aug 5 '13 at 10:40
  • 1
    $\begingroup$ @AmritanshuPrasad Actually, for every nonnegative integer valued random variable. $\endgroup$ – Did Aug 6 '13 at 19:27
  • $\begingroup$ @Did Actually, you prove uniqueness restricted to that space. You don't rule out the possibility that other random variables happen to have the same MGF. $\endgroup$ – user334639 Jul 28 '17 at 3:45

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.