9
$\begingroup$

It's pretty trivial to get a moment-generating function from a p.d.f. (provided $\sum e^{tx}f(x)$ isn't too difficult to evaluate), but since moment-generating functions uniquely determine a probability distribution function, is there a way to "back-generate" the p.d.f from the m.g.f.?

Edit: I'm talking about a discrete distribution.

$\endgroup$
  • $\begingroup$ Are you dealing with discrete distributions? In general, probability distribution need not have a p.d.f. $\endgroup$ – Siméon Apr 7 '13 at 0:22
8
$\begingroup$

The question is to inverse a Laplace/Fourier transform. I take the example of a discrete distribution $f(n)$ on the natural numbers with moment-generating-function $$ M(t) = \sum_{n=0}^\infty f(n)e^{nt} $$ with radius of convergence $R \geq 1$. Fourier inversion here is $$ f(n) = \frac{1}{2\pi}\int_0^{2\pi}M(i\theta)e^{-in\theta}\,d\theta $$

If you prefer to stay in the real realm, there is an interesting formula due to Post.

A related formula would be: $$ f(n) = \left.\frac{d^n}{dt^n}M(\log t)\right|_{t=0} $$

$\endgroup$
  • $\begingroup$ We haven't learned anything about Fourier transforms. $\endgroup$ – ithisa Apr 7 '13 at 1:49
  • $\begingroup$ @user54609 If you were in elementary probability theory, there was no way to do it. You had to recognise a given mgf as an mgf of some random variable $\endgroup$ – BCLC Feb 6 '16 at 7:15
  • $\begingroup$ Siméon, is Post's the same as this one? $\endgroup$ – BCLC Feb 6 '16 at 7:16
  • $\begingroup$ Would you not use the limit as $t \to 0$ in the final equation, since there is trouble with $f(0)$ since $\log(t)$ at $t=0$ is undefined? $\endgroup$ – Alexander Kartun-Giles May 9 '17 at 10:54
4
$\begingroup$

I know that this question is old, but all the answers provided thus far only dealt with the continuous variable cases. In the edit, @user54609 states that he/she was asking how to get the probability distribution of a discrete random variable, and for that case, the answer is surprisingly simple.

The only difference is that it is easier to do what you want using the characteristic function $G(s)$, instead of the moment generating function $E[e^{tx}]$. It is easy to convert between them by looking at the definition of $G(s)$:

$$ G(s) = \sum_{n=0}^{+\infty}p(n)s^{n} $$ where $n$ is the discrete random variable, and $p(n)$ is its probability distribution.Thus, you can convert between $G$ and the m.g.f. by setting $s=e^{t}$.

Now, lets suppose that we know $G(s)$ but not $p(n)$, and we want to find out the functional form of $p(n)$ using what we know about $G(s)$. The way to do that is by evaluating derivatives of $G(s)$ at $s=0$ as:

$$ p(n)=\frac{1}{n!}\left.\frac{d^{n} G(s)}{d s^{n}}\right|_{s=0} $$

For example, the characteristic function of a Poisson distributed random variable is $G(s)=e^{\lambda(s-1)}$. We can recover the probability distribution of the Poisson random variable $n$ as

$$ \frac{1}{n!}\left.\frac{d^{n} G(s)}{d s^{n}}\right|_{s=0}=\left.\frac{\lambda^{n}}{n!}e^{(s-1)\lambda}\right|_{s=0} = \frac{\lambda^{n}}{n!}e^{-\lambda} = p(n) $$

You can ty this with other example distributions, such as the Binomial, whose characteristic function is $G(s)=\left[1+(s-1)p\right]^{N}$ to gain some trust in the method, but it works for all discrete random variables.

$\endgroup$
3
$\begingroup$

Let $\mathcal{M}(g)(s) = \int_0^{\infty} x^{s-1} g(x) dx$ be the Mellin transform; then the moment-generating function of a smooth enough p.d.f $f$ is given by $\mathcal{M}(f(-\log(x))(-s)$;

so given a nice enough moment-generating function $h(s) = E[e^{sX}] = \int_{-\infty}^{\infty} e^{sx} f(x) dx$, we recover $f$ as $$f(x) = \mathcal{M}^{-1}(h(-s))(-e^x)$$ where $\mathcal{M}^{-1}$ is given by the Mellin inversion theorem: $$\mathcal{M}^{-1}h(x) = \frac{1}{2\pi i}\int_{c - i\infty}^{c + i \infty} x^{-s} h(s) ds$$ for an appropriate real number $c$, where the integral is understood to be along a line in $\mathbb{C}$.

$\endgroup$
  • $\begingroup$ We haven't learned anything about Mellin transforms. :P Or how to work with complex numbers in that way. $\endgroup$ – ithisa Apr 7 '13 at 1:50
  • 1
    $\begingroup$ @EricDong I'm not sure you'll be able to do it in a significantly different way. Probably the best thing to do is to use a table. $\endgroup$ – Cocopuffs Apr 7 '13 at 9:16
  • 1
    $\begingroup$ This should work for appropriate discrete functions $f$ as well since piecewise continuity is enough $\endgroup$ – Cocopuffs Apr 7 '13 at 9:17
2
$\begingroup$

Yes, you can. First, convert your mgf into a characteristic function (i.e. replace $t \rightarrow it$). Next, invert the characteristic function to yield the pdf using an inverse Fourier transform.

$\endgroup$
0
$\begingroup$

The logarithm of the mgf is known as the cumulant generating function, and it can be used to get quite good approximations of the original density (or probability mass) function via what is known as the saddlepoint approximation, which can be remarkably good. For an exposition and examples see https://stats.stackexchange.com/questions/191492/how-does-saddlepoint-approximation-work/191781#191781

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.