13
$\begingroup$

It's pretty trivial to get a moment-generating function from a p.d.f. (provided $\sum e^{tx}f(x)$ isn't too difficult to evaluate), but since moment-generating functions uniquely determine a probability distribution function, is there a way to "back-generate" the p.d.f from the m.g.f.?

Edit: I'm talking about a discrete distribution.

$\endgroup$
1
  • $\begingroup$ Are you dealing with discrete distributions? In general, probability distribution need not have a p.d.f. $\endgroup$
    – Siméon
    Commented Apr 7, 2013 at 0:22

5 Answers 5

9
$\begingroup$

The question is to inverse a Laplace/Fourier transform. I take the example of a discrete distribution $f(n)$ on the natural numbers with moment-generating-function $$ M(t) = \sum_{n=0}^\infty f(n)e^{nt} $$ with radius of convergence $R \geq 1$. Fourier inversion here is $$ f(n) = \frac{1}{2\pi}\int_0^{2\pi}M(i\theta)e^{-in\theta}\,d\theta $$

If you prefer to stay in the real realm, there is an interesting formula due to Post.

A related formula would be: $$ f(n) = \left.\frac{d^n}{dt^n}M(\log t)\right|_{t=0} $$

$\endgroup$
5
  • $\begingroup$ We haven't learned anything about Fourier transforms. $\endgroup$
    – ithisa
    Commented Apr 7, 2013 at 1:49
  • $\begingroup$ @user54609 If you were in elementary probability theory, there was no way to do it. You had to recognise a given mgf as an mgf of some random variable $\endgroup$
    – BCLC
    Commented Feb 6, 2016 at 7:15
  • $\begingroup$ Siméon, is Post's the same as this one? $\endgroup$
    – BCLC
    Commented Feb 6, 2016 at 7:16
  • $\begingroup$ Would you not use the limit as $t \to 0$ in the final equation, since there is trouble with $f(0)$ since $\log(t)$ at $t=0$ is undefined? $\endgroup$
    – apg
    Commented May 9, 2017 at 10:54
  • $\begingroup$ Why the integration limits here are from $0$ to $2\pi$, instead of from $-\infty$ to $\infty$? Compare with en.wikipedia.org/wiki/…. $\endgroup$
    – a06e
    Commented Apr 2, 2020 at 11:56
6
$\begingroup$

I know that this question is old, but all the answers provided thus far only dealt with the continuous variable cases. In the edit, @user54609 states that he/she was asking how to get the probability distribution of a discrete random variable, and for that case, the answer is surprisingly simple.

The only difference is that it is easier to do what you want using the characteristic function $G(s)$, instead of the moment generating function $E[e^{tx}]$. It is easy to convert between them by looking at the definition of $G(s)$:

$$ G(s) = \sum_{n=0}^{+\infty}p(n)s^{n} $$ where $n$ is the discrete random variable, and $p(n)$ is its probability distribution.Thus, you can convert between $G$ and the m.g.f. by setting $s=e^{t}$.

Now, lets suppose that we know $G(s)$ but not $p(n)$, and we want to find out the functional form of $p(n)$ using what we know about $G(s)$. The way to do that is by evaluating derivatives of $G(s)$ at $s=0$ as:

$$ p(n)=\frac{1}{n!}\left.\frac{d^{n} G(s)}{d s^{n}}\right|_{s=0} $$

For example, the characteristic function of a Poisson distributed random variable is $G(s)=e^{\lambda(s-1)}$. We can recover the probability distribution of the Poisson random variable $n$ as

$$ \frac{1}{n!}\left.\frac{d^{n} G(s)}{d s^{n}}\right|_{s=0}=\left.\frac{\lambda^{n}}{n!}e^{(s-1)\lambda}\right|_{s=0} = \frac{\lambda^{n}}{n!}e^{-\lambda} = p(n) $$

You can ty this with other example distributions, such as the Binomial, whose characteristic function is $G(s)=\left[1+(s-1)p\right]^{N}$ to gain some trust in the method, but it works for all discrete random variables.

$\endgroup$
3
$\begingroup$

Let $\mathcal{M}(g)(s) = \int_0^{\infty} x^{s-1} g(x) dx$ be the Mellin transform; then the moment-generating function of a smooth enough p.d.f $f$ is given by $\mathcal{M}(f(-\log(x))(-s)$;

so given a nice enough moment-generating function $h(s) = E[e^{sX}] = \int_{-\infty}^{\infty} e^{sx} f(x) dx$, we recover $f$ as $$f(x) = \mathcal{M}^{-1}(h(-s))(-e^x)$$ where $\mathcal{M}^{-1}$ is given by the Mellin inversion theorem: $$\mathcal{M}^{-1}h(x) = \frac{1}{2\pi i}\int_{c - i\infty}^{c + i \infty} x^{-s} h(s) ds$$ for an appropriate real number $c$, where the integral is understood to be along a line in $\mathbb{C}$.

$\endgroup$
3
  • $\begingroup$ We haven't learned anything about Mellin transforms. :P Or how to work with complex numbers in that way. $\endgroup$
    – ithisa
    Commented Apr 7, 2013 at 1:50
  • 1
    $\begingroup$ @EricDong I'm not sure you'll be able to do it in a significantly different way. Probably the best thing to do is to use a table. $\endgroup$
    – Cocopuffs
    Commented Apr 7, 2013 at 9:16
  • 1
    $\begingroup$ This should work for appropriate discrete functions $f$ as well since piecewise continuity is enough $\endgroup$
    – Cocopuffs
    Commented Apr 7, 2013 at 9:17
2
$\begingroup$

Yes, you can. First, convert your mgf into a characteristic function (i.e. replace $t \rightarrow it$). Next, invert the characteristic function to yield the pdf using an inverse Fourier transform.

$\endgroup$
0
$\begingroup$

The logarithm of the mgf is known as the cumulant generating function, and it can be used to get quite good approximations of the original density (or probability mass) function via what is known as the saddlepoint approximation, which can be remarkably good. For an exposition and examples see https://stats.stackexchange.com/questions/191492/how-does-saddlepoint-approximation-work/191781#191781

$\endgroup$

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .