Connection between Taylor Series and Probabilities

I am trying to understand why the coefficients in a PGF represent the probability of $$X=k$$. This is something I always accepted as true without ever actually understanding why it is true.

I start from the basic definition of a PGF:

$$G_X(s) = \mathbb{E}[s^X] = \sum_{k=0}^{\infty} p_k s^k$$

Where:

• $$X$$ is a discrete random variable
• $$p_k = P(X = k)$$ is the probability that $$X$$ takes the value $$k$$ (i.e. $$p_k$$ are the coefficients)

I first tried to expand the expectation:

$$\mathbb{E}[s^X] = \sum_{k=0}^{\infty} s^k P(X = k)$$

I used the notation $$P(X = k) = p_k$$, so we can write:

$$\mathbb{E}[s^X] = \sum_{k=0}^{\infty} p_k s^k$$

Taking the the nth derivative of $$G_X(s)$$ and evaluate it at $$s = 0$$:

$$\frac{d^n}{ds^n} G_X(s) = \frac{d^n}{ds^n} \sum_{k=0}^{\infty} p_k s^k$$

Using the power rule of differentiation:

$$\frac{d^n}{ds^n} G_X(s) = \sum_{k=n}^{\infty} p_k \cdot k(k-1)(k-2)...(k-n+1) s^{k-n}$$

And finally evaluating at $$s = 0$$:

$$\left.\frac{d^n}{ds^n} G_X(s)\right|_{s=0} = n! \cdot p_n$$

Rearranging:

$$p_n = \frac{1}{n!} \left.\frac{d^n}{ds^n} G_X(s)\right|_{s=0}$$

This last equation shows that we can recover the probability $$p_n$$ by taking the nth derivative of the PGF, evaluating it at $$s = 0$$, and dividing by $$n!$$. My interpertation is that the coefficient $$p_k$$ in the PGF expansion directly represents the probability $$P(X = k)$$, which is the probability of the event happening at that point $$k$$.

Is this the correct reason as to why the coefficients in a PGF represent the probability of $$X=k$$ ?

• PS : Taylor Series Connection:

While we are on the topic, I also wanted to solidify my understanding as to how PGF's relate to Taylor series expansion.

In general, a Taylor Series expansion of a function f(x) around a point a is given by:

$$f(x) = f(a) + f'(a)(x-a) + \frac{f''(a)}{2!}(x-a)^2 + \frac{f'''(a)}{3!}(x-a)^3 + ...$$

$$f(x) = \sum_{n=0}^{\infty} \frac{f^{(n)}(a)}{n!}(x-a)^n$$

Keeping this in mind, let's consider the PGF expanded around $$s = 0$$ via Taylor Series:

$$G_X(s) = G_X(0) + G_X'(0)s + \frac{G_X''(0)}{2!}s^2 + \frac{G_X'''(0)}{3!}s^3 + ...$$

$$G_X(s) = \sum_{k=0}^{\infty} \frac{G_X^{(k)}(0)}{k!}s^k$$

Connecting PGF and Probabilities together, we see:

$$p_n = \frac{1}{n!} \left.\frac{d^n}{ds^n} G_X(s)\right|_{s=0} = \frac{G_X^{(n)}(0)}{n!}$$

This means that the coefficient of $$s^k$$ in the Taylor series expansion of $$G_X(s)$$ around $$s = 0$$ is exactly $$p_k$$, the probability that $$X = k$$. It appears that the PGF is essentially a Taylor series expansion around $$s = 0$$, where the coefficients are the probabilities $$p_k$$.

• Bravo! It says we can recover the coef's of a converging power series with non-negative coef's. Commented Aug 1 at 6:43