Deriving Moment Generating Function of the Negative Binomial?

My textbook did the derivation for the binomial distribution, but omitted the derivations for the Negative Binomial Distribution.

I know it is supposed to be similar to the Geometric, but it is not only limited to one success/failure. (i.e the way I understand it is that the negative binomial is the sum of independent geometric random variables). For example: $Y_1 +Y_2 +Y_3+\cdots$ where $Y_i$ is a geometric parameter. I can't seem to find online one for the negative binomial and am having trouble with even doing the geometric.

Can anyone show me a derivation of the negative binomial?

Edit: My book calls the negative binomial as the distribution of the number of trials needed to get a specified number r of successes.

• There are at least two conventions concerning the meaning of the terms "geometric distribution" and "negative binomial distribution". Some books say the negative binomial distribution is the distribution of the number of trials needed to get a specified number $r$ of successes. Others say it's the distribution of the number of failures before $r$ successes. In either case, the geometric distribution is the case where $r=1$. In the latter case, the support is $\{0,1,2,3,\ldots\}$. In the former case, it is $\{r,r+1,r+2,r+3,\ldots\}$. Which do you have in mind here? – Michael Hardy Jun 26 '14 at 15:40
• The advantage of the case where the support is $\{0,1,2,3,\ldots\}$ is that you have an infinitely divisible distribution and $r$ can actually be a non-integer. – Michael Hardy Jun 26 '14 at 15:41
• @MichaelHardy I have edited my post. My book calls it as the distribution of the number of trials needed to get a specified number r of successes. – Junloap Jun 26 '14 at 15:44

The m.g.f. of a sum of independent random variables is just the product of their m.g.f.s, so $$M_{Y_1+\cdots+Y_r}(t) = \left( M_{Y_1}(t) \right)^r.$$ \begin{align} M_{Y_1}(t) & = \operatorname E(e^{tY_1}) = \sum_{y=1}^\infty e^{ty} \Pr(Y_1=y) = \sum_{y=1}^\infty e^{ty} p(1-p)^{y-1} = \frac{p}{1-p} \sum_{y=1}^\infty \left(e^t(1-p)\right)^y \\[10pt] & = \frac{p}{1-p} \cdot \frac{\text{first term}}{1-\text{common ratio}} = \frac{p}{1-p}\cdot\frac{e^t(1-p)}{1-e^t(1-p)} = \frac{e^tp}{1-e^t(1-p)}. \end{align}

(Then remember to raise the whole thing to the power $r$.)

• Thanks but my MGF is: [(pe^t)/(1-(1-p)e^t)]^r. Is there a reason why there is no e^t? It seems like you have t denoted as e^t. Can you explain why? – Junloap Jun 26 '14 at 16:04
• @Junloap : I remain mystified by your comment. What you say is your MFG agrees with what my answer says. As for "why there is no e^t", you need to be specific. Obviously $e^t$ occurs in my answer. – Michael Hardy Nov 17 '17 at 17:24

To derive the mgf of the negative binomial distribution we are going to use the following identity:

$$\binom{-r}{y}=\left( -1 \right)^y \binom{r+y-1}{y}$$

We can prove that in the following way:

\begin{align} \binom{-r}{y} & = \frac{ \left( -r \right) \left(-r-1 \right) \ldots \left(-r-y+1 \right)}{y!}\\ & = \left(-1 \right)^y \frac{ \left(r+y-1 \right) \ldots \left( r+1 \right)r}{y!} \\ & = \left(-1 \right)^y \binom{r+y-1}{y} \end{align}

Now

$$M \left( t \right)=\sum_{y=0}^{\infty} e^{ty} \binom{y+r-1}{r-1} \left( 1-p \right)^y \times p^r$$

Grouping terms and using the above idenity we get:

\begin{align} M \left( t \right)& =p^r \sum_{y=0}^{\infty} \binom{y+r-1}{r-1} \left[ e^t \left( 1-p \right) \right]^y \\&=p^r\sum_{y=0}^{\infty} \binom{-r}{y}\left( -1 \right)^y\left[ e^t \left( 1-p \right) \right]^y \\& =p^r\sum_{y=0}^{\infty} \binom{-r}{y}\left[ -e^t \left( 1-p \right) \right]^y \end{align}

Then using Newton's Binomial Theorem: $\left( x+1 \right)^r= \sum_{i=0}^\infty {r\choose i}x^i$ provided that $|x|<1$, the last term becomes:

$$M \left(t \right)= \frac{p^r}{\left[ 1- \left(1-p \right)e^t \right]^r}$$

provided that $t<-\log(1-p)$

Note that the negative binomial distributioon can come with a slightly different parameterization as well, as it has been pointed out in the comments. I leave it to you to derive the mgf for the other case.

Hope this helps.

• Can you explain what that identity is? I have never seen that before. – Junloap Jun 26 '14 at 15:54
• @Junloap Sure I have edited my post to include a proof of that as well. – JohnK Jun 26 '14 at 16:03
• thanks for the insight into the other way of proving this! – Junloap Jun 26 '14 at 16:09
• @Junloap My pleasure. You should remember these binomial coefficient idenities as they occur freqnuetly. – JohnK Jun 26 '14 at 16:13
• They do occur a lot and are definitely worth knowing, but this m.g.f. can be derived without them. – Michael Hardy Jun 26 '14 at 19:09