Could anyone guide me to a document where they derive the distribution of the difference between two binomial random variables. So $X \sim \mathrm{Bin}(n_1, p_1) $ and $Y \sim \mathrm{Bin}(n_2, p_2) $, what is the distribution of $|X-Y|$.

thank you.

(Also $X$ and $Y$ are independent)

up vote 7 down vote accepted

This question is more tricky than it sounds. To solve it, I will use here a combination of both manual methods and automated methods, in particular computer algebra tools [the mathStatica package (of which I am an author) for Mathematica and the latter itself].

If I may change the notation slightly:

The Problem

Let $X_1$ ~ $\text{Binomial}(n,p)$ and $X_2$ ~ $\text{Binomial}(m,q)$ be independent.

Find the pmf of $|X_1-X_2|$

Given: Due to independence, the joint pmf of $(X_1, X_2)$, say $f(x_1,x_2)$, is:

enter image description here

Solution

Let $Y=X_1-X_2$ and $Z=X_2$. Then, the joint pmf of $(Y,Z)$, say $g(y,z)$, is:

enter image description here

where Transform is a mathStatica function that derives the joint pmf using the Method of Transformations. Deriving the domain of support of $Y$ and $Z$ is a bit more tricky. To make things clearer, here is a rough plot that illustrates the (smoothed continuous version of) the domain of support:

enter image description here

This suggests two cases:

  • Case 1: When $y \ge 0$: $0 \le z \le n-y$

  • Case 2: When $y < 0$: $-y \le z \le m$

The density of $Y=X_1-X_2$ is then obtained by summing out $Z$ in each part of the domain:

enter image description here

Finally, to find the pmf of $|Y|$, the pmf for strictly positive values will be:

enter image description here

and when $Y=0$:

enter image description here

Summary

The pmf of $|X_1-X_2|$, say $\phi(y)$ is:

enter image description here

with domain of support $Y$ = {0, 1, ..., max$(m,n)$}.

All done.


Monte Carlo check

It is always a good idea to check ones work using Monte Carlo methods. Here, for instance, are 100,000 pseudo-random drawings from each of $X_1$ and $X_2$, given some parameter assumptions:

x1data = RandomVariate[BinomialDistribution[12, .1],  100000];
x2data = RandomVariate[BinomialDistribution[ 7, .9],  100000];

Next, compare the empirical distribution of $|X_1-X_2|$ (red triangles) to the theoretical density $\phi(y)$ (blue dots) derived above, given the same parameter assumptions:

enter image description here

Looks good :)

  • For the special case in which $p=q=1/2$ and $n=m$, we get: $Pr[Y=y] = p^{2n} \cdot \text{Binomial}[n,y] \cdot \text{Hypergeometric2F1}[-n,-n+y,1+y,1]$ – Erel Segal-Halevi Feb 3 '15 at 21:11

I doubt there is a special name for the distribution in general. There is one special case of interest: $p_1 = 1 - p_2$. Note that $n_2 - Y \sim {\text Bin}(n_2, 1-p_2)$, and so in this special case $X - Y + n_2 \sim {\text Bin}(n_1 + n_2, p_1)$.

  • This explains the distribution of X−Y (in the special case). How do you get from it to the distribution of |X-Y|? – Erel Segal-Halevi Feb 4 '15 at 6:17
  • 1
    $P(|X-Y|=z) = P(X-Y=z) + P(X-Y=-z)$ for nonnegative integers $z$. – Robert Israel Feb 4 '15 at 6:29

I can give you an answer for the pmf of X-Y. From there |X - Y| is straightforward.

So we start with

$X \sim Bin(n_1, p_1)$

$Y \sim Bin(n_2, p_2)$

We are looking for the probability mass function of $Z=X-Y$

First note that the min and max of the support of Z must be $(-n_2, n_1)$ since that covers the most extreme cases ($X=0$ and $Y=n_2$) and ($X=n_1$ and $Y=0$).

Then we need a modification of the binomial pmf so that it can cope with values outside of its support.

$m(k, n, p) = \binom {n} {k} p^k (1-p)^{n-k}$ when $k \leq n$ and 0 otherwise.

Then we need to define two cases

  1. $Z \geq 0$
  2. $Z \lt 0$

In the first case

$p(z) = \sum_{i=0}^{n_1} m(i+z, n_1, p_1) m(i, n_2, p_2)$

since this covers all the ways in which X-Y could equal z. For example when z=1 this is reached when X=1 and Y=0 and X=2 and Y=1 and X=3 and Y=4 and so on. It also deals with cases that could not happen because of the values of $n_1$ and $n_2$. For example if $n_2 = 4$ then we cannot get Z=1 as a combination of X=4 and Y=5. In this case thanks to our modified binomial pmf the probablity is zero.

For the second case we just reverse the roles. For example if z=-1 then this is reached when X=0 and Y=1, X=1 and Y=2 etc.

$p(z) = \sum_{i=0}^{n_2} m(i, n_1, p_1) m(i+z, n_2, p_2)$

Put them together and that's your pmf.

$ f(z)= \begin{cases} \sum_{i=0}^{n_1} m(i+z, n_1, p_1) m(i, n_2, p_2),& \text{if } z\geq 0\\ \sum_{i=0}^{n_2} m(i, n_1, p_1) m(i+z, n_2, p_2), & \text{otherwise} \end{cases} $

Here's the function in R and a simulation to check it's right (and it does work.)

https://gist.github.com/coppeliaMLA/9681819

  • A great answer from Simon. Can you tell me the inverse cumulative for X - Y? – sjb-sjb Sep 19 at 22:48

Your Answer

 

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Not the answer you're looking for? Browse other questions tagged or ask your own question.