# Covariance of multinomial distribution

Let $X = (X_1,\ldots, X_k)$ be multinomially distributed based upon $n$ trials with parameters $p_1,\ldots,p_k$ such that the sum of the parameters is equal to $1$. I am trying to find, for $i \neq j$, $\operatorname{Var}(X_i + X_j)$. Knowing this will be sufficient to find the $\operatorname{Cov}(X_i,X_j)$.

Now $X_i \sim \text{Bin}(n, p_i)$. The natural thing to say would be that $X_i + X_j\sim \text{Bin}(n, p_i+p_j)$ (and this would, indeed, yield the right result), but I m not sure if this is indeed so.

UPDATE: @grand_chat very nicely answered the question about the distribution of $X_i + X_j$. How would we go about computing the variance of $X_i - X_j$? As @grand_chat correctly points out, this cannot be binomial because it is not guaranteed to be positive. How, then, should one go about computing the variance of this random variable?

• $X=\sum_{i=1}^n Y_i$ where $Y_i$ is the outcome of one draw. Then $cov(X_{i},X_{j})=n\cdot cov(Y_{1,i},Y_{1,j})$ – A.S. Mar 1 '16 at 5:53

$X_i+X_j$ is indeed a binomial variable because it counts the number of trials that land in either bin $i$ or bin $j$. The $n$ trials are independent, and the probability of "success" is $$P(\text{trial lands in i}) + P(\text{trial lands in j}) = p_i+p_j.$$

• Thanks for this concise approach! – Orest Xherija Mar 1 '16 at 8:42
• Let me ask an additional question. Can I say anything about the distribution of $X_i - X_j$? Is this also binomial? – Orest Xherija Mar 1 '16 at 21:40
• $X_i-X_j$ cannot be binomial because it can take negative values. We certainly can say that $X_i-X_j$ is the difference of two correlated binomials, and can calculate its mean and variance. – grand_chat Mar 1 '16 at 21:45
• How would we go about computing the variance? – Orest Xherija Mar 1 '16 at 21:48
• Never mind, I found a relevant result that answers my question. Thanks a lot for your help! – Orest Xherija Mar 1 '16 at 21:56

There are several ways to do this, but one neat proof of the covariance of a multinomial uses the property you mention that $X_i + X_j \sim \text{Bin}(n, p_i + p_j)$ which some people call the "lumping" property

### Covariance in a Multinomial

Given $(X_1,...,X_k) \sim Mult_k(n , \vec{p})$ find $Cov(X_i,X_j)$ for all $i,j$.

\begin{aligned} & If \ i = j, Cov(X_i, X_i) = Var(X_i) = np_i(1 - p_i) \\ \\ & If \ i \neq j, Cov(X_i, X_j) = C \ \ \text{ i.e. what we are trying to find} \\ \\ & Var(X_i + X_j) = Var(X_i) + Var(X_j) + 2Cov(X_i, X_j) \\ \\ & Var(X_i + X_j) = np_i (1 - p_i) + np_j (1 - p_j)+ 2C \\ \\ & \text{By the lumping property } X_i + X_j \sim Bin(n, p_i + p_j) \\ \\ & n(p_i + p_j)(1 - (p_i + p_j)) = np_i(1 - p_i) + np_j(1 - p_j) + 2C \\ & (p_i + p_j)(1 - (p_i + p_j)) = p_i(1 - p_i) + p_j(1 - p_j) + \frac{2C}{n} \\ & C = - n p_i p_j \end{aligned}