Let $X = (X_1,\ldots, X_k)$ be multinomially distributed based upon $n$ trials with parameters $p_1,\ldots,p_k$ such that the sum of the parameters is equal to $1$. I am trying to find, for $i \neq j$, $\operatorname{Var}(X_i + X_j)$. Knowing this will be sufficient to find the $\operatorname{Cov}(X_i,X_j)$.

Now $X_i \sim \text{Bin}(n, p_i)$. The natural thing to say would be that $X_i + X_j\sim \text{Bin}(n, p_i+p_j)$ (and this would, indeed, yield the right result), but I m not sure if this is indeed so.

Suggestions for how to go about this are greatly appreciated!

UPDATE: @grand_chat very nicely answered the question about the distribution of $X_i + X_j$. How would we go about computing the variance of $X_i - X_j$? As @grand_chat correctly points out, this cannot be binomial because it is not guaranteed to be positive. How, then, should one go about computing the variance of this random variable?

UPDATE 2: The answer in this link answers the question in my UPDATE.

  • $\begingroup$ $X=\sum_{i=1}^n Y_i$ where $Y_i$ is the outcome of one draw. Then $cov(X_{i},X_{j})=n\cdot cov(Y_{1,i},Y_{1,j})$ $\endgroup$ – A.S. Mar 1 '16 at 5:53

$X_i+X_j$ is indeed a binomial variable because it counts the number of trials that land in either bin $i$ or bin $j$. The $n$ trials are independent, and the probability of "success" is $$P(\text{trial lands in $i$}) + P(\text{trial lands in $j$}) = p_i+p_j.$$

  • $\begingroup$ Thanks for this concise approach! $\endgroup$ – Orest Xherija Mar 1 '16 at 8:42
  • $\begingroup$ Let me ask an additional question. Can I say anything about the distribution of $X_i - X_j$? Is this also binomial? $\endgroup$ – Orest Xherija Mar 1 '16 at 21:40
  • $\begingroup$ $X_i-X_j$ cannot be binomial because it can take negative values. We certainly can say that $X_i-X_j$ is the difference of two correlated binomials, and can calculate its mean and variance. $\endgroup$ – grand_chat Mar 1 '16 at 21:45
  • $\begingroup$ How would we go about computing the variance? $\endgroup$ – Orest Xherija Mar 1 '16 at 21:48
  • $\begingroup$ Never mind, I found a relevant result that answers my question. Thanks a lot for your help! $\endgroup$ – Orest Xherija Mar 1 '16 at 21:56

There are several ways to do this, but one neat proof of the covariance of a multinomial uses the property you mention that $X_i + X_j \sim \text{Bin}(n, p_i + p_j)$ which some people call the "lumping" property

Covariance in a Multinomial

Given $(X_1,...,X_k) \sim Mult_k(n , \vec{p})$ find $Cov(X_i,X_j)$ for all $i,j$.

\begin{aligned} & If \ i = j, Cov(X_i, X_i) = Var(X_i) = np_i(1 - p_i) \\ \\ & If \ i \neq j, Cov(X_i, X_j) = C \ \ \text{ i.e. what we are trying to find} \\ \\ & Var(X_i + X_j) = Var(X_i) + Var(X_j) + 2Cov(X_i, X_j) \\ \\ & Var(X_i + X_j) = np_i (1 - p_i) + np_j (1 - p_j)+ 2C \\ \\ & \text{By the lumping property } X_i + X_j \sim Bin(n, p_i + p_j) \\ \\ & n(p_i + p_j)(1 - (p_i + p_j)) = np_i(1 - p_i) + np_j(1 - p_j) + 2C \\ & (p_i + p_j)(1 - (p_i + p_j)) = p_i(1 - p_i) + p_j(1 - p_j) + \frac{2C}{n} \\ & C = - n p_i p_j \end{aligned}


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.