5
$\begingroup$

Let $X = (X_1,\ldots, X_k)$ be multinomially distributed based upon $n$ trials with parameters $p_1,\ldots,p_k$ such that the sum of the parameters is equal to $1$. I am trying to find, for $i \neq j$, $\operatorname{Var}(X_i + X_j)$. Knowing this will be sufficient to find the $\operatorname{Cov}(X_i,X_j)$.

Now $X_i \sim \text{Bin}(n, p_i)$. The natural thing to say would be that $X_i + X_j\sim \text{Bin}(n, p_i+p_j)$ (and this would, indeed, yield the right result), but I m not sure if this is indeed so.

Suggestions for how to go about this are greatly appreciated!

UPDATE: @grand_chat very nicely answered the question about the distribution of $X_i + X_j$. How would we go about computing the variance of $X_i - X_j$? As @grand_chat correctly points out, this cannot be binomial because it is not guaranteed to be positive. How, then, should one go about computing the variance of this random variable?

UPDATE 2: The answer in this link answers the question in my UPDATE.

$\endgroup$
1
  • $\begingroup$ $X=\sum_{i=1}^n Y_i$ where $Y_i$ is the outcome of one draw. Then $cov(X_{i},X_{j})=n\cdot cov(Y_{1,i},Y_{1,j})$ $\endgroup$
    – A.S.
    Mar 1, 2016 at 5:53

2 Answers 2

4
$\begingroup$

$X_i+X_j$ is indeed a binomial variable because it counts the number of trials that land in either bin $i$ or bin $j$. The $n$ trials are independent, and the probability of "success" is $$P(\text{trial lands in $i$}) + P(\text{trial lands in $j$}) = p_i+p_j.$$

$\endgroup$
5
  • $\begingroup$ Thanks for this concise approach! $\endgroup$ Mar 1, 2016 at 8:42
  • $\begingroup$ Let me ask an additional question. Can I say anything about the distribution of $X_i - X_j$? Is this also binomial? $\endgroup$ Mar 1, 2016 at 21:40
  • $\begingroup$ $X_i-X_j$ cannot be binomial because it can take negative values. We certainly can say that $X_i-X_j$ is the difference of two correlated binomials, and can calculate its mean and variance. $\endgroup$
    – grand_chat
    Mar 1, 2016 at 21:45
  • $\begingroup$ How would we go about computing the variance? $\endgroup$ Mar 1, 2016 at 21:48
  • $\begingroup$ Never mind, I found a relevant result that answers my question. Thanks a lot for your help! $\endgroup$ Mar 1, 2016 at 21:56
4
$\begingroup$

There are several ways to do this, but one neat proof of the covariance of a multinomial uses the property you mention that $X_i + X_j \sim \text{Bin}(n, p_i + p_j)$ which some people call the "lumping" property

Covariance in a Multinomial

Given $(X_1,...,X_k) \sim Mult_k(n , \vec{p})$ find $Cov(X_i,X_j)$ for all $i,j$.

\begin{aligned} & If \ i = j, Cov(X_i, X_i) = Var(X_i) = np_i(1 - p_i) \\ \\ & If \ i \neq j, Cov(X_i, X_j) = C \ \ \text{ i.e. what we are trying to find} \\ \\ & Var(X_i + X_j) = Var(X_i) + Var(X_j) + 2Cov(X_i, X_j) \\ \\ & Var(X_i + X_j) = np_i (1 - p_i) + np_j (1 - p_j)+ 2C \\ \\ & \text{By the lumping property } X_i + X_j \sim Bin(n, p_i + p_j) \\ \\ & n(p_i + p_j)(1 - (p_i + p_j)) = np_i(1 - p_i) + np_j(1 - p_j) + 2C \\ & (p_i + p_j)(1 - (p_i + p_j)) = p_i(1 - p_i) + p_j(1 - p_j) + \frac{2C}{n} \\ & C = - n p_i p_j \end{aligned}

$\endgroup$
2
  • $\begingroup$ The "lumping" property holds if the variables are independent right? But in the case of the multinomial $X_i$ and $X_j$ are not independent. Why does this still hold? $\endgroup$ Jun 3, 2020 at 22:01
  • $\begingroup$ @N.Pullbacki it does hold because $X_{i} + X_{j}$ represents all results that match with the condition given with probability $p_i + p_j$. They are in fact not independent. $\endgroup$
    – cserpell
    Nov 17, 2022 at 21:29

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .