# Find the covariances of a multinomial distribution

If $(X_1,\cdots, X_n)$ is a vector with multinomial distribution, proof that $\text{Cov}(X_i,X_j)=-rp_ip_j$, $i\neq j$ where $r$ is the number of trials of the experiment, $p_i$ is the probability of success for the variable $X_i$. $$fdp=f(x_1,...x_n)={r!\over{x_1!x_2!\cdots x_n!}}p_1^{x_1}\cdots p_n^{x_n}$$ if $x_1+x_2+\cdots +x_n=r$

I'm trying to use the property: $\text{Cov}(X_i,X_j)=E[X_iX_j]-E[X_i]E[X_j]$ and find that $E[X_i]=rp_i$, but I don´t know the efficient way to calculate $E[X_iX_j].$

• What is $E(X_iX_j)$ for $r=1$? – A.S. Feb 24 '16 at 1:37
• r=x1+x2+...+xn, if r=1 then x1+x2+...+xn=1 – A P Feb 24 '16 at 1:48
• As what A.S. hinted, one common trick is to express $X_i = \sum_{k=1}^r Y_{i,k}, X_j = \sum_{l=1}^r Y_{j,l}$ and use linearity of covariance. By independence across different multinomial trials, you only left the calculate the case with $Cov[Y_{i,k}, Y_{j, k}]$. But those $Y$ are indicators only (i.e. the $r = 1$ case mentioned by A.S.) which is easy to calculate. – BGM Feb 24 '16 at 3:43

We can use indicator random variables to help simplify the covariance expression. We can interpret the problem as $r$ independent rolls of an $n$ sided die. Let $X_i$ be the number of rolls that result in side $i$ facing up, and let $I_{k}^{(i)}$ be an indicator equal to $1$ when roll $k$ is equal to $i$ and $0$ otherwise. Then, we can express $X_i$ and $X_j$ as follows: \begin{equation} X_i = \sum_{k=1}^{r} I_{k}^{(i)}~~~\mathrm{and}~~~X_j = \sum_{k=1}^{r} I_{k}^{(j)} \end{equation} Let's re-write the covariance using indicators: \begin{equation} \mathrm{Cov}(X_i,X_j) = E[X_i X_j] - E[X_i]E[X_j] \end{equation} Let's compute the first term: \begin{eqnarray} E[X_i X_j] &=& E\bigg[(\sum_{k=1}^{r}I_{k}^{(i)}) (\sum_{l=1}^{r}I_{l}^{(j)})\bigg] = \sum_{k=l}E\big[I_{k}^{(i)}I_{l}^{(j)}\big] + \sum_{k\neq l}E\big[I_{k}^{(i)}I_{l}^{(j)}\big] = \\ &=& 0 + \sum_{k\neq l}E\big[I_{k}^{(i)}\big] E\big[I_{l}^{(j)}\big] = \sum_{k\neq l} p_i p_j = (r^2 - r)p_i p_j \end{eqnarray} where we expanded the product of sums, used linearity of expectation and the fact that when $k=l$ we can't simultaneously roll $i$ and $j$ on the same trial $k=l$ (making the product of indicators zero), finally we applied independence of rolls that enabled us to write it as a product of probabilities. Let's compute the remaining term: \begin{equation} E[X_i] = E[\sum_{k=1}^{r}I_{k}^{(i)}] = \sum_{k=1}^{r}E[I_{k}^{(i)}] = rp_i \end{equation} Therefore, the covariance equals: \begin{equation} \mathrm{Cov}(X_i,X_j) = E[X_i X_j] - E[X_i]E[X_j] = (r^2-r)p_ip_j - r^2p_ip_j = -r p_i p_j \end{equation} Notice that $\mathrm{Cov}(X_i, X_j) = -r p_i p_j < 0$ is negative, this makes sense intuitively since for a fixed number of rolls $r$, if we roll many outcomes $i$, this reduces the number of possible outcomes $j$, and therefore $X_i$ and $X_j$ are negatively correlated!