# If ${\bf X}$ is multinomially distributed, then how is $Z_j = \sum_{i} \chi \{X_k = j\}$ distributed?

My question is pretty easy to state, but I can't make any progress on it. Suppose that ${\bf X}$ is multinomially distributed, i.e., ${\bf X} = [X_1,\dots,X_K]\sim \text{Mult}(N,{\bf p})$ such that $\sum_{k=1}^d X_k = N$ for ${\bf p}\in R_+^d$ and $\sum_{i=1}^d p_i = 1$. Then my question is how is $Z_j = \sum_{k=1}^K \chi \{X_k = j\}$ distributed?

This new random variables tells us how many of the entries of ${\bf X}$ are equal to $j$.

[Note that the $\chi$ is meant to represent the indicator function.]

To make things easier, if necessary, we can assume that the vector of probabilities for the multinomial distribution has equal entries, i.e., $p_i = 1/d$ for all $i=1,\dots,d$.

Best, Jake

• Are $d$ and $K$ the same number? And I think $p\in [0,1]^K\subset \mathbb{R}_+^K$? – Sheljohn Oct 26 '17 at 11:36

Let $\{X_1, \dots, X_n\}$ be a collection of $n$ i.i.d. multinomial random variables. Then $p_{j} := \Pr(X_1 = j) = \cdots = \Pr(X_n = j)$. Then,

$\Pr(Z_j = k) = {n \choose k}p_j^k (1 - p_j)^{n-k}$. Plug in the p.d.f. of $p_j$ to get the final distribution.

• Could you please explain in a little more detail? How can the entries of a multinomial be independent (i.i.d)? – Jake P. Taylor-King Oct 24 '17 at 22:40
• Also, do you mean little $n$ to actually be $k$ in your notation above? – Jake P. Taylor-King Oct 24 '17 at 22:41
• @rodms I don't think your formula for $\mathrm{Pr}( Z_j = k )$ is correct. – Sheljohn Oct 26 '17 at 8:35

To expand on @rodms answer, and explain what is going on, it might be useful to forget about probabilities for a minute.

The multinomial distribution can be seen as a tool to study processes that partition a finite set into a fixed number of subsets. If you are familiar with algebra, it helps to think about the distribution of degrees in the monomials of multivariate polynomials (the consonance is not a coincidence). If you are familiar with analysis, think about the derived terms in the multivariate Taylor expansion.

From these analogies, it should be clear that the number of ways to obtain $j$ realisations of a particular event amongst $N$ trials is $\binom{N}{j}$.

Now going back to the probabilities, in order for this particular outcome to happen, we need some event $k$ to happen exactly $j$ times, for which the probability is $p_k^j (1-p_k)^{N-j}$.

Hence the formula that @rodms gave you. But what you are asking goes one step further than that. There are many ways that $Z_j$ can be equal to any number $m$. To clarify: $\mathrm{Pr}(\ Z_j = m\ )$ is the probability that exactly $m$ events occur $j$ times.

In order to obtain this formula, there is a second combinatorial layer; namely the combinations of $m$ events that would occur $j$ time, of which there are exactly $\binom{K}{m}$. Therefore, $\mathrm{Pr}(\ Z_j = m\ )$ is given by the sum over these combinations that the selected $m$ events jointly occur $j$ time.

For any such combination of $m$ events with indices $i_1\neq ...\neq i_m$, the probability that $(X_{i_1} = j\ \text{ and }\ X_{i_2} = j\ ...\ \text{ and }\ X_{i_m} = j)$ if $mj\leq N$ is given by: $$\binom{N}{mj} \pi_{(i_1, ... , i_m)}^j (1- \sigma_{(i_1, ... , i_m)})^{N-mj}$$ where $\pi_{(i_1, ... , i_m)} = \prod_{k=1}^m p_{i_k}$ and $\sigma_{(i_1, ... , i_m)} = \sum_{k=1}^m p_{i_k}$. This yields the (rather ugly) formula: $$\mathrm{Pr}(\ Z_j = m\ ) = \sum_{i_1\neq ...\neq i_m} \binom{N}{mj} \pi_{(i_1, ... , i_m)}^j (1- \sigma_{(i_1, ... , i_m)})^{N-mj}$$ where this sums iterates over the $\binom{K}{m}$ combinations mentioned previously.

If we further assume all $p_i$ equal, then this simplifies to: $$\mathrm{Pr}(\ Z_j = m\ ) = \binom{K}{m} \binom{N}{mj} p^{mj} (1- mp)^{N-mj}$$