My question is pretty easy to state, but I can't make any progress on it. Suppose that ${\bf X}$ is multinomially distributed, i.e., ${\bf X} = [X_1,\dots,X_K]\sim \text{Mult}(N,{\bf p})$ such that $\sum_{k=1}^d X_k = N$ for ${\bf p}\in R_+^d$ and $\sum_{i=1}^d p_i = 1$. Then my question is how is $Z_j = \sum_{k=1}^K \chi \{X_k = j\}$ distributed?

This new random variables tells us how many of the entries of ${\bf X}$ are equal to $j$.

[Note that the $\chi$ is meant to represent the indicator function.]

To make things easier, if necessary, we can assume that the vector of probabilities for the multinomial distribution has equal entries, i.e., $p_i = 1/d$ for all $i=1,\dots,d$.

Best, Jake

  • $\begingroup$ Are $d$ and $K$ the same number? And I think $p\in [0,1]^K\subset \mathbb{R}_+^K$? $\endgroup$ – Sheljohn Oct 26 '17 at 11:36

Let $\{X_1, \dots, X_n\}$ be a collection of $n$ i.i.d. multinomial random variables. Then $p_{j} := \Pr(X_1 = j) = \cdots = \Pr(X_n = j)$. Then,

$\Pr(Z_j = k) = {n \choose k}p_j^k (1 - p_j)^{n-k}$. Plug in the p.d.f. of $p_j$ to get the final distribution.

  • $\begingroup$ Could you please explain in a little more detail? How can the entries of a multinomial be independent (i.i.d)? $\endgroup$ – Jake P. Taylor-King Oct 24 '17 at 22:40
  • $\begingroup$ Also, do you mean little $n$ to actually be $k$ in your notation above? $\endgroup$ – Jake P. Taylor-King Oct 24 '17 at 22:41
  • $\begingroup$ @rodms I don't think your formula for $\mathrm{Pr}( Z_j = k )$ is correct. $\endgroup$ – Sheljohn Oct 26 '17 at 8:35

To expand on @rodms answer, and explain what is going on, it might be useful to forget about probabilities for a minute.

The multinomial distribution can be seen as a tool to study processes that partition a finite set into a fixed number of subsets. If you are familiar with algebra, it helps to think about the distribution of degrees in the monomials of multivariate polynomials (the consonance is not a coincidence). If you are familiar with analysis, think about the derived terms in the multivariate Taylor expansion.

From these analogies, it should be clear that the number of ways to obtain $j$ realisations of a particular event amongst $N$ trials is $\binom{N}{j}$.

Now going back to the probabilities, in order for this particular outcome to happen, we need some event $k$ to happen exactly $j$ times, for which the probability is $p_k^j (1-p_k)^{N-j}$.

Hence the formula that @rodms gave you. But what you are asking goes one step further than that. There are many ways that $Z_j$ can be equal to any number $m$. To clarify: $\mathrm{Pr}(\ Z_j = m\ )$ is the probability that exactly $m$ events occur $j$ times.

In order to obtain this formula, there is a second combinatorial layer; namely the combinations of $m$ events that would occur $j$ time, of which there are exactly $\binom{K}{m}$. Therefore, $\mathrm{Pr}(\ Z_j = m\ )$ is given by the sum over these combinations that the selected $m$ events jointly occur $j$ time.

For any such combination of $m$ events with indices $i_1\neq ...\neq i_m$, the probability that $(X_{i_1} = j\ \text{ and }\ X_{i_2} = j\ ...\ \text{ and }\ X_{i_m} = j)$ if $mj\leq N$ is given by: $$ \binom{N}{mj} \pi_{(i_1, ... , i_m)}^j (1- \sigma_{(i_1, ... , i_m)})^{N-mj} $$ where $\pi_{(i_1, ... , i_m)} = \prod_{k=1}^m p_{i_k}$ and $\sigma_{(i_1, ... , i_m)} = \sum_{k=1}^m p_{i_k}$. This yields the (rather ugly) formula: $$ \mathrm{Pr}(\ Z_j = m\ ) = \sum_{i_1\neq ...\neq i_m} \binom{N}{mj} \pi_{(i_1, ... , i_m)}^j (1- \sigma_{(i_1, ... , i_m)})^{N-mj} $$ where this sums iterates over the $\binom{K}{m}$ combinations mentioned previously.

If we further assume all $p_i$ equal, then this simplifies to: $$ \mathrm{Pr}(\ Z_j = m\ ) = \binom{K}{m} \binom{N}{mj} p^{mj} (1- mp)^{N-mj} $$


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.