Latent Dirichlet allocation I am currently trying to understand Blei, Ng and Jordan 2003 JMLR paper "latent Dirichlet allocation".
In section 3 page 997 I don't understand how to get to equation 3. The paper says "integrating over theta and summing over z". How come they push the summation over z after the product of the words? The sum of products and the product of the sums is not equal in general.
What allows, in that case, to push the sum inside the product?
 A: (There is a copy of the paper here.)
It's the distributive law.  Start with equation $(2)$ in the paper, which is 
$$
p(\theta,\,{\bf z},\,{\bf w}\mid \alpha,\,\beta)=
p(\theta\mid\alpha) \prod_{n=1}^N p(z_n\mid \theta)
p(w_n\mid z_n,\,\beta).
$$
Integrating over $\theta$ then gives
$$
p({\bf z},\,{\bf w}\mid \alpha,\,\beta)=\int 
p(\theta\mid\alpha) \prod_{n=1}^N p(z_n\mid \theta)
p(w_n\mid z_n,\,\beta) \, d\theta.
$$
Now, summing over ${\bf z}=(z_1,\ldots,z_N)$ gives
$$
p({\bf w}\mid \alpha,\,\beta)=\int 
p(\theta\mid\alpha) \sum_{z_1,\ldots,z_N} \prod_{n=1}^N p(z_n\mid \theta)
p(w_n\mid z_n,\,\beta) \, d\theta\ \ (*)
$$
and, using the distributive law in reverse, the sum of products can be changed into a product of sums:
\begin{eqnarray*}
&\ & \sum_{z_1,\ldots,z_N} \prod_{n=1}^N p(z_n\mid \theta)p(w_n\mid z_n,\,\beta)\\
&=& \sum_{z_1}\cdots\sum_{z_N} p(z_1\mid\theta)p(w_1\mid z_1,\,\beta)
\cdots p(z_N\mid\theta)p(w_N\mid z_N,\,\beta)\\
&=& \left(\sum_{z_1} p(z_1\mid\theta)p(w_1\mid z_1,\,\beta)\right)
\cdots
\left(\sum_{z_N} p(z_N\mid\theta)p(w_N\mid z_N,\,\beta)\right)\\
&=& \prod_{n=1}^N \sum_{z_n} p(z_n\mid\theta)p(w_n\mid z_n,\,\beta).
\end{eqnarray*}
Substituting this into $(*)$ gives equation $(3)$ in the paper.
