Why do I need Fubini to evaluate these integrals of products of functions? I am currently reading a book on applied mathematics, and the current chapter is proving some statements related to probability theory. I'm stuck with one step in a derivation, because the author claims to use Fubini('s theorem), but I do not see why he would need it.
Concretely, consider continuous random variables $X, Y$ on $\mathcal{X}$ with probability densities $p(x), q(y)$, respectively, as well as a function $\phi:\mathcal{X}\rightarrow\mathbb{R}^N$. The steps in question are $$\mathbb{E}_p[\phi(X)^T]\mathbb{E}_p[\phi(X)]=\int\phi(x)^Tp(x)dx\int\phi(x')p(x')dx'=\int\int\phi(x)^T\phi(x')p(x)p(x')dxdx'$$ and
$$\mathbb{E}_p[\phi(X)^T]\mathbb{E}_q[\phi(Y)]=\int\phi(x)^Tp(x)dx\int\phi(y)q(y)dy=\int\phi(x)^T\phi(y)p(x)q(y)dxdy$$
I can not figure out why there would be any need to use Fubini in either step. What's more, I think that the order of integration wouldn't matter either way, since according to my understanding I can just do (e.g. in the second case):
$$\int\phi(x)^Tp(x)dx\int\phi(y)q(y)dy=\int\left(\int\phi(y)q(y)dy\right)\phi(x)^Tp(x)dx=\int\left(\int\phi(y)q(y)\phi(x)^Tp(x)dy\right)dx$$ where in the first step I used that the integral in parentheses exists and hence has a constant value, and in the second step that $\phi(x)^Tp(x)$ is constant with respect to $y$. Analogously, $$\int\phi(x)^Tp(x)dx\int\phi(y)q(y)dy=\int\left(\int\phi(x)^Tp(x)dx\right)\phi(y)q(y)dy=\int\left(\int\phi(y)q(y)\phi(x)^Tp(x)dx\right)dy$$
Which means the order of integration does not matter. (I understand that this would be different if the integrand were e.g. a function $h(x,y)$ for which I wouldn't just have $h(x,y)=f(x)g(y)$ for some appropriate $f,g$, and then I would need Fubini.
Does anybody see why the author would write that one needs Fubini?
 A: An important fact about integrals that is sometimes glossed over in calculus courses is that, in general, an iterated integral is not the same thing as an integral over a product domain. Fubini's theorem gives us conditions under which they are equal. Changing the order of integration is really just a byproduct of this subtler idea.
You are actually applying Fubini in your argument for why it's unnecessary. This may be a little clearer if we work in the other direction, that is, start with the integral over the product domain and move to the product of integrals.
Let $f(x,y)=g(x)h(y)$, where $g$ and $h$ are integrable on the domains $\Omega_1$ and $\Omega_2$ respectively, and $\iint_{\Omega_1\times \Omega_2} |f(x,y)| dA <\infty$. Then by Fubini we have the following:
\begin{align}
\iint_{\Omega_1\times \Omega_2} f(x,y) dA  &= \iint_{\Omega_1\times \Omega_2} g(x)h(y) dA \\
  &=  \int_{\Omega_2} \left[ \int_{\Omega_1} g(x) dx \right] h(y)dy
\end{align}
We need Fubini here because the integral over the product $\Omega_1 \times \Omega_2$ is not in general the same as an iterated integral over $\Omega_1$ and $\Omega_2$ separately. Both results you've asked about are just using Fubini to justify that the double integral equals the iterated integral.
