Interchanging expectations of log likelihood I see in papers (here in eq. 3 or here on page 4, for example) that it can be done like this using Fubini
$$\mathbb{E}_x\mathbb{E}_\theta\log f(x|\theta)=\mathbb{E}_\theta\mathbb{E}_x\log f(x|\theta)$$
Where $f(x|\theta)$ is a conditional probability density.
I don't immediately see why $\mathbb{E}_\theta\mathbb{E}_x|\log f(x|\theta)|\leq\infty$. Is it actually the case? Or are some additional conditions on $f$ needed?
 A: The identities
$$\mathbb{E}_x\mathbb{E}_\theta \bigl[{\bf 1}_{\{f(x|\theta)>1\}}\log f(x|\theta)  \bigr]=\mathbb{E}_\theta\mathbb{E}_x\bigl[{\bf 1}_{\{f(x|\theta)>1\}}\log f(x|\theta)  \bigr]\quad \; \; (*)$$
and
$$\mathbb{E}_x\mathbb{E}_\theta \bigl[{\bf 1}_{\{f(x|\theta)\le 1\}}\log f(x|\theta)  \bigr]=\mathbb{E}_\theta\mathbb{E}_x\bigl[{\bf 1}_{\{f(x|\theta)\le 1\}}\log f(x|\theta)  \bigr]\quad \; \; (**)$$
always hold by Tonelli's_theorem [1]. Adding them will give the desired formula,  provided at least one of them is finite.
On the other hand, if $(*)$ yields $+\infty$ and  $(**)$ yields $-\infty$, then the original formula might be undefined. For example,
consider the densities
$$f(x|\theta)=\frac{C}{|x-\theta |\, [1+(\log |x-\theta|)^2]} \,,$$
where $C$ is chosen to ensure $\int f(x|\theta) \, dx=1$.
(One can give an even simpler degenerate example, replacing $\theta$ on the RHS of  this formula by $0$.)
To verify the example, the integration fact needed is that $$\int_1^\infty \frac{dx}{x (1 +(\log  x)^\beta)} $$
is finite for $\beta>1$ but infinite for $\beta=1$. The change of variable $u =\log x$ transforms this to the standard integrals
$$\int_0^\infty \frac{du}{  (1 +u^\beta )} \,. $$
Similarly,
$$\int_0^1 \frac{dx}{x (1 +|\log x|^\beta )} $$
is finite for $\beta>1$ but infinite for $\beta=1$.
[1]
https://en.wikipedia.org/wiki/Fubini%27s_theorem#Tonelli's_theorem_for_non-negative_measurable_functions
