I'm reading about Bayesian inference and there is one derivation I don't understand or see (from my book):
$\textbf{x} = (x_1, ..., x_n)$ is $n$-dimensional vector, $X = (\textbf{x}_1, ..., \textbf{x}_m)$ is a set of training examples and $\theta = (\theta_1, ..., \theta_k)$ is a set of parameters. $$p(\textbf{x}\:|\:X) = \int p(\textbf{x}, \theta\:|\:X)\;\text{d}\theta$$
From the definition of conditional probability densities we can then write
$$p(\textbf{x}, \theta \:|\: X) = p(\textbf{x}\:|\:\theta, X)\:p(\theta\:|\:X)$$
The first factor however is independent of $X$ since it is just our assumed form for the parametrized density, and is completely specified once the values of the parameters $\theta$ have been set. We therefore have
$$p(\textbf{x}\:|\:X) = \int p(\textbf{x}\:|\:\theta)\:p(\theta\:|\:X)\;\text{d}\theta$$
Looking carefully at the formula above and assuming that $p(\theta\:|\:X)$ is known , then $p(\textbf{x}\:|\:X)$ is nothing but the average of $p(\textbf{x}\:|\:\theta)$ with respect to $\theta$, that is,
$$p(\textbf{x}\:|\:X) = E_{\theta}[p(\textbf{x}\:|\:\theta)]$$
Somehow this confuses me :( Can someone clarify me in more detail why is the last part true? Is $Y = p(\textbf{x}\:|\:\theta)$ a random variable here now? What does the average with respect to $\theta$ mean ($E_{\theta}$)?
This confuses me, because if I would use the definition of expected value:
$$E(Y) = \int y\:p(y)\;\text{d}y$$
I would deduce in my case that $y = p(\textbf{x}\:|\:\theta)$ and $p(y) = p(p(\textbf{x}\:|\:\theta))$...and now I start to scratch my head x) I would really also want to know what does $E_{\theta}(Y)$ mean? How is expected value of random variable $\alpha$ with respect to variable $\beta$, $E(\alpha)_{\beta}$ defined?
Thank your for any help! =)