Using probability rules how is the following equation is true

Question

I have found the below equation from this paper and trying to see how it is true.

\begin{equation} p\left(\hat{\boldsymbol{y}}_{i} | \mathbf{x}_{i}, \mathcal{D}\right)=\int_{\theta} p\left(\hat{\boldsymbol{y}}_{i} | \mathbf{x}_{i}, \mathcal{D}, \boldsymbol{\theta}\right) p(\boldsymbol{\theta} | \mathcal{D}) d \boldsymbol{\theta} \end{equation}

\begin{equation} \hat{\boldsymbol{y}}_{i} = output, D = dataset, \theta = parameters, \mathbf{x}_{i} = inputs \end{equation}

I tried the following to prove:

Using the sum rule \begin{equation} p\left(\hat{\boldsymbol{y}}_{i} | \mathbf{x}_{i}, \mathcal{D}\right)= \int_{\theta} p\left(\hat{\boldsymbol{y}}_{i},\boldsymbol{\theta}\right | \mathbf{x}_{i}, \mathcal{D}) d\theta\end{equation}

Using the product rule

\begin{equation} p\left(\hat{\boldsymbol{y}}_{i} | \mathbf{x}_{i}, \mathcal{D}\right)=\int_{\theta} p\left(\hat{\boldsymbol{y}}_{i} | \theta ,\mathbf{x}_{i}, \mathcal{D}\right) p(\boldsymbol{\theta} | \mathbf{x}_{i}, \mathcal{D}) d \boldsymbol{\theta} \end{equation}

assuming $\theta$ is independent of $\mathbf{x}_{i}$ we can rewrite it as:

\begin{equation} p\left(\hat{\boldsymbol{y}}_{i} | \mathbf{x}_{i}, \mathcal{D}\right)=\int_{\theta} p\left(\hat{\boldsymbol{y}}_{i} | \theta ,\mathbf{x}_{i}, \mathcal{D}\right) p(\boldsymbol{\theta} | \mathcal{D}) d \boldsymbol{\theta} \end{equation}

Does the above derivation right? If so, how can the parameters to the Neural network be independent of the inputs? Is it safe to assume?

Answer 1

During inference stage /theta is independent of data points i.e x. Therefore, it is safe to remove that from the condition