0
$\begingroup$

I have found the below equation from this paper and trying to see how it is true.

\begin{equation} p\left(\hat{\boldsymbol{y}}_{i} | \mathbf{x}_{i}, \mathcal{D}\right)=\int_{\theta} p\left(\hat{\boldsymbol{y}}_{i} | \mathbf{x}_{i}, \mathcal{D}, \boldsymbol{\theta}\right) p(\boldsymbol{\theta} | \mathcal{D}) d \boldsymbol{\theta} \end{equation}

\begin{equation} \hat{\boldsymbol{y}}_{i} = output, D = dataset, \theta = parameters, \mathbf{x}_{i} = inputs \end{equation}

I tried the following to prove:

Using the sum rule \begin{equation} p\left(\hat{\boldsymbol{y}}_{i} | \mathbf{x}_{i}, \mathcal{D}\right)= \int_{\theta} p\left(\hat{\boldsymbol{y}}_{i},\boldsymbol{\theta}\right | \mathbf{x}_{i}, \mathcal{D}) d\theta\end{equation}

Using the product rule

\begin{equation} p\left(\hat{\boldsymbol{y}}_{i} | \mathbf{x}_{i}, \mathcal{D}\right)=\int_{\theta} p\left(\hat{\boldsymbol{y}}_{i} | \theta ,\mathbf{x}_{i}, \mathcal{D}\right) p(\boldsymbol{\theta} | \mathbf{x}_{i}, \mathcal{D}) d \boldsymbol{\theta} \end{equation}

assuming $\theta$ is independent of $\mathbf{x}_{i}$ we can rewrite it as:

\begin{equation} p\left(\hat{\boldsymbol{y}}_{i} | \mathbf{x}_{i}, \mathcal{D}\right)=\int_{\theta} p\left(\hat{\boldsymbol{y}}_{i} | \theta ,\mathbf{x}_{i}, \mathcal{D}\right) p(\boldsymbol{\theta} | \mathcal{D}) d \boldsymbol{\theta} \end{equation}

Does the above derivation right? If so, how can the parameters to the Neural network be independent of the inputs? Is it safe to assume?

$\endgroup$
0
$\begingroup$

During inference stage /theta is independent of data points i.e x. Therefore, it is safe to remove that from the condition

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.