Consider the following model:

$$\mathbf{X}=(X_1,...,X_p)|\theta \sim (1-\theta)N_p (\mathbf{0},\mathbf{\Sigma})+\theta N_p (\mathbf{\delta}, \mathbf{\Sigma})$$

$$\theta \sim \textrm{Bernoulli}(1-\pi)$$

for some known $\mathbf{\delta}$, $\mathbf{\Sigma}$ and $\pi$. Show that the Bayes estimator of $\theta$ is given by

$$\frac{(1-\pi) e^{\alpha + \Sigma_{i=1}^p \beta_i X_i}}{\pi + (1-\pi) e^{\alpha + \Sigma_{i=1}^p \beta_i X_i}}$$

for some constants $\alpha$ and $\beta_i, i=1,...,p$. Give formulas for $\alpha$ and $\beta_i$.

My idea:

The posterior distribution of $\theta$ given the data $\mathbf{X}$ is proportional to the product of the likelihood and the prior:

$$\pi(\theta | \mathbf{X}) \propto f(\mathbf{X} | \theta) \pi(\theta),$$

where $$f(\mathbf{X} | \theta) = (1-\theta)N_p (\mathbf{0},\mathbf{\Sigma})+\theta N_p(\mathbf{\delta}, \mathbf{\Sigma})$$ is the likelihood function, and $$\pi(\theta) = (1-\pi)^\theta \pi^{1-\theta}$$ is the prior distribution.

Now, I can calculate the posterior distribution:

$$\pi(\theta | \mathbf{X}) \propto \left[(1-\theta)N_p (\mathbf{0},\mathbf{\Sigma})+\theta N_p (\mathbf{\delta}, \mathbf{\Sigma})\right] \left[(1-\pi)^\theta \pi^{1-\theta}\right].$$

I don't know how to do next.


1 Answer 1


The conditional characteristic function of $X$ is $$E[e^{i\xi X}|\theta]=(1-\theta)e^{-\xi'\Sigma \xi/2}+\theta e^{i\xi'\delta-\xi'\Sigma \xi/2},\,\xi \in \mathbb{R}^p$$ So $$E[e^{i\xi X}]=(1-E[\theta])e^{-\xi'\Sigma \xi/2}+E[\theta] e^{i\xi'\delta-\xi'\Sigma \xi/2}=\pi e^{-\xi'\Sigma \xi/2}+(1-\pi)e^{i\xi'\delta-\xi'\Sigma \xi/2}$$ and therefore $X\sim \pi \mathcal{N}_p(0,\Sigma)+(1-\pi)\mathcal{N}_p(\delta,\Sigma)$. So we have $$P(\theta=t|X=x)=\frac{(1-t) \phi_p(x,0,\Sigma)+t\phi_p(x,\delta,\Sigma)}{\pi \phi_p(x,0,\Sigma)+(1-\pi)\phi_p(x,\delta,\Sigma)}(1-\pi)^t\pi^{1-t}$$ where $\phi_p(x,\mu,\Sigma)$ is the multivariate normal pdf. The mean of the posterior $E[\theta|X=x]=P(\theta=1|X=x)$ so $$E[\theta|X=x]=\frac{(1-\pi)\frac{\phi_p(x,\delta,\Sigma)}{\phi_p(x,0,\Sigma)}}{\pi +(1-\pi)\frac{\phi_p(x,\delta,\Sigma)}{\phi_p(x,0,\Sigma)}}$$ So it remains to compute $$\begin{aligned}2\ln\frac{\phi_p(x,\delta,\Sigma)}{\phi_p(x,0,\Sigma)}&=-(x-\delta)'\Sigma^{-1}(x-\delta)+x'\Sigma^{-1}x\\ &=-\delta'\Sigma^{-1} \delta+2\delta'\Sigma^{-1}x\\ \end{aligned}$$ So we conclude: $$E[\theta|X=x]=\frac{(1-\pi)e^{-\delta'\Sigma^{-1} \delta/2+\delta'\Sigma^{-1}x}}{\pi +(1-\pi)e^{-\delta'\Sigma^{-1} \delta/2+\delta'\Sigma^{-1}x}}$$ so that $\alpha = -\delta'\Sigma^{-1} \delta/2$ and $\beta'=\delta'\Sigma^{-1}$.

  • $\begingroup$ @SofiaDelacruz you're welcome, I'll check that out too $\endgroup$
    – Snoop
    Commented Nov 26, 2023 at 18:04
  • $\begingroup$ The previous link is closed. Can you look into this similar Bayesian question? math.stackexchange.com/questions/4814806/… $\endgroup$ Commented Nov 27, 2023 at 14:06

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .