# Find the Bayes estimator under mixture normal distributions.

Consider the following model:

$$\mathbf{X}=(X_1,...,X_p)|\theta \sim (1-\theta)N_p (\mathbf{0},\mathbf{\Sigma})+\theta N_p (\mathbf{\delta}, \mathbf{\Sigma})$$

$$\theta \sim \textrm{Bernoulli}(1-\pi)$$

for some known $$\mathbf{\delta}$$, $$\mathbf{\Sigma}$$ and $$\pi$$. Show that the Bayes estimator of $$\theta$$ is given by

$$\frac{(1-\pi) e^{\alpha + \Sigma_{i=1}^p \beta_i X_i}}{\pi + (1-\pi) e^{\alpha + \Sigma_{i=1}^p \beta_i X_i}}$$

for some constants $$\alpha$$ and $$\beta_i, i=1,...,p$$. Give formulas for $$\alpha$$ and $$\beta_i$$.

My idea:

The posterior distribution of $$\theta$$ given the data $$\mathbf{X}$$ is proportional to the product of the likelihood and the prior:

$$\pi(\theta | \mathbf{X}) \propto f(\mathbf{X} | \theta) \pi(\theta),$$

where $$f(\mathbf{X} | \theta) = (1-\theta)N_p (\mathbf{0},\mathbf{\Sigma})+\theta N_p(\mathbf{\delta}, \mathbf{\Sigma})$$ is the likelihood function, and $$\pi(\theta) = (1-\pi)^\theta \pi^{1-\theta}$$ is the prior distribution.

Now, I can calculate the posterior distribution:

$$\pi(\theta | \mathbf{X}) \propto \left[(1-\theta)N_p (\mathbf{0},\mathbf{\Sigma})+\theta N_p (\mathbf{\delta}, \mathbf{\Sigma})\right] \left[(1-\pi)^\theta \pi^{1-\theta}\right].$$

I don't know how to do next.

The conditional characteristic function of $$X$$ is $$E[e^{i\xi X}|\theta]=(1-\theta)e^{-\xi'\Sigma \xi/2}+\theta e^{i\xi'\delta-\xi'\Sigma \xi/2},\,\xi \in \mathbb{R}^p$$ So $$E[e^{i\xi X}]=(1-E[\theta])e^{-\xi'\Sigma \xi/2}+E[\theta] e^{i\xi'\delta-\xi'\Sigma \xi/2}=\pi e^{-\xi'\Sigma \xi/2}+(1-\pi)e^{i\xi'\delta-\xi'\Sigma \xi/2}$$ and therefore $$X\sim \pi \mathcal{N}_p(0,\Sigma)+(1-\pi)\mathcal{N}_p(\delta,\Sigma)$$. So we have $$P(\theta=t|X=x)=\frac{(1-t) \phi_p(x,0,\Sigma)+t\phi_p(x,\delta,\Sigma)}{\pi \phi_p(x,0,\Sigma)+(1-\pi)\phi_p(x,\delta,\Sigma)}(1-\pi)^t\pi^{1-t}$$ where $$\phi_p(x,\mu,\Sigma)$$ is the multivariate normal pdf. The mean of the posterior $$E[\theta|X=x]=P(\theta=1|X=x)$$ so $$E[\theta|X=x]=\frac{(1-\pi)\frac{\phi_p(x,\delta,\Sigma)}{\phi_p(x,0,\Sigma)}}{\pi +(1-\pi)\frac{\phi_p(x,\delta,\Sigma)}{\phi_p(x,0,\Sigma)}}$$ So it remains to compute \begin{aligned}2\ln\frac{\phi_p(x,\delta,\Sigma)}{\phi_p(x,0,\Sigma)}&=-(x-\delta)'\Sigma^{-1}(x-\delta)+x'\Sigma^{-1}x\\ &=-\delta'\Sigma^{-1} \delta+2\delta'\Sigma^{-1}x\\ \end{aligned} So we conclude: $$E[\theta|X=x]=\frac{(1-\pi)e^{-\delta'\Sigma^{-1} \delta/2+\delta'\Sigma^{-1}x}}{\pi +(1-\pi)e^{-\delta'\Sigma^{-1} \delta/2+\delta'\Sigma^{-1}x}}$$ so that $$\alpha = -\delta'\Sigma^{-1} \delta/2$$ and $$\beta'=\delta'\Sigma^{-1}$$.