# Bayesian Updating with 1 Signal but 2 Unknowns

Suppose I have an unknown variable $X_i = \alpha_i + \beta_i$ where $\alpha$ is one of 2 different values {${\alpha_1, \alpha_2}$} such that $\alpha = \alpha_1$ with probability $p_1$ and $\beta$ is drawn from a Gaussian distribution: $\beta$~$N(0,\sigma^2)$.

Suppose I am given a "signal" $Y=X+\epsilon$, where $\epsilon$~$N(0,v^2)$ My goal is to update my beliefs about $\alpha$ and $\beta$

Here's my answer so far: It's to treat the two separately.

Suppose I was given $\beta$. Then my "signal" for $\alpha$ would be $=Y-\beta$ and my updated probability that $\alpha=\alpha_1$ is:

$P(\alpha=\alpha_1|Y,\beta)=\frac{p_1\phi(\frac{Y-\beta-\alpha_1}{v})}{p_1\phi(\frac{Y-\beta-\alpha_1}{v})+(1-p_1)\phi(\frac{Y-\beta-\alpha_2}{v})} \space \space$ (1)

where $\phi$ is the standard normal pdf

Similarly, the updated distribution of $\beta$ given $\alpha$ is

$(\beta|\alpha,Y)$ ~ $N(\mu_1,\sigma^2_1)$ where $\mu_1 = \frac{\sigma^2}{\sigma^2+v^2}*(Y-\alpha)$ and $\sigma^2_1 = \frac{\sigma^2}{\sigma^2+v^2}*v^2\space \space$ (2)

I'm not really sure how to proceed next. My goal is to write something like:

$P(\alpha=\alpha_1|Y)=\int P(\alpha=\alpha_1|Y,\beta)dH(\beta)$ But, I'm not sure what distribution $H()$ to use.

Edit: I suppose I could iterate back and forth between the above methods, but Ideally I am looking for a more closed form solution

• Where did the prior ($p_1$) go in (1)? – Daniel Weissman Apr 7 '15 at 21:56
• @DanielWeissman Good Catch, Fixing now – Greg Apr 8 '15 at 0:10

I think you want to update your beliefs about $\alpha$ and $\beta$ jointly, not separately. Then you can just use Bayes' rule to write $p(\alpha,\beta|Y)\propto p(Y|\alpha,\beta)p(\alpha,\beta)$. Assuming that your joint prior for $(\alpha,\beta)$ is just the product of the marginal priors, I think this works out to: $$p(\alpha,\beta|Y) = \frac{\sum_{j=1}^2 \delta(\alpha-\alpha_j)p_j \phi\left(\frac{\alpha_j-Y}{\sqrt{\sigma^2+\nu^2}}\right)\phi\left(\frac{\beta-m_j}{\sigma_1}\right)}{\sum_{j=1}^2 p_j \phi\left(\frac{\alpha_j-Y}{\sqrt{\sigma^2+\nu^2}}\right)},$$ where $m_j\equiv(Y-\alpha_j)\sigma^2/(\sigma^2+\nu^2)$, $\delta$ is the Dirac delta, and $\phi$ and $\sigma_1$ are defined as in the question. If you just want the marginal posterior for $\alpha$, you can integrate over $\beta$ and the answer will be the same without the factor of $\phi\left(\frac{\beta-m_j}{\sigma_1}\right)$.
• I'm a little confused. Doesn't $\mu_1$ depend on $\alpha_1$ and the updated value of $p_1$? If so then how can we solve your equation straight up. Wouldn't we need to find a "fixed point" so that the equation is satisfied? (That is marginal value for $\alpha$ on the RHS that is consistent with the joint distribution on the LHS). – Greg Apr 12 '15 at 21:41
• I'm using your definition of $\mu_1$, where it's a function of $\alpha$. It might be clearer to write it as $\mu_j\equiv (Y-\alpha_j)\sigma^2/(\sigma^2+\nu^2)$ for $j=1,2$ to make it explicit that it depends only on $\alpha_1$ and $\alpha_2$, which are fixed. That notation also makes it clearer that $p(\alpha,\beta|Y)$ doesn't factor, so I'll go ahead and change it in my answer. – Daniel Weissman Apr 12 '15 at 22:58