Suppose I have an unknown variable $X_i = \alpha_i + \beta_i$ where $\alpha$ is one of 2 different values {${\alpha_1, \alpha_2}$} such that $\alpha = \alpha_1$ with probability $p_1$ and $\beta$ is drawn from a Gaussian distribution: $\beta$~$N(0,\sigma^2)$.

Suppose I am given a "signal" $Y=X+\epsilon$, where $\epsilon$~$N(0,v^2)$ My goal is to update my beliefs about $\alpha$ and $\beta$

Here's my answer so far: It's to treat the two separately.

Suppose I was given $\beta$. Then my "signal" for $\alpha$ would be $=Y-\beta$ and my updated probability that $\alpha=\alpha_1$ is:

$P(\alpha=\alpha_1|Y,\beta)=\frac{p_1\phi(\frac{Y-\beta-\alpha_1}{v})}{p_1\phi(\frac{Y-\beta-\alpha_1}{v})+(1-p_1)\phi(\frac{Y-\beta-\alpha_2}{v})} \space \space$ (1)

where $\phi$ is the standard normal pdf

Similarly, the updated distribution of $\beta$ given $\alpha$ is

$(\beta|\alpha,Y)$ ~ $N(\mu_1,\sigma^2_1)$ where $\mu_1 = \frac{\sigma^2}{\sigma^2+v^2}*(Y-\alpha)$ and $\sigma^2_1 = \frac{\sigma^2}{\sigma^2+v^2}*v^2\space \space$ (2)

I'm not really sure how to proceed next. My goal is to write something like:

$P(\alpha=\alpha_1|Y)=\int P(\alpha=\alpha_1|Y,\beta)dH(\beta)$ But, I'm not sure what distribution $H()$ to use.

Edit: I suppose I could iterate back and forth between the above methods, but Ideally I am looking for a more closed form solution

  • 1
    $\begingroup$ Where did the prior ($p_1$) go in (1)? $\endgroup$ – Daniel Weissman Apr 7 '15 at 21:56
  • $\begingroup$ @DanielWeissman Good Catch, Fixing now $\endgroup$ – Greg Apr 8 '15 at 0:10

I think you want to update your beliefs about $\alpha$ and $\beta$ jointly, not separately. Then you can just use Bayes' rule to write $p(\alpha,\beta|Y)\propto p(Y|\alpha,\beta)p(\alpha,\beta)$. Assuming that your joint prior for $(\alpha,\beta)$ is just the product of the marginal priors, I think this works out to: $$ p(\alpha,\beta|Y) = \frac{\sum_{j=1}^2 \delta(\alpha-\alpha_j)p_j \phi\left(\frac{\alpha_j-Y}{\sqrt{\sigma^2+\nu^2}}\right)\phi\left(\frac{\beta-m_j}{\sigma_1}\right)}{\sum_{j=1}^2 p_j \phi\left(\frac{\alpha_j-Y}{\sqrt{\sigma^2+\nu^2}}\right)}, $$ where $m_j\equiv(Y-\alpha_j)\sigma^2/(\sigma^2+\nu^2)$, $\delta$ is the Dirac delta, and $\phi$ and $\sigma_1$ are defined as in the question. If you just want the marginal posterior for $\alpha$, you can integrate over $\beta$ and the answer will be the same without the factor of $\phi\left(\frac{\beta-m_j}{\sigma_1}\right)$.

  • $\begingroup$ I'm a little confused. Doesn't $\mu_1$ depend on $\alpha_1$ and the updated value of $p_1$? If so then how can we solve your equation straight up. Wouldn't we need to find a "fixed point" so that the equation is satisfied? (That is marginal value for $\alpha$ on the RHS that is consistent with the joint distribution on the LHS). $\endgroup$ – Greg Apr 12 '15 at 21:41
  • $\begingroup$ I'm using your definition of $\mu_1$, where it's a function of $\alpha$. It might be clearer to write it as $\mu_j\equiv (Y-\alpha_j)\sigma^2/(\sigma^2+\nu^2)$ for $j=1,2$ to make it explicit that it depends only on $\alpha_1$ and $\alpha_2$, which are fixed. That notation also makes it clearer that $p(\alpha,\beta|Y)$ doesn't factor, so I'll go ahead and change it in my answer. $\endgroup$ – Daniel Weissman Apr 12 '15 at 22:58

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.