I am trying to learn about the Beta Binomial for Bayesian style probability. So far I have:

$p(D|\theta)$ is the likelihood of the data set.

$D$ is the data set.

$p(\theta)$ is the prior probability of the parameter $h$ (I would use $\theta$ instead of $h$, but for this example I'd rather use discrete probability instead of continuous).

So the posterior distribution is proportional to the likelihood times the prior:

$p(\theta|D) \propto p(D|\theta) \times p(\theta)$

My question is this: suppose we first get a data set $D_1$, and we use it to obtain a posterior distribution. But then (like batch style learning), we obtain another data set $D_2$. We first calculate the distribution given $D_1$ using the above equation, but then how do we update $\theta$ before using the equation again to obtain the new posterior from $D_2$?

Thanks in advance


Okay, so initial the posterior is $$p(\theta|D_1)=\frac{p(D_1|\theta)p(\theta)}{\int_{\theta\in\Theta}p(D_1|\theta)p(\theta)\mathrm{d}\theta},\forall\theta\in\Theta$$ where $\Theta$ is the parameter space.

The old posterior becomes the new prior distribution. The updated posterior becomes $$p(\theta|D_1;D_2)=\frac{p(D_2|\theta)p(\theta|D_1)}{\int_{\theta\in\Theta}p(D_2|\theta)p(\theta|D_1)\mathrm{d}\theta},\forall\theta\in\Theta$$

  • $\begingroup$ Hello Dave, when you say p(theta | D1 ; D2), you mean p( (theta|D1) | D2), correct? $\endgroup$ – Josh Jul 20 '17 at 13:31
  • $\begingroup$ One more thing, suppose you wanted to actually modify theta so that the following would be true: p(new_theta) = p(old_theta | D1) ... how would you change theta? $\endgroup$ – Josh Jul 20 '17 at 13:44
  • $\begingroup$ As to first comment, no I meant what I wrote. The probability of theta given both d1 and d2. $\endgroup$ – Dave Harris Jul 20 '17 at 16:30
  • $\begingroup$ As to second comment. It is notationally acceptable to drop the "given d1" from the posterior to make it a prior if you wish. I kept the notation in my answer so you could see the posterior was now a prior. $\endgroup$ – Dave Harris Jul 20 '17 at 16:32

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.