Here is Bayes Rule:

$$P(A\mid B) = \frac{P(B\mid A) P(A)}{P(B)}$$

This paper (http://www.cogsci.northwestern.edu/Bayes/Sivia_1996.pdf) uses Bayes rule on page 21 in the context of model selection where $H$ is the single model parameter, $D_1$ and $D_2$ are two sets of data from different experiments, and $I$ represent all information known prior to the two experiments:

$$P(H\mid D_2, D_1, I) \propto P(D_2, D_1\mid H, I) P(H\mid I)$$

Is valid to select a subset of conditions (in this case $D_1$ and $D_2$) to swap with the event of interest? Or is this just allowed due to the special case where $I$ actually represents all info prior to the experiment?

Thank you!


Yes, as you say $I$ represents all info prior to the experiment: effectively it restricts the sample space. When working with expressions like this I find it convenient to manipulate them without the $I$ and then add it in back afterwards. So you have $$ P(H|D_2,D_1) \propto P(D_2,D_1|H)P(H) $$ Then write $D=D_2\wedge D_1$ and you have Bayes Rule again $$ P(H|D) = \propto P(D|H)P(H) $$

(Because it's using $\propto$ rather than having $P(D)$ on the denominator on the RHS, the probabilities will have to be normalized.)

  • $\begingroup$ "effectively it restricts the sample space" very helpful way to think about it, thank you. $\endgroup$ – andrew Mar 20 '14 at 15:50
  • $\begingroup$ no worries the biggest thing for me was when I realized that if you have horrendous formulas with the same conditioning on every expression, you can take all that conditioning out, work in the simpler system, and then add the conditioning back in at the end. $\endgroup$ – TooTone Mar 20 '14 at 15:58
  • $\begingroup$ for example in your case $P(H\mid D_2, D_1, I) \propto P(D_2, D_1\mid H, I) P(H\mid I)$ I'd say "conditioning everything on $I$, we have $P(H\mid D_2, D_1) \propto P(D_2, D_1\mid H) P(H)$ hence $P(H\mid D_2, D_1) / P(H) \propto P(D_2, D_1\mid H) $ and now writing in the conditioning again explicitly $P(H\mid D_2, D_1, I) / P(H|I) \propto P(D_2, D_1\mid H,I) $" $\endgroup$ – TooTone Mar 21 '14 at 17:47
  • $\begingroup$ What does ∝ mean and by what should they be normalised ? $\endgroup$ – Ciprian Tomoiagă Jan 15 '17 at 15:21

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.