So I've been reading about Bayesian models so I tried I'd have a toy example I could play with.

Consider the following: You are at a bus stop and you observe the bus arriving at various times $t_1, t_2, t_3, t_4, .., t_{10}$. You know that the buses are scheduled to arrive at periodic time intervals of $1 < p < 2$ minutes and you know the first bus is scheduled to arrive at precisely 2 minutes after you start observing the buses. The buses, however, don't arrive precisely on time. For a given expected arrival time, their arrival actually is a draw from a normal distribution centred on the arrival time and variance 1. You know nothing more about $p$ so you can assume it has uniform probability on $[1,2]$.

Now, we can use a simple Bayesian model. I'll walk you through my steps, just to make sure I did not misunderstand something fundamental (and to clear my own thoughts). We use:

$$P(H|D) = \frac{P(D|H)P(H)}{P(D)}$$

$P(H)$ (the probability of the hypothesis) is easy, it's the uniform distribution of $p$ on $[1,2]$.
$P(D|H)$ (the probability of the data, given the hypothesis) is a bit more complicated, but it's the probability of those independent observations $t_1,..,t_{10}$ happening, given a fixed $p$. Since you know the distribution of each $t_i$ and the observations are independent, you can calculate the distribution.

Now, what I am confused about is $P(D)$ (the probability of the data). This is the probability the observations happen, but from which space are we drawing this? The guides online i have seen say that this is the probability of D without any prior. But isn't this the whole thing about Bayesian models, that there always is a prior although sometimes hidden?

Provided my logic here is correct, what should $P(D)$ be here and why?


Since you know $P(D)$ and $P(D|H)$, you can compute $P(D)$ by the Law of Total Probability:

\begin{align} P(D) = \int_{H^{*} \in \Theta}{P(D|H^{*})dP(H^{*})} \end{align}

You usually don't need to compute this value in Bayesian computations, though.

  • $\begingroup$ Thanks for the answer. However, 2 points: 1. Why would we not include this in a Bayesian computation? 2. This still doesn't explain what the meaning of $P(D)$ is. $\endgroup$ – Henry Henrinson Jul 15 '14 at 13:19
  • $\begingroup$ 1) In Bayesian computations it is often not needed to compute $P(D)$. $P(H|D)$ is proportional to $P(H)P(D|H)$. Hence, if you look at the functional form of the latter, you can know the distribution of $H|D$. 2) $P(D)$ is the marginal probability of observing the data. That is, the probability of observing the data without assuming any particular hypothesis $H$ is true. As you can see from the Law of Total Probability, $P(D)$ is the average of each $P(D|H)$ weighted by your prior belief in each $H$, $P(H)$. $\endgroup$ – madprob Jul 15 '14 at 17:23

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.