Take the 2-minute tour ×
Mathematics Stack Exchange is a question and answer site for people studying math at any level and professionals in related fields. It's 100% free, no registration required.

I'm confused by a bayesian way of calculating the probability $\text{Pr}(\theta \in (\theta_1, \theta_2) | y )$, where $\theta$ is assumed to have a (prior) uniform distribution on $[0,1]$, and the likelihood of the random variable $y$ is generated by a binomial sampling model (This example is taken from the textbook Bayesian Data Analysis). Using elementary rules of probability theory, we have

\begin{align} \text{Pr}(\theta \in (\theta_1, \theta_2) | y ) &= \frac{\text{Pr}(\theta \in (\theta_1, \theta_2), y )}{p(y)}\\ &= \frac{\int_{\theta_1}^{\theta_2}p(y|\theta)p(\theta)d\theta}{p(y)} \\ &= \frac{\int_{\theta_1}^{\theta_2}\binom{n}{y}\theta^y(1-\theta)^{n-y}d\theta}{p(y)}. \end{align}

I have two questions about the formula above:

  1. Why $\text{Pr}(\theta \in (\theta_1, \theta_2), y ) = \int_{\theta_1}^{\theta_2}p(y|\theta)p(\theta)d\theta$ ? What's wrong with this way: \begin{align} \text{Pr}(\theta \in (\theta_1, \theta_2), y ) &= \text{Pr}(y | \theta \in (\theta_1, \theta_2))\text{Pr}(\theta \in (\theta_1, \theta_2))\\ &= \int_{\theta_1}^{\theta_2}p(y|\theta)d\theta \int_{\theta_1}^{\theta_2}p(\theta)d\theta. \end{align}
  2. How to calculate the denominator, that is \begin{align} p(y) = \int_0^1 \binom{n}{y}\theta^y(1-\theta)^{n-y}d\theta. \end{align}

Thanks!

share|improve this question

3 Answers 3

up vote 0 down vote accepted

For the first question, be careful that in general $$ \Pr(y|\theta \in (\theta_1,\theta_2)) \neq \int_{\theta_1}^{\theta_2}p(y|\theta)d \theta $$ What is correct is $$ \Pr(y,\theta \in (\theta_1,\theta_2)) = \int_{\theta_1}^{\theta_2}p(y,\theta)d \theta $$

For the second question, you could use the beta function to compute that integral.

share|improve this answer
    
Thanks! Could you give me a quick intuition on the relation between these two equations (the first line in your answer)? –  fishiwhj Dec 31 '12 at 8:32
    
It is really nothing more than applying the law of total probability correctly en.wikipedia.org/wiki/Law_of_total_probability –  Learner Dec 31 '12 at 8:33

For $1$: The first equality in your attempt is correct, but in the second one $\int_{\theta_1}^{\theta_2}p(y|\theta)\mathrm d\theta$ isn't the right way to calculate $\Pr(y|\theta\in(\theta_1,\theta_2))$; it should be

$$ \Pr(y|\theta\in(\theta_1,\theta_2))=\frac{\int_{\theta_1}^{\theta_2}p(y|\theta)p(\theta)\mathrm d\theta}{\int_{\theta_1}^{\theta_2}p(\theta)\mathrm d\theta}\;, $$

and if you use that you can cancel the denominator with the other factor and get $\int_{\theta_1}^{\theta_2}p(y|\theta)p(\theta)\mathrm d\theta$ after all.

For $2$, integrate by parts $y$ or $n-y$ times.

share|improve this answer
  1. In general $\text{Pr}(y | \theta \in (\theta_1, \theta_2)) \not = \int_{\theta_1}^{\theta_2}p(y|\theta)d\theta$: the right hand side increases as $\theta_2$ increases or as $\theta_1$ decreases but there is no reason to suppose the left hand side will. So your alternative will give the wrong answer.

  2. Your denominator should be $$p(y) = {\int_{0}^{1}\binom{n}{y}\theta^y(1-\theta)^{n-y}d\theta} = \frac{1}{n+1}$$ or whatever the actual limits on $\theta$ are (in other cases this may be $\pm \infty$), so that $\text{Pr}(\theta \in (\theta_1, \theta_2) | y )$ is actually a probability, something between $0$ and $1$ for all the possible values of $\theta_1$ and $\theta_2$. In this particular case it is a Beta function and with the prior distribution you were using the implication was that all integer values of $y$ from $0$ to $n$ were initially are equally likely.

share|improve this answer
    
Thanks for further explaining of my fault. –  fishiwhj Dec 31 '12 at 8:46

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.