# Estimating quantities of a posterior distribution.

Consider the following model:

$$\alpha \sim N(0,1)$$ $$\beta \sim N(0,1)$$ $$d_i \mid \alpha, \beta \sim \mathrm{Bernoulli}(\Phi(\alpha + \beta x_i))$$

$d_i$ is $1$ if person $i$ has some property, and $0$ if they do not. $i = 1, \dots, N$.

I have found the posterior for $\alpha$ and $\beta$ conditional on all the $d_i$. I am now asked how I may estimate $\mathbb{P}(\beta > 0 \mid d_1, \dots, d_N)$.

$p(\alpha, \beta \mid d_1, \dots, d_N) \propto \exp{(- \frac{\alpha^2 + \beta^2}{2})}\Pi_{i = 1}^N (\Phi(\alpha + \beta x_i))^{d_i} (1 - \Phi(\alpha + \beta x_i))^{1 - d_i}$

I am familiar with the Gibbs Sampler or Metropolis Hastings Monte Carlo methods. I'm not sure how to go about it in this case however.

Much thanks.

• So what is the posterior for $\alpha$ and $\beta$ conditional on all the $d_i$? – Henry May 25 '15 at 17:55
• Are you given the value of $d_i$ for every $i\in\{1,\ldots,N\}$ or only for one of them? ${}\qquad{}$ – Michael Hardy May 25 '15 at 17:57
• We're given the value for every $d_i$. Sorry for the confusion. I have edited what I found to be the posterior into the question. – user117682 May 25 '15 at 18:21

You could run your MCMC, and after the burn-in period look at the points in the chain where $d_1,...,d_N$ takes some fixed vector of values, say $\mathbf d$. Then calculate the proportion of $\beta$'s which is greater than zero in this set of points, and you have an estimate for $\mathbb P[\beta > 0 | (d_1,...,d_N)=\mathbf d]$.
On a side note, I assume you meant you have the posterior for $\alpha$ conditional on the $d_i$ as well as $\beta$ (similarly for the posterior of $\beta$), in which case you can use the Gibbs sampler (otherwise you would need a more general MH algorithm).
Edit: From your given posterior, it seems that you could consider $(\alpha,\beta)$ to be a single parameter vector. So the Gibbs sampler should cycle through $N+1$ steps: Sample $(\alpha,\beta)$ from this posterior (which I hope it is assumed that you are sample from without problem), then sample the $N$ values of $d_i$ given $(\alpha,\beta)$.
• So if I am understanding correctly, you suggest I obtain posteriors of the form $p(\alpha \mid \beta, d_1, \dots, d_N)$ and $p(\beta \mid \alpha, d_1, \dots, d_N)$, then run the Gibbs Sampler, then use the proportion of beta values greater than 0. – user117682 May 25 '15 at 18:24
• Almost: you need to restrict yourself to consider the proportion within the samples where $d_1,...,d_N$ takes your values of interest (remember that you are answering the question, "Given values of $d_1,...,d_N$, what proportion of the $\beta$'s are greater than zero?"). Basically it would look like this: say your chain obtained via Gibbs sampling is $X_1,X_2,...$, then (burn-in period aside) you will be looking only at some of these points, e.g. $X_3,X_{10},X_{14},...$ etc. because only points in the chain where the $d_i$'s take certain values will be relevant. – Ken Wei May 25 '15 at 18:30
• A remark: Your chain could potentially take very long, because there are $2^N$ values that $d_1,...,d_N$ could possibly take, e.g. if each of these were equally likely, on average you would need a chain of length $k \cdot 2^N$ to obtain a MC estimate of size $k$. – Ken Wei May 25 '15 at 18:41