I'm reading about Bayesian data analysis by Gelman et al. and I'm having big trouble interpreting the following part in the book (note, the rat tumor rate $\theta$ in the following text has: $\theta \sim Beta(\alpha, \beta)$):

Choosing a standard parameterization and setting up a ‘noninformative’ hyperprior dis- tribution. Because we have no immediately available information about the distribution of tumor rates in populations of rats, we seek a relatively diffuse hyperprior distribution for $(\alpha, \beta)$. Before assigning a hyperprior distribution, we reparameterize in terms of $\text{logit}(\frac{\alpha}{\alpha+\beta}) = \log(\frac{\alpha}{\beta})$ and $\log(\alpha+\beta)$, which are the logit of the mean and the logarithm of the ‘sample size’ in the beta population distribution for $θ$. It would seem reasonable to assign independent hyperprior distributions to the prior mean and ‘sample size,’ and we use the logistic and logarithmic transformations to put each on a $(-\infty, \infty)$ scale. Unfortunately, a uniform prior density on these newly transformed parameters yields an improper posterior density, with an infinite integral in the limit $(\alpha+\beta) \rightarrow \infty$, and so this particular prior density cannot be used here.

In a problem such as this with a reasonably large amount of data, it is possible to set up a ‘noninformative’ hyperprior density that is dominated by the likelihood and yields a proper posterior distribution. One reasonable choice of diffuse hyperprior density is uniform on $(\frac{\alpha}{\alpha+\beta}, (\alpha+\beta)^{-1/2})$, which when multiplied by the appropriate Jacobian yields the following densities on the original scale,

$$p(\alpha, \beta) \propto (\alpha+\beta)^{−5/2},$$

and on the natural transformed scale:

$$p\left(\log\left(\frac{\alpha}{\beta}\right), \log(\alpha+\beta)\right)\propto \alpha\beta(\alpha+\beta)^{−5/2}.$$

My problem is especially the bolded parts in the text.

Question (1): What does the author explicitly mean by: "is uniform on $(\frac{\alpha}{\alpha+\beta}, (\alpha+\beta)^{-1/2})$"

Question (2): What is the appropriate Jacobian?

Question (3): How does the author arrive into the original and transformed scale priors?

To me the book hides many details under the hood and makes understanding difficult for a beginner on the subject due to seemingly ambiguous text.

P.S. if you need more information, or me to clarify my questions please let me know.


I figured the solution out myself so I'm gonna share it here if anyone is going to bump into the same part in Gelman's book (pages 110-111).

Answer (1):

The author simply means by this that $$p\left(\frac{\alpha}{\alpha+\beta}, \;(\alpha+\beta)^{-1/2}\right)=\text{constant}\propto 1.$$

Answer (2):

When the author talks about "appropriate Jacobian" he's talking about the determinant of the Jacobian matrix in the change of variables formula for density functions:


Answer (3):

The author simply applies the change of variables formula two times. We know that $$p(\gamma, \delta) = p(\gamma(\alpha, \beta),\, \delta(\alpha, \beta))=p\left(\frac{\alpha}{\alpha+\beta}, \;(\alpha+\beta)^{-1/2}\right)=\text{constant}\propto 1.$$

If we denote $\theta=(\gamma, \delta)$ and $\phi=(\alpha, \beta)$, then:

$$\det\left(\frac{d\theta}{d\phi}\right)=\left|\begin{array}{cc} \frac{d\gamma}{d\alpha} & \frac{d\gamma}{d\beta} \\ \frac{d\delta}{d\alpha} & \frac{d\delta}{d\beta}\end{array}\right|=\left|\begin{array}{cc} \frac{\beta}{(\alpha+\beta)^2} & -\frac{\alpha}{(\alpha+\beta)^2} \\ -\frac{1}{2(\alpha+\beta)^{3/2}} & -\frac{1}{2(\alpha+\beta)^{3/2}}\end{array}\right|=-\frac{1}{2(\alpha+\beta)^{5/2}}.$$

From change of variables formula we get:

$$p(\alpha, \beta) = \underbrace{p\left(\frac{\alpha}{\alpha+\beta}, \;(\alpha+\beta)^{-1/2}\right)}_\text{= constant $\propto \;1$}\cdot \left(-\frac{1}{2(\alpha+\beta)^{5/2}}\right)\propto (\alpha+\beta)^{-5/2},$$

and there it is, i.e. the prior in original scale.

For the alternative scale, by using change of variables in exactly the same manner:

$$p(\alpha, \beta) = p\left(\log\left(\frac{\alpha}{\beta}\right), \log(\alpha+\beta)\right)\,\det\left(\frac{d\theta}{d\phi}\right),$$

where this time $\gamma(\alpha, \beta) = \log\left(\frac{\alpha}{\beta}\right)$ and $\delta(\alpha, \beta) = \log(\alpha+\beta)$. For the Jacobian determinant we get:

$$\det\left(\frac{d\theta}{d\phi}\right)=\left|\begin{array}{cc} \frac{d\gamma}{d\alpha} & \frac{d\gamma}{d\beta} \\ \frac{d\delta}{d\alpha} & \frac{d\delta}{d\beta}\end{array}\right|=\left|\begin{array}{cc} 1/\alpha & -1/\beta \\ (\alpha+\beta)^{-1} & (\alpha+\beta)^{-1}\end{array}\right|=\frac{1}{\alpha\beta},$$

so we get:

$$\underbrace{p(\alpha, \beta)}_\text{$\propto\, (\alpha+\beta)^{-5/2}$} = p\left(\log\left(\frac{\alpha}{\beta}\right), \log(\alpha+\beta)\right)\,\frac{1}{\alpha\beta},$$


$$ p\left(\log\left(\frac{\alpha}{\beta}\right), \log(\alpha+\beta)\right) \propto \alpha\beta(\alpha+\beta)^{-5/2}.$$


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.