So I have the following question from my textbook, the answer I get is slightly different from the book's answer, which I think may be wrong, could someone please confirm?

Question: Suppose $y_{1:T} = (y_1, y_2, \cdots, y_T)'$ is a sample of $T = 20$ Poisson random variables such that:

$y_t|\theta_1 \stackrel{i.i.d}{\sim} Poisson(\theta_1) \ \text{for} \ t=1, 2, \dots, T_1=10$ $y_t|\theta_2 \stackrel{i.i.d}{\sim} Poisson(\theta_2) \ \text{for} \ t=T_1+1, 2, \dots, T=T_1+T_2$ where $T_2 = 10$

Now assume that the parameters of the two Poisson distributions, $\theta_1$ and $\theta_2$, are independent draws from a $Gamma(shape = \alpha, scale = \beta)$ distribution, ie, $$\theta_1, \theta_2 \stackrel{i.i.d}{\sim} Gamma(shape = \alpha, scale = \beta)$$ where $\alpha$ is known but $\beta$ is not known. Finally, place a prior distribution on $\beta$, given by: $$\beta \sim InverseGamma(shape=w, scale = \tau)$$ where $w$ and $\tau$ are both assumed to be known.

Find the full posterior conditional for $\beta$, that is, find the distribution $\beta|\theta_1, \theta_2, y_{1:T}$.

My working:

We have: \begin{align*}\displaystyle p(y_t|\theta_1) = \frac{\theta_1^{y_t}\exp(-\theta_1)}{y_t!} \ \ \ \ \text{for} \ t=1, 2, \cdots, T_1\end{align*} Hence: \begin{align*}p(y_{1:T_1}|\theta_1) = \prod_{t=1}^{T_1} \frac{\theta_1^{y_t}\exp(-\theta_1)}{y_t!} = \frac{\theta_1^{\sum_{t=1}^{T_1} y_t}\exp(-T_1 \theta_1)}{\prod_{t=1}^{T_1}(y_t!)} \end{align*} Likewise: \begin{align*}\displaystyle p(y_t|\theta_2) = \frac{\theta_2^{y_t}\exp(-\theta_2)}{y_t!} \ \ \ \ \text{for} \ t=T_1+1, T_1+2, \cdots, T=T_1+T_2\end{align*} Hence: \begin{align*}p(y_{T_1+1:T_1+T_2}|\theta_2) = \prod_{t=T_1+1}^{T_1+T_2} \frac{\theta_2^{y_t}\exp(-\theta_2)}{y_t!} = \frac{\theta_2^{\sum_{t=T_1+1}^{T_1+T_2} y_t}\exp(-T_2 \theta_2)}{\prod_{t=T_1+1}^{T_1+T_2}(y_t!)} \end{align*} Since $y_t|\theta_1$ for $t = 1, 2, \cdots, T_1$ are independent from $y_t|\theta_2$ for $t=T_1+1, T_1+2, \cdots, T=T_1+T_2$, the likelihood is given by: \begin{align*} p(y_{1:T}|\theta_1, \theta_2) & = p(y_{1:T_1}|\theta_1) \times p(y_{T_1+1:T_1+T_2}|\theta_2) \\ & = \frac{\theta_1^{\sum_{t=1}^{T_1} y_t}\exp(-T_1 \theta_1)}{\prod_{t=1}^{T_1}(y_t!)} \times \frac{\theta_2^{\sum_{t=T_1+1}^{T_1+T_2} y_t}\exp(-T_2 \theta_2)}{\prod_{t=T_1+1}^{T_1+T_2}(y_t!)} \\ & = \frac{\theta_1^{\sum_{t=1}^{T_1}y_t}\theta_2^{\sum_{t=T_1+1}^{T_1+T_2}y_t}\exp(-T_1\theta_1-T_2\theta_2)}{\prod_{t=1}^{T_1+T_2}(y_t!)} \end{align*} The prior is given by: \begin{align*} p(\theta_1, \theta_2, \beta) & = p(\theta_1, \theta_2|\beta)p(\beta) \\ & = p(\theta_1|\beta)p(\theta_2|\beta)p(\beta) \\ & = \frac{\theta_1^{\alpha-1}}{\beta^{\alpha}\Gamma(\alpha)}\exp\left(-\frac{\theta_1}{\beta}\right) \times \frac{\theta_2^{\alpha-1}}{\beta^{\alpha}\Gamma(\alpha)}\exp\left(-\frac{\theta_2}{\beta}\right) \times \frac{\tau^{w}}{\Gamma(w)\beta^{w+1}}\exp\left(-\frac{\tau}{\beta}\right) \\ & = \frac{(\theta_1\theta_2)^{\alpha-1}\tau^w\exp\left(-\frac{1}{\beta}\left(\theta_1+\theta_2+\tau\right)\right)}{\beta^{2\alpha+w+1}\left(\Gamma(\alpha)\right)^2\Gamma(w)} \end{align*} Thus the joint density for $(\theta_1, \theta_2, \beta, y_{1:T})$ is given by: \begin{align*} p(\theta_1, \theta_2, \beta, y_{1:T}) = \frac{\theta_1^{\sum_{t=1}^{T_1}y_t + \alpha -1}\theta_2^{\sum_{t=T_1+1}^{T_1+T_2}y_t + \alpha -1}\tau^w\exp\left(-T_1\theta_1-T_2\theta_2-\frac{1}{\beta}\left(\theta_1+\theta_2+\tau\right)\right)}{\prod_{t=1}^{T_1+T_2}(y_t!)\beta^{2\alpha+w+1}\left(\Gamma(\alpha)\right)^2\Gamma(w)} \end{align*} Hence a kernel for the joint posterior density of $(\theta_1, \theta_2, \beta)$ is given by: \begin{align*} p(\theta_1, \theta_2, \beta|y_{1:T}) \propto \frac{\theta_1^{\sum_{t=1}^{T_1}y_t + \alpha -1}\theta_2^{\sum_{t=T_1+1}^{T_1+T_2}y_t + \alpha -1}\exp\left(-T_1\theta_1-T_2\theta_2-\frac{1}{\beta}\left(\theta_1+\theta_2+\tau\right)\right)}{\beta^{2\alpha+w+1}} \end{align*} The full conditional for $\beta$ is given by: \begin{align*} p(\beta|\theta_1, \theta_2, y_{1:T}) & \propto \frac{\exp\left(-\frac{1}{\beta}\left(\theta_1+\theta_2+\tau\right)\right)}{\beta^{2\alpha+w+1}} \\ & = \frac{1}{\beta^{2\alpha+w+1}}\exp\left(-\frac{\theta_1+\theta_2+\tau}{\beta}\right) \end{align*} Hence $\beta|\theta_1, \theta_2, y_{1:T} \sim InverseGamma\left(shape=2\alpha+w, scale=\theta_1+\theta_2+\tau\right)$

Book's answer: $\beta|\theta_1, \theta_2, y_{1:T} \sim InverseGamma\left(shape=2\alpha+w, scale=\theta_1+\theta_2+1/\tau\right)$

Where'd that $1/\tau$ come from? I don't think it's correct.

  • $\begingroup$ Nice clear work (+1). What book is it? Is there an errata? $\endgroup$ – hejseb Oct 8 '13 at 6:43
  • $\begingroup$ Thanks hejseb, actually it's from a bound of notes I got from my professor, apparently it's from a textbook but it never stated which. I have a feeling the answer was incorrectly transferred to the notes from the textbook :) $\endgroup$ – Trts Oct 8 '13 at 7:14

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.