Suppose that $X_1 ,\ldots ,X_n$ have the one parameter Rayleigh distribution. The probability density function (pdf) is given by $$ f(x |\theta )=\frac{x}{\theta}\exp(-\frac{x^2}{2\theta}), \quad x >0,\; 0<\theta <\infty .$$ The maximum likelihood estimate (MLE) of $\theta$ is $\hat{\theta}=\frac{T}{2n}$, where $T=\sum_{i=1}^{n}x_i^2$. Let conjugate prior distribution has the form $$g(\theta |\alpha,\beta)\propto \theta^{-(\alpha +1)}\exp(-\frac{\beta}{\theta}),\quad \theta >0.$$ I don't know how to get the joint posterior distribution as follows $$\pi (\theta |T)=\frac{(\beta +\frac{T}{2})^{n+\alpha}\theta^{-(n+\alpha +1)}e^{-\frac{1}{\theta}(\beta +\frac{T}{2})}}{\Gamma (n+\alpha)}.$$ By the Bayes theorem, we have $$\pi (\theta |T)=\frac{p(\theta)p(T|\theta)}{p(T)}$$, where $p(\theta)$ is the prior distribution and $p(T|\theta)$ is likelihood function. But I don't understand how to get $\pi (\theta |T) $ like above.
2 Answers
Consider that your likelihood is the following (never mind $x$ because it will become a constant, not depending on $\theta$)
$$p(\mathbf{x}|\theta)\propto \theta^{-n}e^{-T/(2\theta)}$$
... and your prior is
$$\pi(\theta)\propto \theta^{-(\alpha+1)}e^{-\beta/\theta}$$
Thus, just multiplying them you get that
$$\pi(\theta|\mathbf{x})\propto \theta^{-(n+\alpha+1)}\cdot e^{-\frac{1}{\theta}\left( \beta+ T/2\right)}$$
Now looking at this posterior we immediate recognize the kernel of an Inverse Gamma distribution with parameters:
Shape: $(n+\alpha)$
Scale: $\left(\beta+\frac{T}{2}\right)$
As you can see in the link above, this density has exactly the following expression:
$$\pi(\theta|\mathbf{x})=\frac{\left(\beta+\frac{T}{2}\right)^{n+\alpha}}{\Gamma(n+\alpha)}\cdot \theta^{-(n+\alpha+1)}\cdot e^{-\frac{1}{\theta}\left( \beta+ T/2\right)} $$
Which is exactly what you have to show but your $m$ that is a typo, as you have $n$ observations...
In Bayesian Statistics, when possible, the normalizing constant has to be derived recognizing the kernel of your posterior...otherwise it has to be derived by integration...
-
$\begingroup$ Thank you very much for your help. Sorry, I am wrong, actually $m=n$, the number of observations. I understood completely. But what about $p(T)$ in the Bayes Formula? $\endgroup$– M.RamanaApr 30, 2021 at 8:57
-
$\begingroup$ We just multiplied $p(x|\theta)$ and $\pi (\theta)$. Don't we need to divide it by $p(T)$? $\endgroup$– M.RamanaApr 30, 2021 at 9:07
-
$\begingroup$ @M.Ramana: $p(T)$ is a constant w.r.t. $\theta$ and we can derive it without an explicit calculation if we are able (in this case we are...) to recognize the kernel of the posterior. In an alternative way, you can derive this constant by integrating $\int_{\Theta}p(\mathbf{x}|\theta)\pi(\theta)d\theta$ but it can become difficult and, in this case, useless $\endgroup$– tommikApr 30, 2021 at 9:13
-
$\begingroup$ Yes, I understood.you are right. And for the last question, how do you understand that the likelihood is $p(x|\theta )\propto \theta^{-n}e^{-T/(2\theta)}$? $\endgroup$– M.RamanaApr 30, 2021 at 9:16
-
$\begingroup$ @M.Ramana : look at my next answer. If something is not fully clear advise...I can show you a simple example in order to clarify you this important concept $\endgroup$– tommikApr 30, 2021 at 9:23
how do you understand that the likelihood is ...
the likelihood is the product of your densities: $f(x_1)\cdot f(x_2)\cdot \dots\cdot f(x_n)$ that is
$$p(\mathbf{x}|\theta)=\theta^{-n}\cdot \prod_i x_i\cdot e^{-\sum_i x_i^2/(2\theta)}$$
The quantity $\prod_i x_i$ is a constant if you look at it as a function of $\theta$. Your goal is to derive the posterior distribution of $\theta$ thus you can waste it and write directly
$$p(\mathbf{x}|\theta)\propto \theta^{-n}\cdot e^{-\sum_i x_i^2/(2\theta)}$$
-
$\begingroup$ Thank you so much for your valuable help. I got it properly. I am really grateful. $\endgroup$– M.RamanaApr 30, 2021 at 9:24