I was going through a pattern recognition book and in the chapter of Bayesian Parameter Estimation I came across this formula. I cannot understand how the 2nd line is derived from the first line. Please help me any one...D is the sample group containing n number of samples of the random variable x chosen from a pool. Bayesian Parameter Estimation

  • $\begingroup$ As far as I can understand, they are taking products of pmfs by conditioning first, to derive some kind of a joint pmf. The independence may come from sample assumption. Can you be more clear about your question, especially the symbols? $\endgroup$ – Somabha Mukherjee May 18 '13 at 7:36
  • $\begingroup$ yuppp...the n samples chosen from the pool are statistically independent and hence the probability that all the samples occur is the product..that is how we get the second line. But how is this related to the above line of p(D|u)p(u)/(.....) ? can you please clarify that? The way it is written seems the 2nd line is derived from the first line... $\endgroup$ – rotating_image May 18 '13 at 8:20

Consider the denominator first. It is: $$ \int p(\mathbf{D}|\mu)p(\mu)d\mu=c $$ Since $\mu$ is integrated out, it's simply a constant with respect to $\mu$. Let $c^{-1}=\alpha$.

Let also $\mathbf{D}$ be a vector such that $\mathbf{D}=\begin{pmatrix}x_1,&x_2, &\dots, &x_n\end{pmatrix}$. Under the assumption that the samples are independent, we can write the density of the vector $\mathbf{D}$ (conditional on $\mu$) as:

$$ p(\mathbf{D}|\mu)=p(x_1, x_2, \dots, x_n|\mu)=p(x_1|\mu)p(x_2|\mu)\cdots p(x_n|\mu)=\prod_{k=1}^np(x_k|\mu) $$

Putting all of this together:

$$ \frac{p(\mathbf{D}|\mu)p(\mu)}{\int p(\mathbf{D}|\mu)p(\mu)d\mu}=\frac{1}{c}p(\mathbf{D}|\mu)p(\mu)=\alpha \times p(\mathbf{D}|\mu)p(\mu)=\alpha\left(\prod_{k=1}^np(x_k|\mu)\right)p(\mu)=\alpha\prod_{k=1}^np(x_k|\mu)p(\mu) $$

as desired.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.