Take the 2-minute tour ×
Mathematics Stack Exchange is a question and answer site for people studying math at any level and professionals in related fields. It's 100% free, no registration required.

Cited from wikipedia

The likelihood function $L(θ|x)=f(x|θ)$ is not the same as the probability that those parameters are the right ones, given the observed sample. Attempting to interpret the likelihood of a hypothesis given observed evidence as the probability of the hypothesis is a common error, with potentially disastrous real-world consequences in medicine, engineering or jurisprudence. See prosecutor's fallacy for an example of this.

So, is the posterior probability the right ones? Can I think this like posterior probability is just likelihood taking account of prior probability as stated in the Bayes rules?

What confuses more is why maximum-likelihood estimation can be a proper estimate of the model parameters.

share|improve this question
add comment

1 Answer 1

up vote 0 down vote accepted

Yes, the posterior probability is more or less like the likelihood but taking the prior probability into account. From a Bayesian perspective, maximum likelihood is like using a prior proportional to 1 on the entire real line. At best this is an improper prior (e.g. for the mean of a Normal distribution). At worst it doesn't make sense at all--for example, if you are estimating a proportion $p \in [0,1]$, your "prior" is that $p$ is as likely to be in the interval $[0.4,0.5]$ as it is to be in the interval $[6.4,6.5]$, which is clearly nonsense.

One of major successes for Bayesian statistics in the middle of the 20th century was the proof that if your prior is an accurate representation of your beliefs about the parameter before you observe the data, then your posterior as calculated by Bayes' rule not only is an accurate representation of what your beliefs should be after observing the data (if you are perfectly rational), but in a sense is the only accurate representation. So assuming your prior was "correct" (in some sense of the word), posterior probabilities on the values of parameters are also correct.

share|improve this answer
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.