# Books for Understanding Bayesian probability from the Beginning

Lets pretend I have no experience with statistics...

what books, in what order, will get me to the point of being able to understand Bayesian probability fastest.

-
Do you have experience with probability? How much? What do you mean by Bayesian probability? Almost any probability text will mention some form of Bayes Rule. –  deinst Apr 18 '12 at 1:12
Bayesian probability as in Bayes' rule is described in almost any probability textbook and is noncontroversial even among hard core frequentists (or so I believe). On the other hand, Bayesian statistics and Bayesian inference and the like are a different matter. –  Dilip Sarwate Apr 18 '12 at 15:12

The folks at LessWrong have put together a couple of resources that can be helpful for the complete beginner. They are also useful if you've only taken a single course in probability and basically just memorized formulas to get through it.

For two more advanced books that cover practical matters in great detail (and require a bit more mathematical maturity) see:

My own opinion is that learning the formula for Bayes' theorem is elementary; it's something that almost always precedes even a first introduction to common discrete random variables. The real "starting point" for seeing the difference in Bayesian methods is when you start needing to compute Maximum Likelihood Estimators (MLE). Then you can directly compare the output of an MLE with the alternative output if you had set up a prior and chased the computation out all the way to a posterior distribution.

There's nothing special about MLEs, of course, but that just happens to be one of the first places where a student is really forced to say, "Gee, I'd really like to have the whole posterior distribution, but it's really hard to compute." Once a student sees the value in that, they are usually ready to start learning about acceptance/rejection, Metropolis-Hastings, and Gibbs sampling, which are the gateway drugs of Bayesian analysis.

-

EE, Peter M., 2004. Bayesian Statistics: An Introduction, Hodder Arnold.

-
LEE rather then EE –  Henry Apr 18 '12 at 1:33

Reference Book: Bayesian Theory by José M. Bernardo and Adrian F. M. Smith