A Reference Book Justifying Different Distributions Well I am trying to find a book that could come up with a rationale behind different distributions but not only defining them and giving an intuition about the structure of distributions. For example it seems really hard for me to grasp the idea behind beta distribution as to be related to binomial distribution.
Right now I am reading Bayesian Data Analysis by Gelman et. Al:
http://www.amazon.com/Bayesian-Analysis-Edition-Chapman-Statistical/dp/158488388X
And it is not at all covering such a thing. It only defines it which is most of the time the case for the probability reference books.
 A: In Statistical Inference by Casella & Berger, there is one chapter called "Common Families of Distributions". While not perhaps covering all cases, the authors discuss the distributions in words and not only mathematically. There is one example about the Gamma-Poisson relationship, for instance. Also, at the end there is a nice little map showing relationships between distributions (which can also be found here). Their book might help you a little, at least, so have a look at it if you can and see if it's something like what you are looking for.
EDIT: In the link I posted there is a reference to the article in which the map first appeared. The reference is

Leemis, L. M. (1986), “Relationships among common univariate
  distributions,” The American Statistician, Vol. 40, No. 2

and judging by its name it might be worth checking out.
A: I have been working at a compendium of probability distributions that tries to provide some motivation in the introduction to each distribution. The compendium can be found at http://www.statlect.com/distri.htm . For the Beta distribution, in particular, you can read the remarks in the introduction at http://www.statlect.com/beta_distribution.htm
