Mathematics Stack Exchange is a question and answer site for people studying math at any level and professionals in related fields. It's 100% free, no registration required.

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

Can you recommend me a list of good Probability Books for self-studying, with good explanations and introductions for Information Theory and not for the typical statistical subjects?

share|cite|improve this question
up vote 3 down vote accepted


Online resources addressing probability in the context of information theory:

share|cite|improve this answer
The first four links have expired: could you post the last names of the authors so that it could be possible to find the references elsewhere? – William Apr 30 at 0:08

There is the book "Information Theory, Inference, and Learning Algorithms" by David Mackay and available for free on this website

However, I recommend that you complement it by a more traditional approach such as

An Intermediate Course in Probability - Allan Gut

share|cite|improve this answer

Most of the references above are basic texts on Information Theory and not necessary probability theory based. I'm guessing that you're looking for probability theory texts with some emphasis on information theory in preparation for delving more deeply into information theory. For that I'd recommend taking a look at Paul Pfeiffer's Concepts of Probability Theory or either of Alfred Renyi's two books Probability Theory or Foundations of Probability. All either mention information theory specifically or have presentations influenced or working toward the subject in general. All three are reprints that can be had fairly cheaply from Dover Publications.

Alternately, for a bit more money, you might consider Alfredo Leon-Garcia's fairly standard text Probability, Statistics, and Random Processes for Electrical Engineers. As he's also subsequently written texts for communication engineering, he certainly has a very information theory/comm theory flavored presentation. The most recent edition is the 3rd, but the 2nd edition is substantially the same for less money on the used market.

I think all four above are relatively good for self-study, though Renyi's presentations are a tad more sophisticated mathematically and may seem more dense to the beginner.

share|cite|improve this answer

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.