Can you recommend me a list of good Probability Books for self-studying, with good explanations and introductions for Information Theory and not for the typical statistical subjects?
- Anybody can ask a question
- Anybody can answer
- The best answers are voted up and rise to the top
Online resources addressing probability in the context of information theory:
There is the book "Information Theory, Inference, and Learning Algorithms" by David Mackay and available for free on this website
However, I recommend that you complement it by a more traditional approach such as
Most of the references above are basic texts on Information Theory and not necessary probability theory based. I'm guessing that you're looking for probability theory texts with some emphasis on information theory in preparation for delving more deeply into information theory. For that I'd recommend taking a look at Paul Pfeiffer's Concepts of Probability Theory or either of Alfred Renyi's two books Probability Theory or Foundations of Probability. All either mention information theory specifically or have presentations influenced or working toward the subject in general. All three are reprints that can be had fairly cheaply from Dover Publications.
Alternately, for a bit more money, you might consider Alfredo Leon-Garcia's fairly standard text Probability, Statistics, and Random Processes for Electrical Engineers. As he's also subsequently written texts for communication engineering, he certainly has a very information theory/comm theory flavored presentation. The most recent edition is the 3rd, but the 2nd edition is substantially the same for less money on the used market.
I think all four above are relatively good for self-study, though Renyi's presentations are a tad more sophisticated mathematically and may seem more dense to the beginner.