Good books on mathematics of information? Are there any good books on the mathematics of information, under whose umbrella would fall
cryptography, information storing, data processing (i.e. data to information), mathematics of artificial intelligence. I understand that many of these topics have originated in computer science, but I want resources on the mathematical foundations of these topics.
Maybe even some mathematics of information as related to quantum computing.
I've also read a little bit of Marcus Hutter's paper 'AIXI' aka 'Universal Artificial Intelligence'.
 A: I think the following make a good start:

*

*Cover and Thomas, Elements of Information Theory
This is at an intermediate mathematical level, but has quite a broad coverage, including more chapters on network information theory in later editions (e.g., ed. 3)


*R G Gallager, Information Theory and Reliable Communication
A classic, very mathematical, and still very enjoyable to read. Deeper than Cover and Thomas but narrower. However it does cover some algebraic coding theory which TCover and Thomas doesn't.


*D MacKay, Information Theory, Inference and Learning Algorithms
Quite different than the two above, this focuses more on implementations, and links to learning theory and inference. First to cover LDPC codes in detail [which were invented by Gallager in his PhD thesis in the early 60s]


*R E Blahut, Theory of Remote Image Formation

*R E Blahut, Algebraic Codes for Data Transmission

*R E Blahut, Cryptography and Secure Communication

*R E Blahut, Principles and Practice of Information Theory
Blahut's books form a nice ensemble, mathematically similar to Cover and Thomas.


*R Vershynin, High Dimensional Probability
Good for statistics/probability of large data sets.


*D G Luenberger, Information Science
Gentler level of mathematics, interesting broad coverage across different applications of information.
