When I read about Information Theory, for example on Wikipedia, I can never find statements that say if Information Theory is "real" Mathematics with underlying axioms, a notion of "proof beyond doubt" and such.

On the one hand I've read it's an engineering subject with a high level of abstraction (German Wikipedia) and "entropy" is a term used in thermodynamics, but on the other hand I recall reading proofs when seeking an answer to this question (didn't understand the least bit though) and it seems like Information Theory is a branch of probability theory (also, it's part of this website about math), but I'm still confused.

So, is Information Theory Mathematics?

Thanks in advance!

  • $\begingroup$ There are entire Math books written on the subject, see amazon.com/Coding-Information-Theory-Graduate-Mathematics/dp/… . $\endgroup$ – Baby Dragon Dec 28 '14 at 21:02
  • 2
    $\begingroup$ Probability theory is the foundation of Information theory. This is a "theory" because Shannon defined the notion of "information entropy", and using probability theory, for instance, he proved the implications about the distribution of the information in a space and the "conection" wirh topological spaces. I think the Information theory has the same level of statistics, i.e., it's a branch of applied mathematics. $\endgroup$ – Cristhian Gz Dec 28 '14 at 21:03

Yes, Information Theory is a branch of mathematics, although its practitioners are often found in departments of Electrical and Computer Engineering or Computer Science.


Instead of mathematics, I would say it's probability/statistics. The basic idea, entropy, is defined in terms of the probability.

  • 7
    $\begingroup$ Probability is mathematics. Statistics is applied mathematics. $\endgroup$ – Yuval Filmus Dec 28 '14 at 21:30

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.