4
$\begingroup$

When I read about Information Theory, for example on Wikipedia, I can never find statements that say if Information Theory is "real" Mathematics with underlying axioms, a notion of "proof beyond doubt" and such.

On the one hand I've read it's an engineering subject with a high level of abstraction (German Wikipedia) and "entropy" is a term used in thermodynamics, but on the other hand I recall reading proofs when seeking an answer to this question (didn't understand the least bit though) and it seems like Information Theory is a branch of probability theory (also, it's part of this website about math), but I'm still confused.

So, is Information Theory Mathematics?

Thanks in advance!

$\endgroup$
4
  • 1
    $\begingroup$ There are entire Math books written on the subject, see amazon.com/Coding-Information-Theory-Graduate-Mathematics/dp/… . $\endgroup$ Dec 28, 2014 at 21:02
  • 4
    $\begingroup$ Probability theory is the foundation of Information theory. This is a "theory" because Shannon defined the notion of "information entropy", and using probability theory, for instance, he proved the implications about the distribution of the information in a space and the "conection" wirh topological spaces. I think the Information theory has the same level of statistics, i.e., it's a branch of applied mathematics. $\endgroup$ Dec 28, 2014 at 21:03
  • $\begingroup$ It clearly is "real" mathematics. $\endgroup$
    – Peter
    Oct 13, 2022 at 10:58
  • $\begingroup$ @Peter I don't know what this comment's purpose is, since the question already has an 8 year old accepted answer with this content. Your mimicking of my quotation marks and use of the word "clear" simply sounds condescending. I'm in my graduate in probability theory nowadays and to me it is also blatantly clear, but if you took care to look at the wikipedia article, you would notice that the question is justified from a layperson's perspective. Use of mathematical formalism in a wikipedia article does not alone signify a clearly defined area of mathematics (e.g. path integral formalism) $\endgroup$ Oct 14, 2022 at 16:36

3 Answers 3

6
$\begingroup$

Yes, Information Theory is a branch of mathematics, although its practitioners are often found in departments of Electrical and Computer Engineering or Computer Science.

$\endgroup$
0
$\begingroup$

Instead of mathematics, I would say it's probability/statistics. The basic idea, entropy, is defined in terms of the probability.

$\endgroup$
1
  • 9
    $\begingroup$ Probability is mathematics. Statistics is applied mathematics. $\endgroup$ Dec 28, 2014 at 21:30
0
$\begingroup$

Information theory is a mathematical field that deals with the transmission, processing, and storage of information. It was developed by Claude Shannon in the late 1940s, and has become an essential part of telecommunications and data storage systems.

The basic concepts of information theory can be described in simple terms. Information is measured in units called bits, and the amount of information that can be transmitted over a communication channel is limited by the bandwidth of that channel. The efficiency of data transmission is also limited by noise interference and other factors.

Information theory is a complex and fascinating topic, but these basics provide a good introduction to this important area of mathematics.

$\endgroup$

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .