Take the 2-minute tour ×
Mathematics Stack Exchange is a question and answer site for people studying math at any level and professionals in related fields. It's 100% free, no registration required.

I am really confused after reading wikipedia...

What I don't get is how can something "bring" information, and in mathematics, how a mathematical object (like a set) can "have" information.

For example If I think about a set (a collection of objects), the only information that has is his cardinality? When we cansider a $n$-tuple (and is a set to in the sense of Kuratowsky) it has the order too. But we can easily see that two different $n$-tuples can have a very different amount of information.

For example if we have a big book of Category theory it has a huge amount of information.

Let say that that book $C$ is $n$-tuple, in other words an element of $A^n$ (where $A$ are letters of the alphabet + "space" + "new line" + "." + "," ... exc..)

We have that evry element of $A$ in the book $C$ appears $\mathcal l_C(x)$ times ($\mathcal l_C:A\rightarrow \Bbb N$)

We can create a new book $N$ where every letter appears in the the book often as in the book $C$ or in other words $\mathcal l_C(x)=\mathcal l_ N(x)$.

In easy words, the book $N$ is made of the same amount of stuff of the book $C$ but I want to write the book $L$ typing $\mathcal l_C(a)$ times the letter $a$ then we type $\mathcal l_C(b)$ times the letter $b$ and so on...

But the book $N$ does not say anything about the categories ... the only thing (information) that give us is the number of letters used in the category theory's book.

That really make a big confusion in my mind.

So my qestions are:

$1-$ Why the books $C$ and $N$ are made of the same components but $N$ lost all the informations of $C$? Where is all the information of $C$?

$2-$ How we define the information (as a quantity) of a mathematical object? How is it relatad to the Information as physical quantity?

PS:I don't know what is the correct tag.

share|improve this question
information-theory seems like a reasonable tag. –  joriki May 6 '13 at 8:43
add comment

1 Answer

up vote 2 down vote accepted

The book $N$ contains much less information than the book $C$: it tells you how many times each letter appeared but not what order they appeared in (that's where the information is).

A simpler model for understanding what's going on here is to consider binary strings, so words on the alphabet $\{ 0, 1 \}$. There are $2^n$ possible binary strings of length $n$ but only $n+1$ possible counts of $0$s and $1$s in such a binary string.

A very rough description of how information works is that you convey someone $\log_2 N$ bits of information if you send them an object from a set with $N$ elements. But there is more to it than this.

share|improve this answer
But even when we transform the book $N$ in the book $C$ we lose all the information of the order of $N$ (that as you said is where the information is) but $C$ is indeed more meaningful than $C$ he lost the information of the order of $N$, so the "order" of $C$ has more information thand the "order" of $N$... –  MphLee May 6 '13 at 11:59
@Mph: yes. There are more possibilities for the former than the latter. –  Qiaochu Yuan May 6 '13 at 17:59
Let's see if I follow you...My books $C$ and $N$ belong to a set with $|A|^n$ elements (where $A$ is my alphabet and $n$ is the length of the book), so if I send to a friend one of my books I'm going to send him $\log_2(|A|^n)$bits of informations? And if my book is written in base $2$ (and my alphabet is $\{ 0, 1 \}$ ) it will have lengt $b$ because we have $information(C)=\log_2(|A|^n)=\log_2(2^b)=b$? Can you give me some links, of easy free introduction to informaton theory? Thanks in advance, and sorry for all these questions. –  MphLee May 7 '13 at 10:37
add comment

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.