How information works?

I am really confused after reading wikipedia...

What I don't get is how can something "bring" information, and in mathematics, how a mathematical object (like a set) can "have" information.

For example If I think about a set (a collection of objects), the only information that has is his cardinality? When we cansider a $n$-tuple (and is a set to in the sense of Kuratowsky) it has the order too. But we can easily see that two different $n$-tuples can have a very different amount of information.

For example if we have a big book of Category theory it has a huge amount of information.

Let say that that book $C$ is $n$-tuple, in other words an element of $A^n$ (where $A$ are letters of the alphabet + "space" + "new line" + "." + "," ... exc..)

We have that evry element of $A$ in the book $C$ appears $\mathcal l_C(x)$ times ($\mathcal l_C:A\rightarrow \Bbb N$)

We can create a new book $N$ where every letter appears in the the book often as in the book $C$ or in other words $\mathcal l_C(x)=\mathcal l_ N(x)$.

In easy words, the book $N$ is made of the same amount of stuff of the book $C$ but I want to write the book $L$ typing $\mathcal l_C(a)$ times the letter $a$ then we type $\mathcal l_C(b)$ times the letter $b$ and so on...

But the book $N$ does not say anything about the categories ... the only thing (information) that give us is the number of letters used in the category theory's book.

That really make a big confusion in my mind.

So my qestions are:

$1-$ Why the books $C$ and $N$ are made of the same components but $N$ lost all the informations of $C$? Where is all the information of $C$?

$2-$ How we define the information (as a quantity) of a mathematical object? How is it relatad to the Information as physical quantity?

PS:I don't know what is the correct tag.

-
information-theory seems like a reasonable tag. –  joriki May 6 '13 at 8:43

The book $N$ contains much less information than the book $C$: it tells you how many times each letter appeared but not what order they appeared in (that's where the information is).
A simpler model for understanding what's going on here is to consider binary strings, so words on the alphabet $\{ 0, 1 \}$. There are $2^n$ possible binary strings of length $n$ but only $n+1$ possible counts of $0$s and $1$s in such a binary string.
A very rough description of how information works is that you convey someone $\log_2 N$ bits of information if you send them an object from a set with $N$ elements. But there is more to it than this.
But even when we transform the book $N$ in the book $C$ we lose all the information of the order of $N$ (that as you said is where the information is) but $C$ is indeed more meaningful than $C$ he lost the information of the order of $N$, so the "order" of $C$ has more information thand the "order" of $N$... –  MphLee May 6 '13 at 11:59
Let's see if I follow you...My books $C$ and $N$ belong to a set with $|A|^n$ elements (where $A$ is my alphabet and $n$ is the length of the book), so if I send to a friend one of my books I'm going to send him $\log_2(|A|^n)$bits of informations? And if my book is written in base $2$ (and my alphabet is $\{ 0, 1 \}$ ) it will have lengt $b$ because we have $information(C)=\log_2(|A|^n)=\log_2(2^b)=b$? Can you give me some links, of easy free introduction to informaton theory? Thanks in advance, and sorry for all these questions. –  MphLee May 7 '13 at 10:37