# Linked Questions

10 questions linked to/from Intuitive explanation of entropy
3answers
137 views

### How can they come up with the definition of entropy in information theory? [duplicate]

I have read some books about information theory but I don't have any ideas how can they find the definition of entropy? We have $$H(X)=-\sum_{x\in X}p(x)\, \text{log}\, p (x)$$ X is a discrete random ...
1answer
99 views

### Why do I find no clear answer for the log term in Shannon's theorem? [duplicate]

The explanations about Shannon's theorem speed up all of a sudden when they should tell us why he introduced the log term. They usually range from 'it's useful' to 'it's because there are two choices, ...
4answers
2k views

### Integral of odd function doesn't converge?

When I look up $$\int_{-1}^1 \dfrac{1}{x} dx$$ on Wolfram Alpha, it says it doesn't converge. While this is a sum of two diverging integrals, the two areas are clearly symmetric, and I'd assume the ...
3answers
7k views

### How is the formula of Shannon Entropy derived?

From this slide, it's said that the smallest possible number of bits per symbol is as the Shannon Entropy formula defined: I've read this post, and still not quite understand how is this formula ...
5answers
1k views

### Could the Riemann hypothesis be provably unprovable?

I don't know much about foundations and logic, so I ask forgiveness if my question is just plain stupid. Assume we have a statement of the form: There exist no $x\in X$ such that $P(x)$. where $X$...
2answers
723 views

### What does $-p \ln p$ mean if p is probability?

In statistical mechanics entropy is defined with the following relation: $$S=-k_B\sum_{i=1}^N p_i\ln p_i,$$ where $p_i$ is probability of occupying $i$th state, and $N$ is number of accessible ...
1answer
646 views

### What is the most extreme set 4 or 5 nontransitive n-sided dice?

A set of nontransitive dice is a set of dice whose face numbers are such that the relation "is more likely to roll a higher number than" is not transitive. (See wikipedia) For some sets, the ...
1answer
348 views

### Understanding information entropy

I have tried very very hard to get a good intuitive grasp on entropy. I read all the information theoretic intuitions on the web, but I have not exactly been able to understand what information is, ...
2answers
235 views

### Could someone show me a simple example of something being proved unprovable?

Could someone show me a simple example of something being proved unprovable? Pretty much what the title says, I want to understand a proof of some statement being proved unprovable. E: Please read ...
2answers
220 views

### What is the Geometric interpretation of $i^i$?

We know that $i^i$ is real. But how to explain it geometrically maybe in terms of rotation. like we can explain geometrically multiplication of two complex numbers and so on. Can someone show me a ...