10 questions linked to/from Intuitive explanation of entropy
137 views

### How can they come up with the definition of entropy in information theory? [duplicate]

I have read some books about information theory but I don't have any ideas how can they find the definition of entropy? We have $$H(X)=-\sum_{x\in X}p(x)\, \text{log}\, p (x)$$ X is a discrete random ...
99 views

### Why do I find no clear answer for the log term in Shannon's theorem? [duplicate]

The explanations about Shannon's theorem speed up all of a sudden when they should tell us why he introduced the log term. They usually range from 'it's useful' to 'it's because there are two choices, ...
2k views

### Integral of odd function doesn't converge?

When I look up $$\int_{-1}^1 \dfrac{1}{x} dx$$ on Wolfram Alpha, it says it doesn't converge. While this is a sum of two diverging integrals, the two areas are clearly symmetric, and I'd assume the ...
7k views

### How is the formula of Shannon Entropy derived?

From this slide, it's said that the smallest possible number of bits per symbol is as the Shannon Entropy formula defined: I've read this post, and still not quite understand how is this formula ...
1k views

### Could the Riemann hypothesis be provably unprovable?

I don't know much about foundations and logic, so I ask forgiveness if my question is just plain stupid. Assume we have a statement of the form: There exist no $x\in X$ such that $P(x)$. where $X$...
723 views

### What does $-p \ln p$ mean if p is probability?

In statistical mechanics entropy is defined with the following relation: $$S=-k_B\sum_{i=1}^N p_i\ln p_i,$$ where $p_i$ is probability of occupying $i$th state, and $N$ is number of accessible ...
646 views

### What is the most extreme set 4 or 5 nontransitive n-sided dice?

A set of nontransitive dice is a set of dice whose face numbers are such that the relation "is more likely to roll a higher number than" is not transitive. (See wikipedia) For some sets, the ...
348 views

### Understanding information entropy

I have tried very very hard to get a good intuitive grasp on entropy. I read all the information theoretic intuitions on the web, but I have not exactly been able to understand what information is, ...
### What is the Geometric interpretation of $i^i$?
We know that $i^i$ is real. But how to explain it geometrically maybe in terms of rotation. like we can explain geometrically multiplication of two complex numbers and so on. Can someone show me a ...