4
votes
0answers
29 views

Looking for a a measure-theoretic treatment of “differential entropy”

If $X$ is a discrete random variable, its entropy $H(X)$ is usually defined as something along the lines of $-\sum \def\P{\mathbb{P}}\P(x) \log_2( \P(x))$, where the sum ranges over all the possible ...
3
votes
0answers
104 views

Reference for a transformation

Has the (Lebesgue-)ergodic transformation $T: \{0,1\}^{\mathbb{N}} \to \{0,1\}^{\mathbb{N}}$ defined by $T(x(0)x(1)x(2)\cdots) = x(1)x(3)x(5)\cdots$ been well-studied? If so, where? Does it have a ...
1
vote
1answer
52 views

approximate $[0, 1]$ continuous function with 2d basis.

everyone. I've been thinking of this problem when reading papers about Fourier series. I think I can state my question as follows: in the interval $[0, 1]$, I want to approximate an unknown ...
1
vote
1answer
66 views

generalization of base-n notation from naturals to fractions

not exactly sure how to best ask this. base-$n$ notation involves a series of digits written where each digit is a natural number less than $n$. is there some math/theory generalization of ...
0
votes
1answer
93 views

Space : Kolmogorov complexity :: time and space : ___?

It's well-known that the Kolmogorov complexity is uncomputable, essentially because of the halting problem: you can list all programs of length less than one known to generate a given string, but you ...
1
vote
3answers
281 views

Probability books useful for Information Theory?

Can you recommend me a list of good Probability Books for self-studying, with good explanations and introductions for Information Theory and not for the typical statistical subjects?
0
votes
1answer
86 views

How to compute Shannon information?

Given a string of random symbols with yet a priori unknown distribution, what are the known algorithms to compute its Shannon entropy? $$H = - \sum_i \; p_i \log p_i$$ Is there an algorithm to ...