Tagged Questions

6
votes
2answers
137 views

Can the entropy of a random variable with countably many outcomes be infinite?

Consider a random variable $X$ taking values over $\mathbb{N}$. Let $\mathbb{P}(X = i) = p_i$ for $i \in \mathbb{N}$. The entropy of $X$ is defined by $$H(X) = \sum_i -p_i \log p_i.$$ Is it possible ...
1
vote
1answer
216 views

How to calculate/approximate expectation of function of a binomial random variable?

I am stuck at following problem in my research. Suppose that $M=m$ is a random variable with binomial distribution and parameters $n,p$. The constants $r$ and $\gamma$ are greater than zero. ...
3
votes
1answer
208 views

Entropy of $X =\{1,2,\ldots,\infty\}$ with the probability of $\{1/2^1,1/2^2,\ldots,1/2^\infty\}$?

I'm studing for an information theory exam, maybe some of you can help me here with an exercise. What's the entropy of $X$ as $\{1,2,\ldots,n\}$ ($n$=infinity) where the probabilities are $P \{1/2^1, ...