0
votes
2answers
64 views

Entropy of sum of two Uniform random variables

say $X$ and $Y$ are two identical, independent and discrete Uniform random variables and $Z=X+Y$. I do not know more about the random variables. Assuming $H(\cdot)$ to be the entropy of a random ...
1
vote
1answer
65 views

Maximising Entropy of Random Variable taking Positive Integer Values

A random variable $X$ takes positive integer values and $E[X]=6$. What distribution of the random variable $X$ maximises the entropy $H(X)$? What if $X$ can only take a finite number of values? ...
5
votes
0answers
114 views

A tight lower bound for the entropy of the XOR of two random variables

Let $U$ be the uniform random variable over $n$-bit binary strings, and let $X$ be another random variable that is dependent on $U$ and ranges over $n$-bit binary strings. Assuming $I(X;U) \le ...
1
vote
1answer
294 views

Maximum entropy joint distribution from marginals?

How does one find the maximum entropy joint distribution of two random variables X and Y given their marginal probability mass functions? I know: I have the marginals, meaning p(x) and p(y) are ...
0
votes
2answers
55 views

Entropy calculation

Let's say we have an unknown random variable whose entropy is $1.75$ Our job is to find minimum distributions for this random variable. What I wrote was: $$ p_1 \log_2\Big(\frac 1 {p_1}\Big) + \ldots ...
1
vote
3answers
398 views

Entropy of geometric random variable?

I am wondering how to derive the entropy of a geometric random variable? Or where I can find some proof/derivation? I tried to search online, but seems not much resources is available. Here is the ...
1
vote
1answer
51 views

Constructing Distribution By Coin Flipping

I am interested in any example of construction distribution by coin flipping. Actually I want to show the process of construction a random variable $X$ with distribution $(p_1,...,p_n)$ by coin ...
1
vote
0answers
60 views

convergence of discrete random variables with finite entropy

Let $Z$ be the set of discrete random variables on some probability space. Define the quantity $d(X_1,X_2)=h(X_1 \mid X_2)+h(X_2 \mid X_1)$ between two random variables $X_1, X_2 \in Z$. For $X \in Z$ ...
1
vote
1answer
86 views

Conditional entropy of sum of random variables

How can be proven that for random variables $A$ and $B$, and $C = A + B$, $$H(C\mid A) = H(B\mid A).$$ Also, would it be possible to determine if $H(C)$ would be greater than $H(A)$?