1
vote
2answers
45 views

Upper bound on the entropy of a sum two random variables

Let $X$ be a random variable such that $|X| \leq A$ almost surely, for some $A > 0$. Let $Z$ be independent of $X$ such that $Z \sim {\cal N}(0, N)$. My question is: How large can the entropy ...
2
votes
1answer
175 views

Entropy of the product of two random variables

Consider a random matrix $X$ and a random vector $Y$. Let the Shannon entropies $H(X) = H(Y) = n$. Is there a simple upper bound for entropy $H(XY)$? I believe $H(XY) \leq 2n$ as that is a simple ...
5
votes
1answer
128 views

A tight lower bound for the entropy of the XOR of two random variables

Let $U$ be the uniform random variable over $n$-bit binary strings, and let $X$ be another random variable that is dependent on $U$ and ranges over $n$-bit binary strings. Assuming $I(X;U) \le ...
0
votes
1answer
84 views

Entropy Inequality

I have very hard time to prove the following inequality or to show a contradiction. $H(X_1,X_2,X_3) + H(X_1,X_2,X_4)+ H(X_1,X_3,X_4) + H(X_2,X_3,X_4) \leq 3(H(X_1,X_2) + H(X_3,X_4))$ The problem ...
1
vote
0answers
139 views

Inequality involving KL divergence

Following is a part of an answer which was not resolved when I tried to answer a question in mathoverflow. I thought it would be nice to discuss that here. Let $P$ and $Q$ be two distinct ...
22
votes
2answers
1k views

An information theory inequality which relates to Shannon Entropy

For $a_1,...,a_n,b_1,...,b_n>0,\quad$ define $a:=\sum a_i,\ b:=\sum b_i,\ s:=\sum \sqrt{a_ib_i}$. Is the following inequality true?: $${\frac{\Bigl(\prod a_i^{a_i}\Bigr)^\frac1a}a \cdot ...