# Tagged Questions

2answers
37 views

### Upper bound on the entropy of a sum two random variables

Let $X$ be a random variable such that $|X| \leq A$ almost surely, for some $A > 0$. Let $Z$ be independent of $X$ such that $Z \sim {\cal N}(0, N)$. My question is: How large can the entropy ...
1answer
175 views

### Entropy of the product of two random variables

Consider a random matrix $X$ and a random vector $Y$. Let the Shannon entropies $H(X) = H(Y) = n$. Is there a simple upper bound for entropy $H(XY)$? I believe $H(XY) \leq 2n$ as that is a simple ...
1answer
18 views

### how to find the maximum of the cross-entropy of a discrete random variable?

For a discrete random variable $x$, the cross entropy is $$H(x) = -(p_1\log p_1+\cdots+p_n\log p_n)$$ , so what is the maximum of $H(x)$? Here is what I tried, I compute the gradient as follows ...
0answers
195 views

### Entropy of matrix vector product

Consider a random $n$ by $n$ circulant matrix $M$ whose entries are chosen independently and uniformly from $\{0,1\}$. Let $M'$ be the $m$ by $n$ matrix which is formed by taking the first $m$ rows of ...
1answer
104 views

### Finding a specific sequence of digits in pi

Looking at the pifs project on GitHub and this question on SO has made me curious as to how feasible it is to find a specific sequence of digits within Pi. Essentially, on average, how many digits of ...
1answer
29 views

### calculate channel capacity and maximum conditional entropy

i want to know when it is equal channel capacity or $I(X,Y)$ maximum or where $I(X,Y)=H(X)-H(X\mid Y)=H(Y)-H(Y\mid X)$ now if we have two random variable with some specific distribution ...
1answer
32 views

### What's the sum of all events? (not the sum of all probabilites of events)

I need to calculate the entropy $h(X|Y)$, where $Y=X^2$. In this case, I suppose $\mathrm p(x|y)=\frac{1}{2}$. For the entropy \begin{align} h(X|Y) &= \int\limits_y \mathrm p(y)\ h(X|Y=y)\ ...
1answer
24 views

1answer
188 views

### Bits in a coin-toss experiment

This is not homework but an actual problem. We flip a fair coin ten times. This gives A$_1$ to A$_{10}$. Each coin toss = 10 bits. We flip another fair coin ten times. This gives B$_1$ to ...
1answer
119 views

### convergence of entropy and sigma-fields

This question is related to this one. Let $(X_1, X_2, \ldots)$ be a sequence of random variables such that each $X_n$ takes its values in a finite space, say $\{0,1\}$, and the $\sigma$-field ...
1answer
246 views

### Pinsker $\sigma$ Algebra

Let $(X,A,\nu)$ be a probability space and $T:X\to X$ a measure-preserving transformation. The Pinsker sigma algebra is defined as the lower sigma algebra that contains all partition P of measurable ...
1answer
2k views