# Tag Info

## Hot answers tagged information-theory

3

You're looking for erasure coding. Number your group of people $1, \ldots, M$. Let's say your recipe $\mathbf{m}$ is $k$ bits long, i.e., $\mathbf{m} \in \mathbb{F}_2^k$. Pick a $[M, k, d > M-N]_2$ erasure code $c : \mathbb{F}_2^k \to \mathbb{F}_2^M$ and calculate the encoded recipe $\mathbf{t} = (t_1, \ldots, t_M) = c(\mathbf{m})$. Now hand out $t_i$ to ...

2

The notation $I(X;Y)$ does not mean that the information is a function of the variable $X$. In that sense, it's just a number, hence its derivative is zero. If you want an analogy, think of the expectation of a random variable $E(X)$ : it does not "depend on" $X$, it's a number; it would not make sense to ask about $d E(X)/dX$ In other sense, you could say ...

2

$E$ stands for expected value and $d(\cdot, \cdot)$ means distortion, however you define distortion. $X^n$ is a vector or $n$-tuple of random variables $X$. You could write this as $(X_1, X_2, \ldots, X_n)$ where the $X_i$ are random variables with a common distribution (but need not be independent). $f_n(X^n)$ is the result of compressing or encoding ...

1

It seems that I need to give a more detailed answer in the post but not only refer to the references for the main technical ideas. Let me start with a general picture. Estimating the entropy, from a statistical perspective, is by no means a unique problem among the problems of estimating functionals of parameters. The reason why it has attracted so much ...

1

According to your first paragraph, your "outer" channel has only two input symbols: $X\in \{000\cdots 0, 111....1\}=\{{\bf 0},{\bf 1}\}$ And $P(Y|X={\bf 0})=p^w(1-p)^{k-w}$ If they are equiprobable, by symmetry $$H(Y|X)=H(Y|X={\bf 0})=- \sum_{w=0}^k {k \choose w} p^w(1-p)^{k-w} \log \left[p^w(1-p)^{k-w}\right]$$ This coincides with your solution, as far ...

Only top voted, non community-wiki answers of a minimum length are eligible