Mathematics Stack Exchange is a question and answer site for people studying math at any level and professionals in related fields. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

Suppose I know the capacity of channel C1 and the capacity of another channel C2. Both are achieved by a uniform distribution. Both have a binary input.

Now I have a random variable Z which takes values {0,1} uniformly. I build a system that has a binary input, which is multiplexed to C1 if Z=0 and to C2 if Z=1. Outputs can be combined together (i.e. if C1 had outputs {0,1,2} and C2 has outputs {0,2}, then the new system can have outputs {0,1,2}).

What would be the capacity of the combined channel?

share|cite|improve this question
up vote 1 down vote accepted

Let the new channel obtained by multiplexing channels $C_1$ and $C_2$ be called $C_\frac{1}{2}$. With some abuse of notation we'll refer to the capacities of these channels as $C_1$, $C_2$ and $C_\frac{1}{2}$ as well. Let us call the binary input as $X$ and the output as $Y$.

Note that $$p_\frac{1}{2}(y|x) = \frac{1}{2}p_1(y|x) + \frac{1}{2}p_2(y|x)$$ Suppose now that $C_\frac{1}{2}$ achieves its capacity at a certain distribution of $X$, say $p^*(x)$. We know that $I(X;Y)$ is a convex function of $p(y|x)$ for a fixed $p(x)$. Fixing the distribution of$X$ to be $p^*(x)$, we get the inequality $$ C_\frac{1}{2} = I_\frac{1}{2}(X;Y) \leq \frac{1}{2}I_1(X;Y) + \frac{1}{2}I_2(X;Y) \leq \frac{1}{2}C_1 + \frac{1}{2}C_2$$ This upper bound is tight (to see this consider two channels whose outputs don't overlap).

Your question asks the for the exact value of $C_\frac{1}{2}$. The best you can do without knowing the specifics of the channels is the above bound. One can ask the question

Is $p^*(x)$ uniform?

Though it may seem intuitively so, its not true.

Say $C_1$ looks like: $0-->a,b$ and $1 -->c,d$ with some probabilites and for $C_2$ : $0-->c,d$ and $1-->a,b$. Both these channels achieve capacity for a uniform distribution on $X$, however mixing these channels you can create all sorts of ugly looking channels which won't achieve capacity at uniform input distribution.

share|cite|improve this answer

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.