# Tagged Questions

50 views

### How does the presenter in this video derive this formula?

I am watching this coursera video on entropy (in the information theory sense of the word). Right around the two minute mark the presenter shows two forms for H(p). The first (after the equals sign) ...
181 views

### What do two number on top of each other in square brackets mean?

Im currently going through "Universal Portfolios with Side Information" by Cover and Ordentlich [96]. Near the end of the paper, they provide a formula for calculating weights of a Universal Portfolio ...
107 views

### Understanding notation difference between mutual information and information divergance

The mutual information is defined on random variables. That is, $I(X;Y)$ denotes the mutual information between random variables $X$ and $Y$. On the other hand, the the Kullback-Leibler divergence is ...
47 views

### What is being maximised in the channel capacity formula?

The channel capacity formula is given as such: $$C=\max_{p(x)}I(X,Y)$$ Does this mean that it is the maximum probability multiplied by the mutual information, or is something else being maximised ...
Exercise 2.27 in Elements of Information Theory (2nd ed.) reads: Let $\mathbf{p} = (p_{1}, p_{2}, \ldots, p_{m})$ be a probability distribution on $m$ elements (i.e., $p_{i} \geq 0$ and ...