# The mutual information rate spectrum

Definition:

$\mathbf{X}$ denotes the random vector $({X_1},{X_2},...,{X_n})$. The mutual information between $X$ and $Y$, $I(X;Y)$, is determined by the joint law of $p(X,Y)$, Given two random vectors $\mathbf{X}$ and $\mathbf{Y}$, characterized by a joint probability distribution ${p_{_\mathbf{XY}}}$, the probabilty distribution of $\frac{1}{n}I(\mathbf{X};\mathbf{Y})$ is referred to as the mutual information rate spectrum. In addition, the spectral-inf mutual information rate is defined as [1]:

${\rm{p - }}\mathop {\lim }\limits_{n \to \infty }{\rm{inf }}~\frac{1}{n}I(\mathbf{X};\mathbf{Y}) \buildrel \Delta \over = \sup \{ \beta :\mathop {\lim }\limits_{n \to \infty } {\rm{ \mathbb{P}[}}\frac{1}{n}I(\mathbf{X};\mathbf{Y}) < \beta {\rm{] = 0}}\}$

and respectively the spectral-sup mutual information rate is defined as:

${\rm{p - }}\mathop {\lim }\limits_{n \to \infty } {\rm{sup}}~\frac{1}{n}I(\mathbf{X};\mathbf{Y}) \buildrel \Delta \over = \inf \{ \alpha :\mathop {\lim }\limits_{n \to \infty } {\rm{ \mathbb{P}[}}\frac{1}{n}I(\mathbf{X};\mathbf{Y}) > \alpha {\rm{] = 0}}\}$

My Question:

What does this definition say? I am interested in some intuitions and measure-theoretic arguments in terms of convergence, like what we have known about the $lim inf$ or $lim sup$. Any help would be appreciated.

[1] T. S. Han, Information-Spectrum Methods in Information Theory, Springer, 2002.

## migrated from mathoverflow.netApr 20 '15 at 17:28

This question came from our site for professional mathematicians.

Mutual information is defined: $$I(X_1, \ldots, X_n; Y_1, \ldots, Y_n) = H(Y_1, \ldots, Y_n) - H(Y_1, \ldots, Y_n|X_1, \ldots, X_n)$$ There is nothing random about it. It does not make sense to talk about the "probability distribution of $I(X_1, \ldots, X_n; Y_1, \ldots, Y_n)$."
On the other hand, the following quantities do make sense: \begin{align} C_{inf} &= \liminf_{n\rightarrow\infty} \frac{1}{n}I(X_1, \ldots, X_n; Y_1, \ldots, Y_n)\\ C_{sup} &= \limsup_{n\rightarrow\infty} \frac{1}{n}I(X_1, \ldots, X_n; Y_1, \ldots, Y_n) \end{align} They could represent, for example, capacities in a communication channel that uses large blocklength codes with a particular input distribution. The value $C_{inf}$ is the capacity you can expect if you have any "sufficiently large" blocklength, while the value $C_{sup}$ is the optimistic capacity you can get if you use large blocklengths of cleverly chosen sizes.