$\mathbf{X}$ denotes the random vector $({X_1},{X_2},...,{X_n})$. The mutual information between $X$ and $Y$, $I(X;Y)$, is determined by the joint law of $p(X,Y)$, Given two random vectors $\mathbf{X}$ and $\mathbf{Y}$, characterized by a joint probability distribution ${p_{_\mathbf{XY}}}$, the probabilty distribution of $\frac{1}{n}I(\mathbf{X};\mathbf{Y})$ is referred to as the mutual information rate spectrum. In addition, the spectral-inf mutual information rate is defined as [1]:

${\rm{p - }}\mathop {\lim }\limits_{n \to \infty }{\rm{inf }}~\frac{1}{n}I(\mathbf{X};\mathbf{Y}) \buildrel \Delta \over = \sup \{ \beta :\mathop {\lim }\limits_{n \to \infty } {\rm{ \mathbb{P}[}}\frac{1}{n}I(\mathbf{X};\mathbf{Y}) < \beta {\rm{] = 0}}\}$

and respectively the spectral-sup mutual information rate is defined as:

${\rm{p - }}\mathop {\lim }\limits_{n \to \infty } {\rm{sup}}~\frac{1}{n}I(\mathbf{X};\mathbf{Y}) \buildrel \Delta \over = \inf \{ \alpha :\mathop {\lim }\limits_{n \to \infty } {\rm{ \mathbb{P}[}}\frac{1}{n}I(\mathbf{X};\mathbf{Y}) > \alpha {\rm{] = 0}}\}$

My Question:

What does this definition say? I am interested in some intuitions and measure-theoretic arguments in terms of convergence, like what we have known about the $lim inf$ or $lim sup$. Any help would be appreciated.

[1] T. S. Han, Information-Spectrum Methods in Information Theory, Springer, 2002.


migrated from mathoverflow.net Apr 20 '15 at 17:28

This question came from our site for professional mathematicians.


Mutual information is defined: $$I(X_1, \ldots, X_n; Y_1, \ldots, Y_n) = H(Y_1, \ldots, Y_n) - H(Y_1, \ldots, Y_n|X_1, \ldots, X_n)$$ There is nothing random about it. It does not make sense to talk about the "probability distribution of $I(X_1, \ldots, X_n; Y_1, \ldots, Y_n)$."

On the other hand, the following quantities do make sense: \begin{align} C_{inf} &= \liminf_{n\rightarrow\infty} \frac{1}{n}I(X_1, \ldots, X_n; Y_1, \ldots, Y_n)\\ C_{sup} &= \limsup_{n\rightarrow\infty} \frac{1}{n}I(X_1, \ldots, X_n; Y_1, \ldots, Y_n) \end{align} They could represent, for example, capacities in a communication channel that uses large blocklength codes with a particular input distribution. The value $C_{inf}$ is the capacity you can expect if you have any "sufficiently large" blocklength, while the value $C_{sup}$ is the optimistic capacity you can get if you use large blocklengths of cleverly chosen sizes.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy