If it is true then is there a reference of the fact that for discrete RVs X and Y the mutual information is invariant to the invertible mapping i-e $I(X;Y)=I(X';Y')$ where $X'=h(X)$ and $Y'=g(Y)$ with $h$ and $g$ are invertible functions.
$\begingroup$
$\endgroup$
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
This is an almost trivial observation. Due to the invertible mappings considered, the random variables $X$, $X'$, $(X, X')$ are equivalent in terms of entropy as well as information they provide about $Y$ (or, equivalently, $Y'$, $(Y,Y')$).
A bit more formally, it holds $$ \begin{align} I(X;Y) &= I(X, X'; Y, Y')\\ &=H(X,X')-H(X,X'|Y,Y')\\ &=H(X')-H(X'|Y')\\ &=I(X';Y') \end{align} $$
The steps are intuitive, but you may want to explicitly prove them by computing the standard entropy expressions using the (joint) probability distributions of the quantities involved.
