1
$\begingroup$

We have five random variables $N_1,N_2$ and $Z_1, Z_2,Z_3$, such that

  1. $N_1$ and $N_2$ are mutually independent,
  2. $Z_1$ and $Z_2$ are mutually independent,
  3. $N_1$ and $Z_2$ are mutually independent,
  4. $N_2$ and $Z_1$ are mutually independent,
  5. $Z_1$ and $Z_2$ are conditionally independent given $Z_3$.

Given above conditions, is the following mutual information relationship valid:

$I(N_1\: \: N_2 \:;\: Z_3 \: \Big| \: Z_1 \: \: Z_2)=I(N_1 \:;\: Z_3 \: \Big| \: Z_1 )+I(N_2 \:;\: Z_3 \: \Big| \: Z_2).$

$\endgroup$
  • 1
    $\begingroup$ 2 and 5 are the same $\endgroup$ – leonbloy Oct 3 '16 at 23:29
  • $\begingroup$ Question edited. $\endgroup$ – kaka Oct 4 '16 at 0:22
1
$\begingroup$

Chain rule gives you (there is another way of applying chain rule here but same steps can be applied to it.): $$ I(N_1\: \: N_2 \:;\: Z_3 \: \Big| \: Z_1 \: \: Z_2)=I(N_1 \:;\: Z_3 \: \Big| \: Z_1,Z_2 )+I(N_2 \:;\: Z_3 \: \Big| \: Z_2,Z_1,N_1). $$ To have the proposed equality, you need: $$ I(N_1 \:;\: Z_3 \: \Big| \: Z_1,Z_2 )= I(N_1 \:;\: Z_3 \: \Big| \: Z_1) $$ and $$ I(N_2 \:;\: Z_3 \: \Big| \: Z_2,Z_1,N_1)=I(N_2 \:;\: Z_3 \: \Big| \: Z_2). $$ In general, this is not valid. Take $Z_3=(W_1,W_2)$ and $Z_1=N_1+W_1$ and $Z_2=N_2+W_2$ and $W_1,W_2,N_1,N_2$ are independent. You can check that the above equalities will not hold. For instance: $$ I(N_1 \:;\: Z_3 \: \Big| \: Z_1,Z_2 )=I(N_1 \:;\: W_1,W_2 \: \Big| \: W_1+N_1,W_2+N_2 )\\ \neq\\ I(N_1 \:;\: Z_3 \: \Big| \: Z_1)=I(N_1 \:;\: W_1,W_2 \: \Big| \: W_1+N_1). $$ Intuitively $Z_2$ and $Z_1$ are not independent of $Z_3$ and therefore they can contain information about it, so removing them from mutual information can change its value.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.