Does the following identity $$ I(X_1,X_2 ; Y_1, Y_2) = I(X_1; Y_1) + I(X_2; Y_2) $$ hold for mutual information for $(X_1, Y_1)$ and $(X_2, Y_2)$ independent?

Attempt: $p(x_1, x_2, y_1, y_2) = p(x_1, y_1) \cdot p(x_2, y_2)$ implies $p(x_1, x_2) = p(x_1) \cdot p(x_2)$ and $p(y_1, y_2) = p(y_1) \cdot p(y_2)$, where I use lower case $p$ to denote the probability of specific outcomes, e.g. $p(x)$ for $\Pr\{X=x\}$. Thus $$ \sum_{x_1, x_2, y_1, y_2} p(x_1, x_2, y_1, y_2) \cdot \log \frac{p(x_1, x_2, y_1, y_2)}{p(x_1, x_2) \cdot p(y_1, y_2)} = \\ \sum_{x_1, x_2, y_1, y_2} p(x_1, x_2, y_1, y_2) \cdot \log \frac{p(x_1, y_1)}{p(x_1) \cdot p(y_1)} \cdot \frac{p(x_2, y_2)}{p(x_2) \cdot p(y_2)} $$

  • $\begingroup$ Could you show what you tried? It would make it easier for us to guide you towards the answer. Also I believe on the right-hand side you have a typo since you dropped indices on the $Y$. $\endgroup$
    – adrien_vdb
    Jun 23 at 8:22
  • $\begingroup$ @adrien_vdb Thanks! I corrected the typo $\endgroup$
    – Cesare
    Jun 23 at 9:17
  • $\begingroup$ @adrien_vdb: I have added my attempt $\endgroup$
    – Cesare
    Jun 23 at 9:21
  • $\begingroup$ Great thanks for adding it, let me give you two hints. 1) You can use the following property of the logarithm $\log(x\cdot y) = \log(x) +\log(y)$. and 2) Use the fact that probability distributions sum up to 1 --> $\sum_{x, y} p(x, y) = 1$ $\endgroup$
    – adrien_vdb
    Jun 23 at 13:47
  • $\begingroup$ Hi. Thank, but is the derivation up to the point that I got to correct? Cause if it is it does imply that the relationship is correct, no? $\endgroup$
    – Cesare
    Jun 23 at 14:18


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.