How would I go about expanding $I(X_1,...,X_n;Y_1,...,Y_n)$?

The chain rule exists for a single case, i.e.: $I(X_1,...,X_n;Y)=\sum^n_{i=1} I(X_i;Y|X_{i-1},...,X_1)$, but I'm having doubts if this can be applied to solve for the case of 2 sequences of random variables.

EDIT: the condition is that $(X_1,...,X_n)$ are independent, and I want to show that

$I(X_1,...,X_n;Y_1,...,Y_n)\geq \sum^n_{i=1}I(X_i;Y_i)$

  • $\begingroup$ Here's one way: $$I(X_1,...,X_n;Y_1^n)=\sum^n_{i=1} I(X_i;Y_1^n|X_{i-1},...,X_1) = \sum^n_{i=1} \sum^n_{j = 1} I(X_i; Y_j| X_1^{i-1}, Y_1^{j-1})$$ Typically, you expand things on the basis of some relation you know about $(X_i, Y_i)$, or to get the correct form of auxiliary random variable . There's no neat way to do this in general. Perhaps you want to add more context so that we may try to attempt a useful expansion of the same. $\endgroup$ – stochasticboy321 Oct 16 '15 at 0:22

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.