Let's assume I have three probability distributions A,B,C.
The entropy of each is 1.58, with joint entropy 1.58.
Calculating mutual entropy with formula I(A,B,C) = H(A)+H(B)+H(C)-J(A,B,C) results in 3.16.
How could this be explained? This means there is more common information than there is a capacity of a channel. Could you explain this?