I would like to take data columns and compute the multivariate conditional entropy. For instance, suppose I have columns $A, B, C D, E$ and I want to compute the conditional entropy $H(E | A,B,C,D)$. Can someone give a little explanation of all this, pointing out the first joint variable $A,B,C,D$ vs the second variable $E$. Then please give the algorithm and math for calculating the conditional entropy.


suppose I have columns A,B,C,D,E

If by "columns" you mean random variables, for which you know the joint (five dimensional) probability function, then to compute $H(E | A,B,C,D)$ you just use the common defition of conditional entropy

$$H(X | Y) = \sum_Y P(Y=y) H(X | Y=y)$$

replacing $X$ by $E$ and $Y$ by your (multivariate) variable $(A,B,C,D)$.

  • $\begingroup$ Yes, I think random variable is the same as a data column. $\endgroup$ – Ben Sprott Jan 6 at 18:20

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.