Mathematics Stack Exchange is a question and answer site for people studying math at any level and professionals in related fields. It only takes a minute to sign up.
Sign up to join this community
It is well known that mutual information (MI) is a widely-used measure for quantifying statistical dependence between two random variables. Also, I read about other measures such as distance correlation, mutual information dimension, Hilbert-Schmidt independence criterion (HSIC), Kernel-based Constrained Covariance (COCO), etc.
My question is: what are the current state-of-the-art dependence measures other than the ones I mentioned?
I believe there is no such thing as state-of-the-art here, it might completely depend on the problem you are looking at! For example, are you working with images, are you solving an inference problem with a particular set of assumptions, ...? Those question might help guiding towards which measure to use.
What you will often see is that people in the information theory community use Mutual Information (but you can think of it as the KL divergence really since $I(X;Y)=D_{KL}(P_{X, Y}\|P_XP_Y)$), because it is an object that is pretty well understood, and we know how to compute it in many cases.
But as you mentioned, there are so many other possibilities! One measure of independence I find particularly interesting stems from the family of divergences called $f$-Divergences. From these, you can define a measure of independence by looking at $D_f(P_{X, Y}\|P_XP_Y)$, in the same way as the mutual information is simply $D_{KL}(P_{X, Y}\|P_XP_Y)$. The powerful thing is that here, for each choice of convex $f$, $D_{KL}(P_{X, Y}\|P_XP_Y)$ will give you some measure of independence in the end. With this family, you can retrieve many well-known divergences like KL divergence, Total Variation Distance, Hellinger Divergence, ...
I would say as a rule of thumb, look first at the problem you are trying to solve, and then figure out which measure(s) of independence would make sense for that particular case.
