(geometric) intuition of whitening

Question

I found this http://cis.legacy.ics.tkk.fi/aapo/papers/IJCNN99_tutorialweb/node26.html

But I still don't have an intuition of whitening. A diagonal covariance matrix means uncorrelated distributions, ok. But how can you see it from the distribution plot? The parallelogramme became a square with a different orientation as the original distribution. But how can you see that they're uncorrelated now?

Thanks in advance

Answer 1

To see from a distribution plot that two variables are uncorrelated is not easy in general. But in some cases one can deduce it from mere symmetry. First, assume that the mean is zero (if not, then shift the plot so that the origin is at the mean, at least for one variable). Then, the variables will be uncorrelated iff $E(XY)=0$. If the distribution has some symmetry, you can sometimes see that that expectation will be zero because the integral over one cuadrant cancels with another cuadrant. You should see that that is the case, for example for the distributions (always zero mean):

• Uniform over a non-rotated square
• Uniform over a rotated square
• Uniform over a non-rotated rectangle
• Gaussian, with elliptical level curves, non rotated
• Uniform over the triangle (0,1) (1,0) (-1,0)

The following, instead are correlated:

• Uniform over a rotated rectangle
• Uniform over a paralellogram
• Gaussian with elliptical rotated level curves

Another trick that sometimes helps is to draw the regression lines $E(X|Y)$ $E(Y|X)$ (this is very easy for uniform distributions, you should be able to do it for all the above examples), and if you find that $E(X|Y)=E(X)$ or $E(Y|X)=E(Y)$, then the variables are uncorrelated. Note that this is sufficient, not necessary.