Are there any good online resources to learn multivariate statistics? ( With topics such as Multiple Linear Regression, Principal Component Analysis, Factor Analysis, Cluster Analysis and Discriminant Analysis)
$\begingroup$ You may want to also ask this question over at stats.stackexchange.com $\endgroup$– user940Mar 10, 2011 at 14:58
Statsoft has a pretty good comprehensive online guide which I can recommend:
$\begingroup$ Thanks for the link. I didn't read it much yet but it is quite comprehensive. $\endgroup$– user8067Mar 10, 2011 at 15:20
Hopefully I can help with intuitively understanding PCA with my cartoon:
Visual Intuition (zoom in)
Intuition via utility
I think the main usage for PCA is to be able to categorise different distinct "things" e.g. Shiny cells vs. Dark cells in a way that leads to least error. E.g. Imagine sam was hiding behind me and I pinched a cell off the left side of his body then asked you to guess the color of the cell, by looking at the winning photo, or even the winning line, you can make a very good guess it will be a "dark cell".
Intuition via Analogy
So my understanding is that PCA is like taking a "picture" in a lower dimension, but the various methods used out there attempt to make the picture as informative as possible by deciding which "angle" to take the picture from (notice for 1D the angle of "squishing line" also vary).