Principal Component Analysis (PCA) has routinely caused me to question my understanding of mathematics, particularly linear algebra. Once again, PCA is present and I would like to engage to the community to confirm my previous formal education and recent references to refresh my understanding. To put it more succinct, is my interpretation of PCA (below), accurate?
PCA, as a dimension reduction technique is employed to reduce a large data-set (i.e. tons of variables) in to a more coherent and smaller data-set, while maintaining most of the 'principal' information.
This is where I am unsure of my understanding: The outputs from PCA - that is, the principal components are then utilized for further analysis. But to what end?
Let us take a routine example, the Iris data-set. Many programming (R, Python, SPS), focus on this data-set as a practical application of PCA. Note the output from Python's
Understanding that "identifies the combination of attributes..that account for the most variance in the data", what can we do with this output? My interpretation - which I belive is flawed is that - in the case of the PCA output (
sckit-learn), is that there is a strong correlation between 'virginica' and 'versicolor'. But is that all? Is this merely a processing technique which is then fed to machine learning models? It does not seem that the outputs from PCA (e.g. PC1, PC2, PC3) could be used for feature reduction. Plotting principal components, what information are we getting - since we are not really outputting what the principal components hold? If one were to 'present' the output in 2-d (as with the 'PCA of IRIS dataset', below) - would the intent to be to show that there is a stronger correlation between 'virginica' and 'versicolor'?
I have interpreted PCA as a pre-processing technique that allows one to identify the 'most important' (weighted, influential, etc.) features. Thus,the output I would expect from PCA would by something more akin to:
- PC1: Petal Width
- PC2: Petal Length