0
$\begingroup$

What are correlation coefficients used for in PCA?

One can discover them through the PCA formulation, but what are they useful for?

$\endgroup$
0
$\begingroup$

PCA, at least in the data mining context, is usually used for linear dimensionality reduction. If you wanted to compute correlations among the original features of the dataset, there is no need to do that in the reduced space.

In fact, I would probably say that the correlations are actually the inputs to the PCA algorithm, and that the point of PCA is to find a feature space in which those correlations are reduced (in fact I think the principal components are linearly uncorrelated).

Maybe you mean the correlations between variables or features in the original vs reduced space? In this case, one can find that: $$ \rho(X_i,Y_j) = \omega_{ij} \frac{\sigma(Y_j)}{\sigma(X_i)} $$ where $\rho$ is the correlation, $\omega_{ij}$ is the weight of variable $i$ for principal component $j$, $X_i$ is a data feature, $Y_j$ is the $j$th principal component, and $\sigma$ measures the standard deviation.

The interpretation of this is that it tells us how important the $i$th feature is in computing the $j$th principal component, through $\omega_{ij}$. But recall that the PCs are essentially ordered by importance, i.e. how much variability in the data they explain. Hence, if we find that some features contribute heavily to earlier PCs, then we can say that they are more useful in explaining the data than those features which do not contribute much to the more important PCs.

Some useful related questions: [1], [2], [3], [4], [5], [6].

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.