# Interpretation of PCA

I am wondering if there is a practical interpretation of a principal component analysis: Consider you have a data matrix $X\in\mathbb{R}^{N\times p}$ and you perform a principal component analysis where you typically receive certain directions $v_1,...,v_q$, $q<p$, in $\mathbb{R}^N$ that explain the most of the variance in the data. Is there an interpretation of these principal components in terms of the original components, i.e. the variables $x_1,...,x_p$ that constitute the model. Think e.g. of $x_i$ being certain "variables" of a human body such as weight, blood pressure etc. that should be used to predict expected life time. If one now performs a PCA as described a above, one recognizes that certain linear combinations of the columns of $X$ explain most of the variance. If one wants to reduce the model (i.e. reduce the $p$), which variables do you exclude given the information of the PCA?

• Moreover, sparsity of the principal components doesn't necessarily map that well to variable reduction. Even if one of the original variables is dropped from many or almost all of the linear combinations constituting the new variables, you would still have it "in the model" so long as it obtained a non-zero value in a single new predictor variable. (More precisely, if you set up your data matrix such that the rows are samples and the columns are predictors, and you find the SVD of your data matrix as $X=U\Sigma V^T$, then you would need to retain the $i$th variable in your model so long as the $i$th row of V attained a non-zero value in a single column).