Suppose that I have a matrix $X \in \mathbb{R}^{n \times f}$, which I interpret as a dataset where $n$ is the number of samples and $f$ is the number of features. For simplicity, suppose $f=2$, and let $f_1$ and $f_2$ represent the first and second columns of X (and so the first and second features), respectively.

Suppose we have a positive correlation between the features: $r(f_1,f_2)>0$. So then we can regress $f_2$ on $f_1$ to find $f_2 = \beta f_1 + \epsilon$ for some regression weight $\beta >0$ and where we may assume, if we'd like, that $\epsilon \sim N(0,\sigma^2)$.

Now consider the first principal component scores for this dataset -- or, roughly speaking, the "projection" of the data onto the first principal component. By this I mean that if we perform a singular value decomposition on $X$ to obtain $X=UDV'$, then we take the first column of $UD$ to be the score of each sample with respect to the first principal component. We can refer to the first column as $d_1 \vec{u_1}$.

Is it possible to express these first principal component scores, $d_1 \vec{u_1}$, -- mathematically, in closed form -- as a function of the original features and regression parameters (so, in terms of the collection {$f_1,f_2,\beta,\epsilon$})?


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.