Take the 2-minute tour ×
Mathematics Stack Exchange is a question and answer site for people studying math at any level and professionals in related fields. It's 100% free, no registration required.

If you have a matrix:

\begin{pmatrix} 2 & 0 & 0 \\ 0 & 4 & 0 \\ 0 & 0 & 4 \\ \end{pmatrix}

The eigenvalues are $2,4,4$. The eigenvectors are $(0,0,1)$, $(0,1,0)$ and $(1,0,0)$. What is the first principal components, second principal components, and third principal components? Is it just $Y_1 = X_1$, $Y_2 = X_2$ and $Y_3 = X_3$?

share|improve this question

1 Answer 1

up vote 1 down vote accepted

Note that your matrix is $$\begin{pmatrix} 2 & 0 & 0\\ 0 & 4 & 0\\ 0 & 0 & 4 \end{pmatrix} = 4 \begin{pmatrix} 0 \\ 0 \\ 1\end{pmatrix} \begin{pmatrix} 0 & 0 & 1 \end{pmatrix} + 4 \begin{pmatrix} 0 \\ 1 \\ 0\end{pmatrix} \begin{pmatrix} 0 & 1 & 0 \end{pmatrix} + 2 \begin{pmatrix} 1 \\ 0 \\ 0\end{pmatrix} \begin{pmatrix} 1 & 0 & 0 \end{pmatrix}$$ In general, the idea is to get the singular value decomposition of a matrix $A \in \mathbb{C}^{n \times n}$ as $$A = \sigma_1 u_1 v_1^* + \sigma_2 u_2 v_2^* + \cdots + \sigma_n u_n v_n^*$$ where $u_k,v_k \in \mathbb{C}^{n \times 1}$, $\sigma_1 \geq \sigma_2 \geq \cdots \geq \sigma_n \geq 0$, $u_k^* u_k = 1$ and $v_k^* v_k = 1$. The first $\ell$ principal components are the vectors $v_1,v_2,\ldots, v_{\ell}$.

(Typically when you talk of principal component analysis, the underlying matrix is Hermitian and hence $u_k = v_k$. So you could either consider $u_k$'s or $v_k$'s.)

In your case,

$$\text{The first principal component is either $\begin{pmatrix} 0 \\ 0 \\ 1\end{pmatrix}$ or $\begin{pmatrix} 0 \\ 1 \\ 0\end{pmatrix}$}$$

$$\text{The first two principal component are $\begin{pmatrix} 0 \\ 0 \\ 1\end{pmatrix}$ and $\begin{pmatrix} 0 \\ 1 \\ 0\end{pmatrix}$}$$

$$\text{The three principal components are $\begin{pmatrix} 0 \\ 0 \\ 1\end{pmatrix}$, $\begin{pmatrix} 0 \\ 1 \\ 0\end{pmatrix}$ and $\begin{pmatrix} 1 \\ 0 \\ 0\end{pmatrix}$}$$

EDIT You have the eigenvectors and eigenvalues paired up in a wrong order. The eigenvector corresponding to the eigenvalue $2$ is $\begin{pmatrix} 1 \\ 0 \\ 0\end{pmatrix}$ and similarly the others.

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.