0
$\begingroup$

I have some data for which I found the covariance matrix for:

$$\Sigma = \begin{bmatrix}3.33 & −1.00 & 3.33 & 33.00 \\ −1.00 & 1.58 & −1.92 & −13.92 \\ 3.33 & −1.92 & 62.92 & −23.42 \\ 33.00 & −13.92 & −23.42 & 398.92 \end{bmatrix}$$

After performing principal component analysis, I find the 4 PCs and transform my data set to the new coordinate system. However, finding a new covariance matrix on the PC-wise coordinate system data shows off-diagonals as nonzero:

$$\Sigma = \begin{bmatrix}408.92 & 0.42 & -3.92 & 0.00 \\ 0.42 & 60.25 & -0.42 & 0.00 \\ -3.92 & -0.42 & 0.92 & 0.00 \\ 0.00 & 0.00 & 0.00 & 0.00 \end{bmatrix}$$

From what I understand, the goal of PCA is to zero out joint variances. Is this possibly a rounding error? What should a covariance matrix look like after PCA?

$\endgroup$
3
  • $\begingroup$ Have you centered the data ? $\endgroup$
    – nicomezi
    Dec 18, 2018 at 12:06
  • $\begingroup$ @nicomezi Is it necessary? I've seen some PCA tutorials which describe it in terms of geometry and data is centered on the origin before the eigenvectors and eigenvalues are found. I thought it was perhaps optional for ease of illustration. $\endgroup$
    – gator
    Dec 18, 2018 at 12:08
  • $\begingroup$ It depends on the way your perform the PCA. You can read this answer : stats.stackexchange.com/a/189902 $\endgroup$
    – nicomezi
    Dec 18, 2018 at 12:11

1 Answer 1

0
$\begingroup$

I've concluded it's a rounding error. The off-diagonals should be zeroed after PCA confirmed by the below equality:

$$\Sigma - \lambda I = 0$$

$\endgroup$

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .