# Can off-diagonals be nonzero for covariance matrix after PCA?

I have some data for which I found the covariance matrix for:

$$\Sigma = \begin{bmatrix}3.33 & −1.00 & 3.33 & 33.00 \\ −1.00 & 1.58 & −1.92 & −13.92 \\ 3.33 & −1.92 & 62.92 & −23.42 \\ 33.00 & −13.92 & −23.42 & 398.92 \end{bmatrix}$$

After performing principal component analysis, I find the 4 PCs and transform my data set to the new coordinate system. However, finding a new covariance matrix on the PC-wise coordinate system data shows off-diagonals as nonzero:

$$\Sigma = \begin{bmatrix}408.92 & 0.42 & -3.92 & 0.00 \\ 0.42 & 60.25 & -0.42 & 0.00 \\ -3.92 & -0.42 & 0.92 & 0.00 \\ 0.00 & 0.00 & 0.00 & 0.00 \end{bmatrix}$$

From what I understand, the goal of PCA is to zero out joint variances. Is this possibly a rounding error? What should a covariance matrix look like after PCA?

• Have you centered the data ? Dec 18, 2018 at 12:06
• @nicomezi Is it necessary? I've seen some PCA tutorials which describe it in terms of geometry and data is centered on the origin before the eigenvectors and eigenvalues are found. I thought it was perhaps optional for ease of illustration. Dec 18, 2018 at 12:08
• It depends on the way your perform the PCA. You can read this answer : stats.stackexchange.com/a/189902 Dec 18, 2018 at 12:11

$$\Sigma - \lambda I = 0$$