What are the reasons to prefer eigenvalue decomposition over singular value decomposition for numerically computing the pseudo-inverse of a symmetric real matrix? In the case when you want to form the pseudo-inverse a covariance matrix and you also have access to the original data matrix, does using SVD make more sense?
Using SciPy as an example, the documentation for
Compute the (Moore-Penrose) pseudo-inverse of a Hermitian matrix.
Calculate a generalized inverse of a Hermitian or real symmetric matrix using its eigenvalue decomposition and including all eigenvalues with 'large' absolute value.
Why do we prefer eigenvalue decomposition in this case?
Suppose instead of computing the pseudo-inverse of an arbitrary matrix, you start instead with an $m \times n$ matrix $X$ and want to compute the pseudo-inverse of its covariance matrix $C = XX^*$. It seems like there are two obvious and closely related approaches.
Since $C$ is real and symmetric, it can be orthogonally diagonalized: $C = Q \Lambda Q^T$, where $Q$ contains eigenvectors, $\Lambda$ contains eigenvalues, and $QQ^T = I$. We can get the pseudo-inverse of $C$ from the pseudo-inverse of $\Lambda$ as $C^+ = Q \Lambda^+ Q^T$.
Alternatively, start from the singular value decomposition of $X$ as $X = U \Sigma V^*$. Then $C = U \Sigma \Sigma^* U^*$, and we can get the pseudo-inverse again from the pseudo-inverse of $\Sigma \Sigma^*$.
I thought that computing singular values was always a well-conditioned problem, and that this would mean SVD would be a better approach for computing the pseudo-inverse of a covariance matrix. Is that correct? What other concerns might I be overlooking?