Algebraic formula of the pseudoinverse (Moore-Penrose) of symmetric positive semidefinite matrixes

Question

I was reading the Wikipedia page about the Pseudoinverse or Moore-Penrose inverse, where they say that, given a generic matrix $A \in R^{nxm}$, if the matrix is full rank (i.e. rank=$min\{n,m\}$), then the pseudoinverse admits an algebraic formula, which is therefore given for both the right inverse and left inverse. Which is correct. However, the way they express this idea is a little bit too restrictive because there may exist also an algebraic derivation for some special cases of square singular matrixes.

For example, I was thinking that, if a matrix is a square symmetric positive semidefinite matrix $A \in R^{nxn}$, then it admits an orthogonal diagonalization of the kind $A=VDV^{T}$, where $D$ is the diagonal matrix storing the eigenvalues and $V$ is the matrix whose columns are the $n$ orthonormal linearly independent eigenvectors. If some of the eigenvalues are $0$, then this product can be simplified by taking the only non-zero eigenvalues (suppose they are $m<n$ non-zero eigenvalues, and put them on the diagonal of the reduced-size $mxm$ square matrix $D_{m}$) and the corresponding normalized eigenvectors (suppose $V_{m}$ is the $nxm$ matrix whose columns are this subset of eigenvectors). So we have $A=VDV^{T}=V_{m}D_{m}V_{m}^{T}$.

Since the columns of $V_{m}$ are linearly independent and orthonormal (and the same holds for the rows of $V_{m}^{T}$), then we can find the pseudoinverse of $A$ as

$$A^{+}=(V_{m}D_{m}V_{m}^{T})^{+}=V_{m}^{T+}(V_{m}D_{m})^{+}=V_{m}^{T+}D_{m}^{+}V_{m}^{+}=V_{m}D_{m}^{-1}V_{m}^{T}$$

where clearly $D_{m}$ is a $mxm$ square diagonal matrix with non-zero entries along the diagonal, then the pseudoinverse $D_{m}^{+}=D_{m}^{-1}$. And clearly the case where the matrix is positive definite can be seen as a special case (for $m=n$) of the previous one.

So my question is:

1) Is it correct to extend the algebraic derivation to some kinds of special square singular matrixes? In this specific case psd symmetric matrixes (like a covariance matrix), as seen above?

2) The above mentioned inversion holds because the Singular Value Decomposition coincides with the Spectral Decomposition for symetric psd matrixes (as singular values are the absolute values of eigenvalues, thus, if eigenvalues are positive, the eigenvalues coincide with the singular values of the matrix). In case the matrix is still symmetric, but negative semidefinite, is it correct to say that the above mentioned equivalence does NOT hold?

Many thanks!

Answer 1

1. I could not find the specific sentence you are referring to. You are using the word "derivation", which is not what the webpage mentions. Wikipedia states "$A^\dagger$ can be expressed as a simple algebraic formula [..] when $A$ has linearly independent columns, $A^\dagger = (A^* A)^{-1} A^*$" The simple formula is $(A^* A)^{-1} A^*$ which can not be applied to your case, as your matrix is not full rank. But your expression for the pseudo-inverse is covered in the SVD paragraph.
2. When $A$ is symmetric with negative eigenvalues, denote $A = W D W^T$ its eigendecomposition and denote $S = -D$. Then the singular value decomposition will be $A = W S (-W)^T$. And the formula for the pseudo-inverse via the SVD will work and still match your expression, $A = W_m D_m W_m^T$.