4
$\begingroup$

I feel confused about the uniqueness of the Moore-Penrose inverse generated from SVD. For any matrix $A$, if $X$ satisfied $$AXA=A, XAX=X, (AX)^\mathrm{T}=AX, (XA)^\mathrm{T}=XA $$then $X$ is called the Moore-Penrose inverse of $A$.

If $A$ has the SVD(singular value decomposition)$$A=P\left[\begin{matrix}\Lambda_r&0\\0&0\end{matrix}\right]Q^\mathrm{T}$$

then it is easy to prove that$$A^+ = Q\left[\begin{matrix}\Lambda_r^{-1}&0\\0&0\end{matrix}\right]P^\mathrm{T}$$ is a Moore-Penrose inverse.

If $X$ and $Y$ are both Moore-Penrose inverse of $A$, from the equation$$X=XAX=X(AX)^\mathrm{T}=XX^\mathrm{T}A^\mathrm{T}=XX^\mathrm{T}(AYA)^\mathrm{T}=X(AX)^\mathrm{T}(AY)^\mathrm{T}=(XAX)AY=XAY=(XA)^\mathrm{T}YAY=A^\mathrm{T}X^\mathrm{T}A^\mathrm{T}Y^\mathrm{T}Y=A^\mathrm{T}Y^\mathrm{T}Y=(YA)^\mathrm{T}Y=YAY=Y$$ we can see that the Moore-Penrose inverse is unique.

However, the Moore-Penrose inverse depends on the SVD and SVD is not unique. How to explain it?

|cite|improve this question
$\endgroup$
  • 1
    $\begingroup$ A nice conceptual definition of the pseudoinverse of a matrix (or operator) $A$ is that it is the linear transformation $L$ that takes a vector $b$ as input and returns as output the vector $x$ of least norm which satisfies $Ax = \hat b$, where $\hat b$ is the projection of $b$ onto the range of $A$. This linear transformation $L$ is perfectly well defined. If $A$ is a matrix, then the term "pseudoinverse" usually refers to the matrix representation of $L$ (with respect to the standard bases), rather than $L$ itself. Clearly the matrix representation of $L$ is unique. $\endgroup$ – littleO Dec 31 '18 at 12:14
  • 1
    $\begingroup$ @littleO I can understand why the pseudo inverse is unique. But I don’t know how to explain the uniqueness if the inverse is generated from SVD form since SVD is not unique. $\endgroup$ – bregg Dec 31 '18 at 12:28
  • 1
    $\begingroup$ Just because $Q,\Lambda_R,P$ are not unique does not imply that the product $A^+ = Q\left[\begin{matrix}\Lambda_r^{-1}&0\\0&0\end{matrix}\right]P^\mathrm{T}$ is not unique $\endgroup$ – user617446 Dec 31 '18 at 14:04
  • $\begingroup$ The reason is similar to the fact that $1=2\cdot2^{-1}=3\cdot3^{-1}$. By the way, there are other methods for computing the pseudoinverse. If $A=BC$ is a full-rank decomposition, with $B$ left invertible and $C$ right invertible, then $A^+=C^T(CC^T)^{-1}(BB^T)^{-1}B^T$. $\endgroup$ – egreg Dec 31 '18 at 15:38
  • $\begingroup$ @egreg if $B$ is left but not right-invertible, then $BB^T$ fails to be invertible. Something is wrong with your formula $\endgroup$ – Omnomnomnom Dec 31 '18 at 19:18
2
$\begingroup$

The non-uniqueness of SVD can be characterized as follows: suppose that $A = P_0 \Sigma Q_0^T$ is one SVD of $A$. Moreover, suppose that the singular values of $A$ are $s_1$ with multiplicity $k_1$, $s_2$ with multiplicity $k_2$, and so forth, with $s_m = 0$ having multiplicity $k_m = n - r$. That is, we have $$ \Lambda_r = \pmatrix{s_1 I_{k_1} \\ & \ddots \\ && s_{m-1} I_{k_{m-1}}}, \quad \Sigma = \pmatrix{\Lambda_r \\ & 0_{k_m}} $$ Then $A = P\Sigma Q^T$ will be a singular value decomposition of $A$ if and only if there exists an orthogonal matrix $U$ such that $P = P_0 U, Q = Q_0U$, and $U$ is a block-diagonal orthogonal matrix of the form $$ U = \pmatrix{U^{(1)}\\ & \ddots \\ && U^{(m)}} $$ where $U^{(j)}$ is (orthogonal and) of size $k_j \times k_j$.

With that in mind: if you'd like to prove that the pseudoinverse as constructed from SVD is well-defined (that is, uniquely defined regardless of one's choice of SVD), then it suffices to show that for any choice of $U$ of the form prescribed above, we have $$ [Q_0U] \pmatrix{\Lambda_r^{-1} \\ & 0} [P_0U]^T = Q_0 \pmatrix{\Lambda_r^{-1} \\ & 0} P_0 $$ It is straightforward (but in my opinion tedious) to show that this holds if we use the block-structure of $\Lambda_r^{-1}$ and block-matrix multiplication.

|cite|improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.