1
$\begingroup$

I've seen in class and in many different pdf files that the pseudoinverse $-$ i.e for those who don't know functions $f,g$ of finite-dimensional inner product spaces such that $f = fgf$ and $g = gfg$ $-$ are such that $fg$ and $gf$ are by construction self-adjoint.

However the pseudoinverse can also be defined by an SVD, and I would like to used this SVD to show the first definition, in particular to show that $fg$ and $gf$ are indeed self-adjoint.

We can assume that $f = U\sum V^*$ and $f$'s pseudoinverse is $g = V \sum^+U^*$. Then I guess we could try to determine $fg$ and $gf$ by calculation, but I've never used SVD before so I'm note quite sure how to properly use $f$ and $g$ to do so. Any idea?

Thanks for your time

$\endgroup$
9
  • $\begingroup$ What exactly is your definition of the pseudoinverse? As you say, these properties of the pseudoinverse are usually presented as a definition. $\endgroup$ May 18 at 16:33
  • $\begingroup$ Well from what I know there are usually 4 properties, which are: * $f = fgf$ * $g = gfg$ * $fg$ is self-adjoint * $gf$ is self-adjoint $\endgroup$
    – anon
    May 18 at 16:35
  • $\begingroup$ I can't read what you wrote. Are these the four properties that you're referring to? $\endgroup$ May 18 at 16:36
  • $\begingroup$ Yes they are, and I don't know how to show the 2 last one $\endgroup$
    – anon
    May 18 at 16:37
  • $\begingroup$ The fact that $fg$ and $gf$ are self-adjoint as part of the definition of the pseudoinverse means that it is meaningless to "show" or "prove" that the self-adjoint has these properties. If you want something to prove, you would have to start from an alternative definition of the psuedoinverse. For instance, you could instead define the pseudoinverse in terms of its SVD construction. $\endgroup$ May 18 at 16:39
2
$\begingroup$

Let $f = U \Sigma V^*$ be a singular value decomposition, which means that $U,V$ are unitary matrices and $\Sigma$ is a diagonal matrix with the same shape as $f$ and positive diagonal entries. We define the pseudoinverse of $f$ to be $g = f^+ = V \Sigma^+ U^*$, where $\Sigma^+$ is obtained by starting with $\Sigma$ and taking the reciprocal of each non-zero element on the diagonal, leaving the zeros in place, and then transposing the matrix.

Our goal is to show that $f,g$ satisfy the 4 defining properties of the Moore-Penrose pseudoinverse, i.e.

  1. $fgf = f$,
  2. $gfg = g$,
  3. $fg$ is self-adjoint,
  4. $gf$ is self-adjoint.

To that end: first show that the pair $\Sigma,\Sigma^+$ satisfies the required properties that is, show that

  1. $\Sigma\Sigma^+ \Sigma = \Sigma$
  2. $\Sigma^+ \Sigma \Sigma^+ = \Sigma^+$
  3. $\Sigma \Sigma^+$ is self-adjoint
  4. $\Sigma^+ \Sigma$ is self-adjoint.

These are easy to show because $\Sigma,\Sigma^+$ are diagonal matrices. From there, we can show that the conditions hold as follows.

Condition 1:

\begin{align} fgf &= (U \Sigma V^*)(V \Sigma^+ U^*)(U \Sigma V^*) \\ &= U \Sigma (V^*V) \Sigma^+ (U^*U) \Sigma V^* \\ & = U (\Sigma \Sigma^+ \Sigma) V^*= U \Sigma V^* = f \end{align}

Condition 2: \begin{align} gfg &= (V \Sigma^+ U^*)(U \Sigma V^*)(V \Sigma^+ U^*) \\ &= V \Sigma^+ (U^* U) \Sigma (V^*V) \Sigma^+ U^* \\ & = V (\Sigma^+ \Sigma \Sigma^+) U^*= V \Sigma^+ U^* = g \end{align}

Condition 3: \begin{align} fg &= (U \Sigma V^*)(V \Sigma^+ U^*) = U \Sigma (V^*V) \Sigma^+ U^* = U (\Sigma \Sigma^+) U^* \implies\\ (fg)^* &= [U (\Sigma \Sigma^+) U^*]^* = U^{**}(\Sigma \Sigma^+)^* U^* = U (\Sigma \Sigma^+) U^* = fg, \end{align}

and the proof that condition 4 holds is similar.

$\endgroup$
1
  • $\begingroup$ That is very well explained, thank you so much! $\endgroup$
    – anon
    May 18 at 18:45
2
$\begingroup$

(This answer is meant to provide an alternative proof of the existence and uniqueness of Moore-Penrose pseudoinverse that does not assume the existence of SVD. It is not intended to address the OP's question, which asks for an SVD-based proof.)

The existence part is actually quite straightforward. Let $U=(\ker f)^\perp$. Then $f$ is injective on $U$ (because $U\cap\ker f=0$) and zero on $U^\perp\,(=\ker f)$. So, if $r$ is the rank of $f$, we have $\dim U=\dim f(U)=r$.

To prove the existence of $g$, since $f$ is injective on $U$, we may define $$ g(w)= \begin{cases} u\ \text{ when } w=f(u) \text{ for some } u\in U,\\ 0\ \text{ when } w\in f(U)^\perp. \end{cases}\tag{1} $$ With the four given conditions, we have \begin{cases} fgf(u)=f(\color{red}{g(f(u))})=f(u) &\text{ for each } u\in U,\\ fgf(v)=0=f(v) &\text{ for each } v\in U^\perp=\ker f \end{cases} Hence $fgf=f$. In turn, $$ gfg(w)=\begin{cases} g\color{red}{fgf}(u)=gf(u)=g(w) &\text{ when } w=f(u) \text{ for some } u\in U,\\ gf(0)=0=g(w) &\text{ when } w\in f(U)^\perp. \end{cases} $$ Hence $gfg=g$. Moreover, as $(fg)|_{f(U)}=\operatorname{id}$ and $fg|_{f(U)^\perp}=0$, $fg$ has a matrix representation $I_r\oplus0$ in any combined orthonormal basis of $f(U)$ and $f(U)^\perp$. Therefore $fg$ is self-adjoint. Similarly, as $(gf)|_U=\operatorname{id}$ and $gf|_{U^\perp}=0$, $gf$ is also self-adjoint.

The uniqueness part is trickier. A compact algebraic proof can be found in Wikipedia, but I will give a more geometric proof here. We want to prove that if $g$ satisfies the four given conditions, it must be defined as in $(1)$. First, suppose $w=f(u)$ for some $u\in U$. If $gf(u)\not\in U$, then there exists some $k\in U^\perp=\ker f$ such that $$ 0\ne\langle gf(u),k\rangle=\langle u,(gf)^\ast(k)\rangle=\langle u,gf(k)\rangle=0, $$ which is a contradiction. Therefore $gf(u)$ lies inside $U$. But then as $fgf=f$ is injective on $U$, we must have $gf(u)=u$. Hence $g(w)=u$, i.e. $g$ is defined as in the first case in $(1)$.

Next, suppose $w\in f(U)^\perp$. Since $gfg=g$, if $g(w)$ is nonzero, $fg(w)$ must be nonzero too. However, as $fgfg=fg$ is self-adjoint, we arrive at the contradiction that $$ 0<\langle fg(w),fg(w)\rangle=\langle w,fgfg(w)\rangle=\langle w,fg(w)\rangle=0. $$ Therefore $g(w)$ must be zero, i.e. $g$ must be defined as in the second case in $(1)$. Now we are done.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy