0
$\begingroup$

I stumbled upon the limit definition of the Moore-Penrose Pseudoinverse and I'm rather confused.

If we're only considering the left pseudoinverse case, then that definition is

$$A^{+} = \lim_{\alpha \to 0}(A^TA + \alpha I)^{-1}A^T$$

I understand the definition of the pseudoinverse without the limit there - that is, I understand the definition of the pseudoinverse like

$$A^{+} = (A^TA)^{-1}A^T$$

Because if you multiply this on the left of A you see it evaluates into the identity.

My best guess for the use of the limit term is to make sure the definition works for some scenarios where $A^TA$ is not invertible, but isn't $A^TA$ always invertible for any matrix $A$?

So, my question is: What is the point of the limit in the definition of the pseudoinverse?

Thanks

$\endgroup$
1
  • $\begingroup$ You cannot speak of its invertibility if $A$ isn't a square matrix. Even if it is a square matrix, it can be singular. Consider $A=0$ for instance. $\endgroup$
    – user1551
    Jul 4, 2021 at 7:48

1 Answer 1

Reset to default
1
$\begingroup$

$A^TA$ is invertible if $A$ is (just take determinant). Adding a positive matrix like $\alpha I$ makes it positive, with positive eigenvalues. Therefore, it has an inverse and the definition makes sense as long as you can prove that the limit exists and has the right properties.

$\endgroup$
7
  • $\begingroup$ Little confused here. What do you mean by "it" in "Adding a positive matrix like $\alpha I$ makes "it" positive? If "it" is "$A^TA$" why do we care if $A^TA$ is positive? What has this to do with our ability to $A^TA$? Also, would adding $\alpha I$ even gurantee that $A^TA + \alpha I$ is positive, since it only adds positive numbers along the diagonal? $\endgroup$
    – adam dhalla
    Jul 4, 2021 at 1:04
  • $\begingroup$ Yes, "it" means $A^TA$. This matrix is non-negative. When you add a positive multiple of the identity, you make it positive, with all its eigenvalues positive. Hence invertible. $\endgroup$
    – GReyes
    Jul 4, 2021 at 1:27
  • $\begingroup$ Alright, I think I get it now - by adding positive multiples of the identity, you shift the eigenvalues in the positive direction. If all the eigenvalues are positive, you have a positive definite matrix that is guaranteed invertible. Now, the part I'm wondering on is why we take the limit as the multiple approaches zero? What if the $\alpha$ term has to be a lot larger than 0 to gurantee our matrix has all positive eigenvalues? $\endgroup$
    – adam dhalla
    Jul 4, 2021 at 1:54
  • $\begingroup$ @DownstairsPanda The eigenvalues of $A^\top A + \alpha I$ are $\alpha + \lambda_i$ where $\lambda_i$ are the eigenvalues of $A^\top A$. We already know $\lambda_i \ge 0$, so the eigenvalues of $A^\top A + \alpha I$ are strictly positive for any $\alpha > 0$. $\endgroup$
    – angryavian
    Jul 4, 2021 at 1:59
  • 1
    $\begingroup$ Right. As for your previous question, the idea is that you cannot define the pseudoinverse just by setting $\alpha=0$ (it does not make sense), so you appeal to a limiting procedure, this is very typical in Math. $\endgroup$
    – GReyes
    Jul 4, 2021 at 3:56

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .