I need to find the least squares solution using the Pseudo=oinverse, min$_\mathbf x $$_\in$$_\Bbb R$3||A$\mathbf x - \mathbf b$||$_2$, where

A= $\begin{bmatrix} 1 & 3 & 5\\ 1 & 1 & 0\\ 1 & 1 & 2\\ 1 & 3 & 3\\ \end{bmatrix}$ and $\mathbf b$= $\begin{bmatrix} 3\\ 5\\ 7\\ -3\\ \end{bmatrix}$.

And the pseudo inverse $A^\dagger$ = ($A^T$$A$)$^{-1}$$A^T$.

How do we apply $A^\dagger$ to solve the problem?


$A^\dagger b$ provides the vector $x$ that minimizes $\|Ax - b\|_2$ (in the case that $A^TA$ is invertible, this minimum is unique). Thus, the solution to your problem will be $$ \|A(A^\dagger b) - b\|_2 = \|(AA^\dagger - I)b\| $$

| cite | improve this answer | |
  • $\begingroup$ what is the proof that the pseudo-inverse minimizes the least square problem? $\endgroup$ – Pinocchio Oct 22 '17 at 5:40
  • $\begingroup$ @Pinocchio a few proofs are given here $\endgroup$ – Ben Grossmann Oct 22 '17 at 15:01
  • $\begingroup$ thanks I will check it out! But before I do, super quick question (to make sure my point is addressed explicitly somewhere). Does pseudo-inverse minimize least squares even if the matrix is not full column nor full row rank? i.e. $r<n,r<m$ $\endgroup$ – Pinocchio Oct 22 '17 at 16:57
  • $\begingroup$ @Pinocchio here's the way to think about it: among the least-squares solutions to $Ax = b$, $x = A^\dagger b$ is the one with minimal $\|x\|$. If $A$ has full column-rank, then there will be a unique least squares solution. If $A$ has full row-rank, then every least squares solution will be an exact solution. $\endgroup$ – Ben Grossmann Oct 23 '17 at 0:30

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.