Using pseudo inverse to solve least squares problem

I need to find the least squares solution using the Pseudo=oinverse, min$_\mathbf x $$_\in$$_\Bbb R$3||A$\mathbf x - \mathbf b$||$_2$, where

A= $\begin{bmatrix} 1 & 3 & 5\\ 1 & 1 & 0\\ 1 & 1 & 2\\ 1 & 3 & 3\\ \end{bmatrix}$ and $\mathbf b$= $\begin{bmatrix} 3\\ 5\\ 7\\ -3\\ \end{bmatrix}$.

And the pseudo inverse $A^\dagger$ = ($A^T$$A)^{-1}$$A^T$.

How do we apply $A^\dagger$ to solve the problem?

$A^\dagger b$ provides the vector $x$ that minimizes $\|Ax - b\|_2$ (in the case that $A^TA$ is invertible, this minimum is unique). Thus, the solution to your problem will be $$\|A(A^\dagger b) - b\|_2 = \|(AA^\dagger - I)b\|$$
• thanks I will check it out! But before I do, super quick question (to make sure my point is addressed explicitly somewhere). Does pseudo-inverse minimize least squares even if the matrix is not full column nor full row rank? i.e. $r<n,r<m$ – Pinocchio Oct 22 '17 at 16:57
• @Pinocchio here's the way to think about it: among the least-squares solutions to $Ax = b$, $x = A^\dagger b$ is the one with minimal $\|x\|$. If $A$ has full column-rank, then there will be a unique least squares solution. If $A$ has full row-rank, then every least squares solution will be an exact solution. – Ben Grossmann Oct 23 '17 at 0:30