Left inverses and numbers of solutions to Ax = b If an $m\times{}n$ matrix $A$ has linearly independent columns, it is left invertible. Only tall and square matrices can have this property.
If $A$ has linearly independent columns, $m > n$ (tall), and $Ax = b$, then $b$ is out of the range space of $A$, and so we need to approximate the solution $x$ (e.g. least squares). There are no solutions to this problem. I understand this.
What confuses me is that here, $A$ is left invertible, so some $Y$ exists such that $YA = I$. Then, $YAx = x = Yb$. This says that there are infinitely many solutions to $Ax = b$, for each left inverse $Y$. This is the opposite result that I would expect. 
Where have I gone wrong?
 A: This says that there is at most one solution.
Suppose $Ax_1=b$ and $Ax_2=b$. If $L$ is a left inverse of $A$, we get
$$
LAx_1=Lb,\qquad
LAx_2=Lb.
$$
Since $LA=I$, we obtain
$$
x_1=Lb,\qquad
x_2=Lb.
$$
Therefore $x_1=x_2$.
The fact that there may be infinitely many left inverses of $A$ has no role here. Note that we assume $x_1$ and $x_2$ are solution; there may be none. Hence the linear system either has no solution or it has a unique one.
A: The value of $x$ obtained pre-multiplying $b$ by the left pseudo-inverse of $A$ is just an approximation (obtained as $\hat x=\arg\min\|b-Ax\|$ over $x\in\mathbb{R}^n$), and, in general, not a solution of the system of linear equations (it can be a solution if and only if the space spanned by the columns of $A$ plus the vector $b$ has dimension $n$);
if you try to substitute $\hat x=A^{\dagger}b$, into $Ax$, you get just $AA^{\dagger}b$, and, in general, this is not equal to $b$ because, by definition the left pseudo-inverse does not satisfy the rule of the right pseudo-inverse, i.e. $AA^{\dagger}\ne I$.
