Why do rectangular matrices have no inverse that can solve Ax = v? We can reverse a linear transformation using an inverse matrix to find the input vector used in the original linear transformation:
$$A^{-1}A\vec{x} = A^{-1}\vec{v}$$
Let's say we have a 2D vector that we transform into a 1D "vector":
$$\begin{pmatrix} 
a & b 
\end{pmatrix}*\begin{pmatrix} 
x  \\ 
y   
\end{pmatrix} = \begin{pmatrix} 
ax + by
\end{pmatrix}       $$
Isn't there some matrix that can then convert that 1D vector back into the original 2D vector? Something like this:
$$\begin{pmatrix} 
c  \\ 
d 
\end{pmatrix}*\begin{pmatrix} 
ax + by
\end{pmatrix}  = \begin{pmatrix} 
x  \\ 
y  
\end{pmatrix}      $$
My TA says that the original transformation does not have an inverse because a single input vector would map to multiple  output vectors. But I'm not seeing those multiple outputs in my algebra.
 A: The point is that there are infinitely many vectors $\pmatrix{x\cr y\cr}$ that would give the same $ax+by$.  For example, if $a=1$ and $b=1$, $\pmatrix{1 & 1} \pmatrix{t\cr -t\cr} = 0$ for all $t$.  So how is your inverse going to take that $0$ and find out what $t$ it came from?
A: Some rectangular matrices have left inverses which can be used to solve the equation $Ax=v$ where $v$ is in the span of the columns of $A$. Specifically, if $A$ is a rectangular matrix of full rank having more rows than columns, then the "left inverse" of $A$ equals $(A^TA)^{-1}A^T$ so that $$Ax=v \implies x=(A^TA)^{-1}A^Tv$$ This is only true if $v$ belongs to the column space of $A$. On the other hand, if $v\notin \text{Col}(A)$, then $x^*=(A^TA)^{-1}A^Tv$ happens to be the least squares solution to the inconsistent system $Ax=v$ i.e. it's "the best you can do," and $Ax^*$ is the projection of $b$ onto the span of the columns of $A$.
A: When it exists, the solution of a linear system
$$\eqalign{
AX &= B \\
}$$
can be written using the Moore-Penrose inverse $A^+$ and an arbitrary matrix $Y$ as
$$\eqalign{
X &= A^+B + (I_n-A^+A)Y \\
}$$
When the dimensions of the matrices $(A,B,X,Y)$ are
$\Big((1\times 2),\,(1\times 1),\,(2\times 1),\,(2\times1)\Big)$ you'd rewrite this in terms of vectors and scalars $(a,\beta,x,y)$ as
$$\eqalign{a^Tx = \beta}$$
Since the pseudoinverse of a vector has a closed-form $\Big(a^+ = \frac{a^T}{a^Ta}\Big)$ so does the solution
$$\eqalign{
x = \beta\left(\frac{a}{a^Ta}\right) + \left(I_2-\left(\frac{aa^T}{a^Ta}\right)\right)y \\
}$$
Which is great -- except that there are an infinite number of possible
$y$ vectors, and so there are an infinite number of solutions.
