Least squares invariant under transformation? Suppose that $\hat{x}$ is the least squares solution of $Ax = b$.  That is, $\min{|r|}, r = Ax - b$ with respect to $x$.
Is $\hat{x}$ also the least squares solution of $JAx = Jb$?  I suspect so, but I'm not sure how to prove it.  If $J$ is invertible, it's easy enough to prove as you can just cancel it out from both sides.  But what if it's not invertible?
 A: The answer is NO. The vector $\hat{x}$ won't generally be a solution of the transformed least squares problem unless, e.g., $J$ is identity or orthogonal or $Ax=b$ is consistent. 
Also, note that if $J$ is nonsingular, then of course, you can "cancel out" $J$ somewhat but it won't give you the same least squares solution! Recall that you do not solve an "ordinary" consistent system but a least squares problem whose solution depends on what you minimize and in which norm. Which leads to the following remark:

Multiplying the system from left changes the norm, so instead of minimizing $\|b-Ax\|$, you minimize $\|J(b-Ax)\|$ which can be something completely different. The two problems can have different solutions.

Example:
Consider 
$$
A=\begin{bmatrix}1&1\\1&0\\1&0\end{bmatrix},
\quad
b=\begin{bmatrix}1\\0\\1\end{bmatrix},
\quad
J=\begin{bmatrix}1&1&0\\0&1&0\\0&0&1\end{bmatrix}.
$$
The solutions of the problems $Ax\approx b$ and $JAy\approx Jb$ are
$$
\hat{x} = \begin{bmatrix}1/2\\1/2\end{bmatrix}\quad\text{and}\quad
\hat{y} = \begin{bmatrix}1/2\\0\end{bmatrix}.
$$
