# Linear Algebra, Least Squares and Error (Matlab)

"Let $A$ be a $4 \times 4$ random matrix with rank $2$ (check that its rank is $2$). Let $b$ be a random vector in $\mathbb R^4$.

Check that the system $Ax = b$ is inconsistent and then find a least squares solution $x_0$ of $Ax = b$ (is this solution unique?).

Compute the error vector $b-Ax_0$. Check that this error vector is perpendicular to the column space of $A$.

Compute the error is the length $\|b - Ax_0\|$. Check that this error is minimized for the least squares solution by computing $\|b - Ax\|$ for several random vectors $x$ and seeing that it is larger than the error."

So far I have:

A = rand(4,2)*rand(2,4);
rank(A);
b=rand(4,1);
C=inv(A)*b;
x = (inv(A'*A))*A'*b;
E = b-A*x;
D = norm(E);


The first three lines define $A$ and $b$ and prove the rank is 2. C is intended to prove $Ax=b$ is inconsistent (it returns a strange answer). x should be the least squares solution, this apparently is not unique because of the last question, but I don't know how else to find an x. E is the error vector; I do not know how to check it's perpendicular to the column space of $A$. D is the length of the error vector; I do not know how to find other x's to test.

-
Holy everloving... please please please use C=A\b; and x = (A'*A)\(A'*b); (and even this second one could be improved upon!), instead of C=inv(A)*b; and x = (inv(A'*A))*A'*b;! – J. M. Dec 3 '11 at 5:48
@J.M. or better leave it to MATLAB with linsolve ;) – user13838 Dec 3 '11 at 6:07
@percusse: I like backslash more than linsolve(), but whatever floats your boat, no? ;) – J. M. Dec 3 '11 at 6:21
@J.M. inv(A'*A) is bad, but inv(A), provided it exists, is not necessarily bad. A\b is no better if there are multiple right hand sides to solve for. Please see stackoverflow.com/a/35150369/517835 . – Milind R Feb 14 at 13:49

$E$ will be perpendicular to the column space of $A$ if and only if it is perpendicular to each of the columns of $A$, so if you know how to check whether two vectors are perpendicular, that part should be easy. As for how to find other values of $x$ to test, the problem statement calls for "several random vectors $x$", so presumably you just generate several random vectors (of the appropriate number of components) and test them.