3
$\begingroup$

Here's the entire question: Let $A$ be an 8 $\times$ 5 matrix of rank 3, and let $b$ be a nonzero vector in $N(A^T)$.

a) Show that the system $Ax = b$ must be inconsistent. Gonna take a wild stab at this one... If the rank is 3, that means the dimension of the column space is 3. But $A$ has 5 columns, so they are not all linearly independent and therefore $Ax = b$ is inconsistent.

b) How many least squares solutions will the system $Ax = b$ have? Explain.

On previous problems, I found the best least squares linear fit, where the approximation of $x$ was a vector that contained sometimes regular numbers, and sometimes variables. Does this mean that there must be either 1 linear solution or infinite (because you can always find an approximation)? In the example that apparently had an infinite number of least squares solutions, it appeared that one row of $A^TA$ was a constant multiple of another row, leading to a row of zeros in reduced row echelon form. From this problem I know that $A^TA$ is a 5x5 matrix, but I don't think I can prove that any rows are a scalar multiple of other rows, so I'm guessing I have to use some other means of figuring this out.

Sorry if I sound like I have no idea what I'm talking about. Just wanted to try out the problem to my best ability before asking about it.

$\endgroup$
  • $\begingroup$ For (a), that's not good enough, because your argument doesn't mention what $b$ is. You need to use the fact that the column space of $A$ is the set of all vectors $c$ such that $Ax=c$ is consistent. ... The vector $b$ being in the null space of $A^\top$ means $b$ is perpendicular to every vector in the column space (why?); hence, $b$ can't be in the column space, unless $b$ is zero (why?). $\endgroup$ – Christopher Carl Heckman Nov 11 '15 at 7:11
  • $\begingroup$ For (b), use the fact that $b$ is in the null space of $A^\top$ means $A^\top b = 0$, and use the fact that the normal equation is $(A^\top A)x = (A^\top b)$. $\endgroup$ – Christopher Carl Heckman Nov 11 '15 at 7:13
  • $\begingroup$ So $R(A)$ = $N(A^T)^⊥$ (just found that formula in my book), and because b is in the latter, that means it must be perpendicular to R(A). Though I can't find anything about b having to be 0 because of this. I'll keep looking, though. $\endgroup$ – Chris Nov 11 '15 at 7:37
  • $\begingroup$ The book says that $Ax = b$ is consistent if and only if $b$ is an element of $R(A)$. But is that not possible because $b$ doesn't contain the 0 vector? I remember doing that when checking to see if something is a subspace. $\endgroup$ – Chris Nov 11 '15 at 7:49
  • $\begingroup$ And for the part $b$, I said $A^Tb$ = $0$, meaning $(A^TA)x$ = $0$, and because the rank of $A$ is equal to the rank of $A^TA$, that means $A^TA$ has a rank of 3 and therefore has a nullity of 5, which means there are 5 free variables in $A^Tb$ = $0$ and that means $Ax=b$ has infinitely many least squares solutions. Is that right? $\endgroup$ – Chris Nov 11 '15 at 8:06
0
$\begingroup$

a

The system is inconsistent. Example: $$ \left[ \begin{array}{cc} 1 & 0 \\ 0 & 0 \end{array} \right] % \left[ \begin{array}{cc} x_{1} \\ x_{2} \end{array} \right] = \left[ \begin{array}{c} 0 \\ 1 \end{array} \right] $$

b

No, the system is not unique. The solution is an affine space: infinite solutions. In linear algebra the number of solution is either 0 (no existence), 1 (existence and uniqueness), or infinite (existence, no uniqueness).

A pencil and paper exercise: Unique least square solutions

Theoretical treatment: Is a least squares solution to Ax=bAx=b necessarily unique

To see a diagram Is the unique least norm solution to Ax=b the orthogonal projection of b onto R(A)?

How the SVD produces the Moore-Penrose pseudoinverse: How does the SVD solve the least squares problem?

| cite | improve this answer | |
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.