4
$\begingroup$

I'm following along Ch. 4 entitled "Orthogonality" in Gilbert Strang's Introduction to Linear Algebra. Here are some of the initial results in that chapter

  • Definition: two subspaces of a vector space are orthogonal if every vector in the first subspace is perpendicular to every vector in the second subspace

  • Every vector $x$ in the nullspace of a matrix $A$ is perpendicular to every row of $A$.

  • Similarly, every vector $y$ in the nullspace of $A^T$ is perpendicular to every column of $A$.

  • Definition: The orthogonal complement of a subspace $V$ contains every vector that is perpendicular to $V$

By this definition, the nullspace is the orthogonal complement of the row space. Every $x$ that is perpendicular to the rows satisfies $Ax=0$.

My question is about this last quote.

How do we know that all vectors not in the nullspace are in the row space?

Every vector $x$ that satisfies $Ax=0$ is in the nullspace of $A$, and is perpendicular to every row of $A$. Thus, $x$ is also perpendicular to linear combinations of rows of $A$ and is thus perpendicular to every vector in the row space.

If $A$ is $m$ by $n$ with rank $r$, the nullspace has dimension $n-r$ and the row space has dimension $r$. Vectors in the nullspace and in the row space are in $\mathbb{R}^n$.

How do we know that not only are these two subspaces orthogonal, they are also orthogonal complements because there is no vector in $\mathbb{R}^n$ that is perpendicular to the vectors in the nullspace that is not in the row space, and there is no vector perpendicular to the vectors in the row space that is not in the null space?

$\endgroup$
3
  • 1
    $\begingroup$ "How do we know all vectors not in the nullspace are in the row space?" We do not; that's a false statement, and it does not follow from "the rowspace is the orthogonal complement of the nullspace". The orthogonal complement of the rowspace is the set of all vectors $v$ such that $r\cdot v=0$ for all vectors $r$ in the rowspace. $\endgroup$
    – Arturo Magidin
    Dec 25, 2022 at 23:13
  • $\begingroup$ Do you have an example in which there is a vector in $\mathbb{R}^n$ that is not in the nullspace or the rowspace of an $m$ by $n$ matrix $A$ of rank $r$? $\endgroup$
    – evianpring
    Dec 25, 2022 at 23:23
  • 2
    $\begingroup$ @evianspring: Sure: take the diagonal matrix that has a $0$ in the first diagonal coordinate and $1$s in the others. The rowspace are the vectors of the form $(0,x_2,\ldots,x_n)$. The nullspace are the vectors of the form $(x_1,0,\ldots,0)^t$. The vector $(1,1,1,\ldots,1)$ is in neither. More generally, take a nonzero vector $x$ in the rowspace, a nonzero vector $y$ In the nullspace, then $x+y$ is in neither. $\endgroup$
    – Arturo Magidin
    Dec 25, 2022 at 23:34

2 Answers 2

Reset to default
3
$\begingroup$

The boldface question/statement is incorrect. No one is asserting that the complement of the nullspace is the rowspace. The claim is that the nullspace is the orthogonal complement of the rowspace.

As you note, anything in the nullspace is orthogonal to the rowspace. If $v$ is orthogonal to the rowspace, then it is orthogonal to each row, so we also have $Av=\mathbf{0}$. Thus, we have that the nullspace is contained in the orthogonal complement of the rowspace (by the first observation), and that the orthogonal complement of the rowspace is contained in the nullspace (by the second observation). Thus, the nullspace is equal to the orthogonal complement of the rowspace.

Alternatively, we know by the Rank-Nullity Theorem that the dimension of the rowspace plus the dimension of the nullspace is $n$. In addition, the dimension of the rowspace plus the dimension of the orthogonal complement of the rowspace also add up to $n$. Since the nullspace is contained in the orthogonal complement, and they must have the same dimension by the previous computations, we conclude they have the same dimension and thus are equal, since one is contained in the other.

$\endgroup$
6
  • $\begingroup$ What is the "second observation" that leads you to say the orthogonal complement of the rowspace is contained in the nullspace? $\endgroup$
    – evianpring
    Dec 25, 2022 at 23:29
  • $\begingroup$ @evianspring: "if $v$ is orthogonal to the rowspace, then it is orthogonal to each row, so $Av=0$". $\endgroup$
    – Arturo Magidin
    Dec 25, 2022 at 23:34
  • 2
    $\begingroup$ @evian: First, there is no "admission"; there are facts and deductions. Second, no. Think about $\mathbb{R}^2$. The orthogonal complement of the $x$-axis is the $y$-axis. The dimension of the space spanned by the $x$-axis is $1$. The dimension of the space spanned by the $y$ axis is $1$. They add up to $2$. But it is not the case that every vector not in the $x$-axis is in the $y$-axis. There are lots of vectors that are in neither the $x$-axis nor the $y$-axis. $\endgroup$
    – Arturo Magidin
    Dec 25, 2022 at 23:38
  • 2
    $\begingroup$ @evianpring "if there were a vector not in the nullspace and not in the rowspace, it would have an extra dimension, no?" Absolutely not. What you are forgetting is that the union of two subspace, neither of which contains the other, is not a subspace. I've given you explicit examples. Use them to try to figure out where the error in your thinking and understanding is lurking, because you are absolutely, utterly, and completely wrong about this. $\endgroup$
    – Arturo Magidin
    Dec 25, 2022 at 23:41
  • 1
    $\begingroup$ Okay, I see now, especially with the example with the subspaces represented by the x and y axes. Dimension seems to be the source of confusion still. Adding a dimension isn't the same as adding all the vectors with that added dimension. The nullspace being the orthogonal complement of the rowspace simply means it contains all vectors perpendicular to the rowspace. Such vectors have dimensions not present in the rowspace vectors, and additionally, among all vectors with such dimensions, these specific vectors are perpendicular to the rowspace. $\endgroup$
    – evianpring
    Dec 25, 2022 at 23:51
2
$\begingroup$

The way that we know that all vectors orthogonal to the null space are in the row space is by counting dimension. In particular, as you noted, the nullspace has dimension $n-r$. It follows that the orthogonal complement of the null space has dimension $n-(n-r) = r$.

Now, the row-space is an $r$ dimensional subspace of the orthogonal complement of the null space, which in turn has dimension $r$. The only $r$ dimensional subspace of an $r$-dimensional space is the entirety of the space itself. So, the row-space is not only a subspace of the orthogonal complement but comprises the entirety of the orthogonal complement.

$\endgroup$

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .