What is the difference between orthogonal subspaces and orthogonal complements?

I am learning linear algebra through professor Giblert Strang's lectures on MIT OCW.

The professor says that the row space and the null space of a matrix are orthogonal subspaces. This I can follow, since any vector in the nullspace takes any linear combination of the rows to zero.

He then says that the row space and null space are more than just orthogonal subspaces, they are orthogonal complements, Because The "nullspace contains all the vectors that are perpendicular to the row space", and vice versa.

Consider: $$A= \begin{bmatrix} 1 & 0 & 0 \\ 0 & 1 & 0 \end{bmatrix} \begin{bmatrix} x_1 \\ x_2 \\ x_3 \end{bmatrix}=0$$

A is rank 2, Dimension of nullspace of A=1

Null space: $$X=c \begin{bmatrix} 0 \\ 0 \\ 1 \end{bmatrix}$$

Graphically, the row space of A is the x-y plane, and the null space is the z-axis.

It's easy to see that the nullspace does not contain all the vectors that are perpendicular to a vector in the row space. If we look at $[1 ~0~ 0]$, it has the entire y-z plane perpendicular to it, not just the z-axis.

All I can see is two orthogonal subspaces. I do not understand what additional property has earned these subspaces the term orthogonal complements.

Refer:
Lecture Video (from 31:16 to 33:00)
Lecture Notes Page 2 paragraph 1

• complements means that the direct sum of these subspaces is $\mathbb{R^3}$ – user392395 Jun 7 '17 at 13:12
• Can you comment on this statement by the professor: "nullspace contains all the vectors that are perpendicular to the row space"? Shall I disregard that statement and move on? – jumpmonkey Jun 7 '17 at 13:31
• @jumpmonkey "It contains ... vectors that are perpendicular to the row space [and no vectors which are not]" = nullspace and row space are orthogonal. "It contains all [such perpendicular vectors]" (emphasis mine) = the direct sum of the nullspace and row space is $\Bbb R^n$. Hence the nullspace and the row space are orthgonal complements. – user137731 Jun 7 '17 at 13:34
• Two perpendicular lines through the origin are orthogonal spaces in $\mathbb{R}^3$, but they'er not orthogonal complements because they're not as large as possible. In fact, the two lines span a plane and not all of $\mathbb{R}^3$. The orthogonal complement of a line in $\mathbb{R}^3$ is a plane (the plane passing through the origin whose normal vector is given by the direction of the line.). @Bye_World Thanks. – Michael Burr Jun 7 '17 at 13:35

Let $$U= \operatorname{span}\{(1,0,0)\} \\ V = \operatorname{span}\{(0,1,1)\} \\ W=\operatorname{span}\{(0,1,0),(0,0,1)\}$$ Then clearly every vector in $U$ is orthogonal to every vector in $V$ and to every vector in $W$. Hence $U$ and $V$ are orthogonal subspaces and $U$ and $W$ are orthogonal subspaces.

But $U$ and $V$ together don't make up all of the vectors in $\Bbb R^3$ (the ambient space). For instance $(0,0,1)$ is not in either one. However $U$ and $W$ together do in fact make up all of the vectors in $\Bbb R^3$. In the lingo of linear algebra, we say that the direct sum of $U$ and $W$ is $\Bbb R^3$ -- or symbolically $U\oplus W = \Bbb R^3$.$^\dagger$ It's this extra property that makes $U$ and $W$ not just orthogonal subspaces -- but each other's orthogonal complement.

$\dagger$: I'm being a little loose with my language here. Technically the condition for $U\oplus W=\Bbb R^3$ (which is the required extra condition) is that every vector in $\Bbb R^3$ can be written in a unique way as a sum of an element of $U$ and of $W$. I.e. for every $v\in \Bbb R^3$, there exists a unique $u\in U$ and a unique $w\in W$ such that $u+w=v$. Nevertheless, the statements I give in the paragraph above are how we intuitively think of orthogonal complements.

• It might be good to explicitly say that "$U$ and $W$ together make up $\Bbb R^3$" means not literally that $U \cup W = \Bbb R^3$, instead that every vector in $\Bbb R^3$ is a sum of something in $U$ and something in $W$. – pjs36 Jun 7 '17 at 13:41
• @pjs36 OK. I've added an addendum. – user137731 Jun 7 '17 at 13:49

There are two conceptual issues here:

(1) The orthogonal complement $$X^{\perp}$$ of a vector subspace $$X \leq \Bbb V$$ consists of the vectors in $$\Bbb V$$ that are perpendicular to all of the vectors in $$X$$, not just at least one vector.

In practice, it is convenient to use the characterization that $$v \in \Bbb V$$ is orthogonal to the subspace $$X \leq \Bbb V$$ iff for any (equivalently every) basis $$(E_a)$$ of $$X$$ we have $$v \perp E_a$$ for all basis elements $$E_a$$. Using this characterization, it is straightforward to check that the row space and null space are orthogonal complements.

(2) Given a vector space $$\Bbb V$$, the vector subspaces $$X, Y \leq \Bbb V$$ are complementary iff (a) $$X$$ and $$Y$$ are transverse, that is, $$X \cap Y = \{ 0 \}$$ and (b) $$X$$ and $$Y$$ together span $$\Bbb V$$, that is, $$X + Y = \Bbb V$$. Given the first condition, the second condition is equivalent to $$X \oplus Y = \Bbb V$$.

Given an inner product space $$(\Bbb V, \,\cdot\,)$$, two vector spaces $$X, Y \leq \Bbb V$$ of are orthogonal (we denote $$X \perp Y$$) iff every element of $$X$$ is orthogonal to every element of $$Y$$.

If $$X$$ and $$Y$$ are orthogonal and complementary, they are orthogonal complements