# Orthogonal Subspaces

I am reading orthogonality in subspaces and ran into confusion by reading this part:

Suppose S is a six-dimensional subspace of nine-dimensional space $\mathbb R^9$.
a) What are the possible dimensions of subspace orthogonal to $S$? Answer: Sub spaces orthogonal to S can have dimensions $0,1,2,3.$
b) What are the possible dimensions of the orthogonal complement $S^{\perp}$ of $S$? Answer: Complement $S^{\perp}$ is the largest orthogonal subspace with dim $3$.

Where I am having trouble is understanding how the answers make sense to the question, or how the answers are pretty much the answers. In other words, for a, how is the dimensions $0,1,2,3$? But maybe I am not understanding the question. Any assistance with helping me understand the answer would be appreciated.

-

The orthogonal complement of $S$ is $$S^{\perp}=\{v\in\mathbb{R}^9:\langle v, w\rangle=0,\text{ for all }w\in S\}$$ (where $\langle v,w\rangle$ denotes the inner product). It's immediate that $S^{\perp}$ is a subspace of $\mathbb{R}^9$.

If a vector $u$ belongs to $S\cap S^{\perp}$, then it must be orthogonal to all vectors in $S$, in particular to $u$ itself, so $u=0$. Therefore $$S\cap S^{\perp}=\{0\}.$$ In particular, if $B$ is a basis of $S$ and $C$ is a basis of $S^\perp$, then $B\cup C$ is linearly independent.

Now, another theorem tells us that $$S+S^{\perp}=\mathbb{R}^9$$ which means that $B\cup C$ is a basis of $\mathbb{R}^{9}$. So $C$ must have three elements, $C=\{v_1,v_2,v_3\}$. Hence $$\{0\}\subset\operatorname{Span}\{v_1\} \subset\operatorname{Span}\{v_1,v_2\} \subset\operatorname{Span}\{v_1,v_2,v_3\}=S^{\perp}$$ exhibits subspaces of $S^{\perp}$ of dimensions $0$, $1$, $2$, $3$ respectively. They are orthogonal to $S$ because they are included in $S^\perp$.

Finally, since $\dim S^{\perp}=3$, no subspace with dimension greater than $3$ can be included in it. And any subspace orthogonal to $S$ is included in $S^{\perp}$ by definition.

-

Take all the vectors linearly independent vectors in $S$ and put them in a matrix (as rows). Since $S$ has dim=6, so there are 6 linearly independent vectors in S. Thus the matrix will have size 6x9. Now, rank of this matrix is $6$, and the orthogonal complement to $S$ is the rank of its null space. So, by rank nullity theorem, dimension of orthogonal complement is $3$.

Now, subspaces orthogonal to $S$ consists of vectors that belong to the null space of the above matrix, and are subspaces of the null space of the matrix (any subspace will do). These subspaces can only have dimension $0,1,2$ or $3$.

Hope this helps.

-

Suppose $S$ is a six-dimensional subspace of $\mathbb{R}^9$.
a) What are the possible dimensions of subspace orthogonal to S?
Before we get to the answer, let's review the definition of an orthogonal subspace. It is a subspace whose intersection $S$ is exactly the zero vector. Now, how many dimensions could such a subspace have, given that $\dim S = 6$? Certainly the zero vector alone qualifies, which is a subspace of dimension zero. Now, I claim that any such orthogonal subspace could not exceed three dimensions. If it did, it would necessarily contain a nontrivial overlap with $S$.

Let a basis for $S \subset \mathbb{R}^9$ and let $U \subset \mathbb{R}^9$ be a four dimensional subspace. Let $\{b_1, ..., b_9\}$ form a basis for $\mathbb{R}^9$. Then we can pick six of these to form a basis for $S$ and four of these to form a basis for $U$. But this necessarily implies that $U$ and $S$ have a nontrivial intersection! Using this same reasoning, you should be able to understand that the possible dimensions for a subspace orthogonal to $S$ are $0, 1, 2, 3$.

b) What are the possible dimensions of the orthogonal complement $S^{\perp}$ of S?
The orthogonal complement of a set is everything within the stated universe that is not in $S$ itself, plus the zero vector. More formally, it is the set that completes the universal space via direct sum with $S$. Since $S$ is six dimensional, and our universe is $\mathbb{R}^9$, this implies that the orthogonal complement must have three dimensions.

-
Is there another way to think about this using the rank of S, which could be a maximum of 6? Then 9-6, 9-7, 9-8, and 9-9 make up the possible dimensions of subspaces orthogonal to S. – grayQuant Jan 14 '14 at 15:23

Since the other (very nice) answers here have not satisfied you, let me take a different route. I will illustrate what is going on in a simpler situation, and hopefully it will shed light on your question by analogy. Also, since the algebraic formalism doesn't seem to be to your liking, we'll think very geometrically.

Let's consider a 1-dimensional subspace $S$ of $\mathbf{R}^3$. Such a subspace is simply a line through the origin. In fact, for concreteness, let's take $S$ to be the familiar $x$-axis in $\mathbf{R}^3$.

What are all the possible dimensions of orthogonal subspaces to $S$?

Well, the empty-set is orthogonal to $S$ for trivial reasons, and that provides the unique zero-dimensional example. This is also the case in your problem.

Next, we have one-dimensional subspaces which are orthogonal to $S$. These are lines through the origin spanned by vectors making a "right angle" with the $x$-axis. There are many such lines. The $y$-axis and the $z$-axis provide the most familiar examples. So $1$-dimensional orthogonal subspaces are also possible, there are lots of them, and if you take $S$ together with a one-dimensional orthogonal subspace, there's still plenty of $\mathbf{R}^3$ that remains "untouched".

Now, take any two distinct one-dimensional orthogonal subspaces from the previous example. These lines are spanned by two linearly independent vectors. Thus, taken together they span a $2$-dimensional subspace of $\mathbf{R}^3$, and since both of the basis vectors are orthogonal to $S$, the entire space is orthogonal to $S$. A $2$-dimensional subspace of $\mathbf{R}^3$ is a plane. This plane is going to be the $y-z$-plane, which is probably familiar to you from multivariable calculus. You can probably visualize how the $x-$axis "points out of the $y-z$-plane orthogonally".

But now we're done, because the next dimension to consider would be dimension $3$, but the only $3$-dimensional subspace of $\mathbf{R}^3$ is all of $\mathbf{R}^3$ itself, which would certainly overlap with $S$, and hence not be orthogonal to it.

So the possible dimensions for orthogonal subspaces were 0, 1, and 2. Now, a subspace and its orthogonal complement, taken together, "fill up" the entire ambient vector space. What we noticed in our example was that the $x-$axis and the $y-z$-plane are not only orthogonal, but taken together they fill up $\mathbf{R}^3$. So the orthogonal complement of our one-dimensional subspace of $\mathbf{R}^3$ was two-dimensional.

Now, you can play the same sort of game with the problem in your question...it just won't be so easy to visualize what's going on. If $S$ is six-dimensional in $\mathbf{R}^9$, then you can find orthogonal subspaces of dimensions 0, 1, 2, or 3, and if you take a 3-dimensional orthogonal subspace, then you've filled up all of $\mathbf{R}^9$, so you've found the orthogonal complement.

-
This was helpful but the explanation following Strang's style is most relevant to me. – grayQuant Jan 18 '14 at 4:47
Appreciate your nice post very much. The illustration indeed provides additional insight which may not be seen by the formal explanation. – C. Y. Cheng Jan 21 '14 at 20:21

I use Gilbert Strang's notation in the discussion below. For an $m\times n$ matrix $A$, $\, C(A^T)$ stands for its row space, and $N(A)$ the nullspace, respectively. In fact, the answer below is mainly borrowed from Strang's "Linear Algebra and Its Applications," 4th Edition.

(a) Subspace $S$'s dimension is 6, and it is a subspace of $\mathbf{R}^9$, thus we can say that there are 6 basis vectors (each vector is $9\times 1$) for $S$. Furthermore, we can construct an $m\times n$ ($m=6$, $n=9$) matrix $A$ with these basis vectors as the rows of matrix $A$. We know the rank $r$ of $A$ is $6$.

Now the original problem can be investigated by using the theorems of the fundamental subspaces of matrix $A$.

From the theorems of four fundamental subspaces in linear algebra, we know the nullspace of $A$ is orthogonal to $S=C(A^T)$, and $N(A)$ has dimension $3$ ($n-r = 9-6=3$). Note that $3$ is not the only dimension of a subspace which can be orthogonal to $C(A^T)$. Subspaces of $N(A)$ with dimensions from $0$ up to $3$ are also possible. Therefore we conclude that the possible dimensions of subspaces orthogonal to $S$ are $0,1, 2, 3$.

(b) Not all subspaces orthogonal to $S$ are its orthogonal complements. The size (dimension) must be right. For example, a 1-D subspace $V$ spanned by $(1,0,0)$ in $\mathbf{R}^3$ can be orthogonal to a 1-D subspace $W$ spanned by a perpendicular vector $(0,1,0)$, yet $V$ cannot be $W^\perp$ because we can find plenty of lines which are perpendicular to $W$ but lie outside $V$. (In fact, $W^\perp$ is a 2-D plane, not a line.)

The proper size of $S^\perp$ is readily deduced, again, from the fundamental theorem of linear algebra: row space ($C(A^T)=S$) and nullspace ($N(A)=S^\perp$) of $A$ are orthogonal complements, and their dimensions must satisfy the following equation:

$$\mathrm{dim}\, C(A^T) + \mathrm{dim}\, N(A) =\mathrm{dim}\, S + \mathrm{dim}\, S^\perp = r + (n-r)= n.$$

Therefore $\mathrm{dim}\, S^\perp = n- \mathrm{dim}\, S=6-3=3$.

(In other words, the orthogonal subspaces of dimensions $0$, $1$ and $2$ are too small to be orthogonal complements to $S$.)