# Finding an orthogonal basis w.r.t an inner product

I am trying to figure out a question: Let $W$ be the subspace $\mathbb{R^4}$ spanned by vectors

$$v_1=(1,1,1,1), v_2=(1,0,1,0) \text{ and }v_3=(1,1,0,0)$$

equipped with the standard Euclidean inner product (i.e. dot product).

a) Find the dimension of $W$

b) Construct an orthogonal basis for $W$.

Any help would be greatly appreciated as I am struggling to understand the difference between orthonormal bases, orthogonal bases, dimension of subspaces versus dimension of column row spaces etc!

Thanks

• Are the v_i linearly independent? Have you heard about Gram-Schmidt? – Tzimmo Sep 13 '15 at 7:31

We have $\dim W$ since $(v_1, v_2, v_3)$ is linearly independent. To see that, suppose $$c_1v_1 + c_2v_2 + c_3v_3 = 0$$ for some $c_1,c_2,c_2\in\mathbb R$. This implies \begin{align} c_1 + c_2 + c_3 &= 0\tag 1\\ c_1 + c_3 &= 0\tag 2\\ c_1 + c_2 &= 0\tag 3\\ c_1 &= 0\tag 4 \end{align} $(4)$ directly implies $c_1=0$, which in turn implies $c_2=c_3 = 0$ from $(2)$ and $(3)$.

To construct an orthogonal basis for $W$, there is a standard inductive algorithm called Gram-Schmidt. In general, suppose we have a basis $(v_1, \ldots, v_n)$ for a subspace $V$ of $\mathbb R^m$ where $n<m$ (the case where $n=m$ isn't interesting because we can just take the standard basis). The algorithm proceeds as follows:

1. Let $V_1 = \operatorname{Span}(\{u_1\})$, where $u_1=v_1$.
2. Given $V_i$, $i\leqslant 1\leqslant n-1$, set $V_{i+1} = V_i \cup \{u_{i+1}\}$ where $$u_{i+1} = v_{i+1} - P_{V_i}(v_{i+1}),$$ where $P_{\cdot}$ denotes the projection operator, i.e. $$P_{V_i}(v_{i+1}) = \sum_{j=1}^i \frac{\langle v_{i+1}, v_j \rangle}{\langle v_j, v_j \rangle} v_j.$$

In this example, we have $u_1=v_1=(1,1,1,1)$, and then \begin{align} u_2 &= v_2 - P_{V_1}(v_2) = (1,0,1,0) - \frac{\langle (1,0,1,0), (1,1,1,1) \rangle}{\langle (1,1,1,1), (1,1,1,1) \rangle} \\ &= (1,0,1,0) - \frac24(1,1,1,1)\\ &= \left(\frac12,-\frac12,\frac12,-\frac12\right). \end{align} Finally,

\begin{align} u_3 =&\ v_3 - P_{V_2}(v_2)\\ =&\ (1,1,0,0)\\ &- \left[\frac{\langle (1,1,0,0), (1,1,1,1) \rangle}{\langle (1,1,1,1), (1,1,1,1) \rangle}(1,1,1,1)+\right.\\ &\quad\quad\left.\frac{\langle (1,1,0,0), \left(\frac12,-\frac12,\frac12,-\frac12\right) \rangle}{\langle \left(\frac12,-\frac12,\frac12,-\frac12\right), \left(\frac12,-\frac12,\frac12,-\frac12\right) \rangle} \left(\frac12,-\frac12,\frac12,-\frac12\right) \right]\\ =&\ (1,1,0,0) - \left(\frac24 (1,1,1,1) + 0 \right)\\ =&\ \left(\frac12, \frac12, -\frac12, -\frac12\right). \end{align}

(Notice that conveniently $v_3\perp u_2$, saving some computation). Hence, our orthogonal basis is $$\left(u_1, u_2, u_3 \right) = \left(\left(1,1,1,1\right), \left(\frac12,-\frac12,\frac12,-\frac12\right),\left(\frac12, \frac12, -\frac12, -\frac12\right) \right)$$

To convert this orthogonal basis, we need only divide the basis elements by their norm, i.e. compute $e_i = \frac1{\|u_i\|}$ where $\|u_i\| = \langle u_i,u_i\rangle^{\frac12}$. I'll spare the computation, but the result is

$$(e_1, e_2, e_3) = \left(\left(\frac14, \frac14, \frac14, \frac14 \right),\left(\frac12,-\frac12,\frac12,-\frac12\right),\left(\frac12, \frac12, -\frac12, -\frac12\right) \right).$$

• Why has this answer not been accepted? This is brilliantly written and very extensive. @Angie01, please accept. :) – user860374 Sep 13 '15 at 20:08

Two vectors $\mathbb u, \mathbb v$ are said to be orthogonal if their dot product $\mathbb u \cdot \mathbb v$ is $0$.
An orthogonal basis for a vector space $V$, with respect to an inner product $(\cdot)$, is a basis for $V$ whose vectors are mutually orthogonal.
An orthonormal basis for a vector space $V$ is a basis for $V$ whose vectors are not only mutually orthogonal, but also normalized (i.e. having norm $1$).
The dimension of a vector space is the number of elements of one of its bases. It follows from a standard theorem that every basis of a vector space must have the same numbers of elements.
You mention the dimension of columns/rows spaces as if it were a different thing, but it really is the dimension of the vector space spanned by the columns or the rows.

To solve the problem, you should firstly find out if those three vectors are linearly independent. This will also give you the dimension of the space $W$. Using the Gaussian elimination algorithm, we get $$\begin{pmatrix} 1 & 1 & 1 & 1\\ 1 & 0 & 1 & 0\\ 1 & 1 & 0 & 0\\ \end{pmatrix} \longrightarrow \begin{pmatrix} 1 & 1 & 1 & 1\\ 0 & -1 & 0 & -1\\ 0 & 0 & -1 & -1\\ \end{pmatrix}$$

Since it's not possible to reduce it further, we conclude that the three vectors are linearly independent. It follows that they are also a basis for $W$ and then $\dim W = 3$.

For the second part of the problem, you can use the Gram–Schmidt algorithm, which is fairly simple to apply. On the linked page there is also a really helpful animation which I found to be really useful in order to completely understand the process.