Mathematics Stack Exchange is a question and answer site for people studying math at any level and professionals in related fields. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

I have to solve the following problem (exact wording):

Given is a Matrix $A_{m \times n} \in \mathbb{R}^{m \times n}$ and $r = \mathrm{rank}(A)$. The goal is to find the vectors $V=\{v_1,v_2,\ldots,v_n\}$ of the orthogonal basis of $\mathrm{im}(A)$ by using the Gram Schmidt algorithm.

May $a_1,\ldots,a_n \in \mathbb{R}^m$ be the column vectors of A. We define $u_1,\dots,u_n \in \mathbb{R}^m$ by means of the Gram Schmidt algorithm,

$$ u_r = a_r - \sum_{s=1}^{r\;-1}\;\mathrm{proj}_{u_s}(a_r)\quad \text{for r = 1,...,n} $$


$$ \mathrm{proj}_{u_s}(a_r) = \begin{cases} \frac{u_s\cdot a_r}{u_s\cdot u_s}u_s & \text{for}~u_s \neq 0\\ \phantom| \\ \qquad0 & \text{for}~u_s=0 \end{cases} $$

a) What is the value of $u_r$, if $a_r\in \mathrm{span}(a_1,\dots,a_{r\,-1})$ with $r \leq n$? How can $\{v_1,\ldots,v_k\}$ be expressed in terms of $\{u_1,\ldots,u_n\}$?

What I don't understand is how $\mathrm{proj}_{u_s}(a_r)$ is defined, as far as I understand $u_s$ and $a_r$ are column vectors of size $m$, thus how can they be multiplied with one another?

My hunch is that $u_r$ is zero if $a_r \in \mathrm{span}(a_1,\dots,a_{r\,-1})$ as $a_r$ is not linearly independent in this case and can thus be expressed as sum of $(u_1,\dots,u_{r\,-1})$.

share|cite|improve this question
up vote 2 down vote accepted

The vector product being described here is the dot product, which is an important way to take a product of vectors. For column vectors, the dot product can be computed as

$$ \mathbf a \cdot \mathbf b = \mathbf a^{\mathsf T} \mathbf b = a_1 b_1 + a_2 b_2 + \cdots + a_m b_m\;.$$

It is possible to show that $\mathbf a \cdot \mathbf b = \| \mathbf a \| \| \mathbf b \| \cos(\theta) $, where $ \| \mathbf v \|$ is the length of a vector $\mathbf v$ (e.g. the distance of the point $\mathbf v$ from the origin in Euclidean space), and $\theta$ is the angle between the two vectors $\mathbf a$ and $\mathbf b$ (i.e. between the line segment connecting the origin to the point $\mathbf a$, and the similar segment for $\mathbf b$). As a result, for any vector $\mathbf a$ and any unit vector $\mathbf u$, the vector $(\mathbf a \cdot \mathbf u) \; \mathbf u$ represents the projection of $\mathbf a$ onto the line passing through the origin and pointing in the direction of $\mathbf u$. This property is implicitly the basis on which the Gram-Schmidt process operates.

share|cite|improve this answer

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.