Mathematics Stack Exchange is a question and answer site for people studying math at any level and professionals in related fields. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

I am trying to implement the steepest descent algorithm for linear systems. The equation is below:

$\begin{align*} Ax &= b\\ x_0 &= [0]*[m,n]\\ x_k &= x_{k-1} + \frac{|d_{k-1}|^2}{d_{k-1} * Ad_{k-1}}d_{k-1}\\ d_{k-1} &= -(Ax_{k-1} - b) \end{align*}$

My problem is that I think $d_{k-1}$ should be an $m\times1$ matrix and when you try and dot the denominator of $x_k$ it does not work because you end up trying to dot an $m\times1$ by an $m\times1$ matrix.

What am I doing wrong?

share|cite|improve this question
up vote 5 down vote accepted

What you have written as "$d_{k-1} * Ad_{k-1}$" is more properly written as $$\color{red}{d_{k-1}^\top}\color{blue}A\color{magenta}{d_{k-1}}$$

that is; a quadratic form. You are taking the product of $\color{red}{1\times m}$, $\color{blue}{m\times m}$, and $\color{magenta}{m\times 1}$ matrices, and you wind up with a scalar result as it should be, since it's in the denominator.

share|cite|improve this answer
Thanks. Thats exactly what I was missing. – Jeffrey Nov 22 '11 at 6:30

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.