Mathematics Stack Exchange is a question and answer site for people studying math at any level and professionals in related fields. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

Does this expression $\frac{da^{\intercal}a}{da}$ evaluates to $2*a$ or $2*a^{\intercal}$ ? Here $a$ is column vector. Derivation steps would be appreciated.

share|cite|improve this question

There is no universally accepted convention about whether the gradient is a row vector or a column vector. In optimization literature, $\nabla f(x)$ is a column vector. In Terence Tao's Real Analysis books, $\nabla f(x)$ is a row vector.

In Calculus on Manifolds by Spivak, if $f:\mathbb R^n \to \mathbb R^m$, then $f'(x)$ is an $m \times n$ matrix. In particular, if $f:\mathbb R^n \to \mathbb R$, then $f'(x)$ is a $1 \times n$ matrix, a row vector.

share|cite|improve this answer

This actually confused me also when I was learning. For instance in my signal processing course, I learned it as a row vector, where as in my optimization course I learned it as a column vector. Then later I came to know its just the difference of how you choose to denote the derivative of a scalar with respect to a vector. Here $a^{T}a$ is a scalar and $a$ is a vector. You can choose it either a column vector or to be row vector. The choice should depend on your domain of study and the prevalent notation in that. You can read more about it in wikipedia. The fact how the constant $2$ comes up should be obvious I guess.

share|cite|improve this answer

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.