Take the 2-minute tour ×
Mathematics Stack Exchange is a question and answer site for people studying math at any level and professionals in related fields. It's 100% free, no registration required.

Does this expression $\frac{da^{\intercal}a}{da}$ evaluates to $2*a$ or $2*a^{\intercal}$ ? Here $a$ is column vector. Derivation steps would be appreciated.

share|improve this question
add comment

2 Answers 2

There is no universally accepted convention about whether the gradient is a row vector or a column vector. In optimization literature, $\nabla f(x)$ is a column vector. In Terence Tao's Real Analysis books, $\nabla f(x)$ is a row vector.

In Calculus on Manifolds by Spivak, if $f:\mathbb R^n \to \mathbb R^m$, then $f'(x)$ is an $m \times n$ matrix. In particular, if $f:\mathbb R^n \to \mathbb R$, then $f'(x)$ is a $1 \times n$ matrix, a row vector.

share|improve this answer
add comment

This actually confused me also when I was learning. For instance in my signal processing course, I learned it as a row vector, where as in my optimization course I learned it as a column vector. Then later I came to know its just the difference of how you choose to denote the derivative of a scalar with respect to a vector. Here $a^{T}a$ is a scalar and $a$ is a vector. You can choose it either a column vector or to be row vector. The choice should depend on your domain of study and the prevalent notation in that. You can read more about it in wikipedia. The fact how the constant $2$ comes up should be obvious I guess.

share|improve this answer
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.