# The Gradient as a Row vs. Column Vector

Kaplan's Advanced Calculus defines the gradient of a function $f:\mathbb{R^n} \rightarrow \mathbb{R}$ as the $1 \times n$ row vector whose entries respectively contain the $n$ partial derivatives of $f$. By this definition then, the gradient is just the Jacobian matrix of the transformation.

We also know that using the Riesz representation theorem, assuming $f$ is differentiable at the point $x$, we can define the gradient as the unique vector $\nabla f$ such that

$$df(x)(h) = \langle h, \nabla f(x) \rangle, \; h \in \mathbb{R}^n$$

Assuming we ignore the distinction between row vectors and column vectors, the former definition follows easily from the latter. But, row vectors and column vectors are not the same things. So, I have the following questions:

1. Is the distinction here between row/column vectors important?
2. If (1) is true, then how can we know from the second defintion that the vector in question is a row vector and not a column vector?
• The gradient as a row vector seems pretty non-standard to me. I'd say vectors are column vectors by definition (or usual convention), so $df(x)$ is a row vector (as it is a functional) while $\nabla f(x)$ is a column vector (the scalar product is a product of two vectors. And yes, the distinction is important. – t.b. Jul 28 '11 at 21:46
• Kaplan seems to really be describing $df$, not $\nabla f$. – Qiaochu Yuan Jul 28 '11 at 22:03
• @Qiaochu Near the top of page 94 he writes "The Jacobian matrix of $f$ is the row vector $(\partial_x f, \partial_y f, \partial_z f)$. We call this vector the gradient vector of $f$ and write $\nabla f$". – ItsNotObvious Jul 28 '11 at 22:12
• @3Sphere: yes, and...? I don't think that's the standard definition of the gradient in general. – Qiaochu Yuan Jul 28 '11 at 22:17
• @3Sphere: well, "incorrect" is slightly overstating it. There are two reasonable things one might naively call the gradient (a certain family of tangent vectors, or the family of linear functionals it defines) on $\mathbb{R}^n$ and these things happen to generalize differently. The definition is probably fine for working on $\mathbb{R}^n$ although I personally get confused when people don't distinguish between vectors and covectors (e.g. in intro special relativity courses) since they transform differently under a change of variables. – Qiaochu Yuan Jul 28 '11 at 22:32

Yes, the distinction between row vectors and column vectors is important. On an arbitrary smooth manifold $M$, the derivative of a function $f : M \to \mathbb{R}$ at a point $p$ is a linear transformation $df_p : T_p(M) \to \mathbb{R}$; in other words, it's a cotangent vector. In general the tangent space $T_p(M)$ does not come equipped with an inner product (this is an extra structure: see Riemannian manifold), so in general we cannot identify tangent vectors and cotangent vectors.
So on a general manifold one must distinguish between vector fields (families of tangent vectors) and differential $1$-forms (families of cotangent vectors). While $df$ is a differential form and exists for all $M$, $\nabla f$ can't be sensibly defined unless $M$ has a Riemannian metric, and then it's a vector field (and the identification between differential forms and vector fields now depends on the metric).
If one thinks of tangent vectors as column vectors, then $\nabla f$ ought to be a column vector, but the linear functional $\langle -, \nabla f \rangle$ ought to be a row vector. A major problem with working entirely in bases is that distinctions like these are frequently glossed over, and then when they become important students are very confused.
Some remarks about non-canonicity. The tangent space $T_p(V)$ to a vector space at any point can be canonically identified with $V$, so for vector spaces we don't run into quite the same problems. If $V$ is an inner product space, then in the same way it automatically inherits the structure of a Riemannian manifold by the above identification. Finally, when people write $V = \mathbb{R}^n$ they frequently intend $\mathbb{R}^n$ to have the standard inner product with respect to the standard basis, and this equips $V$ with the structure of a Riemannian manifold.