Kaplan's Advanced Calculus defines the gradient of a function $f:\mathbb{R^n} \rightarrow \mathbb{R}$ as the $1 \times n$ row vector whose entries respectively contain the $n$ partial derivatives of $f$. By this definition then, the gradient is just the Jacobian matrix of the transformation.

We also know that using the Riesz representation theorem, assuming $f$ is differentiable at the point $x$, we can define the gradient as the unique vector $\nabla f$ such that

$$ df(x)(h) = \langle h, \nabla f(x) \rangle, \; h \in \mathbb{R}^n $$

Assuming we ignore the distinction between row vectors and column vectors, the former definition follows easily from the latter. But, row vectors and column vectors are not the same things. So, I have the following questions:

  1. Is the distinction here between row/column vectors important?
  2. If (1) is true, then how can we know from the second defintion that the vector in question is a row vector and not a column vector?
  • 1
    $\begingroup$ The gradient as a row vector seems pretty non-standard to me. I'd say vectors are column vectors by definition (or usual convention), so $df(x)$ is a row vector (as it is a functional) while $\nabla f(x)$ is a column vector (the scalar product is a product of two vectors. And yes, the distinction is important. $\endgroup$ – t.b. Jul 28 '11 at 21:46
  • $\begingroup$ Kaplan seems to really be describing $df$, not $\nabla f$. $\endgroup$ – Qiaochu Yuan Jul 28 '11 at 22:03
  • $\begingroup$ @Qiaochu Near the top of page 94 he writes "The Jacobian matrix of $f$ is the row vector $(\partial_x f, \partial_y f, \partial_z f)$. We call this vector the gradient vector of $f$ and write $\nabla f$". $\endgroup$ – ItsNotObvious Jul 28 '11 at 22:12
  • 1
    $\begingroup$ @3Sphere: yes, and...? I don't think that's the standard definition of the gradient in general. $\endgroup$ – Qiaochu Yuan Jul 28 '11 at 22:17
  • 1
    $\begingroup$ @3Sphere: well, "incorrect" is slightly overstating it. There are two reasonable things one might naively call the gradient (a certain family of tangent vectors, or the family of linear functionals it defines) on $\mathbb{R}^n$ and these things happen to generalize differently. The definition is probably fine for working on $\mathbb{R}^n$ although I personally get confused when people don't distinguish between vectors and covectors (e.g. in intro special relativity courses) since they transform differently under a change of variables. $\endgroup$ – Qiaochu Yuan Jul 28 '11 at 22:32

Yes, the distinction between row vectors and column vectors is important. On an arbitrary smooth manifold $M$, the derivative of a function $f : M \to \mathbb{R}$ at a point $p$ is a linear transformation $df_p : T_p(M) \to \mathbb{R}$; in other words, it's a cotangent vector. In general the tangent space $T_p(M)$ does not come equipped with an inner product (this is an extra structure: see Riemannian manifold), so in general we cannot identify tangent vectors and cotangent vectors.

So on a general manifold one must distinguish between vector fields (families of tangent vectors) and differential $1$-forms (families of cotangent vectors). While $df$ is a differential form and exists for all $M$, $\nabla f$ can't be sensibly defined unless $M$ has a Riemannian metric, and then it's a vector field (and the identification between differential forms and vector fields now depends on the metric).

If one thinks of tangent vectors as column vectors, then $\nabla f$ ought to be a column vector, but the linear functional $\langle -, \nabla f \rangle$ ought to be a row vector. A major problem with working entirely in bases is that distinctions like these are frequently glossed over, and then when they become important students are very confused.

Some remarks about non-canonicity. The tangent space $T_p(V)$ to a vector space at any point can be canonically identified with $V$, so for vector spaces we don't run into quite the same problems. If $V$ is an inner product space, then in the same way it automatically inherits the structure of a Riemannian manifold by the above identification. Finally, when people write $V = \mathbb{R}^n$ they frequently intend $\mathbb{R}^n$ to have the standard inner product with respect to the standard basis, and this equips $V$ with the structure of a Riemannian manifold.

  • $\begingroup$ It is important to keep track of what things are and what extra structures they depend on, but distinctions like row vector versus column vector are essentially cosmetic until you have a whole family of objects, and then only because you want natural operations on your vectors to correspond to geometric operations and not nonsense (and you want the geometric operations to correspond to vector operations which are as simple as possible). If you are considering "row vector vs. column vector" you have already fixed a basis, so much of the intrinsic is already lost. $\endgroup$ – Aaron Jul 28 '11 at 22:06
  • $\begingroup$ @Aaron: well, I am just using "column vector" as a euphemism for "tangent vector" and "row vector" as a euphemism for "cotangent vector." I prefer that nobody use these terrible terms, but as long as the OP is... $\endgroup$ – Qiaochu Yuan Jul 28 '11 at 22:08
  • $\begingroup$ Looking at the Jacobian matrix makes it really look like row vectors dont it? i.e thinking of it as a linear map on the space on which it's function act. $\endgroup$ – user415535 Feb 4 '18 at 19:27

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.