Could anyone explain in simple words (and maybe with an example) what the difference between the gradient and the Jacobian is?

The gradient is a vector with the partial derivatives, right?

  • 7
    Actually, the gradient is subset of the Jacobian. – cheesyfluff Nov 8 '15 at 19:10
  • 2
    @cheesyfluff: it would be better to say that the gradient can be seen as special case of the Jacobian, i.e. when the function is scalar. The Jacobian is not a set. – Yves Daoust Aug 31 at 9:19

These are two particular forms of matrix representation of the derivative of a differentiable function $f,$ used in two cases:

  • when $f:\mathbb{R}^n\to\mathbb{R},$ then for $x$ in $\mathbb{R}^n$, $$\mathrm{grad}_x(f):=[\frac{\partial f}{\partial x_1}\frac{\partial f}{\partial x_2}\dots\frac{\partial f}{\partial x_n}]|_x$$ is the matrix $1\times n$ of the linear map $Df(x)$ exprimed from the canonical base of $\mathbb{R}^n$ to the canonical base of $\mathbb{R}$ (=(1)...). Because in this case this matrix would have only one row, you can think about it as the vector $$\nabla f(x):=(\frac{\partial f}{\partial x_1},\frac{\partial f}{\partial x_2},\dots,\frac{\partial f}{\partial x_n})|_x\in\mathbb{R}^n.$$ This vector $\nabla f(x)$ is the unique vector of $\mathbb{R}^n$ such that $Df(x)(y)=\langle\nabla f(x),y\rangle$ for all $y\in\mathbb{R}^n$ (see Riesz representation theorem), where $\langle\cdot,\cdot\rangle$ is the usual scalar product $$\langle(x_1,\dots,x_n),(y_1,\dots,y_n)\rangle=x_1y_1+\dots+x_ny_n.$$
  • when $f:\mathbb{R}^n\to\mathbb{R}^m,$ then for $x$ in $\mathbb{R}^n$, $$\mathrm{Jac}_x(f)=\begin{bmatrix}\frac{\partial f_1}{\partial x_1}&\frac{\partial f_1}{\partial x_2}&\dots&\frac{\partial f_1}{\partial x_n}\\\frac{\partial f_2}{\partial x_1}&\frac{\partial f_2}{\partial x_2}&\dots&\frac{\partial f_2}{\partial x_n}\\ \vdots&\vdots&&\vdots\\\frac{\partial f_m}{\partial x_1}&\frac{\partial f_m}{\partial x_2}&\dots&\frac{\partial f_m}{\partial x_n}\\\end{bmatrix}|_x$$ is the matrix $m\times n$ of the linear map $Df(x)$ exprimed from the canonical base of $\mathbb{R}^n$ to the canonical base of $\mathbb{R}^m.$

For example, with $f:\mathbb{R}^2\to\mathbb{R}$ such as $f(x,y)=x^2+y$ you get $\mathrm{grad}_{(x,y)}(f)=[2x \,\,\,1]$ (or $\nabla f(x,y)=(2x,1)$) and for $f:\mathbb{R}^2\to\mathbb{R}^2$ such as $f(x,y)=(x^2+y,y^3)$ you get $\mathrm{Jac}_{(x,y)}(f)=\begin{bmatrix}2x&1\\0&3y^2\end{bmatrix}.$

  • how is your answer different from this one - math.stackexchange.com/questions/336640/… ? – gansub Jan 20 '17 at 4:30
  • @gansub : The two ones are really similar indeed! I think that it has not been tagged as duplicate since the two questions sound differently, but except an example my answer does not bring other informations than Branimir Ćaćić's one. – Balloon Jan 20 '17 at 8:06
  • not really similar IMHO. He says it is a column vector. You are saying gradient is a row vector. Clarification ? – gansub Jan 20 '17 at 8:07
  • @ gansub : I think he does the same clarification that me : you can think about the gradient as a matrix (more precisely a row vector), or as a vector of $\mathbb{R}^n$ (which is often written as $(x_1,\dots,x_n)\in\mathbb{R}^n$ to not overload the paper, but which means $$\begin{pmatrix}x_1\\\vdots\\x_n\end{pmatrix}\in\mathbb{R}^n,$$ and so corresponds indeed to a column vector). – Balloon Jan 20 '17 at 8:10
  • 1
    Yes, we have the equality $\mathbb{R}^n\simeq\mathcal{M}_{n\times 1}(\mathbb{R})$. I would prefer to write "we have the canonical isomorphism" between the two spaces, since the first notation really makes me think about $\mathbb{R}^n$ as a vector space whereas the second one makes me think about $\mathbb{R}^n$ as a coordinate space. – Balloon Jan 20 '17 at 8:20

The gradient vector of a scalar function $f(\mathbf{x})$ that maps $\mathbb{R}^n\to\mathbb{R}$ where $\mathbf{x}=<x_1,x_2,\ldots,x_n>$ is written as $$\nabla f(\mathbf{x})=\frac{\partial f(\mathbf{x})}{\partial x_1}\hat{x}_1+\frac{\partial f(\mathbf{x})}{\partial x_2}\hat{x}_2+\ldots+\frac{\partial f(\mathbf{x})}{\partial x_n}\hat{x}_n$$

Whereas the Jacobian is taken of a vector function $\mathbf{f}(\mathbf{x})$ that maps $\mathbb{R}^n\to\mathbb{R}^m$, where $\mathbf{f}=<f_1,f_2,\ldots,f_m>$ and $\mathbf{x}=<x_1,x_2,\ldots,x_n>$. The Jacobian is written as

$$J_\mathbf{f} = \frac{\partial (f_1,\ldots,f_m)}{\partial(x_1,\ldots,x_n)} = \left[ \begin{matrix} \frac{\partial f_1}{\partial x_1} & \cdots & \frac{\partial f_1}{\partial x_n} \\ \vdots & \ddots & \vdots \\ \frac{\partial f_m}{\partial x_1} & \cdots & \frac{\partial f_m}{\partial x_n} \end{matrix} \right]$$

Note that when $m=1$ the Jacobian is same as the gradient because it is a generalization of the gradient.

The Jacobian determinant can be used for changes of variables because it can be viewed as the ratio of an infinitesimal change in the variables of one coordinate system to another. This requires that the function $\mathbf{f}(\mathbf{x})$ maps $\mathbb{R}^n\to\mathbb{R}^n$, which produces an $n\times n$ square matrix for the Jacobian. For example

$$\iiint_R f(x,y,z) \,dx\,dy\,dz = \iiint_S f(x(u,v,w),y(u,v,w),z(u,v,w))\left|\frac{\partial (x,y,z)}{\partial(u,v,w)}\right|\,du\,dv\,dw$$

where the Jacobian $J_\mathbf{g}$ is taken of the function

$$\mathbf{g}(u,v,w)=x(u,v,w)\hat{\imath}+y(u,v,w)\hat{\jmath}+z(u,v,w)\hat{k}$$

and the areas $R$ and $S$ correspond to each other.

The gradient and Jacobian are both disguise names for what is really "the derivative" of respectively a real-valued function of several real variables and a vector field, respectively. Or in better terms the gradient and Jacobian are exactly to functions of type $\mathbb{R}^n \rightarrow \mathbb{R}$ and $\mathbb{R}^m \rightarrow \mathbb{R}^n$ what the ordinary derivative $\frac{d}{dx}$ is to function of type $\mathbb{R} \rightarrow \mathbb{R}$, as opposed to partial and directional derivatives and other such derivative-like concepts. They are the "straight-up" honest-to-goodness derivatives - something all the calc texts I've seen seem to just quietly omit despite that it is incredibly important to developing an intuitive grasp of these concepts!

The derivative of an ordinary function at some point gives you, intuitively, the following things:

  1. the slope of the line that is approximated by the graph of the function when you zoom very far into the point in question - this is the airtight intuitive meaning of a tangent line with which you virtually can't go wrong (and in fact can be directly translated into a rigorous definition of a tangent line for which the fact the derivative is the slope can be proven as a theorem!),
  2. the linear map (i.e. $L(x) = ax$, not $L(x) = ax + b$, the latter is an affine map) that the function "looks like" near the point after accounting for a possible shift of coordinates (that's the "affine" bit), as given by its characterizing number $a$, which is the same as the slope above.

When you go to a real-valued function of $n$ variables, the derivative becomes something of a different kind than the original map. (*) This can be seen by noting that if, say, we're considering a real-valued function of two variables, what the direct generalization of 1) above is is the "slope of the tangent plane" to the surface formed by the function when graphed as a surface plot in 3D, but a plane doesn't just have a steepness, it also has a direction in which it slopes upward or which it "points" (along the horizontal plane) with its upward climb. Thus to specify its slope, we need both a magnitude and a direction - that is, a vector, and thus it should be no surprise that the derivative of our function of two variables, $\mathrm{grad}[f]$, is just such a vector. The corresponding linear map is a linear functional (linear map that outputs a number), and it acts upon the input vector by this acting through the dot product. Equivalently the linear map is the covector $(\mathrm{grad}[f])^T$, where the little $T$ is chosen purposefully to coincide with, as it actually is, the linear-algebraic transpose.

When you get to a function of several variables to several variables, now you are talking a general linear map between spaces, and the derivative must be described by a matrix at each point. This matrix is just the Jacobian matrix. It is the exact equivalent of the number $a$ above, or the slope $m$ of the tangent line - only now it's the "slope" of some weird hyperdimensional tangenty-thing on another such hyperdimensional surfacey-thing: a "thing" we'd more properly call, respectively, the tangent space to the manifold that is formed by the hyperdimensional "graph" of the function (who lives in $\mathbb{R}^{n+m}$) - concepts that we distinguish because there is a way to describe them on their own terms without actually needing to reference them as being embedded in any kind of hyperdimensional space to begin with.

(*) Actually, technically so is the ordinary derivative, at least under the second interpretation above. A linear map is not a number, and actually if we want to get real picky (in mathese: stop identifying by isomorphisms) the "first" notion of derivative is, for consistency, a 1-vector, not a scalar.

The gradient is the vector formed by the partial derivatives of a scalar function.

The Jacobian matrix is the matrix formed by the partial derivatives of a vector function. Its vectors are the gradients of the respective components of the function.

E.g., with some argument omissions,

$$\nabla f(x,y)=\begin{pmatrix}f'_x\\f'_y\end{pmatrix}$$

$$J \begin{pmatrix}f(x,y),g(x,y)\end{pmatrix}=\begin{pmatrix}f'_x&&g'_x\\f'_y&&g'_y\end{pmatrix}=\begin{pmatrix}\nabla f;\nabla g\end{pmatrix}.$$

If you want, the Jacobian is a generalization of the gradient to vector functions.

Your Answer

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Not the answer you're looking for? Browse other questions tagged or ask your own question.