I understand the proof that the Jacobian matrix is the best linear approximation of a function in the limit. But since it's a matrix it has different dimensions than the original function, shouldn'the it be a vector too? Given the Jacobian how would I find or plot where it takes some given 'input'? Since the derivative is a function I suppose one could transform it into something like f'(x1,.....,×n)=(y1,.....,yn)

I tried to be as clear as possible but I don't think I put it well into words.

EDIT: Clarifiyng, my question is how do I find, explicitly, an arbitray element of image of the derivative. Does it need two vectors/points to be defined? If so, why not only one point as in single variable calculus?

  • $\begingroup$ Have you ever heard of the differential of a function? $\endgroup$ – ClassicStyle Sep 29 '16 at 1:47
  • $\begingroup$ A matrix can be thought of as a function that takes a vector as input, and returns a vector as output. I.e., for a matrix $A$ and vector $v$, think of the action $v \mapsto Av$. $\endgroup$ – Nick Alger Sep 29 '16 at 1:52
  • $\begingroup$ I haven'the got to differential forms yet. I understand it is a linear transformation. I suppose my questions is a bit more 'practical'. In single-variable calculus the derivative for a arbitrary x is clear. But here how I'm supposed to find the derivative of an arbitrary point (x1,...,xn). If I take the entrances of the jacobian with the point (x1,...,xn) and multiply the matrix by the same arbitrary point (x1,...,xn) wouldn'the I be 'doubling' the output? If not, which vector should I do the product with the jacobian? $\endgroup$ – GoedelGrimm Sep 29 '16 at 2:14

There’s a subtle misunderstanding in your question that might be the source of some confusion. The Jacobian matrix (i.e., the differential) isn’t the best linear approximation to a function near a point $\mathbf x_0$. It’s the best linear approximation to the change in the function’s value near the point. That is, $$\Delta f_{\mathbf x_0}=f(\mathbf x_0+\Delta\mathbf x)-f(\mathbf x_0)=\operatorname{d}f_{\mathbf x_0}[\Delta\mathbf x]+o(\|\Delta \mathbf x\|).$$ Observe that $\operatorname{d}f_{\mathbf x_0}$ operates on a displacement $\Delta\mathbf x$ (technically, on a vector in the tangent space at $\mathbf x_0$), not on an element of $f$’s domain. If $f:\mathbb R^n\to\mathbb R^m$, then $\operatorname{d}f:\mathbb R^n\to\mathbb R^m$, and in matrix terms this linear transformation is represented by multiplication by the $m\times n$ matrix $\mathbf{Jac}_f(\mathbf{x_0})$, i.e., $$\operatorname{d}f_{\mathbf x_0}:\mathbf h\mapsto\mathbf{Jac}_f(\mathbf{x_0})\,\mathbf h,$$ where $\mathbf h$ is the $1\times n$ column vector corresponding to $\Delta\mathbf x$. One way to think of $\operatorname{d}f$ is as a rule that assigns an $m\times n$ matrix, $\mathbf{Jac}_f(\mathbf{x})$, to each point $\mathbf x$ of the domain of $f$. The matrix can vary from point to point, of course.

As a concrete example, take $f:(x,y)\mapsto x^2-y^2$ as in your comments to another answer. The Jacobian at the point $(x_0,y_0)$ is the $1\times 2$ matrix $\begin{bmatrix}2x_0&-2y_0\end{bmatrix}$, so the best linear approximation to $f$ at $(x_0,y_0)$ is $$\begin{align}f(x,y)&\approx f(x_0,y_0)+\begin{bmatrix}2x_0&-2y_0\end{bmatrix}\begin{bmatrix}x-x_0\\y-y_0\end{bmatrix} \\ &= x_0^2-y_0^2+2x_0(x-x_0)-2y_0(y-y_0).\end{align}$$ To illustrate this with some concrete values, $$f(3.2,-5.1)\approx3^2-(-5)^2+\begin{bmatrix}6&10\end{bmatrix}\begin{bmatrix}0.2&-0.1\end{bmatrix}^T=9-25+0.2=-15.8.$$ The actual value is $-15.77$.

A somewhat more interesting example is $f:\begin{bmatrix}x\\y\\z\end{bmatrix}\mapsto\begin{bmatrix}x^2-y^2+3z\\2xy\end{bmatrix}$. The Jacobian is $\begin{bmatrix}2x&-2y&3\\2y&2x&0\end{bmatrix}$, and so $$f(x,y,z)\approx\begin{bmatrix}x_0^2-y_0^2+3z_0\\2x_0y_0\end{bmatrix}+\begin{bmatrix}2x_0&-2y_0&3\\2y_0&2x_0&0\end{bmatrix}\begin{bmatrix}x-x_0\\y-y_0\\z-z_0\end{bmatrix}$$ and $$f(3.2,-5.1,1.7)\approx\begin{bmatrix}3^2-(-5)^2+3\cdot1\\2\cdot3\cdot(-5)\end{bmatrix}+\begin{bmatrix}6&10&3\\-10&6&0\end{bmatrix}\begin{bmatrix}0.2\\-0.1\\0.7\end{bmatrix}=\begin{bmatrix}-13\\-30\end{bmatrix}+\begin{bmatrix}2.3\\-2.6\end{bmatrix}=\begin{bmatrix}-10.7\\-32.6\end{bmatrix}.$$ The actual value is $\begin{bmatrix}-10.67&-32.64\end{bmatrix}^T$.

  • $\begingroup$ Thank you this clarified things a lot. My question is then, how do I choose $\delta x$? In single-variable calculus you talk about the derivative at a point, but don't explicitly state the change in a tangent line. $\endgroup$ – GoedelGrimm Sep 29 '16 at 9:39
  • $\begingroup$ It was supposed to be $\Delta x$. I'm on my phone I can't see if I'm typing mathjax correctly. $\endgroup$ – GoedelGrimm Sep 29 '16 at 10:05
  • $\begingroup$ @GoedelGrimm You don’t choose $\Delta\mathbf x$ per se. It’s the offset to the point at which you want to find the approximate value of $f$ from the one at which you know the differential. In single-variable terms, it’s the $h$ that goes to zero in the limit definition of the derivative. In little-oh notation, that would be expressed as $f(x+h)=f(x)+hf'(x)+o(h)$. The point $(x+h, f(x+h))$ is on the curve, while $(x+h, f(x)+hf'(x))$ is on the line tangent to the curve at $(x,f(x))$. As $h$ gets smaller, the distance between these two points gets smaller “faster” than $h$... $\endgroup$ – amd Sep 29 '16 at 20:03
  • $\begingroup$ @GoedelGrimm Similarly, for a surface defined by $f:\mathbb R^2\to\mathbb R$, the point $(x+\Delta x,y+\Delta y ,f(x+\Delta x,y+\Delta y))$ is on the surface, while $(x+\Delta x,y+\Delta y,f(x,y)+\operatorname{d}f[(\Delta x,\Delta y)])$ is on the tangent plane. The distance between these points—the error in the approximation—vanishes “faster” than the displacement $(\Delta x,\Delta y)$ from $(x,y)$. $\endgroup$ – amd Sep 29 '16 at 20:11

Let $\ f: \mathbb{R}^n \to \mathbb{R}^m$ and $p \in U \subset \mathbb{R}^n$ then the differential of $f$ at $p$ is given by the matrix;

$$\textbf{Jac}_f = \left[\frac{\partial f^i}{\partial x_j}\right]$$

which is $n \times m$ matrix and so it takes as input, vectors of the form $[x_1 \ \cdots x_m]^T$

  • $\begingroup$ For a arbitrary point x=(x1...xm), [Jf][x] would be the output. But to compute it, should the entrances of the jacobian be 'in function' of the input vector? If so, isn't it doing something twice? $\endgroup$ – GoedelGrimm Sep 29 '16 at 2:20
  • $\begingroup$ The jacobian is the matrix, the actual map we denote as $d_pf$ and we have $d_pf( \textbf{x}) = \textbf{Jac}_f( \textbf{x})$ does that answer your question? $\endgroup$ – Faraad Armwood Sep 29 '16 at 2:22
  • $\begingroup$ Say $d_p f: \mathbf x \mapsto \mathbf u$ How do I find $\mathbf u$ for an arbitray $\mathbf x$? Take $f(x,y)=x^2-y^2$ the jacobian is $[2x, -2y]$ the product by $\mathbf x$ is $[Jf(\mathbf x)][\mathbf x]= \mathbf u= 2x^2 -2y^2$. Is this $\mathbf u$? $\endgroup$ – GoedelGrimm Sep 29 '16 at 2:41
  • $\begingroup$ No one said the map was onto. Do you need me to do an explicit example? $\endgroup$ – Faraad Armwood Sep 29 '16 at 3:04
  • $\begingroup$ Actually, it would help. $\endgroup$ – GoedelGrimm Sep 29 '16 at 12:57

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.