How is the Jacobian the derivative if they have different dimensions

I understand the proof that the Jacobian matrix is the best linear approximation of a function in the limit. But since it's a matrix it has different dimensions than the original function, shouldn'the it be a vector too? Given the Jacobian how would I find or plot where it takes some given 'input'? Since the derivative is a function I suppose one could transform it into something like f'(x1,.....,×n)=(y1,.....,yn)

I tried to be as clear as possible but I don't think I put it well into words.

EDIT: Clarifiyng, my question is how do I find, explicitly, an arbitray element of image of the derivative. Does it need two vectors/points to be defined? If so, why not only one point as in single variable calculus?

• Have you ever heard of the differential of a function? – ClassicStyle Sep 29 '16 at 1:47
• A matrix can be thought of as a function that takes a vector as input, and returns a vector as output. I.e., for a matrix $A$ and vector $v$, think of the action $v \mapsto Av$. – Nick Alger Sep 29 '16 at 1:52
• I haven'the got to differential forms yet. I understand it is a linear transformation. I suppose my questions is a bit more 'practical'. In single-variable calculus the derivative for a arbitrary x is clear. But here how I'm supposed to find the derivative of an arbitrary point (x1,...,xn). If I take the entrances of the jacobian with the point (x1,...,xn) and multiply the matrix by the same arbitrary point (x1,...,xn) wouldn'the I be 'doubling' the output? If not, which vector should I do the product with the jacobian? – GoedelGrimm Sep 29 '16 at 2:14

There’s a subtle misunderstanding in your question that might be the source of some confusion. The Jacobian matrix (i.e., the differential) isn’t the best linear approximation to a function near a point $\mathbf x_0$. It’s the best linear approximation to the change in the function’s value near the point. That is, $$\Delta f_{\mathbf x_0}=f(\mathbf x_0+\Delta\mathbf x)-f(\mathbf x_0)=\operatorname{d}f_{\mathbf x_0}[\Delta\mathbf x]+o(\|\Delta \mathbf x\|).$$ Observe that $\operatorname{d}f_{\mathbf x_0}$ operates on a displacement $\Delta\mathbf x$ (technically, on a vector in the tangent space at $\mathbf x_0$), not on an element of $f$’s domain. If $f:\mathbb R^n\to\mathbb R^m$, then $\operatorname{d}f:\mathbb R^n\to\mathbb R^m$, and in matrix terms this linear transformation is represented by multiplication by the $m\times n$ matrix $\mathbf{Jac}_f(\mathbf{x_0})$, i.e., $$\operatorname{d}f_{\mathbf x_0}:\mathbf h\mapsto\mathbf{Jac}_f(\mathbf{x_0})\,\mathbf h,$$ where $\mathbf h$ is the $1\times n$ column vector corresponding to $\Delta\mathbf x$. One way to think of $\operatorname{d}f$ is as a rule that assigns an $m\times n$ matrix, $\mathbf{Jac}_f(\mathbf{x})$, to each point $\mathbf x$ of the domain of $f$. The matrix can vary from point to point, of course.

As a concrete example, take $f:(x,y)\mapsto x^2-y^2$ as in your comments to another answer. The Jacobian at the point $(x_0,y_0)$ is the $1\times 2$ matrix $\begin{bmatrix}2x_0&-2y_0\end{bmatrix}$, so the best linear approximation to $f$ at $(x_0,y_0)$ is \begin{align}f(x,y)&\approx f(x_0,y_0)+\begin{bmatrix}2x_0&-2y_0\end{bmatrix}\begin{bmatrix}x-x_0\\y-y_0\end{bmatrix} \\ &= x_0^2-y_0^2+2x_0(x-x_0)-2y_0(y-y_0).\end{align} To illustrate this with some concrete values, $$f(3.2,-5.1)\approx3^2-(-5)^2+\begin{bmatrix}6&10\end{bmatrix}\begin{bmatrix}0.2&-0.1\end{bmatrix}^T=9-25+0.2=-15.8.$$ The actual value is $-15.77$.

A somewhat more interesting example is $f:\begin{bmatrix}x\\y\\z\end{bmatrix}\mapsto\begin{bmatrix}x^2-y^2+3z\\2xy\end{bmatrix}$. The Jacobian is $\begin{bmatrix}2x&-2y&3\\2y&2x&0\end{bmatrix}$, and so $$f(x,y,z)\approx\begin{bmatrix}x_0^2-y_0^2+3z_0\\2x_0y_0\end{bmatrix}+\begin{bmatrix}2x_0&-2y_0&3\\2y_0&2x_0&0\end{bmatrix}\begin{bmatrix}x-x_0\\y-y_0\\z-z_0\end{bmatrix}$$ and $$f(3.2,-5.1,1.7)\approx\begin{bmatrix}3^2-(-5)^2+3\cdot1\\2\cdot3\cdot(-5)\end{bmatrix}+\begin{bmatrix}6&10&3\\-10&6&0\end{bmatrix}\begin{bmatrix}0.2\\-0.1\\0.7\end{bmatrix}=\begin{bmatrix}-13\\-30\end{bmatrix}+\begin{bmatrix}2.3\\-2.6\end{bmatrix}=\begin{bmatrix}-10.7\\-32.6\end{bmatrix}.$$ The actual value is $\begin{bmatrix}-10.67&-32.64\end{bmatrix}^T$.

• Thank you this clarified things a lot. My question is then, how do I choose $\delta x$? In single-variable calculus you talk about the derivative at a point, but don't explicitly state the change in a tangent line. – GoedelGrimm Sep 29 '16 at 9:39
• It was supposed to be $\Delta x$. I'm on my phone I can't see if I'm typing mathjax correctly. – GoedelGrimm Sep 29 '16 at 10:05
• @GoedelGrimm You don’t choose $\Delta\mathbf x$ per se. It’s the offset to the point at which you want to find the approximate value of $f$ from the one at which you know the differential. In single-variable terms, it’s the $h$ that goes to zero in the limit definition of the derivative. In little-oh notation, that would be expressed as $f(x+h)=f(x)+hf'(x)+o(h)$. The point $(x+h, f(x+h))$ is on the curve, while $(x+h, f(x)+hf'(x))$ is on the line tangent to the curve at $(x,f(x))$. As $h$ gets smaller, the distance between these two points gets smaller “faster” than $h$... – amd Sep 29 '16 at 20:03
• @GoedelGrimm Similarly, for a surface defined by $f:\mathbb R^2\to\mathbb R$, the point $(x+\Delta x,y+\Delta y ,f(x+\Delta x,y+\Delta y))$ is on the surface, while $(x+\Delta x,y+\Delta y,f(x,y)+\operatorname{d}f[(\Delta x,\Delta y)])$ is on the tangent plane. The distance between these points—the error in the approximation—vanishes “faster” than the displacement $(\Delta x,\Delta y)$ from $(x,y)$. – amd Sep 29 '16 at 20:11

Let $\ f: \mathbb{R}^n \to \mathbb{R}^m$ and $p \in U \subset \mathbb{R}^n$ then the differential of $f$ at $p$ is given by the matrix;

$$\textbf{Jac}_f = \left[\frac{\partial f^i}{\partial x_j}\right]$$

which is $n \times m$ matrix and so it takes as input, vectors of the form $[x_1 \ \cdots x_m]^T$

• For a arbitrary point x=(x1...xm), [Jf][x] would be the output. But to compute it, should the entrances of the jacobian be 'in function' of the input vector? If so, isn't it doing something twice? – GoedelGrimm Sep 29 '16 at 2:20
• The jacobian is the matrix, the actual map we denote as $d_pf$ and we have $d_pf( \textbf{x}) = \textbf{Jac}_f( \textbf{x})$ does that answer your question? – Faraad Armwood Sep 29 '16 at 2:22
• Say $d_p f: \mathbf x \mapsto \mathbf u$ How do I find $\mathbf u$ for an arbitray $\mathbf x$? Take $f(x,y)=x^2-y^2$ the jacobian is $[2x, -2y]$ the product by $\mathbf x$ is $[Jf(\mathbf x)][\mathbf x]= \mathbf u= 2x^2 -2y^2$. Is this $\mathbf u$? – GoedelGrimm Sep 29 '16 at 2:41
• No one said the map was onto. Do you need me to do an explicit example? – Faraad Armwood Sep 29 '16 at 3:04
• Actually, it would help. – GoedelGrimm Sep 29 '16 at 12:57