0
$\begingroup$

I have been reading some lecture notes, which have been somewhat confusing for me.

What the lecture notes state:


Let $f:\Omega \subseteq \mathbb{R}^n \rightarrow \mathbb{R}^m$ be a continuously differentiable function, remembering analysis we know that

$Df: \Omega \rightarrow Lin(\mathbb{R}^n,\mathbb{R}^m)$ with $x_{\ast} \in \Omega \mapsto (Df)_{x_{\ast}}$, where the linear map $(Df)_{x_{\ast}}$ evaluated in $v \in \mathbb{R}^n$ can be written as:

$(Df)_{x_{\ast}} \cdot (v):=Jacf(x_{\ast}) \cdot v=$ $ \begin{pmatrix} \partial_{x_1}f_1(x_{\ast}) & ... & \partial_{x_n}f_1(x_{\ast}) \\ ... & & \\ \partial_{x_1}f_m(x_{\ast}) & ... & \partial_{x_n}f_m(x_{\ast}) \end{pmatrix} \cdot \begin{pmatrix} v_1\\ ...\\ v_n\\ \end{pmatrix}$


As far as I can remember $D(f(x_1,...,x_n))=\begin{pmatrix} \partial_{x_1}f_1(x_{\ast}) & ... & \partial_{x_n}f_1(x_{\ast}) \\ ... & & \\ \partial_{x_1}f_m(x_{\ast}) & ... & \partial_{x_n}f_m(x_{\ast}) \end{pmatrix}$, but if I want to evaluate the matrix $Df(x_1,...,x_n)$ in some point $p=(p_1,...,p_n)$, I just take the Jacobi matrix and insert $p_1$ for $x_1$,..., $p_n$ for $x_n$.

For example: If $f(x,y)=(\cos(x),\sin(x),x)$, then the Jacobi Matrix is $\begin{pmatrix} -\sin(x) & 0\\ 0 & \cos(y) \\ 1 & 0 \end{pmatrix}$. If I now want to evaluate it at for example in $(3,2)$, I would get:

$\begin{pmatrix} -\sin(3) & 0\\ 0 & \cos(2) \\ 1 & 0 \end{pmatrix}$

But, the lecture notes would state, to take the matrix $\begin{pmatrix} -\sin(x) & 0\\ 0 & \cos(y) \\ 1 & 0 \end{pmatrix}$ and multiply it by $\begin{pmatrix} 3 \\ 2 \end{pmatrix}$ , which yields a different result.

My Question is: Have I been understanding it wrong, or did I misunderstand what the mentioned text means?

$\endgroup$
2
  • $\begingroup$ The Jacobian is the linear map (matrix) which approximates the differentiable function. When you multiply that matrix by $(3,2)$ you are rather getting the approximated output value from the matrix. $\endgroup$ Nov 12, 2023 at 4:26
  • $\begingroup$ Take the function $f(x) = x^2$. Then the (Fréchet) derivative at $x$ is the linear map $h \mapsto 2xh$, or $Df(x)h = 2xh$. In one dimension this is not very exciting. $\endgroup$
    – copper.hat
    Nov 12, 2023 at 6:31

2 Answers 2

2
$\begingroup$

$Df$ has two arguments. The first argument is the point in $\Omega$, where you evaluate your Jacobian matrix. This matrix then acts (via multiplication) on your vector of increments $v\in\mathbb{R}^n$ to produce the linear part of the increment of your function (a vector in $\mathbb{R}^m$).

$Df$ is linear in its second argument, so it acts via multiplication by a matrix. It is not linear, in general, in the first argument, so you can consider higher order differentials.

$\endgroup$
3
  • $\begingroup$ I am sorry, but I am not sure if I understand what you mean. I remember the definition of a function $f: \mathbb{R}^n \rightarrow \mathbb{R}^m$ being differentiable in some point $\xi \in \mathbb{R}^n$. This is the case if there exists an $n \times m$ matrix $A$, such that $\forall \xi+h$ in some neighborhood $U$ of $\xi$, we can write the increment $f(\xi+h)-f(\xi)$, as $f(\xi+h)-f(\xi)=Ah+r(h)$, with $\lim_{h \rightarrow 0} \frac{r(h)}{||h||}$. $\endgroup$
    – Peter
    Nov 12, 2023 at 4:55
  • $\begingroup$ In what you wrote in the first section, this would mean that multiplying the Jacobi Matrix $A$ with the vector $h$ would give me the vector $f(\xi+h)-f(x)$.Did I understand that correct? $\endgroup$
    – Peter
    Nov 12, 2023 at 4:55
  • 1
    $\begingroup$ That is correct. Up to $o(h)$. $\endgroup$
    – GReyes
    Nov 12, 2023 at 17:33
1
$\begingroup$

I think what you're missing here is that the derivative at a point is not a scalar, or a vector, or even (technically speaking) a matrix. It is a linear function. It's a function in the sense that it acts on the space $\Bbb{R}^n$ and produces a vector in $\Bbb{R}^m$. We are used to representing (naturally) as a matrix, or even as a scalar in the $\Bbb{R} \to \Bbb{R}$ case, but it should be thought of as a function.

You can see this in the notation in your notes: the codomain of $Df$ is $\operatorname{Lin}(\Bbb{R}^n, \Bbb{R}^m)$, the set of linear functions from $\Bbb{R}^n$ to $\Bbb{R}^m$.

This space of functions can be naturally represented by matrices. For any $T \in \operatorname{Lin}(\Bbb{R}^n, \Bbb{R}^m)$, there exists a unique matrix $M_T \in \Bbb{R}^{n \times m}$ such that: $$Tv = M_Tv$$ for all $v \in \Bbb{R}^n$. So, for this reason, you will often see derivatives represented as matrices.

In your example, $f(x, y) = (\cos x, \sin y, x)$, the derivative at a point, say, $(\pi/2, \pi)$, will be the linear map: $$\pmatrix{a \\ b} \mapsto \pmatrix{-a \\ -b \\ a} = \pmatrix{-\sin \frac{\pi}{2} & 0 \\0 & \cos \pi \\ 1 & 0}\pmatrix{a \\ b}.$$ This is represented by the above $3 \times 2$ matrix.

The particular linear function you get will depend on the point at which you're taking the derivative (remember, $Df$ is a function, and depends on the input from its domain $\Bbb{R}^n$, or more precisely, the set of points where $f$ is differentiable). So, choosing another point will produce a different linear function under $Df$, and a different corresponding matrix.

It's worth paying a bit of attention to the $\Bbb{R} \to \Bbb{R}$ case. In this case, the linear map is also from $\Bbb{R}$ to $\Bbb{R}$, leading to a $1 \times 1$ matrix. Linear maps from $\Bbb{R}$ to $\Bbb{R}$ take the form $x \mapsto mx$ for some $m$. When we start teaching derivatives, we identify the linear map $x \mapsto mx$ more simply by the coefficient $m$, which is a real number. This is why, when we take the derivative of a single-variable function, e.g. $f(x) = x^3 + x$, we produce a scalar-valued derivative $f'(x) = 3x^2 + 1$. When I say the derivative is a function, this is not the function I mean. What we are saying, in terms of the more general structure, is that, at a given point $x_0$, the derivative $Df$, evaluated at $x_0$, is the linear map $$x \mapsto (3x_0^2 + 1)x.$$ This is the linear map, and it will change depending on the point $x_0 \in \Bbb{R}$ we substitute in.

So, let's look back at $f(x, y) = (\cos(x), \sin(y), x)$. What you computed was the matrix representation of the derivative of $f$ at the point $(3, 2)$. What the question was asking you to do was to compute the matrix representation at an arbitrary point $(x, y)$ (giving us a linear map from $\Bbb{R}^2$ to $\Bbb{R}^3$), then evaluate this linear map at the point $\pmatrix{3 \\ 2}$. This corresponds to simply multiplying the vector by its matrix representation.

As you can see, it's a different question to how you interpreted it. You answered a perfectly reasonable question, but it wasn't the question that was asked.

$\endgroup$
1
  • $\begingroup$ Thank you very much! $\endgroup$
    – Peter
    Nov 12, 2023 at 6:23

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .