continuously differentiable multivariable functions

Question

What does it really mean to say a function $f:\mathbb{R}^n\rightarrow \mathbb{R}^n$ is continuously differentiable?

A function $f:\mathbb{R}\rightarrow \mathbb{R}$ is continuously differentiable if $f$ is differentiable and $f':\mathbb{R}\rightarrow \mathbb{R}$ is continuous..

But what is $f'$ in case of $f:\mathbb{R}^n\rightarrow \mathbb{R}^n$..

Is it the jacobian $\begin{bmatrix}\frac{\partial f_1}{\partial x_1}& \frac{\partial f_1}{\partial x_2}& \cdots &\frac{\partial f_1}{\partial x_n} \\ \frac{\partial f_2}{\partial x_1}& \frac{\partial f_2}{\partial x_2}& \cdots &\frac{\partial f_2}{\partial x_n} \\ \vdots \\ \frac{\partial f_n}{\partial x_1}& \frac{\partial f_n}{\partial x_2}& \cdots &\frac{\partial f_n}{\partial x_n} \end{bmatrix}$

I could not understand how does this matrix act on $\mathbb{R}^n$..

As an example, for $f(x,y)=(xy,x+y)$ we see that jacobian is $\begin{bmatrix}y&x\\1&1\end{bmatrix}$.. I do not understand how do i define $Df: \mathbb{R}^2\rightarrow \mathbb{R}^2$..

In case of derivative at a point we have $Df((a,b))=\begin{bmatrix}b&a\\1&1\end{bmatrix}$ and we define

$Df(a,b)(x,y)=\begin{bmatrix}b&a\\1&1\end{bmatrix} \begin{bmatrix}x\\y\end{bmatrix}=\begin{bmatrix}bx+ay\\x+y\end{bmatrix}$

I mean there are already variables in $Df$ and so i am getting confused how to act on $\mathbb{R}^n$.. But in case of $Df(a,b)$ there are no variables.. So, it seems to be natural.

Answer 1

Given a function $f:\>\Omega\to{\mathbb R}^m$ with domain $\Omega\subset{\mathbb R}^n$ its derivative is a map $$df:\quad\Omega\to{\cal L}({\mathbb R}^n,{\mathbb R}^m),\qquad x\mapsto df(x)\ .$$ The "coordinates" of $df(x)$ are the entries of the Jacobian $$J(x):=\left[\matrix{f_{1.1}&\cdots& f_{1.n}\cr\vdots\cr f_{m.1}&\cdots& f_{m.n}\cr}\right]_x\ ,\qquad f_{i.k}(x):={\partial f_i\over\partial x_k}(x)\ .$$ The space ${\cal L}({\mathbb R}^n,{\mathbb R}^m)$ has a natural metric induced by the norm $\|A\|:=\sup_{|x|=1}|Ax|$. It so happens that the function $df$ is continuous on $\Omega$ iff all entries $f_{i.k}$ of the Jacobian are continuous on $\Omega$. This fact belongs to elementary limit geometry in ${\mathbb R}^d$ and has nothing to do with differentiation per se.

Answer 2

There is a general theory of differentiation for functions between two normed space. However, you may be happy to learn that a function $f \colon \mathbb{R}^n \to \mathbb{R}^m$ is continuously differentiable if and only if each component $f_i \colon \mathbb{R}^n \to \mathbb{R}$ is continuously differentiable, for $i=1,\ldots,m$.

Answer 3

$ \newcommand{\d}{\mathrm{d}\,} \newcommand{\df}{\mathrm{d}\,f} \newcommand{\dfx}{\mathrm{d}\,f_{\x}} \newcommand{\x}{ \vec{\mathbf{p}}} \newcommand{\h}{ \vec{\mathbf{x}}} \newcommand{\p}{ \vec{\mathbf{p}}} \newcommand{\y}{ \vec{\mathbf{y}}} \newcommand{\xz}{ \vec{\mathbf{x}}^{0}} $ I believe that the best way to interpret Jacobian matrix is think of it as a total derivative of a function $f:\Bbb R^n \to \Bbb R^n$. In other words, $f' $ is the best linear approximation of $f$. The linear approximation depends on a point at which we trying to expand $f$.

Recall the definition of differentiable function $\left( \text{ at a point } \ \x = \left[\begin{smallmatrix} x^p_1, & x^p_2, & , \dots, & x^p_n\end{smallmatrix}\right]^T\,\right)$:

$f: \Bbb R^n \to \Bbb R^n$ is called (totally) differentiable, at a point $x_0 \in\Bbb R^n $ if there exists a linear map $\dfx : \Bbb R^n \to \Bbb R^n$ such that $$ \lim_{\x\to \y} \frac{\left\|\, f\left(\x\right) - f\left(\y\right) - \dfx \left(\x - \y\right) \,\right\|}{\left\|\, \left(\x - \y\right) \,\right\|} = 0 $$

In this case the linearization of $f$ at a point $\x$ will look like: $$ f\left(\x + \y \right) = f\left(\x \right) + \dfx\cdot \y + R\left( \y \right) $$ where the remainder $R\left( \y \right) = o\left( \left\|\y \right\|\right)$, i.e. $\lim_{\left\|\y \right\|\to 0} \frac{\left\|\, R\left(\y\right) \,\right\|}{\left\|\, \y \,\right\|} = 0$.

Since $\dfx: \Bbb R^n \to \Bbb R^n $ is a linear map, it can be represented as a matrix. Using notation $$ \h = \begin{bmatrix} x_1 \\ x_2 \\ \vdots \\ x_n \end{bmatrix}, \qquad f\left(\h\right) = \begin{bmatrix} f_1 \left(\h \right) \\ f_2 \left(\h \right) \\ \vdots \\ f_n \left(\h \right) \end{bmatrix} = \begin{bmatrix} f_1 \left(x_1, x_2, \dots, x_n \right) \\ f_2 \left(x_1, x_2, \dots, x_n \right) \\ \vdots \\ f_n \left(x_1, x_2, \dots, x_n \right) \end{bmatrix}, $$ the total derivative of $f$ can be expressed as Jacobian matrix: $$ \dfx =\frac{\mathrm{D}\,f\left(\h \right)}{\mathrm{D}\,\h}\bigg\lvert_{\h = \x} = \begin{bmatrix} \frac{\partial f_1\left(\h \right) }{\partial x_1} \big\lvert_{\h = \x} & \frac{\partial f_1\left(\h \right) }{\partial x_2} \big\lvert_{\h = \x} & \cdots & \frac{\partial f_1\left(\h \right) }{\partial x_n} \big\lvert_{\h = \x} \\ \frac{\partial f_2\left(\h \right) }{\partial x_1} \big\lvert_{\h = \x} & \frac{\partial f_2\left(\h \right) }{\partial x_2} \big\lvert_{\h = \x} & \cdots & \frac{\partial f_2\left(\h \right) }{\partial x_n} \big\lvert_{\h = \x} \\ \vdots & \vdots & \ddots & \vdots \\ \frac{\partial f_n\left(\h \right) }{\partial x_1} \big\lvert_{\h = \x} & \frac{\partial f_n\left(\h \right) }{\partial x_2} \big\lvert_{\h = \x} & \cdots & \frac{\partial f_n\left(\h \right) }{\partial x_n} \big\lvert_{\h = \x} \end{bmatrix} \label{*}\tag{*} $$

You have to keep in mind that strictly speaking the derivative $\dfx$ is defined only at a certain point $\x\in \Bbb R^n$, and that every entry in the matrix $\eqref{*}$ is a constant.

Therefore the Jacobian matrix can act on a vector $\y \in \Bbb R^n$, and the result will also be a vector in $\Bbb R^n$: $$ \dfx \cdot \y = \begin{bmatrix} \frac{\partial f_1\left(\h \right) }{\partial x_1} \big\lvert_{\h = \x} & \frac{\partial f_1\left(\h \right) }{\partial x_2} \big\lvert_{\h = \x} & \cdots & \frac{\partial f_1\left(\h \right) }{\partial x_n} \big\lvert_{\h = \x} \\ \frac{\partial f_2\left(\h \right) }{\partial x_1} \big\lvert_{\h = \x} & \frac{\partial f_2\left(\h \right) }{\partial x_2} \big\lvert_{\h = \x} & \cdots & \frac{\partial f_2\left(\h \right) }{\partial x_n} \big\lvert_{\h = \x} \\ \vdots & \vdots & \ddots & \vdots \\ \frac{\partial f_n\left(\h \right) }{\partial x_1} \big\lvert_{\h = \x} & \frac{\partial f_n\left(\h \right) }{\partial x_2} \big\lvert_{\h = \x} & \cdots & \frac{\partial f_n\left(\h \right) }{\partial x_n} \big\lvert_{\h = \x} \end{bmatrix} \cdot \begin{bmatrix} y_1 \\ y_2 \\ \vdots \\ y_n \end{bmatrix} \implies \dfx \cdot \y \in \Bbb R^n $$

https://en.wikipedia.org/wiki/Total_derivative#The_total_derivative_as_a_linear_map

Answer 4

If you use the Euclidean norm on $\mathbb{R}^n$ then continuity of $Df$ means that for all $\epsilon>0$, there exists a $\delta >0$ such that if $||u - v|| <\delta$ then $||Dfu - Dfv|| < \epsilon$.

Edit

Using your mapping that you have above, if we let $u = \left [ \begin{array}{c} u_1 \\ u_2 \\ \end{array} \right ]$ and $v = \left [ \begin{array}{c} v_1 \\ v_2\\ \end{array} \right ]$, then whenever $||u-v|| < \delta$ we would find

$$ ||Dfu - Dfv|| \;\; =\;\; ||Df(u-v)|| \;\; =\;\;\left | \left | \left [ \begin{array}{cc} y & x \\ 1 & 1 \\ \end{array} \right ] \left [ \begin{array}{c} u_1 - v_1 \\ u_2 - v_2\\ \end{array} \right ] \right | \right | \;\; =\;\; \left | \left | \; \left [ \begin{array}{c} (u_1 - v_1)y + (u_2 - v_2)x \\ u_1 - v_1 + u_2 - v_2 \\ \end{array} \right ] \; \right | \right | $$

which under the Euclidean norm turns into

$$ ||Dfu - Dfv|| \;\; =\;\; \sqrt{[(u_1 - v_1)y + (u_2 - v_2)x]^2 + (u_1 + u_2 - v_1 - v_2)^2 } \;\; < \;\; \epsilon. $$