As the question suggests, I need help understanding the concepts around the following differentiation of vector-valued functions:

1) The Norm. I understand that Norm to be defined as follows:

In the metric space defined on $\Bbb R^n$ to each vector $x = (x_1, x_2, ... , x_n)$ one can associate the nonnegative number $$\|x\| = d(x,0)$$ what on earth does this mean?

2) The Jacobian Matrix and its use in the definitions of differentiability and continuity. I understand the Jacobian matrix to be the gradient matrix i.e. a matrix whose elements are the partial derivatives of a function $f: \Bbb R^n \to \Bbb R^m$ at a point $(a,b) \in \Bbb R^n$.

Now then a function is said to be differentiable if $$ \lim_{h \to 0} \frac{\|f(u+h) - f(u) - Ah\|}{\|h\|} = 0$$ where $u$ and $h$ are vectors in $\Bbb R^n$ and --this is where I'm perplexed -- $Ah = J_f(u)h \in \Bbb R^m$

What are we saying here? That $Ah$ is the derivative of said function? And are we saying that essentially a vector valued function is differentiated into a vector of its partial derivatives?


Don't worry, multidimensional analysis is a shock to the system the first time that you see it.

1) The norm is the distance to the origin $d(x,0)$, assumed to be Euclidean, so $$\lVert x \rVert = \sqrt{\sum x_i^2}$$

2) Okay, let's get a grip on this. $$f:\mathbb{R}^n \to \mathbb{R}^m$$ so we have functions $f_1(x_1,\cdots,x_n), \ldots, f_m(\cdots)$. There are really $m$ unrelated functions $f_i:\mathbb{R}^n \to \mathbb{R}$

Great; now, what is a derivative for a function like $f_i$? It's the gradient $\nabla f_i$. Hence the whole lot of derivative information is encoded in a $m\times n$ matrix $A=J_f(u)$ with components $$A_{ij} = \partial_j f_i$$

Right! But now we want a formal definition of differentiability. How do we do it for a single $f_i=g$? We say $\nabla g$ is the derivative if it tells us what the *directional * derivatives are. How does it do this?

We want $g(x+dx) = g(x) + dx \cdot \nabla g$ - small changes get dotted with the derivative. So formally, we want a vector $v$ such that $$\lim_{h\to 0} |g(x+h) - g(x) - h \cdot v | / \lVert h \rVert = 0$$ so that the error is smaller than $h$.

But now for all the $f_i$ we get a different vector $v_i$. Putting these all together into $A$ and choosing the denominator to bound the error for any one $f_i$ gives the result you have!

Edit: To summarize, $h$ is a small change in the coordinates, $Ah$ is the directional derivative of all the separate $f_i$s in this direction, and $A$ is the 'gradient' containing all derivative information.

  • $\begingroup$ OOOOoooohhhh! Thank you so, so much. That's how I needed to here it. :) $\endgroup$ – Siyanda Apr 20 '13 at 23:39
  • $\begingroup$ That's okay! (: $\endgroup$ – Sharkos Apr 20 '13 at 23:45

Regarding the norm:

A norm is a different concept to a metric, although the two are related.

A norm on a (real) vector space $V$ is a function $\|\cdot\|: V \rightarrow [0,\infty)$ satisfying:

  1. $\|\lambda u\| = |\lambda|\|u\|$, $\forall \lambda \in \mathbb{R}$

  2. $\|u\| = 0 \Leftrightarrow u = 0$

  3. $\|u + v\| \leq \|u\| + \|v\|$ (the triangle inequality)

Conceptually, a norm is a "length" - the norm of $u$ is like the "length" of $u$.

The definition in the question ($\|u\| := d(u,0)$) is not true for all metrics. For instance, the discrete metric (where the distance between two different points is always 1) will not satisfy the first property of a norm above.

The converse is always possible though: for any normed vector space, we can define a metric $d(x,y) := \|x-y\|$ (which is called the metric induced by the norm)

For your second question, you're using a norm of elements of the space $\mathbb{R}^n$. Although there isn't one specific norm for a given space, but there are lots of results that you can use without actually knowing what the norm is, especially on a finite-dimensional space (in fact, any two norms on $\mathbb{R}^n$ can be shown to be equivalent).

  • $\begingroup$ Thanks for helping me understand the norm. $\endgroup$ – Siyanda Apr 20 '13 at 23:30

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.