Consider a real function $f$ of one variable. Suppose the second order derivative exists. To find the second order derivative of $f$, I usually derivate $f$ two times. I start with $f$, and derivate to get the function $\frac{d}{dx} f(x)$ of one variable, which I then derivate again. I would succinctly describe this by writing

$$\frac{d}{dx} \frac{df}{dx}.$$

But when I consider a real multivariable function which maps elements into $\mathbb{R}^k$, with $k \geq 1$, I do not always know how to find the second order derivative. In the case $k=1$ we can consider the derivative of some function $g$ as the gradient $\nabla g$ of $g$. Then the second order derivative of $g$ would be the hessian matrix $H(g)$ of $g$.

But when $k \geq 2$, for example $h(x, y)=(x^2, y^2)$, I do not find it easy to compute the second order derivative, and our coursebook does not discuss this topic.

I know that

$$[h'(x, y)]=\begin{pmatrix} 2x & 0 \\ 0 & 2y \end{pmatrix}.$$

The difference between finding the derivative of $f$ and $g$ or $h$ is that when derivating $f$ we get a function of the same structure, we still map elements from $\mathbb{R}$ into $\mathbb{R}$, but when derivating $g$ or $h$ the structure changes and instead of, for example, mapping elements from $\mathbb{R}^2$ into $\mathbb{R}^2$, we map elements from $\mathbb{R}^2$ into $\mathbb{R}^{2 \times 2}$ (the set of 2 by 2 matrices with real elements). To me, it seems that the problem of finding the second order derivative of $g$ is to derivate a matrix in which all elements are real functions.

I would be grateful if you could, for example, direct me to any material that would help me solve this general problem.



If we have some function $f \ \colon \mathbb R^n \to \mathbb R ^m$ then the derivative of $f(\underline{x})$ (if it exists) is an $m \times n$ matrix. Since this derivative varies as we change the input, we can describe it as a function $f' \ \colon \mathbb R ^n \to \mathbb R ^{m \times n}$ which assigns to each input its derivative matrix. Note that while the matrix will be a linear operator, its entries will not depend linearly on $\underline{x}$. For example the matrix $ \left( \begin{array}{ccc} 3x^2 & 0& 0\\ 0& 3y^2& 0\\ 0 & 0 & 3z^2 \end{array} \right) $ Is the derivative of the function $f(x,y,z) = (x^3, y^3, z^3)$. The entries are clearly not linear functions.

The derivative of $f'(\underline x)$ is the derivative of a function $\mathbb R ^n \to \mathbb R ^{m \times n}$, so it will be a $(m \times n) \times n$ matrix. This is easier to imagine as a "cube" of mixed partial derivatives. In our previous example the first derivative could be thought of as a function that assigns to each pair $(i,j)$ the entry $\displaystyle \frac{\partial f_j}{\partial x^i}$ of the corresponding matrix. The second derivative will assign to each pair $(i,j,k)$ the entry $\displaystyle \frac{\partial}{\partial x^i} \frac{\partial f_k}{\partial x^j}$ in the "cube". The "cube" is called a second rank tensor, and a matrix is called a first rank tensor. A $n ^{th}$ rank tensor is a linear operator which inputs $n$ vectors and outputs one.

Ultimately, computing $f^{(n)}(\underline{x})$ amounts to computing each $\displaystyle \frac{\partial}{\partial x^{i_1}} \frac{\partial}{\partial x^{i_2}} \ldots \frac{\partial f_{i_{n+1}}}{\partial x^n}$. Depending on your function several of these can be done at once . It is rarely helpful or useful to try and illustrate all of the mixed partial derivatives as some form of "hypercube" however.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.