Take the 2-minute tour ×
Mathematics Stack Exchange is a question and answer site for people studying math at any level and professionals in related fields. It's 100% free, no registration required.

We have to use the Hessian to calculate the second order derivatives of a function. While that is okay if the function is mapped from $\mathbb{R}^n$ to $\mathbb{R}$, how does one proceed if it is mapped from $\mathbb{R}^n \longrightarrow \mathbb{R}^m$, where $m > 1$? How does one take the derivative of f with respect to each variable when f is itself in more than one dimension?

For example, if there is a function f : $\mathbb{R}^2 \longrightarrow \mathbb{R}^3$ with $(x,y)$ mapped to $(x^2 + y, y^3, \cos(y))$, how does one calculate the Hessian?

Thank you

share|improve this question
    
Do it component-wise. –  Willie Wong Jul 11 '12 at 12:17
    
Then how does the hessian look like? Normally it would look like this: [del^2(f)/delx^2 del^2(f)/delx dely;del^2(f)/dely^2 del^2(f)/delx dely ] . Now how will it look like? –  johnathan Jul 11 '12 at 12:30
add comment

2 Answers

I guess what you are looking for is the following:

Let $\vec{f}(x,y) = \begin{pmatrix} u(x,y) \\ v(x,y) \\ w(x,y)\end{pmatrix}$. Its "Hessian" is simply the "vector-valued matrix"

$$ H_{\vec{f}} = \begin{pmatrix} \begin{pmatrix} u_{xx} \\ v_{xx} \\ w_{xx}\end{pmatrix} & \begin{pmatrix} u_{xy} \\ v_{xy} \\ w_{xy}\end{pmatrix} \\ \begin{pmatrix} u_{yx} \\ v_{yx} \\ w_{yx}\end{pmatrix} & \begin{pmatrix} u_{yy} \\ v_{yy} \\ w_{yy}\end{pmatrix} \end{pmatrix} $$

where $ u_{xy} = \frac{\partial^2}{\partial x\partial y} u$ etc. Note that each entry of the "matrix" is now a vector in its own right.

share|improve this answer
    
So to apply it to any vector, we need two vectors p and q, right? –  johnathan Jul 11 '12 at 12:53
    
What do you mean "to apply it to any vector"? If you are trying to compute the second order derivative in the mixed directions $\vec{p},\vec{q}$, you plug it in as $\vec{p}^T H_{\vec{f}} \vec{q}$ which will return a vector. –  Willie Wong Jul 11 '12 at 13:15
add comment

The importance of the Hessian comes from the following fact:

Let $X$ and $Y$ be normed spaces, and let $L(X,Y)$ denote the space of continuous linear maps from $X$ to $Y$. There is a canonical isomorphism (and actually an isometry) $$L(X,L(X,Y)) \simeq \operatorname{Bil}(X\times Y\to Y),$$ where Bil is the set of bilinear continuous maps from $X \times Y$ to $Y$.

Since the second derivative belongs to $L(X,L(X,Y))$, we can identify it with a bilinear map. If $f \colon \mathbb{R}^n \to \mathbb{R}$, then the bilinear maps from $\mathbb{R}^n \times \mathbb{R}^n$ to $\mathbb{R}$ are just $(n\times n)$ matrices, and you get the Hessian matrix. If the codomain of $f$ has higher dimension, matrices no longer suffice. I guess you'd need tensors. See also here.

Additional material: assume that $\mathbf{f} \colon \mathbb{R}^n \to \mathbb{R}^m$ is of class $C^2$. Then the second differential at $\mathbf{x}_0 \in \mathbb{R}^n$ acts like $$\mathrm{d}^2 \mathbf{f}(\mathbf{x}_0)\colon (\mathbf{v}_1,\mathbf{v}_2) \mapsto \sum_{i,j=1}^n \frac{\partial}{\partial x_{i}} \frac{\partial \mathbf{f}}{\partial x_{j}}(\mathbf{x}_0) v_{1,i}v_{2,j}$$ on the generic vectors $\mathbf{v}_1$, $\mathbf{v}_2 \in \mathbb{R}^n$, and $v_{1,i}$, $v_{2,j}$ are the components of $\mathbf{v}_1$ and $\mathbf{v}_2$, respectively.

share|improve this answer
    
I can't understand how to start solving for the matrix. Could you please give me an idea as to how to do it. I need to calculate the second order derivative of the aforementioned function. –  johnathan Jul 11 '12 at 12:32
    
If you codomain has dimension higher than $1$, there is no hessian matrix at all. The second derivative is a tensor of order $3$; you can compute the components of this tensor by second order partial derivatives of $f$, but the action of the second derivative of $f$ is the action of a tensor. Remember that you are differentiating twice a vector field, and this is not that trivial. –  Siminore Jul 11 '12 at 12:39
    
So how does this tensor look in this case? I have not done tensor theory, actually. How does one calculate the second derivative of the function? –  johnathan Jul 11 '12 at 12:42
    
I've added an explicit formula for the second differential. –  Siminore Jul 11 '12 at 13:18
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.