Take the 2-minute tour ×
Mathematics Stack Exchange is a question and answer site for people studying math at any level and professionals in related fields. It's 100% free, no registration required.

Consider $f:\mathbb{R}^n\rightarrow\mathbb{R}^m$ and $g:\mathbb{R}^m\rightarrow\mathbb{R}^k$. Then $(g\circ f):\mathbb{R}^n\rightarrow\mathbb{R}^k$ and, if both of them are differentiable, $[D(g\circ f)_p]=(Dg)_{f(p)}\cdot (Df)_p$. If these functions are two times differentiable, then

$D(D(g\circ f))_p=(D^2g)_{f(p)}\cdot (Df)^2_p + (Dg)_{f(p)}\cdot (D^2 f)_p$.

I'm trying to figure out what $(Df)^2_p$ means. Since it is a $m\times n$ matrix I cannot multiply them. Can someone help me?

share|improve this question
1  
It's a tensor. To get an idea of what's going on: say we have $f(x,y,z)$ (scalar valued). Then $Df=\nabla f = \langle f_x,f_y,f_z \rangle$ (a vector of functions). $D^2f=H$ (the Hessian matrix) which is a $3 \times 3$ matrix filled with second partials. $D^3f$ is a $3 \times 3 \times 3$ cube filled with third partials. etc. So if your first derivative $Df$ is an $m \times n$ matrix, $D^2f$ is a vector of matrices (or better to start using tensors). –  Bill Cook Apr 8 '12 at 22:09
1  
Are you sure? Because what I have in my equation is $(Df)_p\cdot (Df)_p$, not $(D^2 f)_p$. I know what $(D^2 f)_p$ is, but this product is what is taking my sleep away at night :) –  Marra Apr 8 '12 at 22:31
    
Yes. I'm sure. To make sense out of "multiplying" these things using matrices and such, you'll need to pull apart $D(f)$ into a big column vector so that $D^2(f)$ is again a matrix. But in the end this is not a great way to handle such matters. –  Bill Cook Apr 8 '12 at 22:47
    
@Bill Gustavo is asking about $(Df)_p^2$, not $(D^2f)_p$. –  Jim Belk Apr 8 '12 at 23:18
add comment

1 Answer

Let $p = (x_1,\ldots,x_m)$, $f(p) = (y_1,\ldots,y_n)$, and $(g\circ f)(p)= (z_1,\ldots,z_k)$. Then the chain rule can be written $$ \frac{\partial z_j}{\partial x_i} \;=\; \sum_{\alpha=1}^m \frac{\partial z_j}{\partial y_\alpha} \frac{\partial y_\alpha}{\partial x_i} $$ What you have written as $(Dg)_{f(p)}$ is the matrix of partial derivatives $\dfrac{\partial z_j}{\partial y_\alpha}$, and what you have written as $(Df)_p$ is the matrix of partial derivatives $\dfrac{\partial y_\alpha}{\partial x_i}$.

Taking the derivative again yields $$ \frac{\partial^2 z_j}{\partial x_h \partial x_i} \;=\; \sum_{\alpha=1}^m\sum_{\beta=1}^m \frac{\partial^2 z_j}{\partial y_\beta\partial y_\alpha}\frac{\partial y_\beta}{\partial x_h}\frac{\partial y_\alpha}{\partial x_i}\;+\;\sum_{\alpha=1}^m \frac{\partial z_j}{\partial y_\alpha}\frac{\partial^2 y_\alpha}{\partial x_h \partial x_i} $$ What you have written as $(D^2g)_{f(p)}$ is the rank-three $k\times m\times m$ tensor of second partial derivatives $\dfrac{\partial^2 z_j}{\partial y_\beta \partial y_\alpha}$. As you can see, what you have written as $(Df)_p^2$ is the rank four $m\times n\times m\times n$ tensor whose entries are $\dfrac{\partial y_\beta}{\partial x_h}\dfrac{\partial y_\alpha}{\partial x_i}$. That is, $(Df)_p^2$ is the tensor product (or Kronecker product) of the matrix $(Df)_p$ with itself.

In general, given an $m\times n$ matrix $A$ and a $q\times r$ matrix $B$, their tensor product is the $m\times n\times q \times r$ tensor whose $(i,j,k,\ell)$-th entry is $a_{i,j}b_{k,\ell}$. This operation is analogous to the outer product of two vectors. More generally, it is possible to take the tensor product of any rank $R$ tensor with any rank $S$ tensor to get a tensor of rank $R+S$.

From a more algebraic point of view, $(Df)_p$ is a linear transformation from $\mathbb{R}^n$ to $\mathbb{R}^m$, and the second derivative $(D^2f)_p$ is a linear tranformation $$ (D^2f)_p : \mathbb{R}^n\otimes\mathbb{R}^n \to \mathbb{R}^m $$ where $\otimes$ denotes the tensor product of vector spaces. The object $(Df)_p^2$ is the linear transformation $$ (Df)_p^2 : \mathbb{R}^n\otimes\mathbb{R}^n \to \mathbb{R}^m\otimes\mathbb{R}^m $$ defined by $$ (Df)_p^2(v\otimes w) \;=\; (Df)_p(v) \,\otimes\, (Df)_p(w) $$ Since $(D^2g)_{f(p)}$ goes from $\mathbb{R}^m \otimes \mathbb{R}^m$ to $\mathbb{R}^k$, the composition $(D^2g)_{f(p)}\cdot (Df)_p^2$ is defined, and is a linear transformation from $\mathbb{R}^n\otimes\mathbb{R}^n$ to $\mathbb{R}^k$.

share|improve this answer
    
The first line seem to be incorrect , i guess you meant $\frac {\partial (f o g)}{\partial x_i}$ –  Theorem Nov 13 '13 at 12:59
    
@Theorem I don't see a problem. Can you be more specific? –  Jim Belk Nov 13 '13 at 19:27
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.