7
$\begingroup$

I'm thinking that the second directional derivative, if both dd's are evaluated in the same direction, will just give you the concavity (the second scalar derivative) in that direction. Is that right?

But what if the second directional derivative is evaluated in a different direction? As in $D_{\vec v}D_{\vec u} f(\vec x)$ where $\vec v\ne \vec u$. Then what would this thing mean? Does it still have to do with concavity?

$\endgroup$
12
  • 1
    $\begingroup$ If you want to reduce to the scalar setting, I think you are basically stuck with some linear algebra. Specifically, for ease of visualization, consider a point where the gradient is zero. You have a matrix of the 4 second partial derivatives, called the Hessian. Along the lines given by the eigenvectors of this matrix, provided the corresponding eigenvalue $\lambda$ isn't zero, the function looks like the parabola $\lambda x^2$. $\endgroup$
    – Ian
    May 21, 2016 at 2:00
  • $\begingroup$ Geometrically, when the mixed partials are zero, these lines are exactly along the coordinate axes; when the unmixed partials are zero, these lines are exactly the lines $y=x$ and $y=-x$. In between the lines are in between these two cases (but one can prove with linear algebra that they are always perpendicular). $\endgroup$
    – Ian
    May 21, 2016 at 2:00
  • $\begingroup$ note that in $\mathbb{R}^2$ : the $\frac{\partial^2 }{\partial x \partial y}$ operator is of that type. and it is useful for finding the Taylor expansion of $f$. without, you cannot say what will look like $f(\vec{x}+h \vec{v}) - f(\vec{x}) - h \vec{\Delta} f . \vec{v}$ which is $= h^2 \vec{v}^T H_{f}(\vec{x}) \vec{v} + o(h^2)$ where $H_{f}(\vec{x})$ is the Hessian matrix $H_{f}(\vec{x})_{ij} = \frac{\partial^2 f}{\partial x_i \partial x_j}(\vec{x})$ i.e. the matrix of the second order directionnal derivatives $\endgroup$
    – reuns
    May 21, 2016 at 2:00
  • 1
    $\begingroup$ @user341126 The concavity in a given direction $d$ ($d$ is some unit vector) is $d^T H d$ where $H$ is the Hessian. $\endgroup$
    – Ian
    May 21, 2016 at 2:06
  • 1
    $\begingroup$ Well I know that $v^T(Hf)v = D_vD_vf$ (when $Hf$ exists) so then I guess the second directional derivative only represents concavity when both are taken in the same direction? Then I still wouldn't know how to interpret (geometrically or even analytically) what $D_vD_u f(x)$ is. $\endgroup$
    – user341126
    May 21, 2016 at 2:09

1 Answer 1

2
+50
$\begingroup$

The answer to this question rests in linear algebra. If $f$ is twice continuously differentiable (just to avoid pathological things) then we have that partial derivatives of $f$ commute. I'll skip the proof, but, the same goes for the directional derivatives of $f$. We have $D_u(D_v f) = D_v(D_u(f))$. Furthermore, if $u,v$ are linearly independent then they form a basis for $\mathbb{R}^2$ and we can study the behavior of the quadratic term in the multivariate Taylor expansion in terms of the theory of quadratic forms. In particular, $$ Q(h,k) = [h,k]\left[ \begin{array}{cc} D_uD_uf & D_uD_v f \\ D_uD_v f & D_vD_v f \end{array} \right]\left[ \begin{array}{c}h \\ k \end{array} \right] = \lambda_1\bar{x}^2+\lambda_2\bar{y}^2$$ where $\bar{x}, \bar{y}$ are eigencoordinates and $\lambda_1, \lambda_2$ are the eigenvalues of $A = \left[ \begin{array}{cc} D_uD_uf & D_uD_v f \\ D_uD_v f & D_vD_v f \end{array} \right]$. Using linear algebra and the specific form of $A$, $$ \text{trace}(A) = \lambda_1+\lambda_2 = D_uD_uf+D_vD_vf $$ and $$ \text{det}(A) = \lambda_1\lambda_2 = (D_uD_uf)(D_uD_uf)-(D_uD_vf)^2$$ If $\nabla f =0$ at a point then $f(p+(h,k)) = f(p) + Q(h,k)+ \cdots $ which means that the nature of $z = f(x,y)$ near $p$ is totally governed by $Q$. It's easy to see

  1. If $\lambda_1\lambda_2 <0$ then $f$ is both increasing and decreasing near $p$ hence we face a saddle point. Specifically, in terms of directional derivatives, $(D_uD_uf)(D_uD_uf)-(D_uD_vf)^2 < 0$ provides $\text{det}(A)<0$ and hence the eigenvalues differ in sign.
  2. If $\lambda_1, \lambda_2>0$ then $f$ increases as we travel away from $p$ this is manifest from the formula $\lambda_1\bar{x}^2+\lambda_2\bar{y}^2$ for $Q$. This requires $(D_uD_uf)(D_uD_uf)-(D_uD_vf)^2 > 0$ and $\text{trace}(A) >0$, but, if you think about it, given $\text{det}(A)>0$ we have $D_uD_u f>0$ implies $D_vD_v f>0$. In short, if $(D_uD_uf)(D_uD_uf)-(D_uD_vf)^2 > 0$ and $D_uD_u f>0$ then $f$ has a local minimum at $p$.
  3. If $\lambda_1, \lambda_2<0$ then $f$ decreases as we travel away from $p$ this is manifest from the formula $\lambda_1\bar{x}^2+\lambda_2\bar{y}^2$ for $Q$. This requires $(D_uD_uf)(D_uD_uf)-(D_uD_vf)^2 > 0$ and $\text{trace}(A) < 0$, but, if you think about it, given $\text{det}(A)>0$ we have $D_uD_u f<0$ implies $D_vD_v f<0$. In short, if $(D_uD_uf)(D_uD_uf)-(D_uD_vf)^2 > 0$ and $D_uD_u f<0$ then $f$ has a local maximum at $p$.

At this point, you may complain, I meant to study $f: \mathbb{R}^n \rightarrow \mathbb{R}$. What is the significance of mixed directional derivatives in that context for $n>3$. It's more complicated, to give the complete picture, I need $n$-LI directions and all $n(n+1)/2$-independent mixed directional derivatives. If I have all that data then I can construct the $n \times n$ analog of $A$ and again characterize the nature of the graph $x_{n+1} = f(x_1, \dots , x_n)$ at a critical point in terms of the eigenvalues of $A$ (assuming $A \neq 0$, in the case $A=0$, we'd need higher derivative data...)

Another way to look at my answer is this: the mixed directional derivatives also have to do with concavity. The eigenvalues tell us the concavity of the sections of the graph in the direction of eigenvectors. If the point is critical, then that concavity reveals the extremal nature of the point.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.