Given a multivariable real-valued function $f$ whose first partials all exist (but which aren't all continuous) at $p$, it is possible that $f$ is not (totally) differentiable at $p$. But since the first partials all exist, grad $f$ is defined at $p$. So we have a non-differentiable function at $p$ that nonetheless has a gradient at $p$. Does this make sense?

The usual definition of stationary point at $p$ is that grad $f(p)=0$, but some texts define stationary point to be $Df(p)=0$. Which is the more fundamental definition? The two definitions are equivalent when $f$ is differentiable, but in the pathological case I painted above where $Df(p)$ doesn't exist but grad $f(p)=0$, does the stationary point for $f$ still exist at $p$? In other words, does stationary point require total differentiability or merely for the gradient to be zero?

  • 1
    $\begingroup$ You need $DF(p)$ to exist. Otherwise you could create pathological functions which have derivatives along the axes and go haywire elsewhere. $\endgroup$ – copper.hat Nov 1 '13 at 20:49
  • $\begingroup$ @copper.hat Thanks. Most places seem to define the existence of stationary point at $p$ as grad $f(p) = 0$ though. $\endgroup$ – Ryan G Nov 1 '13 at 20:57
  • $\begingroup$ I think of the gradient as the representation of $DF(p)$ from the Reisz representation theorem, so I don't really distinguish the two. I think to be 'useful' you need differentiability, otherwise axis-wise differentiability has little value. $\endgroup$ – copper.hat Nov 2 '13 at 0:31
  • $\begingroup$ I do not, at least in terms of differentiability. I think of the gradient as the representative in $X$ corresponding to an element of $X^*$. I don't think it make sense to think of the gradient as a collection of stacked Gâteaux derivatives. $\endgroup$ – copper.hat Nov 2 '13 at 7:14
  • $\begingroup$ I do not distinguish, but in the context here, my understanding is that by gradient you mean the stacked Gâteaux derivatives, which may exist even if $DF(p)$ does not. I am saying that I don't really see the value of such a relaxed gradient, and that one should stipulate differentiability at $p$, not just the partials. In the context of 'stationary', I think only $Df(p) = 0$ make sense. $\endgroup$ – copper.hat Nov 2 '13 at 7:24

While one can define $\nabla f(x) = ( \frac{\partial f(x)}{\partial x_1}, \cdots , \frac{\partial f(x)}{\partial x_n} )^T$, which depends only on the existence of the partials, it is not generally useful in the context of stationary points unless $f$ is actually differentiable at the relevant point. For example, the function $f(x) = \begin{cases} -1, & \text{all} \ x_i \text{ are irrational} \\ 1, & \text{otherwise}\end{cases}$ has a 'gradient' at $x=0$, but it says very little about $f$ in a neighbourhood of $x=0$.

Instead of thinking of $\nabla f(x)$ and $Df(x)$ as essentially different objects, it is better to think of $\nabla f(x)$ as a different representation of $Df(x)$. In a Hilbert space $\mathbb{H}$, the Reisz representation theorem establishes that for any element $\gamma \in \mathbb{H}^*$, there is some $g \in \mathbb{H}$ such that $\gamma(x) = \langle g, x \rangle$. Then for a scalar valued $f$, the derivative $Df(x)$ is a continuous linear operator, and hence some $g$ exists such that $Df(x)(h) = \langle g, h \rangle$. We call this element $g$ the gradient, and use the usual notation $\nabla f(x) = g$.

So, when I think of a stationary point, the expressions $Df(p) = 0$ and $\nabla f(p) = 0$ are essentially synonymous.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.