# Which is the correct definition of stationary point for real-valued functions in Euclidean space?

Given a multivariable real-valued function $f$ whose first partials all exist (but which aren't all continuous) at $p$, it is possible that $f$ is not (totally) differentiable at $p$. But since the first partials all exist, grad $f$ is defined at $p$. So we have a non-differentiable function at $p$ that nonetheless has a gradient at $p$. Does this make sense?

The usual definition of stationary point at $p$ is that grad $f(p)=0$, but some texts define stationary point to be $Df(p)=0$. Which is the more fundamental definition? The two definitions are equivalent when $f$ is differentiable, but in the pathological case I painted above where $Df(p)$ doesn't exist but grad $f(p)=0$, does the stationary point for $f$ still exist at $p$? In other words, does stationary point require total differentiability or merely for the gradient to be zero?

• You need $DF(p)$ to exist. Otherwise you could create pathological functions which have derivatives along the axes and go haywire elsewhere. – copper.hat Nov 1 '13 at 20:49
• @copper.hat Thanks. Most places seem to define the existence of stationary point at $p$ as grad $f(p) = 0$ though. – Ryan G Nov 1 '13 at 20:57
• I think of the gradient as the representation of $DF(p)$ from the Reisz representation theorem, so I don't really distinguish the two. I think to be 'useful' you need differentiability, otherwise axis-wise differentiability has little value. – copper.hat Nov 2 '13 at 0:31
• I do not, at least in terms of differentiability. I think of the gradient as the representative in $X$ corresponding to an element of $X^*$. I don't think it make sense to think of the gradient as a collection of stacked Gâteaux derivatives. – copper.hat Nov 2 '13 at 7:14
• I do not distinguish, but in the context here, my understanding is that by gradient you mean the stacked Gâteaux derivatives, which may exist even if $DF(p)$ does not. I am saying that I don't really see the value of such a relaxed gradient, and that one should stipulate differentiability at $p$, not just the partials. In the context of 'stationary', I think only $Df(p) = 0$ make sense. – copper.hat Nov 2 '13 at 7:24

While one can define $\nabla f(x) = ( \frac{\partial f(x)}{\partial x_1}, \cdots , \frac{\partial f(x)}{\partial x_n} )^T$, which depends only on the existence of the partials, it is not generally useful in the context of stationary points unless $f$ is actually differentiable at the relevant point. For example, the function $f(x) = \begin{cases} -1, & \text{all} \ x_i \text{ are irrational} \\ 1, & \text{otherwise}\end{cases}$ has a 'gradient' at $x=0$, but it says very little about $f$ in a neighbourhood of $x=0$.
Instead of thinking of $\nabla f(x)$ and $Df(x)$ as essentially different objects, it is better to think of $\nabla f(x)$ as a different representation of $Df(x)$. In a Hilbert space $\mathbb{H}$, the Reisz representation theorem establishes that for any element $\gamma \in \mathbb{H}^*$, there is some $g \in \mathbb{H}$ such that $\gamma(x) = \langle g, x \rangle$. Then for a scalar valued $f$, the derivative $Df(x)$ is a continuous linear operator, and hence some $g$ exists such that $Df(x)(h) = \langle g, h \rangle$. We call this element $g$ the gradient, and use the usual notation $\nabla f(x) = g$.
So, when I think of a stationary point, the expressions $Df(p) = 0$ and $\nabla f(p) = 0$ are essentially synonymous.