Which is the correct definition of stationary point for real-valued functions in Euclidean space? Given a multivariable real-valued function $f$ whose first partials all exist (but which aren't all continuous) at $p$, it is possible that $f$ is not (totally) differentiable at $p$. But since the first partials all exist, grad $f$ is defined at $p$. So we have a non-differentiable function at $p$ that nonetheless has a gradient at $p$. Does this make sense?
The usual definition of stationary point at $p$ is that grad $f(p)=0$, but some texts define stationary point to be $Df(p)=0$. Which is the more fundamental definition? The two definitions are equivalent when $f$ is differentiable, but in the pathological case I painted above where $Df(p)$ doesn't exist but grad $f(p)=0$, does the stationary point for $f$ still exist at $p$? In other words, does stationary point require total differentiability or merely for the gradient to be zero?
 A: While one can define $\nabla f(x) = ( \frac{\partial f(x)}{\partial x_1}, \cdots , \frac{\partial f(x)}{\partial x_n} )^T$, which depends only on the existence of the partials, it is not generally useful in the context of stationary points unless $f$ is actually differentiable at the relevant point.
For example, the function $f(x) = \begin{cases} -1, & \text{all} \ x_i \text{ are irrational} \\ 1, & \text{otherwise}\end{cases}$ has a 'gradient' at $x=0$, but it says very little about $f$ in a neighbourhood of $x=0$.
Instead of thinking of $\nabla f(x)$ and $Df(x)$ as essentially different objects, it is better to think of $\nabla f(x)$ as a different representation of $Df(x)$. In a Hilbert space $\mathbb{H}$, the Reisz representation theorem
establishes that for any element $\gamma \in \mathbb{H}^*$, there is some $g \in \mathbb{H}$ such that $\gamma(x) = \langle g, x \rangle$. Then for a scalar valued $f$, the derivative $Df(x)$ is a continuous linear operator, and hence some $g$ exists such that $Df(x)(h) = \langle g, h \rangle$. We call this element $g$ the gradient, and use the usual notation $\nabla f(x) = g$.
So, when I think of a stationary point, the expressions $Df(p) = 0$ and $\nabla f(p) = 0$ are essentially synonymous.
