# Is the normal cone well defined?

I am reading Mordukhovich's Variational Analysis and Generalized Differentiation, and the first definition there is that of a $\epsilon-$normals Given a Banach space $X,$ and $\Omega\subseteq X,$ consider a point $x \in \Omega.$ The $\epsilon-$ normal cone of $\Omega$ at $x$ is the set

$$\hat{N}_{\epsilon}(x,\Omega)=\{x^{*} \in X^*: \limsup_{u\to x,\; u\in \Omega}\frac{<x^*,u-x>}{\|u-x\|}\leq \epsilon\}.$$ Is this definition correct? There should be no problem if there exists a sequence $\{u_k\}\subseteq \Omega$ such that $u_k\to x$ and $u_k\neq x.$ However, my concern would be when $x$ is an isolated point of $\Omega.$ In that case, there exists a neighborhood $U$ of $x$ such that $\Omega\cap U=\{x\},$ and hence the $\limsup$ it is not well defined. Am I mising something? I guess we should take $\hat{N}_{\epsilon}(x,\Omega)=\emptyset$ in this case.

This depends on a proper interpretation of the $\limsup$. Unfortunately, Mordukhovich's book does not contain a definition of this expression. I would define it as $$\limsup_{u \to x, u \in \Omega} f(u) := \limsup_{\varepsilon \searrow 0} \; \sup\{f(u) \mid u \in \Omega \cap B_\varepsilon(x)\} ,$$ where $f : X \to \mathbb R$ is some function and $B_\varepsilon(x)$ is a ball around $x$ with radius $\varepsilon$.

For $f(u) = \frac{\langle x^*, u - x\rangle}{\|u-x\|}$ one has to exclude $u = x$, therefore, I would take $$\limsup_{u \to x, u \in \Omega} \frac{\langle x^*, u - x\rangle}{\|u-x\|} := \limsup_{\varepsilon \searrow 0} \; \sup\biggl\{\frac{\langle x^*, u - x\rangle}{\|u-x\|} \biggm| u \in \Omega \cap B_\varepsilon(x) \setminus \{x\}\biggr\}$$

In case that $x$ is an isolated point of $\Omega$, $\Omega \cap B_\varepsilon(x) \setminus \{x\}$ is empty for $\varepsilon$ small enough.

By convention, the supremum over the empty set is $-\infty$, hence, $$\limsup_{u \to x, u \in \Omega} \frac{\langle x^*, u - x\rangle}{\|u-x\|} = -\infty$$ in this case. And $-\infty \le \epsilon$ for all $\epsilon > 0$ independently of $x^*$.

Hence, $\hat N_\epsilon(x,\Omega) = X^*$ in this case. This also makes sense from another point of view: Around $x$, $\Omega$ is locally convex and the normal cone (in the sense of convex analysis) of $\{x\}$ in $x$ is $X^*$.

• thanks @gerw. That clears the question – John D Jan 2 '18 at 15:30