# Counterexample of Separation Theorem in topological vector space

The Separation Theorem states that: If $A$ and $B$ be two disjoint convex subsets in a vector space $X$ and one of them has nonempty core (algebraic interior) then there exists a linear functional $f\neq 0$ such that $$\sup\{f(x):\; x\in A\} \leq \inf\{f(x):\; x\in B\},$$ where $x\in \mbox{Core}(A)$ if and only if for any $v\in X$, there exists $\epsilon>0$ such that $x+\lambda v\in X$ for any $\lambda \in (-\epsilon, \epsilon)$. The proof of this can be, for e.g., based on Hahn-Banach Themrem with sub-additive and homogeneous function there in is Minkowski function of an absorb and convex subset.

I also know, the assumption on non-emptiness of core is very important by some counterexamples.

Moreover, in a topological vector space $X$, if a linear functional $f\neq 0$ separates two sets and one of them has nonempty topological interior then $f$ is continuous. This fact is due to if $f$ is bounded in a neighborhood of the origin then it must be continuous.

But it is always true that $\mbox{int}(A) \subset \mbox{Core}(A)$, combining two facts above, we have: If $A$ and $B$ be two disjoint convex subsets in a topological vector space $X$ and one of them has nonempty (topological) interior then they can be separated by a continuous linear functional $f\in X^*\setminus\{0\}.$

I would like to construct an example on a topological vector space $X$, there exists two disjoint convex subsets one of them has nonempty core, both of them have empty topological interior and they can not be separate by any CONTINUOUS linear functional $f\in X^*\setminus\{0\}.$

There are a set in finite dimensional space which has it's core is not equal to it's interior, e.g., $$A=\{(x,y)\in \mathbb R^2:\;\; y\geq x^2\} \cup \{(x,y)\in \mathbb R^2:\;\; y\leq -x^2\} \cup \{(x,y)\in \mathbb R^2:\;\; y=0\},$$ then $0$ is in core $A$ but not an interior point of $A$.

In infinite dimensional Banach space $X$, let $f$ be a discontinuous linear functional and $$B=\{x\in X:\;\; |f(x)|\leq 1\},$$ then $\mbox{int}(B)=\emptyset$ but $0$ belongs to core of $B$.

Could anyone can help me to construct such an example? I will highly appreciate your helps.

The principle is easy. Take a topological vector space $(X,\tau)$ (over $\mathbb{R}$), and a discontinuous linear form $f \colon X \to \mathbb{R}$. Then the two sets $A = f^{-1}((0,+\infty))$ and $B = f^{-1}((-\infty,0))$ are disjoint and convex, they have nonempty core, since they are open in some vector space topology finer than $\tau$ (for example the finest TVS topology on $X$) in which $f$ is continuous, and they can be separated only by multiples of $f$, hence not by any linear form that is continuous with respect to $\tau$.

For a concrete example, let $Y = \mathbb{R}^\mathbb{N}$ the space of all real sequences, endowed with the product topology, and let

$$X = \left\{ x\in Y : \sum_{k=0}^\infty x_k^2 < \infty \right\}$$

the subspace of square summable sequences. Then take

$$f(x) = \sum_{k=0}^\infty \frac{x_k}{k+1}.$$

A continuous linear form on $X$ can only involve finitely many components, so $f$ is discontinuous. $f$ would however be continuous in the $\ell^2$-topology.

• That's great! Thank you so much for your construction! – Richkent Mar 29 '14 at 11:56