To say that two convex sets $C$ and $D$ in $\mathbb{R}^n$ can be separated by a hyperplane $\mathcal{H}$ is to say that there is a linear functional $f$ on $\mathbb{R}^n$ such that $\mbox{sup} \{f\left(x\right)\: : \: x\in C\} \leq \mbox{inf} \{f\left(x\right)\: : \: x \in D\}$. $\mathcal{H}$ will then be the
hyperplane $\{x \in \mathbb{R}^n \: : \: f\left(x\right) = c\}$, which is a coset
of the kernel of $f$.
We say that $C$ and $D$ can be strictly separated if a linear functional $f$ can be chosen so that the above inequality is strict. Your intuition about what sorts of closed convex sets can be separated but not strictly separated is correct. This can happen when $C$ and $D$ are not compact and approach arbitrarily close to each other without meeting. So if you let $C$ be the points in $\mathbb{R}^2$ such that $y \geq \frac{1}{x}$ with $x > 0$ and $D$ be the points such that $y \leq -\frac{1}{x}$ with $x > 0$, then $C$ and $D$ are separated by the functional that projects onto the $y$ axis, but they are not strictly separated by this functional.
Noah's answer gives another good example. joriki's answer clears up a lot of things, and the paper he links to is a good introduction to this topic.