basis functions do not lie in the space they form For example, any continuous function in $\mathbb{L}^2(-\infty,\infty)$ space can be expanded by delta functions $\delta(x-a)$ or Fourier basis $e^{ikx}$. However, the basis functions, both $\delta(x-a)$ and $e^{ikx}$, are not square-integrable, thus not in $\mathbb{L}^2(-\infty,\infty)$ space. It's hard to understand there is a space whose has basis that are not in the space. Is there an intuitive explanation?
 A: I think there is a confusion between two similar-sounding statements:


*

*every element of space $X$ can be written in terms of functions in the set $S$

*the set $S$ spans $X$ 


The second statement has a precise meaning, and it does require $S$ to be a subset of $X$. The meaning of the first is determined from context: "written in terms of" means whatever the writer wants it to mean. Here is a simple example without exotic things like delta functions:
Fact. Every function $f\in C[0,1]$ is the sum of a uniformly convergent series $f=\sum_{k=1}^\infty c_k g_k$, where $g_1,g_2,\dots$ are characteristic functions of the dyadic intervals $[(j-1)/2^m,j/2^m]$ enumerated in some way. 
Note that the topology of $C[0,1]$ is indeed the topology of uniform convergence, so the infinite sum is almost like an expansion of $f$ with respect to a spanning set of $C[0,1]$. But it's not, precisely because $g_k$ do not belong to $C[0,1]$. Not being contained in $C[0,1]$, the set $\{g_k\}$ does not satisfy the definition of a spanning set or of a basis.
A: When talking about $L^2$-spaces, the domain of definition of the member functions matters. $L^2(\mathbb{R})$ is different from $L^2(a,b)$ for example. What the theory of Fourier sequences asserts is that the functions $e^{ikx}$ are a basis for $L^2(-\pi, \pi)$ and they definitely do lie in this space since
$$\int_{-\pi}^\pi \! |e^{ikx}|^2\, dx = 2\pi$$
and thus $\|e^{ikx}\|_2 = \sqrt{2\pi}$. It is not asserted that they are a basis for $L^2(\mathbb{R})$, and this is as you have pointed out obviously not true.
As for the $\delta$s, these are not actually functions. (It should be an easy exercise to show that every measurable $f: \mathbb{R} \to \mathbb{C}$ whose support is a single point has integral equal to $0$.) A rigorous treatment of these "functions" comes from the theory of distributions, a thorough discussion of which is too long for this post. As an accessible reference, I recommend Appendix A and Chapter 2 of the book Partial Differential Equations by J. Rauch.
A: I am not sure if this analogy is right: 
Every real number $x$ can be written as $x=\frac{(z+z^*)}{2}$, where $z=x+yi$ is a complex number, $y$ is a non-zero real number, $*$ denotes complex conjugate. So every real number can be written in terms of a pair of conjugate complex numbers. Could this explain that every element of a space $X$ could be written in terms of elements outside $X$?
EDIT: Elements in a vector space $V_1$ could be expanded by basis of $V_2$ if $V_1\subset V_2$; on the other hand, Elements that are not in a vector space $V_1$ cannot be expanded by basis of $V_1$ since $V_1$ is spanned by its basis.
