Is $\int |x\rangle \langle x|dx$ Mathematical? I am enrolling in a Quantum Mechanics class. As we all know, the formulation of the basic ideas from QM relies heavily on the notion of Hilbert Space. I decide to take the course since it might help me understand the motivation underlying the theory of unbounded operators.
However, things started to get confusing when the teacher introduced the inner product of the quantum states. Suppose there are two vectors $|\phi \rangle$,$|\psi \rangle$ represented by two column vectors $v$,$u$ respectively, he then defined their inner product to be
$$
\langle \psi  |\phi \rangle = \bar u^t v
$$
which makes perfect sense to me. He then went on to explain the continuous case by first introducing the thing called "completeness relation"
$$
1=\sum |i\rangle\langle i|
$$ 
where $|i\rangle$'s are the normalized eigenvectors of an Hermitian Operator. I, as the only math major in the class, recognize the term on the RHS as the projector into the subspace spanned by the orthonormal sequence $(|i\rangle)_{i=1}^\infty$, which is equal to identity since the sequence is complete.
He then went on to find a way to calculate $\langle \psi  |\phi \rangle$ in the case where $|\phi \rangle$,$|\psi \rangle\in L^2[-a,a]$, the so-called infinite square well. He said that since the state is continuous (whatever that means) the above summation approach an integration, so we have
$$
1=\int |x\rangle \langle x|dx
$$
instead. Then he demonstrated
$$\begin{align}
\langle \psi|\phi \rangle &= \langle \psi|1|\phi \rangle \\
&= \langle \psi|(\int |x\rangle \langle x|dx)|\phi \rangle \\
&= \int \langle \psi|x\rangle \langle x|\phi \rangle dx \\
&= \int \overline{\langle x|\psi\rangle} \langle x|\phi \rangle dx \\
&= \int \bar\psi(x)\phi(x) dx
\end{align}$$
, which makes very little sense to me. I had always seen the relation as THE definition of inner product in $L^2$ space, not something to be derived. When I asked him questions about the derivation he tried to justify it by saying something about Dirac's delta being an element of the Hilbert space (the irony) and the family of shifted Dirac's delta constitutes a basis (in some vague sense) of $L^2[-a,a]$.
While knowing that my teacher's statement makes little sense in the theory of Hilbert space, which is not untypical of a physicist by the way (This is not meant to be an accusation by any mean, I really respect him and he's a good physicist. However the word "physicist" and "rigor" are usually mutually exclusive), I've learned about existence of Rigged Hilbert space and heard that it partially resolve some foundational issues with using Dirac's delta in QM.  
Here's my questions:
1.) I wonder if the notation $\int |x\rangle\langle x|dx$
has definite meaning in the Rigged Hilbert space?
2.) Could anyone please explain to me if the derivation is sound in ANY mathematical theory?  
Note that I'm an undergraduate so I'd really appreciate some not-to-advanced answers :) Thank you in advance.
Edit: What does $\langle x|\phi \rangle = \phi(x)$ means anyway? At first I think it looks like the evaluation map but now I'm not quite sure.
 A: Yes, this does make sense in the context of "rigged Hilbert spaces", e.g., something like what is occasionally called a Gelfand triple $H^{+1}\subset L^2\subset H^{-1}$ of Sobolev spaces on an interval in $\mathbb R$. Somehow Dirac had a wonderful intuition in this direction already prior to 1930. Also, the possibility of writing "integral kernels" for all mappings was eventually systematized into L. Schwartz' Kernel Theorem, and A. Grothendieck's nuclear spaces. Perhaps the most direct way to make things completely precise is as follows.
Let $L^2=L^2[a,b]$ be the usual space of square-integrable functions, which we know is also the completion of the space of test functions on $[a,b]$ with respect to the $L^2$ norm. Let $H^1=H^1[a,b]$ be the completion of test functions with respect to the (Sobolev) norm $|f|^2_{H^1}=\langle f-f'',f\rangle$. The injection $j:H^{-1}\to L^2$ has an adjoint $j^*$, and we identify $L^2$ with its own dual (but not the others!), obtaining $j^*:L^2\to H^{-1}$ where $H^{-1}=(H^1)^*$.
Dirac delta at a point $x_o$ in $[a,b]$ is provably in $H^{-1}$.
As a small part of some version of Schwartz' Kernel Theorem in this setting, with some vector-valued integral justification, the computation you quote is exactly the verification that the kernel for the identity map is "Dirac delta on the diagonal" in $[a,b]\times[a,b]$.
(The bra-ket notation can be rewritten in terms of tensors and tensor products if one desires, making it look less physics-y.)
A: Here is my understanding. But I know very little about QM, so this explanation may be incorrect.

Consider the multiplication operator $A$ defined on $L^2[-a, a]$ by
$$A\varphi(x) = x\varphi(x). $$
(In quantum mechanics, $A$ corresponds to the position operator.) Since this operator is bounded with the spectrum $\sigma(A) = [-a, a]$, by the spectral theorem there corresponds a projection-valued measure $E$ on $\sigma(A)$ such that
$$ A = \int_{[-a, a]} \lambda \, dE_{\lambda}. $$
This measure admits the resolution of the identity:
$$ 1 = \int_{[-a, a]} dE_{\lambda}. $$
Now heuristically, Dirac delta $\delta_{\lambda}$ is an eigenfunction of $A$ corresponding to the eigenvalue $\lambda$. Since we know that $E_{\lambda} = \mathbf{1}_{(-\infty, \lambda]}(A)$, we can think that $dE_{\lambda} \approx \mathbf{1}_{\{\lambda\}}(A)$ is a projection to the eigenspace corresponding to $\lambda$, which should be spanned by $\delta_{\lambda}$. Thus it is not unreasonable to carelessly write
$$ dE_{\lambda} \approx | \delta_{\lambda} \rangle\langle \delta_{\lambda} | \, d\lambda, $$
where $d\lambda$ comes from the normalization.
