Prob. 8, Sec. 3.5 in Erwin Kreyszig's Introductory Functoinal Anlaysis With Applications Erwin Kreyszig's Introductory Functoinal Anlaysis With Applications
Prob. 8, Sec. 3.5
$\DeclareMathOperator{\span}{span}$Let $(e_k)$ be an orthonormal sequence in a Hilbert space $H$, and let $M = \span (e_k)$. Let $x \in H$.
If
$$x = \sum_{k=1}^\infty \langle x, e_k \rangle e_k,$$
then $x \in \overline{\span(e_k)}$ because in this case the sequence
$(s_n)$ in $\span(e_k)$, where $s_n =  \sum_{k=1}^n \langle x, e_k \rangle e_k$, converges to $x$.
How to show the converse?
That is, how to show that if $x \in \overline{\span(e_k)}$, then the series
$\sum_{k=1}^\infty \langle x, e_k \rangle e_k$ converges (in the norm induced by the inner product on $H$) and has sum $x$?

My effort:
Suppose $x \in \overline{\span(e_k)}$. Then there is a sequence $(x_n)$ in $\span(e_k)$ that converges to $x$. Let $x_n = \sum_{k=1}^{m_n} \alpha_{nk} e_k$ for each $n= 1, 2, 3, \ldots$, where $\alpha_{nk}$ are scalars and the $m_n$ are natural numbers.
Then, using the orthonormality of the $e_k$, we can conclude that $\alpha_{nk} = \langle x_n, e_k \rangle$ for each $n=1, 2, 3, \ldots$ and for each $k= 1, \ldots, m_n$. So
$$x_n = \sum_{k=1}^{m_n} \langle x_n, e_k \rangle e_k. $$
What next?
Can we say the following?
For each fixed $k$,
$$\langle x_n, e_k \rangle \to \langle x, e_k \rangle \  \mbox{ as } \ n \to \infty. $$
How to show that
$$x = \sum_{k=1}^\infty \langle x, e_k \rangle e_k?$$
I also know that the series $\sum \langle x, e_k \rangle e_k$  does converge.
 A: Hint: note that, with the norm defined via the inner product, we have
$$
\left\| 
\sum_{k=1}^N \langle x,e_k \rangle e_k
\right\|^2 = 
\sum_{k=1}^N |\langle x,e_k \rangle|^2
$$
because the vectors $e_k$ are orthonormal.  Also, note that for all $N$, 
$$
\|x\|^2 \geq 
\left\| 
\sum_{k=1}^N \langle x, e_k \rangle e_k
\right\|^2
$$
We now know that the sum $\sum_{k=1}^\infty |\langle x,e_k \rangle|^2$ converges, which means that $\sum_{k=1}^N \langle x,e_k \rangle e_k$ converges.
However, we must now show that its limit is $x$.  In order to do this, it suffices to show that $x - \sum_{k=1}^\infty \langle x,e_k \rangle e_k$ is orthogonal to each $e_k$.
A: The orthogonal projection $P_{N}x$ of $x$ onto the subspace $M_{N}$ spanned by $\{ e_1,e_2,\cdots,e_N\}$ is given by $P_{N}x=\sum_{n=1}^{N}(x,e_n)e_n$. The orthogonal projection onto $M_{N}$ is the same as the closest point projection onto $M_{N}$ (just like in the good 'ole days of your Calculus class.) Therefore
$$
                  \|x-P_{N}x\| \le \|x-(\alpha_1 e_1 + \cdots +\alpha_N e_N)\|
$$
holds for all choices of scalars $\{\alpha_n\}_{n=1}^{N}$. The orthogonal (equivalently, closest-point) projection onto a larger subspace is at least as close. Hence,
$$
          \|x-P_{N'}x\| \le \|x-P_{N}x\| \le \|x-(\alpha_1 e_1 + \cdots +\alpha_N e_N)\|,\;\;\; N' \ge N.
$$
Therefore, if you can approximate $x$ to within a distance of $\epsilon$ by some $m \in M$, then the orthogonal series is within $\epsilon$ of $x$ for large enough $N$.
A: We can express each series representing $x_n$ as $x_n=\sum_{k\geq 1} \alpha_{nk} e_k$ by considering the Fourier Coefficients $\alpha_{nk} = 0$ for $k > m_n$. Then each of the sequences of Fourier Coefficients i.e. $(\alpha_{nk})_{k\geq 1}$ for $n \in \Bbb{N}$ lies in $l^2$(space of all square summable sequences). 
Let $s_n=(\alpha_{nk})_{k\geq 1}$. Then $(s_n)_{n\geq 1}$ is a Cauchy Sequence in $l^2$. This can be checked from the fact that since $x_n$ converges, it is a Cauchy Sequence and $||x_n-x_m||=||s_n-s_m||$. This makes $(s_n)_{n>=1}$ a Cauchy Sequence. Since the space $l^2$ is complete the sequence $(s_n)_{n \geq 1}$ will converge to some $l^2$ sequence. Let it be $s=(\alpha_k)_{k\geq 1}$. 
As $s \in l^2$, the series $\tilde{x}=\sum_{k\geq 1} \alpha_k e_k$ where $\alpha_k=\langle \tilde{x},e_k \rangle$ converges and $\tilde{x} \in \overline{span(e_k)}$
Only thing left is to prove $x_n \rightarrow \tilde{x}$ which is observable from the fact $||x_n-\tilde{x}||=||s_n-s||$.
