# Erwin Kreyszig's Introductory Functoinal Anlaysis With Applications

## Prob. 8, Sec. 3.5

$$\DeclareMathOperator{\span}{span}$$Let $$(e_k)$$ be an orthonormal sequence in a Hilbert space $$H$$, and let $$M = \span (e_k)$$. Let $$x \in H$$.

If $$x = \sum_{k=1}^\infty \langle x, e_k \rangle e_k,$$ then $$x \in \overline{\span(e_k)}$$ because in this case the sequence $$(s_n)$$ in $$\span(e_k)$$, where $$s_n = \sum_{k=1}^n \langle x, e_k \rangle e_k$$, converges to $$x$$.

How to show the converse?

That is, how to show that if $$x \in \overline{\span(e_k)}$$, then the series $$\sum_{k=1}^\infty \langle x, e_k \rangle e_k$$ converges (in the norm induced by the inner product on $$H$$) and has sum $$x$$?

## My effort:

Suppose $$x \in \overline{\span(e_k)}$$. Then there is a sequence $$(x_n)$$ in $$\span(e_k)$$ that converges to $$x$$. Let $$x_n = \sum_{k=1}^{m_n} \alpha_{nk} e_k$$ for each $$n= 1, 2, 3, \ldots$$, where $$\alpha_{nk}$$ are scalars and the $$m_n$$ are natural numbers.

Then, using the orthonormality of the $$e_k$$, we can conclude that $$\alpha_{nk} = \langle x_n, e_k \rangle$$ for each $$n=1, 2, 3, \ldots$$ and for each $$k= 1, \ldots, m_n$$. So $$x_n = \sum_{k=1}^{m_n} \langle x_n, e_k \rangle e_k.$$

What next?

Can we say the following?

For each fixed $$k$$, $$\langle x_n, e_k \rangle \to \langle x, e_k \rangle \ \mbox{ as } \ n \to \infty.$$

How to show that $$x = \sum_{k=1}^\infty \langle x, e_k \rangle e_k?$$

I also know that the series $$\sum \langle x, e_k \rangle e_k$$ does converge.

• Hi Saaqib. Take note of the changes I made to your post. One change was in the typesetting (I needed to define \span), and the other was the title (titles should describe the problem). – Ben Grossmann May 6 '15 at 14:42
• If you could tell us anything about how you were planning to approach or tried to approach this problem, it would be very helpful. – Ben Grossmann May 6 '15 at 14:42
• @Omnomnomnom, I would like to retain the original title. You see, if I come back to the stuff I'm studying now in, say, two years' time and if I've unlearnt something I've learnt now, then I'll have somewhere specific to look. Otherwise I might end up posting too many duplicate questions. Don't you agree? – Saaqib Mahmood May 6 '15 at 14:53
• I've left your original title (i.e. the reference to the problem number/text) at the top of your question. So if, in two years' time, you click on this question, you'll see where exactly you should look. With this title, you (and others) can see what the question is about before clicking the question. – Ben Grossmann May 6 '15 at 15:01
• But @Omnomnomnom, it's from the Kreyszig's text that I've got this question in the first place, and not vice versa. So, it is more likely that I'll encounter this question during another reading of Kreyszig at some point of time in the future. – Saaqib Mahmood May 6 '15 at 15:05

The orthogonal projection $P_{N}x$ of $x$ onto the subspace $M_{N}$ spanned by $\{ e_1,e_2,\cdots,e_N\}$ is given by $P_{N}x=\sum_{n=1}^{N}(x,e_n)e_n$. The orthogonal projection onto $M_{N}$ is the same as the closest point projection onto $M_{N}$ (just like in the good 'ole days of your Calculus class.) Therefore $$\|x-P_{N}x\| \le \|x-(\alpha_1 e_1 + \cdots +\alpha_N e_N)\|$$ holds for all choices of scalars $\{\alpha_n\}_{n=1}^{N}$. The orthogonal (equivalently, closest-point) projection onto a larger subspace is at least as close. Hence, $$\|x-P_{N'}x\| \le \|x-P_{N}x\| \le \|x-(\alpha_1 e_1 + \cdots +\alpha_N e_N)\|,\;\;\; N' \ge N.$$ Therefore, if you can approximate $x$ to within a distance of $\epsilon$ by some $m \in M$, then the orthogonal series is within $\epsilon$ of $x$ for large enough $N$.

• can you please have a careful look through my post, especially my effort, and then complete the argument there? That would be more instructive as far as I'm concerned. – Saaqib Mahmood May 7 '15 at 5:24
• @SaaqibMahmuud : I'm not sure that would more instructive. You are missing the point of why the series $\sum_{n=1}^{N}(x,e_n)e_n$ converges to $x$ if $x$ is in the closure of the linear span of $M$. This holds even if the space is not complete. It simply has to do with the fact that orthogonal projection and closest-point projection are the same thing in infinite and in finite dimensional inner product spaces. Until you see that point, you cannot get what you want. – Disintegrating By Parts May 7 '15 at 5:49
• yes, I do know that if $Y$ is a complete subspace of an inner product space $X$, then, for every $x \in X$, there exists a unique point $y \in Y$ such that $\inf_{v \in Y} \Vert x- v \Vert = \Vert x-y \Vert$. (This holds even if $Y$ is any complete convex subset of $X$) and that this point $y$ also satisfies $\langle x-y, v\rangle = 0$ for all $v \in Y$. – Saaqib Mahmood May 23 '15 at 4:48
• how do you obtain the inequality $\Vert x- P_{N^\prime} x \Vert \leq \Vert x - P_N x \Vert$ if $N^\prime > N$? – Saaqib Mahmood May 23 '15 at 4:53
• If you have a sequence $x_{n}=\sum_{k=1}^{m_{n}}a_{nk}e_k$ that converges to $x$, then $\|x-P_{m_{n}}x\| \le \|x-x_{n}\|$ by the previous remark, which means that $\|x-P_{l}x\| \le \|x-x_{n}\|$ for all $l \ge m_{n}$. That's enough to see that $\lim_{l}\|x-P_{l}x\|=0$. – Disintegrating By Parts May 23 '15 at 22:53

Hint: note that, with the norm defined via the inner product, we have $$\left\| \sum_{k=1}^N \langle x,e_k \rangle e_k \right\|^2 = \sum_{k=1}^N |\langle x,e_k \rangle|^2$$ because the vectors $e_k$ are orthonormal. Also, note that for all $N$, $$\|x\|^2 \geq \left\| \sum_{k=1}^N \langle x, e_k \rangle e_k \right\|^2$$ We now know that the sum $\sum_{k=1}^\infty |\langle x,e_k \rangle|^2$ converges, which means that $\sum_{k=1}^N \langle x,e_k \rangle e_k$ converges. However, we must now show that its limit is $x$. In order to do this, it suffices to show that $x - \sum_{k=1}^\infty \langle x,e_k \rangle e_k$ is orthogonal to each $e_k$.

• I've corrected something in your answer. Do you agree? Although I know these results, I haven't been able to figure out how these results lead to what I'm trying to prove. – Saaqib Mahmood May 6 '15 at 15:02
• I agree, good catch. After looking at the question again, this seems trickier than I at first expected. I'll see if I can make this into something useful. – Ben Grossmann May 6 '15 at 15:04
• See my latest edit. – Ben Grossmann May 6 '15 at 15:22
• please look at my edit in my original question as well and then see how you can take me toward completion of the argument. – Saaqib Mahmood May 6 '15 at 15:41
• how do we show that if $x \in \overline{M} = \overline{\span \{e_1, e_2, \ldots\}}$, then the convegeent series $\sum \langle x, e_k \rangle e_k$ necessarily has the sum $x$? – Saaqib Mahmood May 23 '15 at 5:19

We can express each series representing $$x_n$$ as $$x_n=\sum_{k\geq 1} \alpha_{nk} e_k$$ by considering the Fourier Coefficients $$\alpha_{nk} = 0$$ for $$k > m_n$$. Then each of the sequences of Fourier Coefficients i.e. $$(\alpha_{nk})_{k\geq 1}$$ for $$n \in \Bbb{N}$$ lies in $$l^2$$(space of all square summable sequences).

Let $$s_n=(\alpha_{nk})_{k\geq 1}$$. Then $$(s_n)_{n\geq 1}$$ is a Cauchy Sequence in $$l^2$$. This can be checked from the fact that since $$x_n$$ converges, it is a Cauchy Sequence and $$||x_n-x_m||=||s_n-s_m||$$. This makes $$(s_n)_{n>=1}$$ a Cauchy Sequence. Since the space $$l^2$$ is complete the sequence $$(s_n)_{n \geq 1}$$ will converge to some $$l^2$$ sequence. Let it be $$s=(\alpha_k)_{k\geq 1}$$.

As $$s \in l^2$$, the series $$\tilde{x}=\sum_{k\geq 1} \alpha_k e_k$$ where $$\alpha_k=\langle \tilde{x},e_k \rangle$$ converges and $$\tilde{x} \in \overline{span(e_k)}$$

Only thing left is to prove $$x_n \rightarrow \tilde{x}$$ which is observable from the fact $$||x_n-\tilde{x}||=||s_n-s||$$.