# Convergence of the random harmonic series $\sum_{n=1}^{\infty}\frac{X_{n}}{n}$

Let $$(X_{n})_{n \in \mathbb{N}}$$ be independent with Rademacher distribution:

$$\begin{equation} \mathbb{P}(X_{n} = -1) = \frac{1}{2} = \mathbb{P}(X_{n} = 1). \end{equation}$$

I have to investigate

$$\begin{equation} \sum_{n=1}^{\infty}\frac{X_{n}}{n} \end{equation}$$ for convergence. It was given in a textbook and I'm very interested in the solution. It is something between the harmonic series $$\sum_{n=1}^{\infty}\frac{1}{n}$$ and the series $$\sum_{n=1}^{\infty}\frac{(-1)^{n}}{n}$$, but I know the sign changes randomly.

• @BCLC You are right. – RideTheWavelet Jul 15 '18 at 5:40

Define for $N \in \mathbb{N}$ \begin{equation} Y_{N} = \sum_{n=1}^{N}X_{n} \end{equation} Now \begin{equation} \mathbb{E}(\frac{1}{n}X_{n}) = 0. \end{equation} Therefore \begin{equation} \mathbb{E}(Y_{N}) = 0. \end{equation} Also \begin{align} var(Y_{N}) &= var(\sum_{n=1}^{N}X_{n})\\ &= \sum_{n=1}^{N}\frac{1}{n^{2}}var(X_{n})\\ &=\sum_{n=1}^{N}\frac{1}{n^{2}}\\ &\leq\sum_{n=1}^{\infty}\frac{1}{n^{2}} = \frac{\pi^{2}}{6}. \end{align}

For $M < N$ we have \begin{align} \mathbb{E}[|Y_{N}-Y_{M}|^{2}] &= \mathbb{E}[|\sum_{n=M+1}^{N}\frac{1}{n}X_{n}|^{2}]\\ &=var(\sum_{n=M+1}^{N}\frac{1}{n}X_{n})\\ &=\sum_{n=M+1}^{N}\frac{1}{n^{2}} \xrightarrow{N,M \to \infty} 0, \end{align} i.e. $(Y_{N})_{N \in \mathbb{N}}$ is a cauchy-sequence with respect to the $\mathcal{L}^{2}$-norm. Now the Riesz–Fischer theorem says, that $\mathcal{L}^{2}$ is a banach-space. Especially every cauchy sequence in $\mathcal{L}^{2}$ converges. Now $\mathcal{L}^{2}$-convergence implies onvergence in probability and distribution. We also have almost sure convergence, since if $(X_{n})_{n \in \mathbb{N}}$ are independet random variables with $S_{n} = \sum_{i=1}^{n}X_{i} \xrightarrow{p} S$ then $S_{n} \rightarrow S$ almost surely.

• I don't know if there was a typo somewhere or what, but in fact $var(\sum X_n)=\sum var(X_n)$; that $1/n^2$ some from nowhere. – David C. Ullrich May 17 '18 at 15:12
• I believe $Y_{N}$ is meant to be $\sum_{n=1}^{N}\frac{1}{n}X_{n}$. I have a bigger problem with the last statement, since typically convergence in probability does not imply convergence a.s. – RideTheWavelet May 17 '18 at 15:15
• @RideTheWavelet. You can prove the last statement. See e.g in the book of Chung, A course in probability theory, page 126 – wayne May 17 '18 at 15:22
• Great reference in Chung, I'm really pleased to know a new LLN! – RideTheWavelet May 17 '18 at 15:43
• @RideTheWavelet It's true that if a sum of independent random variables converges in $L^p$ then it converges almost surely. The relevant "maximal inequality" is fairly simple in this case... – David C. Ullrich May 17 '18 at 15:44

It's clear that the sum converges in $L^2$. Because the terms are independent it follows that the sum converges almost surely. One can give a simple "trick proof" of this using Khinchine's Inequality:

Say $$X=\sum\frac{X_n}{n}$$(where the convergence is in $L^2$). We need to show that $E_n\to0$ almost surely, where $$E_n=X-S_n,$$ $$S_N=\sum_{n=1}^N\frac{X_n}{n}.$$

Khinchine's inequality (see below) says that $$||E_N||_4\le c||E_N||_2=c\left(\sum_{n=N+1}^\infty\frac1{n^2}\right)^{1/2}\sim\frac c{N^{1/2}}.$$So $$\Bbb E(E_n^4)\le\frac c{n^2}.$$So monotone convergence implies $\Bbb E\sum E_n^4<\infty$; hence $\sum E_n^4<\infty$ almost surely, which implies $E_n\to0$ almost surely.

Below: In fact the relevant special case of Khinchine's inequality is trivial:

Lemma. If $a_j$ is real then $\Bbb E (\sum a_jX_j)^4\le c(\sum a_j^2)^2$.

Proof: If you multiply out $(\sum a_jX_j)^4$ you get a lot of terms. By independence all the terms have expected value $0$ except for terms of the form $a_j^4X_j^4$ and $a_j^2a_k^2X_j^2X_k^2$. Noting that the first form is just the second form for $j=k$ you see that $$\Bbb E(\sum a_jX_j)^4\le c\sum_j\sum_ka_j^2a_k^2=c(\sum a_j^2)^2.$$

• How are my answers please? – BCLC Jun 3 '18 at 11:22

Here's a way with summation by parts: $$\sum_{k=1}^n \frac{X_k}{k} = \frac{S_n}{n} + \sum_{k=1}^{n-1} \frac{S_k}{k(k+1)}$$

where $$S_k =\sum_{i=1}^k X_i$$. Note that $$(S_k)_k$$ is the symmetric random walk on $$\mathbb Z$$. It's an irreducible recurrent Markov chain, so $$(S_k)_k$$ is unbounded a.s. and the Dirichlet test does not apply.

However, by the law of the iterated logarithm, $$\limsup_k \frac{|S_k|}{\sqrt{2k\log \log k}} = 1$$ a.s, hence $$\left(\frac{|S_k|}{\sqrt{k\log \log k}}\right)_k$$ is bounded a.s. As a result, $$S_k=O(k^{1/2+\epsilon})$$ a.s., thus the series $$\sum_{k\geq 1} \frac{S_k}{k(k+1)}$$ converges a.s and simultaneously $$\frac{S_n}{n}\to 0$$ a.s.

For more elementary asymptotics, note that for any $$\epsilon, \epsilon'>0$$, Hoeffding's inequality yields $$P\left(\frac{S_n}{n^{1/2+\epsilon}}\geq \epsilon'\right) \leq \exp\left(-\frac{\epsilon'^2n^{2\epsilon}}{2}\right)$$ so $$\sum_n P\left(\frac{S_n}{n^{1/2+\epsilon}}\geq \epsilon'\right)$$ converges, thus $$\displaystyle \frac{S_n}{n^{1/2+\epsilon}}\xrightarrow[]{a.s.} 0$$. Hence $$S_n=o(n^{1/2+\epsilon})$$ a.s. and one concludes similarly.

Use the following theorem about sums of zero-mean independent variables in $\mathcal L^2$ from David Williams' Probability with Martingales Chapter 12.2.

Let $\{X_n\}_{n \in \mathbb N}$ be a sequence of random variables. We have $$\sum_n X_n < \infty \ \text{a.s.}.$$ if the following 4 conditions hold:

1. $$\{X_n\}_{n \in \mathbb N} \text{is independent.}$$

2. $$E[X_n] = 0 \ \forall n \in \mathbb N$$

3. $$\sigma_n^2 := Var[X_n] < \infty \ \forall n \in \mathbb N$$

4. $$\sum_{n \in \mathbb N} \sigma_n^2 < \infty$$

In our case:

1. $$\{\frac{X_n}{n}\}_{n \in \mathbb N} \ \text{is independent} \ \because \{X_n\}_{n \in \mathbb N} \text{is independent.}$$

2. $$E[\frac{X_n}{n}]=\frac{1}{n}E[X_n] = \frac{1}{n} 0 = 0 \ \forall n \in \mathbb N$$

3. $$Var[\frac{X_n}{n}] = \frac{1}{n^2} Var[\frac{X_n}{1}] = \frac{1}{n^2} (1) = \frac{1}{n^2} < \infty \ \forall n \in \mathbb N$$

4. $$\sum_{n \in \mathbb N} Var[\frac{X_n}{n}] = \sum_{n \in \mathbb N} \frac{1}{n^2} < \infty \ \text{by integral test}$$

$\therefore,$ by theorem in Chapter 12.2 of David Williams' Probability with Martingales, $$\sum_n \frac{X_n}{n} < \infty \ \text{a.s.}.$$ QED