# Independent random variables with infinite expectation and central limit theorem

I’m trying to construct a sequence of independent random variables $$X_1, X_2, \ldots$$ with $$\mathbb E[|X_n|] = \infty$$ for every $$n$$, and for which we have $$S_n^* := \frac{X_1 + \cdots + X_n}{\sqrt n}$$ converges in distribution to the standard normal distribution with density $$e^{-x^2/2}/\sqrt{2\pi}$$.

The proof of the Central Limit Theorem with which I’m most familiar involves taking the characteristic functions $$\varphi_{S_n^* }(t)$$ of $$S_n^*$$, showing they converge pointwise to $$e^{-t^2/2}$$ as $$n \to \infty$$, and using Lévy’s Continuity Theorem to show that the distributions of $$S_n^*$$ converge weakly to the standard normal distribution $$\mathcal N_{0,1}$$. The only way I know to do this involves using the Taylor expansion of $$\varphi_{S_n^* }$$, which requires that $$\mathbb E[|X_n|] <\infty$$ for all $$n$$. (More specifically, the Taylor expansion requires that $$\varphi_{S_n^* }(t)$$ be at least twice differentiable, which is equivalent to $$\mathbb E[|S_n^* |^2]<\infty$$, which we don’t have if $$\mathbb E[|X_n|] = \infty$$ for all $$n$$).

The other ideas I’ve tried have involved either (a) taking a distribution of the form $$\mathbb P_{X_n} = \sum_{k \in \mathbb Z} p_{n,k} \delta_k^*$$, where $$p_{n,k} = \mathbb P[X_n = k]$$ and $$\delta_k$$ is the Dirac mass at $$k \in \mathbb Z$$, or (b) taking $$X_n$$ with a continuous density $$f_n$$ with respect to Lebesgue measure. In both cases, we want $$\mathbb E[|X_n|] = \infty$$. We know $$\varphi_{S_n^* }(t) = \prod_{i = 1}^n \varphi_{X_i}(t/\sqrt{n})$$ by independence. In the discrete case (a), $$\varphi_{X_i}(t) = \sum_{k \in \mathbb Z} p_{i,k} \cos(kt),$$ and in the continuous case (b), $$\varphi_{X_i}(t) = \int_{\mathbb R} \cos(xt) f_i(x) dx.$$ In both cases I’m not sure how to find a nice expression for $$\prod_i\varphi_{X_i}(t/\sqrt{n})$$, and using a Taylor approximation for $$\varphi_{X_i}$$ is essentially a non-starter if $$\mathbb E[|X_i|] = \infty$$ for all $$i$$.

Any suggestions for how I might proceed?

• They should. I’ve edited the problem accordingly Sep 12, 2022 at 20:16
• are you certain that it's possible ? AFAIK, a mean of infinity rules out the use of any CLT's but I'm not an expert and could be wrong. Sep 12, 2022 at 20:27
• @markleeds a priori there's nothing that stops a stochastic process with infinite expectation from having variance-normed means whose distributions converge to a Gaussian. You certainly can't apply the hypotheses of the CLT but it's far from obvious that its conclusion shouldn't hold anyways. Sep 12, 2022 at 21:04

The following is an example of sequence $$\{X_n,n\ge 1 \}$$ of independent random variables with $$\mathsf{E}[|X_n|]=\infty$$ and $$S_n^\ast := \frac{X_1+\cdots+X_n}{\sqrt{n}} \overset{d}{\to} N(0,1)$$.

Suppose $$\{\xi_n,n\ge1\}$$ is a sequence of i.i.d. random variables and $$\xi_n\overset{d}{=} U(-1,1)$$. Denote \begin{align*} X_n&=\sqrt{3}\xi_n + \frac{1_{\{|\xi_n|\le n^{-2}\}}}{|\xi_n|}. \\ &=\sqrt{3}\xi_n +Z_n. \tag{1} \end{align*} Then $$\begin{gather*} \mathsf{E}[|Z_n|]=\infty, \quad \mathsf{E}[|X_n|]=\infty,\\ \frac{1}{\sqrt{n}}\sum_{j=1}^n\sqrt{3}\xi_j\overset{d}{\to} N(0,1). \tag{2} \end{gather*}$$ Due to $$\sum\limits_{n\ge 1}\mathsf{P}(Z_n\ne0)<\infty$$, $$\begin{gather*} \sum_{n=1}^{\infty}Z_n <\infty,\qquad \text{a.s.}.\\ \frac{1}{\sqrt{n}}\sum_{j=1}^n Z_j\to 0, \qquad \text{a.s.}.\tag{3} \end{gather*}$$ From (1)-(3) get $$\begin{equation*} \frac{1}{\sqrt{n}}\sum_{j=1}^n X_j \overset{d}{\to} N(0,1). \end{equation*}$$

• JGWang: I follow most of it but what was the reasoning for having to make the numerator of $Z_{n}$ be an indicator function involving $\frac{1}{n^2}$. I think it has to do with the use of the Borel-Cantelli lemma later on ? Thanks for your nice answer. Sep 15, 2022 at 6:35
• Also,, does the summation above 3 imply 3 or does the "Due to" imply 3 ? Thanks. Sep 15, 2022 at 13:12
• Thank you for your comments. $$\{Z_n\ne 0\}=\{\xi_n\le n^{-2} \} " \implies \mathsf{P}(Z_n\ne0)=\frac{1}{n^2} \implies \sum\limits_{n\ge 1}\mathsf{P}(Z_n\ne0)<\infty$$ and $\sum\limits_{n=1}^{\infty}Z_n <\infty \implies \frac{1}{\sqrt{n}}\sum\limits_{j=1}^n Z_j\to 0$. Sep 16, 2022 at 1:07

Let $$Y_1,Y_2,\dots$$ and $$N_1,N_2,\dots$$ be jointly independent random variables with $$N_i \sim N(0,1)$$. Also, assume that the distribution of $$Y_i$$ is such that $$\mathbb{E}[|Y_i|] = \infty$$ but $$\mathbb{P}(Y_i \neq 0) \leq 2^{-i}$$ (I will leave the existence of such $$Y_i$$ to you). Finally, set $$X_i := Y_i + N_i$$. It is easy to see that the $$X_i$$ are independent and satisfy $$\mathbb{E}|X_i| = \infty$$.

Now, by the Borel Cantelli Lemma, since $$\sum_i \mathbb{P}(Y_i \neq 0) < \infty$$, we see that almost surely, $$Y_i \neq 0$$ only holds for finitely many $$i$$. This easily implies that $$\overline{Y}_n := n^{-1/2} (Y_1 + \dots + Y_n)$$ satisfies $$\overline{Y_n} \to 0$$ almost surely (why?!).

Now, let $$N \sim N(0,1)$$. To show that $$\overline{X}_n := n^{-1/2} (X_1+\dots+X_n)$$ satisfies $$\overline{X}_n \to N(0,1)$$ in distribution, it suffices to show for every bounded Lipschitz function $$g$$ that $$\mathbb{E}[g(\overline{X_n})] \to \mathbb{E}[g(N)]$$. To show this, first note that $$\overline{N}_n := n^{-1/2} (N_1 + \dots + N_n)$$ satisfies $$\overline{N}_n \sim N$$. Next, note that $$|g(\overline{N}_n) - g(\overline{X}_n)| \leq \min \{ 2 C, L \cdot |\overline{N}_n - \overline{X}_n| \} = \min \{ 2 C, L \cdot |\overline{Y}_n| \} \to 0$$ as $$n \to \infty$$, where $$L$$ is the Lipschitz constant of $$g$$ and $$|g(x)| \leq C$$ for all $$x$$. Hence, by dominated convergence, $$\big| \mathbb{E}[g(\overline{X}_n)] - \mathbb{E}[g(N)] \big| = \big| \mathbb{E}[g(\overline{X}_n)] - \mathbb{E}[g(\overline{N}_n)] \big| \leq \mathbb{E} |g(\overline{N}_n) - g(\overline{X}_n)| \to 0 .$$ As noted above, this shows that $$\overline{X}_n \to N(0,1)$$ in distribution.

• Thanks for your nice answer. I just have two questions about it. 1) Did you mean that $X_{i} = Y_{i} + N_{i}$ rather than $X_{i} = Y_{i} + Z_{i}$. 2) What is $g$ in this case ? Is it just any function that is a bounded Lipschitz function ? Sep 15, 2022 at 6:39
• @markleeds: Yes and yes. Feel free to fix the typo (i am on my phone right now) Sep 15, 2022 at 6:49
• I fixed the typo . Also, I meant to ask if you could answer your "why ?". Thanks. Sep 15, 2022 at 13:10
• @markleeds: For almost every "elementary event" $\omega$, there is some $N = N(\omega)$ such that $Y_i = 0$ for all $i \geq N$. Hence for every $n \geq N$, we have $\overline{Y}_n(\omega) = n^{-1}\cdot (Y_1(\omega)+...+Y_N(\omega))$ and this converges to zero for $n \to \infty$. Sep 16, 2022 at 14:41
• Nice and thanks. Sep 17, 2022 at 14:42

Alternative solution that doesn’t use Borel-Cantelli:

Let $$Y_n$$ be Cauchy distributed with parameter $$a = 1/n$$, and $$Z_n$$ be uniformly distributed over $$[-\sqrt 3, \sqrt 3]$$. Assume all $$Y_n$$ and $$Z_n$$ are pairwise independent. Let $$X_n = Y_n + Z_n$$ and $$S^*_n = \frac 1{\sqrt n}(X_1 + \cdots + X_n)$$.

By independence we have that $$\varphi_{S^*_n}(t) = \prod_{j=1}^n \left( \varphi_{Y_j}\left(\frac{t}{\sqrt n}\right) \varphi_{Z_j}\left(\frac{t}{\sqrt n}\right)\right) = \varphi_{Z_1}\left(\frac t{\sqrt n}\right)^n \prod_{j=1}^n \varphi_{Y_j}\left(\frac{t}{\sqrt n}\right).$$ Since $$\mathbb E\left[|Z_1|\right] < \infty$$ and $$\mathbb{Var}[Z_1] = 1$$,it follows from the classical Central Limit Theorem and Lévy’s Continuity Theorem that $$\varphi_{Z_1} \left(\frac{t}{\sqrt n}\right)^n = \varphi_{\frac{1}{\sqrt n}\left(Z_1 + \cdots + Z_n\right)}(t) \xrightarrow{n \to \infty} e^{-t^2/2}.$$ Meanwhile, $$Y_j$$ has characteristic function $$\varphi_{Y_j}(t) = e^{-|t|/j}$$, so: $$\prod_{j=1}^n \varphi_{Y_j} \left(\frac{t}{\sqrt n}\right) = \exp\left(-\frac{|t|}{\sqrt n}\sum_{j=1}^n \frac 1 j\right) \xrightarrow{n \to \infty} 1$$ since $$\frac{1}{\sqrt n}\sum_{j=1}^n \frac 1 j \leq \frac 1{\sqrt n}(1+\log n) \xrightarrow{n \to \infty} 0$$. It follows that $$\varphi_{S^*_n}(t) \to e^{-t^2/2}$$. By Lévy’s Continuity Theorem, $$S^*_n$$ converges to $$\mathcal N_{0,1}$$ in distribution. Since $$\mathbb E[|Y_n|] = \infty$$ and $$\mathbb E[|Z_n|] = \sqrt 3/2$$ for every $$n$$, it’s straightforward to argue that $$\mathbb E[|X_n|] = \infty$$.

• Okay. I'll have to print these out and see if I can make sense of them. Thanks to all who sent the examples. D. Ford, PhoemueX and JGWang. Sep 14, 2022 at 5:21
• @D Ford: Thanks for your nice answer. I just didn't follow where you got that $\frac{1}{\sqrt{n}} \sum_{j=1}^{n} \frac{1}{j} \le \frac{1}{\sqrt{n}}( 1 + log (n))$. Sep 15, 2022 at 6:42
• @markleeds I'm using the fact that $\sum_{j=1}^n \frac 1 j < 1 + \int_1^n \frac 1 x dx$. You can show this graphically, eg with a Riemann sum. Sep 15, 2022 at 7:45
• Gotcha. Thanks. Sep 15, 2022 at 13:07