3
$\begingroup$

Recently, I am self-studying Stanford Stat 300B and encounter an exercise in the problem set, which is to prove the following:

Suppose that $X_i$ are independent mean-zero random variables with $\mathbb{E}[|X_i|^k]<\infty$, let $S_n=\sum_{i=1}^n X_i$ and prove that: $$ \mathbb{E}[|S_n|^k]\leqslant C_k \mathbb{E}\left[\left(\sum_{i=1}^n X_i^2\right)^{\frac{k}{2}}\right]. $$ I have tried to expand left hand side into terms of $X_{i_1}^{\alpha_1}\cdots X_{i_t}^{\alpha_t}$ and use the fact of independency, but in vain.

Can any body give me some hints on this problem, or provide the solution provided in https://web.stanford.edu/class/stats300b/exercise.html, since I don't have access to the solution link. Thank you very much!

$\endgroup$
2
  • $\begingroup$ Id imagine Holders inequality would be the way to go here. Holders gives you $E |S_n|^k \leq (E|S_n|^2)^{k/2}$. Perhaps try to use that the $X_i$ are centered to go from here $\endgroup$
    – Andrew
    Jun 17, 2023 at 7:58
  • $\begingroup$ Just some initial thoughts…try at your own risk :) $\endgroup$
    – Andrew
    Jun 17, 2023 at 8:04

1 Answer 1

2
$\begingroup$

The short answer is that it follows from the Marcinkiewicz–Zygmund inequality. However, to add some value to this answer, let me break down the proof and go through some of the details$^1$.

  1. Your idea of expanding $|S_n|^k$ using the Multinomial Theorem is a good idea, and is in fact used to show what is known as Khintchine inequality. It states that given i.i.d. Rademacher random variables $V_1, \dots, V_n$, some numbers $x_1, \dots, x_n\in\mathbb{R}$, and an integer $k\geq 1$, $$\mathbb{E}\left[\left|\sum_{i=1}^n V_i x_i\right|^{k}\right]^{1/k}\leq B_k\left(\sum_{i=1}^n x_i^2\right)^{1/2},$$ where $B_k$ is a constant depending only on $k$.

Proof: Assuming $k=2m$, we have (using the Multinomial Theorem and that $V_i$'s are independent) $$\mathbb{E}\left[\left|\sum_{i=1}^n V_i x_i\right|^{2m}\right] = \sum_{\substack{\alpha_1, \dots, \alpha_n:\\ \alpha_1+ \dots + \alpha_n=2m}} \binom{2m}{\alpha_1, \dots, \alpha_n} \prod_{i=1}^n x_i^{\alpha_i}\mathbb{E}\left[V_i^{\alpha_i}\right].$$ Now since $\mathbb{E}[V_i^p] = \mathbb{1}\{p\text{ is even}\}$, this means the non-zero terms in the sum above must have all their coefficients $\alpha_1, \dots, \alpha_n$ even. Hence, re-writing using $\alpha_j = 2\beta_j$ reads \begin{align} \mathbb{E}\left[\left|\sum_{i=1}^n V_i x_i\right|^{2m}\right] &= \sum_{\substack{\beta_1, \dots, \beta_n:\\ \beta_1+ \dots + \beta_n=m}} \binom{2m}{2\beta_1, \dots, 2\beta_n} \prod_{i=1}^n x_i^{2\beta_i}\\ &\leq \sum_{\substack{\beta_1, \dots, \beta_n:\\ \beta_1+ \dots + \beta_n=m}} \frac{\binom{2m}{2\beta_1, \dots, 2\beta_n}}{\binom{m}{\beta_1, \dots, \beta_n}}\binom{m}{\beta_1, \dots, \beta_n} \prod_{i=1}^n x_i^{2\beta_i}\\ &\leq \left(\sup \frac{\binom{2m}{2\beta_1, \dots, 2\beta_n}}{\binom{m}{\beta_1, \dots, \beta_n}}\right) \sum_{\substack{\beta_1, \dots, \beta_n:\\ \beta_1+ \dots + \beta_n=m}} \binom{m}{\beta_1, \dots, \beta_n} \prod_{i=1}^n x_i^{2\beta_i}\\ &=D_{2m} \left(\sum_{i=1}^n x_i^2\right)^{m}, \end{align} where $D_{2m} = \sup_{\substack{\beta_1, \dots, \beta_n:\\ \beta_1+ \dots + \beta_n=m}} \frac{\binom{2m}{2\beta_1, \dots, 2\beta_n}}{\binom{m}{\beta_1, \dots, \beta_n}}$. Raising each side to the power $1/(2m)$, we get the desired result, $\mathbb{E}\left[\left|\sum_{i=1}^n V_i x_i\right|^{2m}\right]^{1/(2m)}\leq B_{2m}\left(\sum_{i=1}^n x_i^2\right)^{1/2}$ with $B_{2m} = D_{2m}^{1/(2m)}$. Finally, for the case where $k$ is odd, (we can write $k=2m-1)$, one should note that the LHS in Khintchine inequality is an $L_p$-norm. Since the $L_p$-norm is increasing in $p$, $$\mathbb{E}\left[\left|\sum_{i=1}^n V_i x_i\right|^{2m-1}\right]^{1/(2m-1)} \leq \mathbb{E}\left[\left|\sum_{i=1}^n V_i x_i\right|^{2m}\right]^{1/(2m)}\leq B_{2m}\left(\sum_{i=1}^n x_i^2\right)^{1/2},$$ establishing the result. $\blacksquare$

  1. With Khintchine inequality proven, we can move on to the proof of Marcinkiewicz–Zygmund inequality. In this case, we have i.i.d. zero-mean random variables $X_1, \dots, X_n$ with $\mathbb{E}[|X_i|^k]<\infty$, $k\geq 1$. The statement is $$\mathbb{E}\left[\left|\sum_{i=1}^n X_i\right|^{k}\right]\leq C_k\mathbb{E} \left[\left(\sum_{i=1}^n X_i^2\right)^{k/2}\right] ,$$ where $C_k$ depends only on $k$.

Proof: To prove the statement, you need to play a bit with conditional expectations, use some symmetrization argument, and introduce Rademacher random variables to invoke Khintchine inequality. More formally, let $X_1^{\prime}, \dots, X_n^{\prime}$ be independent of and identically dstributed to $X_1, \dots, X_n$. With this, consider the symmetrized version of $X_i$'s, i.e., let $X_i^{\star} = X_i-X_i^{\prime}$. Moreover, let $V_1, \dots, V_n$ be i.i.d. Rademacher random variables, independent of both $X_i$'s and $X_i^{\prime}$'s.

We have \begin{align} \mathbb{E}\left[\left|\sum_{i=1}^n X_i\right|^{k}\right] &= \mathbb{E}\left[\left|\mathbb{E}\left[\sum_{i=1}^n X_i^{\star}|X_1, \dots, X_n\right]\right|^{k}\right]\\ &\stackrel{\text{(Jensen's inequality)}}{\leq} \mathbb{E}\left[\mathbb{E}\left[\left|\sum_{i=1}^n X_i^{\star}\right|^{k}|X_1, \dots, X_n\right]\right]\\ &=\mathbb{E}\left[\left|\sum_{i=1}^n X_i^{\star}\right|^{k}\right]\\ &\stackrel{\text{($X_i^{\star}$ symmetric)}}{=} \mathbb{E}\left[\left|\sum_{i=1}^n V_i X_i^{\star}\right|^{k}\right]\\ &=\mathbb{E}\left[\left|\sum_{i=1}^n V_i X_i-\sum_{i=1}^n V_i X_i^{\prime}\right|^{k}\right]\\ &\stackrel{(|x-y|^k\leq 2^{k-1}(|x|^k+|y|^k))}{\leq} 2^{k-1}\mathbb{E}\left[\left|\sum_{i=1}^n V_i X_i\right|^k+\left|\sum_{i=1}^n V_i X_i^{\prime}\right|^{k}\right]\\ &=2^k \mathbb{E}\left[\left|\sum_{i=1}^n V_i X_i\right|^k\right]\\ &=2^k \mathbb{E}\left[\mathbb{E}\left[\left|\sum_{i=1}^n V_i X_i\right|^k|X_1, \dots, X_n\right]\right]\\ &\stackrel{\text{(Khintchine inequality)}}{\leq} 2^k B_k^k\mathbb{E}\left[\left(\sum_{i=1}^n X_i^2\right)^{k/2}\right]. \end{align} Hence, we have $\mathbb{E}\left[\left|\sum_{i=1}^n X_i\right|^{k}\right]\leq C_k\mathbb{E} \left[\left(\sum_{i=1}^n X_i^2\right)^{k/2}\right]$ with $C_k = 2^k B_k^k$, as desired. $\blacksquare$

$^1:$ The proof is adapted from the material given in Section 10.3 of the book Probability Theory: Independence, Interchangeability, Martingales (Second Edition), by Yuan Shih Chow and Henry Teicher.

$\endgroup$

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .