# Supremum of average of i.i.d. stochastic processes

Suppose that $$(X_t)_{t\in [0, T]}$$ is a stationary stochastic process with continuous sample paths such that $$\mathbb E[X_t] = 0$$ and $$\mathbb E[|X_t|^2] < \infty$$, and assume that $$S := \mathbb E \left[ \sup_{t \in [0, T]} |X_t|\right] < \infty.$$ Take $$J$$ independent copies $$(X^1_t), \dotsc, (X^J_t)$$ of the process $$(X_t)$$. Does it hold that $$\mathbb E \left[\sup_{t \in [0, T]} \left| \frac{1}{J} \sum_{j=1}^JX^j_t \right| \right] \leq \frac{S}{\sqrt{J}} \, ?$$ possibly up to a constant on the right-hand side. In other words, does the left-hand side decrease at the usual Monte Carlo rate?

Particular case. If $$X_t$$ is a Gaussian process, then the answer is yes, and both sides are in fact equal. Indeed, it holds in this case that $$\frac{1}{\sqrt{J}} \sum_{j=1}^J X^j = X \qquad \text{in law as stochastic processes,}$$ because both sides are Gaussian processes with the same mean and covariance functions. Can we say something on the case of a general mean-zero stationary stochastic process $$X$$?

Possible idea: The result could perhaps be proved by using a central limit theorem for function-valued random variables. Indeed it seems possible that $$\frac{1}{\sqrt{J}} \sum_{j=1}^{J} X^j \to Z \qquad \text{in some appropriate weak sense},$$ where $$Z$$ is a stationary Gaussian process with mean 0 and covariance variance $$k(s) = \mathbb E[Z_0 Z_s] = \mathbb E [X_0 X_s].$$ This seems to be the content of this paper. However, even assuming that a central limit theorem holds, I do not see how a conclusion could be reached, as the supremum is not a bounded function, and uniform integrability seems hard to prove.

Here is a stupid counter-example: Let $$X = (X_t)_{t\in[0,T]}$$ be such that

• $$X_t = X_0$$ for all $$t$$, and

• $$\mathbf{E}[\exp(i\xi X_0)] = e^{-|\xi|^{\alpha}}$$ for some $$\alpha \in (1, 2)$$. That is, $$X_0$$ has a symmetric stable distribution with shape parameter $$\alpha$$, zero location parameter, and unit scale parameter.

Then $$X$$ is stationary, and $$\mathbf{E}[X_t] = \mathbf{E}[X_0] = 0$$. Moreover, if $$X^1, \ldots, X^J$$ are i.i.d. copies of $$X$$, then with $$\beta = 1-\frac{1}{\alpha} \in (0, \frac{1}{2})$$,

$$\mathbf{E}\biggl[ \exp\biggl( i\xi \cdot \frac{1}{J} \sum_{j=1}^{J} X_t^j \biggr) \biggr] = \mathbf{E}[\exp(i\xi X_0 / J)]^J = \exp(-|\xi / J^{\beta}|^{\alpha}),$$

hence we have

$$\frac{1}{J} \sum_{j=1}^{J} X_t^j \stackrel{d}{=} \frac{X_t}{J^{\beta}}.$$

In particular, this implies

$$\mathbf{E}\biggl[ \sup_{t \in [0, T]} \left| \frac{1}{J} \sum_{j=1}^{J} X_t^j \right| \biggr] = \frac{S}{J^{\beta}},$$

which clearly decays slower than $$J^{-1/2}$$.

• Hmm, here you are in the usual Monte Carlo setting, so I do not see how the scaling could be any different than $1/\sqrt{J}$. Does $X_0$ have second a second moment in this example? I just added this constraint in the main post, but it was not included when you answered. I'll award the bounty to you in case there are no other answers by the end of the period. Commented Feb 10 at 17:40
• @RobertoRastapopoulos, In my example, indeed $X_0$ does not have a second moment. I also have a hunch that the result is likely true with the finiteness of second moment, but I don't have a good lead. Commented Feb 10 at 18:17
• Many thanks in any case! Commented Feb 12 at 17:25

Here is a sketch of counterexample.

For $$n\ge 2$$, take a continuous function $$f_n\colon [0,1]\to \mathbb R$$ such that $$f_n(t) = \begin{cases} n!,\quad & |t - 1/2|<1/(2n!),\\ 0, &|t-1/2|>1/n! \end{cases}$$ and $$f_n$$ takes intermediate values in other points. Then, $$\int_0^1 f_n(t)^2 dt \asymp n!$$, so there exist some probabilities $$p_n \asymp \big((n+1)!\log^2 n\big)^{-1}, n\ge 2,$$ such that $$\sum_{n\ge 2} p_n \cdot \int_0^1 f_n(t)^2 dt = 1$$.

Now define $$X_t = \kappa \cdot f_N\big((t+U) \mod 1\big), t\in [0,1],$$ where $$U,N,\kappa$$ are independent, $$U$$ is uniformly distributed on $$[0,1]$$, $$N$$ has distribution $$\mathrm P(N=n) = p_n$$, $$n\ge 2$$, and $$\kappa$$ has the Rademacher distribution: $$\mathrm P(\kappa = \pm 1) = 1/2$$.

Then the process $$X$$ is stationary with $$\mathrm E[X] = 0$$, $$\mathrm{Var}(X) = 1$$, and $$\mathrm E[\sup_{t\in [0,1]} |X_t|] = \sum_{n\ge 2} n!\cdot p_n <\infty$$.

Now take $$J\sim (n+1)! \log^3 n$$, and let $$X^j = \kappa^j\cdot f_{N^j}(t+U^j), j=1,\dots, J,$$ be independent copies of $$X$$. Write $$\sum_{j=1}^J X^j = \sum_{j\le J: N^j\ge n} X^j + \sum_{j\le J: N^j< n} X^j.$$ The first sum (say, $$X'$$) contains about $$\log n$$ summands, and most of the time they are disjoint (the birthday paradox says that we need about $$\sqrt{n!}\gg \log n$$ summands for a "collision"). So $$|X'_t| \ge n!$$ on approximately $$\log n$$ intervals. The second sum (say, $$X''$$) contains much more summands, which are "independent" from $$X'$$, so with overwhelming probability $$X''$$ will have the same sign as $$X'$$ at least somewhere inside these intervals.

Therefore, $$\mathrm E\left[\sup_{t\in [0,1]} \bigg|\sum_{j=1}^J X^j_t\bigg| \right]\gtrsim n!,$$ which is much more than $$\sqrt{J}$$. Actually, this example shows that $$\frac1J \sup_{t\in [0,1]} \left|\sum_{j=1}^JX^j_t\right|$$ can decrease arbitrarily slowly (under the given assumptions).

• Beautiful example ! Commented Feb 15 at 17:13
• I should have added an addiitonal restriction that $X_t$ was Markov, in which case the general answer is unclear to me. Commented Feb 15 at 17:14

Here is an example where the usual Monte Carlo rate is obtained. This does not answer my initial question, but solves the problem I was actually facing. I'll write it here as it could help someone else in the future.

Suppose that $$(X_t)$$ has moments of all orders and is the solution to a stochastic differential equation $$d X_t = b(X_t) d t + \sigma(X_t) d W_t,$$ for drift and diffusion coefficient satisfying $$b(x) \leq 1 + |x|^q$$ and $$\sigma(x) \leq 1 + |x|^q$$. Since $$(X_t)$$ is stationary, it holds that $$\mathbb E[b(X_t)] = 0$$. Let $$Y_t = \frac{1}{J} \sum_{j=1}^J X^j_t,$$ It holds that $$d Y_t = \frac{1}{J} \sum_{j=1}^J b(X^j_t) d t + \frac{1}{J} \sum_{j=1}^J \sigma(X^j_t) d W^j_t,$$ and so $$\forall t \in [0, T], \qquad \lvert Y_t - Y_0 \rvert^p \leq 2^{p-1} \left\lvert \int_0^t \frac{1}{J} \sum_{j=1}^{J} b(X^j_s) d s \right\rvert^p + 2^{p-1} \left\lvert \frac{1}{J} \sum_{j=1}^{J} \int_{0}^{t} \sigma(X^j_s) d W^j_s \right\rvert^p$$ $$\leq (2T)^{p-1} \int_0^T \left\lvert \frac{1}{J} \sum_{j=1}^{J} b(X^j_s) \right\rvert^p d s + 2^{p-1} \left\lvert \frac{1}{J} \sum_{j=1}^{J} \int_{0}^{t} \sigma(X^j_s) d W^j_s \right\rvert^p.$$ The argument of the second absolute value is a martingale, so by the Burkholder-Davis-Gundy inequality (using this formula for the quadratic variation of the sum of independent martingales), we deduce that $$\mathbb E \left[ \sup_{t \in [0, T]} \lvert Y_t - Y_0 \rvert^p \right] \leq (2T)^{p-1} \int_0^T \mathbb E \left[ \left\lvert \frac{1}{J} \sum_{j=1}^{J} b(X^j_s) \right\rvert^p \right] d t + 2^{p-1} \frac{C_{BDG}}{J^p} \mathbb E \left[ \left\lvert \sum_{j=1}^{J} \int_0^T \bigl\lvert \sigma(X^j_s) \bigr\rvert^2 d s \right\rvert^{\frac{p}{2}} \right]$$ $$\leq (2T)^{p-1} \int_0^T \mathbb E \left[ \left\lvert \frac{1}{J} \sum_{j=1}^{J} b(X^j_s) \right\rvert^p \right] d t + 2^{p-1} \frac{C_{BDG}}{J^{p/2}} T^{\frac{p}{2}-1} \mathbb E \left[ \int_0^T \bigl\lvert \sigma(X^1_s) \bigr\rvert^p d s \right].$$ Using standard estimates for the first term, we finally obtain that $$\mathbb E \left[ \sup_{t \in [0, T]} \lvert Y_t - Y_0 \rvert^p \right] \leq C J^{- \frac{p}{2}}.$$ So in this case, we recover the usual Monte Carlo rate.