I'm trying to understand this proof of the uniform law of large numbers. The proof I mean is on page 2, the theorem titled "ULLN1".

First of all, I don't understand the first inequality:

$$\max_k \sup_\theta\left[ \frac1n\sum g(X_i, \theta)-Eg(X_i,\theta) \right] \leq \max_k\left[\frac1n\sum\sup_\theta g(X_i, \theta)-E\inf_\theta g(X_i,\theta)\right]$$

Is the idea to forget about the $\max$, and bound each $\sup$ separately? It seems like we're just saying $\sup (a-b)\leq \sup a - \inf b$, is that basically it?

Secondly, two lines down, I get that we're applying the standard LLN, but how do we pull the $o_P(1)$ out of the $\max$?

Finally, I'm not sure I really understand the overall structure of the proof. We basically want to prove $f_n\to f$ uniformly, but instead of trying to bound $\sup|f_n-f|$, it seems like we're separately bounding $\sup(f_n-f)$ and $\inf(f_n-f)$, is that right? Can the proof not be rewritten in a more standard way where we just bound an absolute value directly?

  • $\begingroup$ What does "this proof" refer to in the first line? Page 2 of what? Did you forget to stick in a URL? My understanding of the "ULLN" is that it is the LLN for vectors in the Banach space $C(\Theta)$: see math.stackexchange.com/questions/2469152. $\endgroup$ Dec 9, 2017 at 14:23
  • $\begingroup$ @kimchilover Yes, I forgot the link. $\endgroup$
    – Jack M
    Dec 9, 2017 at 14:35

1 Answer 1


To answer your last paragraph: yes, I think the proof sketch given in the answer to this question does describe the overall structure of the proof you are reading.

The rest of this answer is predicated on knowledge of that answer. I will keep it short (unusually short for me), just giving a hint.

A key step is covering the space of functions with a collection of finitely many compact sets of functions that catch most of the probability mass. Recall the Arzelà–Ascoli theorem, which describes relatively compact subsets of continuous functions: they have to bounded, and they have to be equicontinuous. That is, they cannot wiggle too much, oscillate too much. The mysterious sup - inf terms in the $\max_k$ expression you ask about has to do with bounding the oscillation of the summand functions over certain $\delta$-balls covering $\Theta$. The $\max$ picks the worst such $\delta$-ball, and guess what, it's tame enough, after all.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.