# Understanding a proof of the Uniform LLN

I'm trying to understand this proof of the uniform law of large numbers. The proof I mean is on page 2, the theorem titled "ULLN1".

First of all, I don't understand the first inequality:

$$\max_k \sup_\theta\left[ \frac1n\sum g(X_i, \theta)-Eg(X_i,\theta) \right] \leq \max_k\left[\frac1n\sum\sup_\theta g(X_i, \theta)-E\inf_\theta g(X_i,\theta)\right]$$

Is the idea to forget about the $\max$, and bound each $\sup$ separately? It seems like we're just saying $\sup (a-b)\leq \sup a - \inf b$, is that basically it?

Secondly, two lines down, I get that we're applying the standard LLN, but how do we pull the $o_P(1)$ out of the $\max$?

Finally, I'm not sure I really understand the overall structure of the proof. We basically want to prove $f_n\to f$ uniformly, but instead of trying to bound $\sup|f_n-f|$, it seems like we're separately bounding $\sup(f_n-f)$ and $\inf(f_n-f)$, is that right? Can the proof not be rewritten in a more standard way where we just bound an absolute value directly?

• What does "this proof" refer to in the first line? Page 2 of what? Did you forget to stick in a URL? My understanding of the "ULLN" is that it is the LLN for vectors in the Banach space $C(\Theta)$: see math.stackexchange.com/questions/2469152. Dec 9, 2017 at 14:23
• @kimchilover Yes, I forgot the link. Dec 9, 2017 at 14:35

A key step is covering the space of functions with a collection of finitely many compact sets of functions that catch most of the probability mass. Recall the Arzelà–Ascoli theorem, which describes relatively compact subsets of continuous functions: they have to bounded, and they have to be equicontinuous. That is, they cannot wiggle too much, oscillate too much. The mysterious sup - inf terms in the $$\max_k$$ expression you ask about has to do with bounding the oscillation of the summand functions over certain $$\delta$$-balls covering $$\Theta$$. The $$\max$$ picks the worst such $$\delta$$-ball, and guess what, it's tame enough, after all.