Understanding a proof of the Uniform LLN

Question

I'm trying to understand this proof of the uniform law of large numbers. The proof I mean is on page 2, the theorem titled "ULLN1".

First of all, I don't understand the first inequality:

$$\max_k \sup_\theta\left[ \frac1n\sum g(X_i, \theta)-Eg(X_i,\theta) \right] \leq \max_k\left[\frac1n\sum\sup_\theta g(X_i, \theta)-E\inf_\theta g(X_i,\theta)\right]$$

Is the idea to forget about the $\max$, and bound each $\sup$ separately? It seems like we're just saying $\sup (a-b)\leq \sup a - \inf b$, is that basically it?

Secondly, two lines down, I get that we're applying the standard LLN, but how do we pull the $o_P(1)$ out of the $\max$?

Finally, I'm not sure I really understand the overall structure of the proof. We basically want to prove $f_n\to f$ uniformly, but instead of trying to bound $\sup|f_n-f|$, it seems like we're separately bounding $\sup(f_n-f)$ and $\inf(f_n-f)$, is that right? Can the proof not be rewritten in a more standard way where we just bound an absolute value directly?

Answer 1

To answer your last paragraph: yes, I think the proof sketch given in the answer to this question does describe the overall structure of the proof you are reading.

The rest of this answer is predicated on knowledge of that answer. I will keep it short (unusually short for me), just giving a hint.

A key step is covering the space of functions with a collection of finitely many compact sets of functions that catch most of the probability mass. Recall the Arzelà–Ascoli theorem, which describes relatively compact subsets of continuous functions: they have to bounded, and they have to be equicontinuous. That is, they cannot wiggle too much, oscillate too much. The mysterious sup - inf terms in the $\max_k$ expression you ask about has to do with bounding the oscillation of the summand functions over certain $\delta$-balls covering $\Theta$. The $\max$ picks the worst such $\delta$-ball, and guess what, it's tame enough, after all.