Prove $\lim_{n\rightarrow \infty}\left(\sup_{\theta\in \Theta}\left| \frac{1}{n}\sum_{i=1}^n f(X_i,\theta)-E(f(X,\theta)) \right|\right)=0$ 
$\Theta\in\mathbb{R}^d$ is a compact set, $f(x,\theta):\mathbb{R}^p\times Y\in \mathbb{R}^+$ are continous functions in $\theta$ for every $x$. Let $X,X_1,\dots,X_n,\dots$ be i.i.d random vectors.
Then $\displaystyle E\left( \sup_{\theta\in \Theta}f(X,\theta)\right)<\infty\implies \lim_{n\rightarrow \infty}\left(\sup_{\theta\in \Theta}\left| \frac{1}{n}\sum_{i=1}^n f(X_i,\theta)-E(f(X,\theta)) \right|\right)=0$

I don't see an immediate way of doing this rather than working with the definition of limit. The idea is to show that there is $N$ such that $\sup_{\theta\in \Theta}\left| \frac{1}{n}\sum_{i=1}^n f(X_i,\theta)-E(f(X,\theta)) \right|<\epsilon$ for a given $\epsilon$.
I think the compactness of $\Theta$ might be used to argue that there is a $\theta_0$ for which $\sup_{\theta\in \Theta}f(X,\theta) = f(X,\theta_0)$. Then I would apply the same to the second equation $\left| \frac{1}{n}\sum_{i=1}^n f(X_i,\theta_0)-E(f(X,\theta_0)) \right|<\epsilon$.
But I find the $ \frac{1}{n}$ in $\displaystyle\frac{1}{n}\sum_{i=1}^n f(X_i,\theta_0)$ problematic because it will make the term smaller meanwhile $E(f(X,\theta_0))$ doesn't seem to decrease.
 A: This is an immediate consequence of the SLLN for separable Banach spaces (see here for the first publication and here for a dumbed down statement), in particular, for $S=C(\Theta),$ the space of continuous real-valued functions on $\Theta$, with the  topology induced by the sup norm $\|h\| =  \sup_{\theta\in\Theta}|h(\theta)|$.  Let $Y\in S$ be the function $\theta\mapsto f(X,\theta)$; you need to check that this defines $Y$ as a random element of $C(\Theta)$, and similarly for $Y_i:\theta\mapsto f(X_i,\theta)$.  Then you need to check that $E\|Y\| = E\sup_{\theta\in\Theta}|f(X,\theta)| \lt \infty$.  Let $m=EY:\theta\mapsto E(X,\theta)$.  Then the desired result follows directly from the SLLN: $\overline Y_n \to m$ almost surely, which is to say, $\|\overline Y_n-m\|\to0$ almost surely.
The $S=C(\Theta)$ SLLN is easy to prove, in principle.  Here is a sketch of one line of argument. Let $Y$ be a random vector in $S$, with distribution $\mu$, for which $\int _S\|Y\|\mu(dY)<\infty$.  Let $m=EY$. Define another measure on $S$ by $\nu(A)=\int_A\|Y\|\mu(dY).$ Note that $\nu$ is finite, and hence for any given real $\epsilon>0$ there exists a compact $K\subset S$ such that $\nu(A^c)<\epsilon$.  Cover $K$ with open $\epsilon$-balls, extract a finite sub-cover, and let $T(\epsilon)$ denote the centers of the balls in the finite sub-cover.  Now define  the random vector $T_i$ to be a closest element of $T(\epsilon)$ if $Y_i\in K$, and $0$ otherwise.  Note that $E\|Y_i - T_i\| <\epsilon$, observe that the $T_i$ obey the finite dimensional SLLN and so $\overline T_n\to ET_1$ a.s.  But $\|ET_1 -m \|<\epsilon$, so we have $\|\overline Y_n - m\|<2\epsilon$ for all $n$ sufficiently large.  Since $\epsilon>0$ was arbitrary we are done.  In particular, let $G(\epsilon)$ be the event that $\|\overline Y_n - m\|<2\epsilon$ holds for all $n$ sufficiently large. We have just seen that $P(G(\epsilon))=1$ for $\epsilon>0$, so $G = \bigcap_{k>0}G(1/k)$ also has probability $1$.  But conditional on $G$, we have $\overline Y_n \to m.$
