Continuity of sup-norm for random functions I have a problem when reading through Theoretical Statistics by Robert. W. Kenner, Theorem 9.1 in Chapter 9, pp. 152-153. It is about the continuity of random functions.
By random functions, they are defined as:

Let $X_1, X_2, \dots$ i.i.d., and $K$ be a compact set in $\mathbb{R}^p$. Define
$$
  W_i(t) = h(t, X_i), \quad t \in K,
$$
where $h$ is a continuous function of $t$ for all $x$. Then $W_1, W_2, \dots$ are i.i.d. random functions in $C(K)$, the space of continuous functions on $K$.

In my question, the randomness should not be so important, and I think it is safe to treat them as continuous functions on $C(K)$.
The theorem states that:

Theorem 9.1 Let $W$ be a random function in $C(k)$ and define
$$
  \mu(t) = \mathbb{E}W(t), \quad t \in K
$$
($\mathbb{E}$ is the expectation). If $\lvert \lvert W \rvert \rvert _\infty < \infty$ (the sup norm of $W$), then $\mu(t)$ is continuous. Also
$$
  \sup _{t \in K} \: \{ \mathbb{E} \sup _{s: \lvert \lvert s - t \rvert \rvert < \epsilon}
  |W(s) - W(t)| \} \to 0 \quad \text{as } \epsilon \to 0.
$$

The part that I am struggling with is proof on the second part,

Define
$$
  M _\epsilon (t) = \sup _{s: \lvert \lvert s - t \rvert \rvert < \epsilon} |W(s) - W(t)|,
$$
... . $\color{red}{\text{Because $W$ is continuous, $M_\epsilon$ is continuous}}.$

I cannot figure out why this claim is true. If I write it out,
$$
  |M_\epsilon(t_1) - M_\epsilon(t_2)|
  = \left| 
      \sup _{s_1: \lvert \lvert s_1 - t_1 \rvert \rvert < \epsilon} |W(s_1) - W(t_1)|
      - \sup _{s_2: \lvert \lvert s_2 - t_2 \rvert \rvert < \epsilon} |W(s_2) - W(t_2)|
    \right|.
$$
Usually the ineqaulity $\left| |x| - |y| \right| \leq |x - y|$ will be applied, but in this case there are two balls with different centers. Even though one can control the distance between $t_1$ and $t_2$, the distance between $s_1$ and $s_2$ cannot since it depends on $\epsilon$ which is a parameter of the function $M$.
I also tried considering only small $\epsilon$'s. A brief outline is suppose continuity of $W$ means for any $\gamma > 0$ there is $\delta > 0$ such that
$$
  \lvert \lvert t_1 - t_2 \rvert \rvert < \delta
  \implies |W(t_1) - W(t_2)| < \frac{\gamma}{2},
$$
then for $\epsilon < \delta$,
$$
  |M_\epsilon(t_1) - M_\epsilon(t_2)| \leq \gamma.
$$
However, the choice of $\epsilon$ depends on $\gamma$, and to show continuity we need $\epsilon$ to be independent of $\gamma$.
Please share your thoughts about this problem. Any help is appreciated.

Edit: After reading through the comments from @Porufes, I come up with the following incomplete argument. The problem is simplified to 1D case, i.e. to show that

If $f$ is a continuous function on $K$, where $K \subset R$ is compact, then
$$
  d_\epsilon (t) = \sup _{x \in (t-\epsilon, t+\epsilon)} |f(x) - f(t)|
$$
is continuous on $K$.

My argument: Let $\epsilon > 0$ be fixed. Suppose $d_\epsilon$ is not continuous at $x \in K$, i.e. there is $\gamma_0 > 0$, such that for all $\delta > 0$, there is $y$ which $|x - y| < \delta$ and $|d_\epsilon(x) - d_\epsilon(y)| > \gamma_0$.
We first pick $\delta > 0$ according to the uniform continuity of $f$, such that
$$
  x, y \in K, \: |x - y| < \delta \implies |f(x) - f(y)| < \frac{\gamma_0}{3}.
$$
Let $\gamma > 0$ and $\epsilon > 0$ be fixed, and find $\delta > 0$ according to the uniform continuity of $f$, such that
$$
  x, y \in K, \: |x - y| < \delta \implies |f(x) - f(y)| < \gamma.
$$
For any $y$ satisfying the requirements, the compactness of $K$ and continuity of $f$ implies the existence of
$$
\begin{align*}
  p \in [x - \epsilon, x + \epsilon] \cap K, &\quad |f(x) - f(p)| = d_\epsilon(x), \\
  q \in [y - \epsilon, y + \epsilon] \cap K, &\quad |f(y) - f(q)| = d_\epsilon(y).
\end{align*}
$$
The discontinuity at $x$ can be considered as two separate cases.
Case 1: $d_\epsilon (x) \geq \gamma_0 + d_\epsilon (y)$.
By the choice of $\delta$,
$$
\begin{align*}
  \big||f(x) - f(p)| - |f(y) - f(p)|\big| &\leq |f(x) - f(y)| 
  < \frac{\gamma_0}{3} < \gamma_0 ,
\\
  \implies |f(y) - f(p)| &> |f(x) - f(p)| - \gamma_0 \\
  &= d_\epsilon(x) - \gamma_0 \\
  &\geq d_\epsilon(y).
\end{align*}
$$
Because $d_\epsilon(y)$ is the supremum over $I = (y - \epsilon, y + \epsilon) \cap K$, the above implies that $p \not \in I$, or $p \in [x - \epsilon, y - \epsilon) \cap K$. $\color{red}{\text{Suppose that there is}}$
$$
  \color{red}
  {
    z \in (p, p + \delta) \cap (y - \epsilon, y + \epsilon) \cap K, 
  }
$$
then since $|z - p| < \delta$,
$$
  |f(p) - f(z) | < \frac{\gamma_0}{3}.
$$
Moreover, as $z \in (y - \epsilon, y + \epsilon)$ and $|f(y) - f(t)|$ attains maximum at $t = q$,
$$
  |f(y) - f(p)| \geq |f(y) - f(q)| \geq |f(y) - f(z)|.
$$
Therefore
$$
\begin{align*}
  | f(q) - f(y) | - | f(z) - f(y) |
  &\leq |f(p) - f(y)| - |f(z) - f(y)| \\
  &\leq |f(p) - f(z)| \\
  &< \frac{\gamma_0}{3}.
\end{align*}
$$
Hence
$$
\begin{align*}
  |d_\epsilon(y) - d_\epsilon(x)|
  &= \big| |f(x) - f(p)| - |f(y) - f(q)| \big| \\
  &\leq |f(x) - f(y) + f(p) - f(z) + f(z) - f(q)| \\
  &\leq |f(x) - f(y)| + |f(p) - f(z)| + |f(z) - f(q)| \\
  &< \frac{\gamma_0}{3} + \frac{\gamma_0}{3} + \frac{\gamma_0}{3} \\
  &= \gamma_0.
\end{align*}
$$
This contradicts with $d_\epsilon (x) \geq \gamma_0 + d_\epsilon (y)$, thus the assumption does not hold.
For the second case $d_\epsilon (x) \leq d_\epsilon (y) - \gamma_0$,
I have not worked out the detail but I assume it is similar to the first case.
The problem of this argument is that the $\color{red}{\text{line in red}}$ may not hold if $K$ is not an interval, since such $z$ may not exists.
Another insight I got is that the size of $\epsilon$ cannot be too large. For instance, if $K = [0, 1] \cup [2, 3]$, and
$$
  f = \begin{cases}
    1 & \text{if } x \in [0, 1], \\
    0 & \text{if } x \in [2, 3].
  \end{cases}
$$
and $\epsilon = 1.5$, then
$$
  d _{1.5}(x) = 
  \begin{cases}
    0 & \text{if } x \in [0, 0.5] \cup (2.5, 3], \\
    1 & \text{if } x \in (0.5, 1] \cup [2, 2.5],
  \end{cases}
$$
which is not continuous on $K$. Thus it is needed to limit $\epsilon$ so that it splits $K$ into, roughly speaking, "connected components" so that $f$ is "smooth enough".
Here are my thoughts after thinking about this for a week. Again, any help is appreciated.
 A: Assume $W$ continuous $K\to R$ and first assume $K$ convex. Let $M_\epsilon(t) = \sup_{s:|t-s|\le \epsilon} |W(t)-W(s)|$. Since the intersection of $K$ with the closed ball around $t$ is compact, there is $s_t$ such that $M_\epsilon(t)=|W(t)-W(s_t)|$. Now for $x,t$,
$$
M_\epsilon(t) - M_\epsilon(x)
\le |W(t)-W(s_t)| - |W(x)-W(s_x)| \le |W(t)-W(s_t)|-|W(x)-W(S)|
$$
where we define $S$ to be $S=P_K(x+(s_t-t))$ where $P_K$ is the convex projection onto $K$. By the triangle inequality,
$$
M_\epsilon(t) - M_\epsilon(x)
\le |W(t)-W(x) + W(S)-W(s_t)|\le |W(s_t)-W(S)| + |W(t)-W(x)|.
$$
By Heine's theorem, for any $e>0$ there exists $\delta_e$ such that $|a-b|\le \delta \Rightarrow |W(a)-W(b)|\le e$.
If $|t-x|\le \delta$ then $|W(t)-W(x)|$ and (drawing a parallelogram picture helps) we have because $P_K$ is 1-Lipschitz
$$
|S-s_t| = |P_K(x+(s_t-t)) - O_K(s_t)| \le |x+(s_t-t)-s_t|=|x-t|\le \delta
$$
and $|W(s_t)-W(S)|\le e$ as well.

Now assume that for any $\bar K$ convex compact and any continuous $\bar h:K\times R\to R$, using what we did above thanks to convexity of $\bar K$, we can prove that
$$
\sup_{t\in \bar K} E[\sup_{s\in \bar K: |s-t|\le \epsilon} |\bar W(s)-\bar W(t)|] \to 0
$$
as $\epsilon\to 0$, where $\bar W_i(u) = h(u,X_i)$.
If $K$ is compact but not convex and $h:K\times R\to R$ is continuous, we can always find an extension $\bar h:R^p\times R\to R$ that is continuous and is an extension, in the sense $\bar h(t,x)= h(t,x)$ for all $t\in K, x\in R$ (see https://en.wikipedia.org/wiki/Tietze_extension_theorem). Now choose $\bar K$ to be a large enough compact and convex set (for instance a large Euclidean ball) such that $K\subset \bar K$. If thanks to convexity you are able to prove
$$
\sup_{t\in \bar K} E[\sup_{s\in \bar K: |s-t|\le \epsilon} |\bar W(s)-\bar W(t)|] \to 0,
$$
this implies
$\sup_{t\in K} E[\sup_{s\in K: |s-t|\le \epsilon} |W(s)-W(t)|]\to 0$ as well as desired, because the suprema are taken over smaller sets and $W(s)=\bar W(s),\bar W(t)=W(t)$ for $s,t\in K$.
A: $W$ is a continuous function on a compact set $K\subset \mathbb{R}^p$. By the Heine–Cantor theorem, it is uniformly continuous. Let $\epsilon'$ be given. Then choose $\epsilon'' < \frac{\epsilon'}{2}$, for which $\exists \delta:$
$$(\forall s,t \in K)\left(|s - t| < \delta \implies |W(s) - W(t)| < \epsilon'' < \frac{\epsilon'}{2}\right)\\
\implies (\forall s,t \in K) \left(\sup_{s:|s-t|<\delta} |W(s) - W(t)| = \epsilon'' <\frac{\epsilon'}{2}\right)$$
Therefore $(\forall x, t \in K)$ we have:
$$|x - t| < \delta=\epsilon \implies |M_\epsilon(x) - M_\epsilon(t)|\\
=\left| \sup_{s:||s-x|| < \epsilon} |W_1(s) - W_1(x)| - \sup_{s:||s-t|| < \epsilon} |W_1(s) - W_1(t)|\right| \\
\left| \sup_{s:||s-x|| < \delta} |W_1(s) - W_1(x)| - \sup_{s:||s-t|| < \delta} |W_1(s) - W_1(t)|\right| \\
\leq 2\sup_{s:||s-x|| < \delta} |W_1(s) - W_1(x)| \\
< 2\frac{\epsilon'}{2}= \epsilon'$$
Which means
$$(\forall x, t \in K) \left( |x - t| < \delta = \implies |M_\epsilon(x) - M_\epsilon(t)| < \epsilon'\right)$$
Which proves that $M_\epsilon$ is uniformly continuous on $K$ and uniform continuity $\implies$ continuity so $M_\epsilon$ is continuous on $K$.
