The epsilon delta definition requires there to be a delta for all epsilon but at https://openstax.org/books/calculus-volume-1/pages/2-5-the-precise-definition-of-a-limit example 2.41 it says:
Prove $\lim\limits_{x \to 2} x^2 = 4$
Without loss of generality, assume $\epsilon \leq 4$ (since $\delta \leq 2 - \sqrt{4 - \epsilon}$), this is allowed because if we can find $\delta>0$ that “works” for $\epsilon \leq 4$, then it will “work” for any $\epsilon>4$ as well. Keep in mind that, although it is always okay to put an upper bound on $\epsilon$, it is never okay to put a lower bound (other than zero) on $\epsilon$.
I don't understand why a delta for a restricted range of epsilon implies there exist a delta for all epsilon as said above.