$\delta$ and $\epsilon$ in the continuity definition Considering the following definition of continuity (there is nothing unusual yet here):
$$\forall \varepsilon > 0\ \exists \delta > 0\ \text{s.t. } 0 < |x - x_0| < \delta \implies |f(x) - f(x_0)| < \varepsilon $$
I always thought $\delta$, $\epsilon$ $\rightarrow$ 0. 
However I was told today (and my assignment wash downgraded accordingly) that $\delta$ and $\epsilon$ are not necessary infinitesimally small numbers.
Can someone please share an insight into this idea and provide a detailed explanation on how I can show that a piece wise continuous function contains discontinuities without resorting to infinitesimally small $\delta$ and $\epsilon$ in the above continuity definition. 
I am really struggling with this new removal of restriction on $\delta$ and $\epsilon$, so a really detailed explanation would be much appreciated.
 A: The point is that $\epsilon$ can be made arbitrarily small and this definition encapsulates that. But it still has to be a number. 
So if a function is discontinuous at a point $x$, then you can find an $\epsilon>0$ for which there is no $\delta>0$ that makes $|f(x)-f(x_0)|<\epsilon$ for all $|x-x_0|<\delta$. Consider for example the piecewise function $f(x)=0$ for $x\leq 0$ and $f(x)=1$ for $x>0$. There's clearly a discontinuity at 0 and it can be captured as follows: $f(0)=0$, and $|f(x)-f(0)|=|f(x)-0|$. For any $\delta>0$, the inequality $|x-0|<\delta$ is satisfied by $x=\delta/2$ (say), and $|f(\delta/2)-f(0)|=|1-0|=1$. So if you pick $\epsilon=1/2$, you will never find a $\delta>0$ such that $|f(x)-f(0)|<\epsilon$ whenever $|x-x_0|<\delta$. On the other hand, notice that $x=-\delta/2$ gives $|f(-\delta/2)-f(0)|=0$. So think of continuity as the ability to probe how much the function changes in very small neighborhoods around the point you are interested in. Continuity is scale invariant: no matter how much you zoom into the point that your function is continuous at, it will still look continuous. This is the "arbitrarily small" idea at work.
A: There's no such thing as an infinitesimally small number (not at this level, anyway). The definition of continuity says nothing about the magnitudes of $\epsilon$ and $\delta$, except that they are strictly positive.
Having said that, it might give you a better sense of the motivation behind the definition if you start it like this:
"For all $\epsilon > 0$ (no matter how small),..."
A: There are no "infinitesimally small numbers". There are positive real numbers and an order, and given two different numbers you can say which is smaller and which is bigger. And that's all.
Intuitively you can think something like: "for any $\epsilon>0$, no matter how small is", etc. But this is simply an aid to understanding. That comment, "no matter how small is", has no mathematical meaning at all.
But, why some people think things like $\epsilon$ is a very small, tiny, puny, fine number? Because if we prove that
$$|f(x)-f(x_0)|<\epsilon$$
for a small, tiny, puny, fine number $\epsilon$, then it is proven for not-so-small numbers.
But there is no magic in those $\epsilon$. They are positive numbers, like $3.8$, $75!$, $1$, $1/2$ or, $10^{-10^{10}}$.
A: Consider $f$ defined by $f(x) = 1$ when $x = 0$ and $f(x) = 0$ when $x \neq 0$.  Here is a demonstration that $f$ is not continuous at $x = 0$.  
If $x \neq 0$,
$$|f(x) - f(0)| = |0 - 1| = 1$$ 
So if we take $\epsilon = \frac{1}{2}$ we cannot find any $\delta > 0$ such that $|x - 0| < \delta$ will guarantee that $|f(x) - f(0)| \leq \frac{1}{2}$.
Discussion: Continuity requires that for any positive number $\epsilon$ your function can meet the definition - that there is a $\delta$ so that if $x$ is within $\delta$ of $a$, $f(x)$ is within $\epsilon$ of $f(a)$.  The opposite of something holding for every $\epsilon$ is that it does not hold for at least one particular $\epsilon$.
