I came across the question: Evaluate$f(x)=\lim_{m \to \infty}\lim_{n \to \infty}[\cos(n!\pi x)]^{2m}$. I simplified this to: $$f(x)= \begin{cases} 1 & \text{if $x \in Q$} \\ 0 & \text{if $x\notin Q$} \end{cases}$$ Is this function continuous?
As far as I know, if the limit of the function at a point $x=a$ equals $f(a)$, then the function is continuous at $x=a$. In this case however, how does one decide whether the limit is 1, 0, or does not exist?
In the comments to your question, we have discussed that your first $f$ cannot be your second $f$. However, if we just look at $$ f(x)= \begin{cases} 1 & \text{if $x \in Q$} \\ 0 & \text{if $x\notin Q$} \end{cases}, $$ this is a well-defined function and we can think about whether it's continuous in some point $x$ or not.
First, assume that $x \in \mathbb Q$, so $f(x) = 1$. Then, for any $\epsilon>0$, you can find an irrational number $y$ with $|x-y| < \epsilon$ (for instance by choosing $y = x + \frac1n \sqrt2$ with a large enough natural number $n$). But then $|f(x) - f(y)| = 1$ even though $|x-y|$ can be arbitrarily small.
On the other hand, if $x \notin \mathbb Q$, we can use that the rational numbers are dense in the real numbers to find a $y \in \mathbb Q$ with $|x-y| < \epsilon$ for any small $\epsilon$ we like. Again, $|f(x) - f(y)| = 1$.
In both cases, if you refer to the $\epsilon$-$\delta$ definition of continuity, this shows that $f$ is not continuous in $x$. Thus, $f$ is a function which is continuous nowhere.
