How can I find in wich points $f(x)$ continuous? and how can I find in which points $f(x)$ differentiable? given $f: \mathbb{R} \to \mathbb{R}$ so that:
$$f(x) = \begin{cases}x & x  \in \mathbb{Q}  \\ ax(x-1) & x  \not \in \mathbb{Q} \end{cases}$$  
How can I find in which points $f(x)$ continuous? And how can I find in which points $f(x)$ is differentiable?
 A: Continuity of $f$
As the rationals and the irrationals are both dense in $\mathbb R$, and the functions $f_1: x \mapsto x$ and $f_2: x \mapsto ax(x-1)$ are both continuous real functions, $f$ is continuous at $x$ if and only if
$$x = ax(x-1)$$
If $a =0$, $0$ is the only solution of this equation.
And for $a \neq 0$, the solutions are $0$ and $a +\frac{1}{a}$.
Differentiability of $f$
As a map can only be differentiable where it is continuous, we are left to study differentiability at points find above.
If $a=0$, we have
$$f(x) = \begin{cases}x & x  \in \mathbb{Q}  \\ 0 & x  \not \in \mathbb{Q} \end{cases}$$ and for $ h \neq 0$
$$\frac{f(h)-f(0)}{h-0}= \begin{cases}1 & x  \in \mathbb{Q}  \\ 0 & x  \not \in \mathbb{Q} \end{cases}$$
hence $\lim\limits_{h \to 0} \frac{f(h)-f(0)}{h-0}$ can’t exist by unicity of a limit. Consequently $f$ is nowhere differentiable for $a=0$.
And for $a \neq 0$, we have $f(0) = 0$ and $f(a +\frac{1}{a})= a +\frac{1}{a}$.
According to a similar argument than the one used for continuity, $f$ is differentiable at $0$ if and only if$f_1^\prime(0)=f_2^\prime(0)$. And we have $f_1^\prime(0)=1$ , while $f_2^\prime(0)=-a$. $f$ is differentiable at $0$ if and only if $a=-1$.
Last thing to study is the differentiability of $f$ at $a +\frac{1}{a}$. We have $f_1^\prime(a +\frac{1}{a})=1$ and $f_2^\prime(a +\frac{1}{a})=2a^2-a+2$.Those two values are equal if and only if 
$$2a^2-a+1=0$$ which has no real solution.
Conclusion
If $a=0$, $f$ is only continuous at $0$ and is nowhere differentiable.
If $a = -1$, $ f$ is continuous at $0$ and $-2$ and differentiable only at $0$.
If $a \notin \{0,-1\}$, $ f$ is continuous at $0$ and $a +\frac{1}{a}$ and is nowhere differentiable.
A: $\forall x \in \mathbb{Q} \forall \epsilon >0 \exists y \notin \mathbb{Q}: |x-y|<\epsilon $ thats why $f(x)$ will be continuous only when $x=ax(x-1)$.
$$x_1=0, x_2=\frac{1}{a}+1$$
it means that $lim_{x \rightarrow 0}f(x)=0=f(0)$ and $lim_{x \rightarrow \frac{1}{a}+1}f(x)=\frac{1}{a}+1=f(\frac{1}{a}+1)$
A: The answer depends on the value of $a$. If $a\ne 0$ the function  $$f(x) = \begin{cases}x & x  \in \mathbb{Q}  \\ ax(x-1) & x  \not \in \mathbb{Q} \end{cases}$$is continuous at $x=0$ and at $x=1+\frac {1}{a} $       where the two definitions result in the same value. For $a=0$ the only point of continuity is $ x=0$.
We know that a function which is not continuous at a point does not have a derivative at that point. 
If $a=0$ we look at the only point of continuity of $f(x)$ namely $x=0$. The function does not have a derivative at $x=0$ because the difference quotient for rationals will approach $1$ while for irrationals it will approach $0$.
For $a\ne 0$, we have two points to look at .
For $x=0$, the  difference quotient approaches $1$ for rationals and approaches $a(2x-1)$ for irrationals. The results match only if $a=-1.$
For $x=1+\frac {1}{a}$, the  difference quotient approaches $1$ for rationals and approaches $a(2x-1)$ for irrationals. The results match again if $a=-1$ which takes us back to $x=0.$
