This is a soft question. I'm searching for examples of mathmatical statements (preferably in number theory, but other topics are also fine), that seem to be true, but are actually not. Statements where observing some examples would let you think it is always true, but then there is a well hidden counter-example.
If Riemann's $\zeta$ function had a zero beside the critical line, this would be such an example I'm looking for. Or if Fermat's Last Theorem would be false.
Do you know any such surprising counter-exmaples?