I am searching for some good book which section is devoted to modular arithmetic. I am self learner so I strongly prefer that book has exercises best with answers or solutions. I have CS background and has taken course on discrete mathematics but besides some basic facts on modulo operation it lacked some introduction to modular arithmetic.
You might want to check out the Khan Academy on modular arithmetic. (Scroll down to "modular arithmetic"). There's a substantial series of tutorials there; you'll also be able to find practice problems at the Academy.
May I ask what your goals are? Modular arithmetic gets fun when you get to deal with symmetries; the study of symmetry in this context is called group theory. From there you can look at such things as public key cryptography. There is also quite a lot of fun to be had just working with mod 2. If you want to look at such things as Galois shift registers, which you may know as CRC32, that's another direction you can go in. For an entry into the world of finite mathematics I like Peter Cameron's book on combinatorics. It has lots of games ad fun situations to examine. You may want to start with something simpler, such as any introductory level book on group theory. That always deals with modular arithmetic.
Concepts of Modern Mathematics by Ian Stewart is a good introduction to many pure mathematical concepts, including modular arithmetic.