If you've seen a bit of set theory (presumably in calculus) and done linear algebra - then you should be prepared for an introductory course in group theory.
The only way to develop skills with writing proofs is by experience. Abstract algebra is filled with enough interesting proofs to give you good examples for building intuition and plenty of challenging problems to keep you motivated.
You might consider seeing if there are any courses in the computer science department you could take concurrently, since it is typical for introductory level courses on data structures and algorithms to cover various proof techniques and strategies quite thoroughly. This is usually taught in conjunction with first order propositional logic and basic set theory.
If there isn't such a course, or if you cannot take one for some reason, then I recommend finding a copy of The Art of Computer Programming by Donald Knuth - which will help build the intuition needed for really thinking about proofs.
I recommend looking into computer science a bit because proving the correctness of an algorithm (and analyzing its complexity) requires the same level of rigor and employs the same strategies as the proofs in pure mathematics - but when proving an algorithm, you have something concrete to work with, whereas this is often not the case when proving a theorem.
Also, knowing how to prove the Euclidean algorithm and picking up some number theory from Knuth would certainly put you at an advantage in abstract algebra.