First, I would recommend learning some formal logic and proof theory directly. In particular, you'd want a text that focuses on proving and not on semantics or metatheory. It should use something like Natural Deduction or the Fitch system, and not a Hilbert-style approach. You don't have to go very deep or spend a lot of time on this. You're looking at it as a tool, not as a field of study.
Second, you should rebuild your understanding of the field, abstract algebra in this case, from the ground up. Go back to some introductory book and read through it again. Only this time, every time you get to a proposition or a theorem, prove it yourself before looking at the proof in the book (if any is given). Prove everything. You should be able to skim past most of the text, so the "reading" shouldn't take very long, but, on the other hand, thinking up your own proof of a statement is a lot more challenging and time-consuming.
Typically, the proof you come up with will be similar to the one in the book. If it is similar, but the book's proof is simpler than yours then this is an opportunity to refine your proof writing skills. If it is similar, but the book's proof is more complicated, this often means that you've missed some detail or are making unwarranted assumptions. If it is different, this could be due to several reasons: 1) your proof could be wrong, 2) the book proof restricts itself to an "elementary" approach while you do not, 3) your proof is "right" but circular (e.g. using the fundamental theorem of finite abelian groups to prove a result that will later be used to prove the fundamental theorem of abelian groups); you're not "solving problems", you're rebuilding, 4) the book's prof is wrong, 5) your proof is just different. Differentiating case 1 from case 5 is where studying proof theory and such comes in. You need to be able to tell when your proofs are wrong or right. When you're uncertain about a proof, make it more formal. In fact, you should try to be as formal as is tolerable during this rebuilding.
You probably don't want to do this quite so slavishly as I've presented it. First, for the "heavy-duty" theorems, you may want to flesh out a proof sketch rather than formulate your own proof from scratch. Second, you should feel free to bring in your understanding from other fields and prove things your own way. Third, and related to the previous, reorganize the presentation to suit your tastes, don't just follow the book's presentation. That said, if you are not confident in the correctness of your own proofs, it can still be worthwhile to prove the statements in the book in the style/way the book expects so you can compare apples to apples. Feel free to prove the statements in multiple ways. Often seemingly different proofs are different ways of packaging the same idea and you should get used to translating between the different views.
Obviously, none of this is specific to abstract algebra. This "rebuilding" process is something you should do multiple times throughout your career. (Often it is a way to refresh your knowledge of a field that you've been away from for a while.) Later "rebuildings" will likely rely much less on books. Instead, you'll be more interested in integrating it with knowledge you've acquired in the meantime or fitting it into higher-level frameworks.