From my limited experience, it seems like many people loose their ability to do contest-like math as they go trough college. For example, before I posted it here I asked a math undergrad student (who was about to finish) this and similar questions, and he went off in multiple completely different directions and still didn't solve it. So how do I make sure I still retain my very small knowledge of this kind of math? I have a feeling that after years and years of very formal, advanced, and abstract math I will loose some of my ability and just learn math by memorizing certain approaches and such.
I'm midway through my degree at the moment, and I am suffering from the same problem. I found that the best way is the simplest and hardest way, practice!
My friend suggested The Art and Craft of Problem Solving to me by Paul Zeitz. It's a great problem book and there are plenty more out there. Anything by Titu Andreescu is great, as is Problem Solving Strategies by Arthur Engel. Geometry Revisited by Coxeter and Greitzer is always a hit.
There are two keys to using any of these books I find:
1) Pick one, and only one, and focus on it. If you don't like it, move on but try and give it a fair chance.
2) Be hard on yourself. Try as hard as you can at a problem without looking at the answers. This might involve leaving a problem simmer for days (if not weeks) but it will be worth it in the end. When I started problem solving again, it felt completely different from hopping back on a bike; I most certainly did forget! So be patient but stick at it.
I have a feeling that after years and years of very formal, advanced, and abstract math I will loose some of my ability and just learn math by memorizing certain approaches and such.
Er, if this is the approach you take to advanced mathematics, you will not get very far.
I should think the opposite would happen, that as you learn advanced mathematics, you will get better at seeing imaginative ways to surmount problems.
I'm not sure exactly what aspect of such competitions you are most interested in anyway. Most people who enjoy problem solving enjoy doing so at their own leisure and in whatever way they want to do it socially. I don't know what the contest circuit has to offer other than time limits and prize money (and some sort of fame? I don't see it really.) These contests sound more like some sort of "blitz chess" for the problem solving world.
Do the contest problems every year. Most appear online. But the sharpness required for contests gets applied to more advanced problems and new topics as life goes on.
The other way is to teach students preparing for contests.
Practice, practice, practice.
I have encountered this situation before, and found an effective way to keep the skills current and reasonably fresh was to buy/access books, websites etc about that particularly type of maths and practice it regularly.
Also, a technique (not always possible) is a bit of problem solving, is to link those skills with new ones you are learning.