One thing that should be made clear here, for those of you who don't know (possibly even the OP), is a masters in actuarial science is not at all necessary. The whole purpose of such a degree is for someone who already has an undergrad degree but does not have the necessary background to pass the exams. A masters in actuarial science is basically equivalent to an undergrad degree in actuarial science at one of the top undergrad act sci schools. A masters will probably cover all the background needed for the first 4 or 5 exams.
But, you don't need to pass 4 or 5 exams to get a job. You need to pass 1 or 2, or sometimes 0 is enough. And, Exam FM isn't all that heavy on math. If you could pass that one exam on your own, you would show companies that you are interested in the field and are capable of passing exams and you could get a job somewhere (assuming you have other necessary skills). And, then you would get paid study time to study for the other exams. This is another viable option. A masters in actuarial science will cost you a lot of money but help you pass exams faster.
I googled "masters in actuarial science" and Boston University's program was first. The prereqs for admission are just 3 semesters of calculus. That's it. I have no idea how hard it is to get in to such a program and I have no idea if you would get in. But, you can self study most of Calc 1 through 3 by yourself and be plenty prepared to start such a program. If you think this is too hard, then go take these classes at a community college. I must disagree with Mark Higgins. If you have taken "advanced calculus", then I believe you could relearn the basics of 3 semesters of calculus. When I say basics, I mean basics because you don't need to know Stoke's theorem or how to do surface integrals or anything.
Just to be clear, I'm currently teaching an Exam FM test prep course and the math we have used in it are some derivatives and integrals, but no trig has ever shown up. We have dealt with geometric sums (finite and infinite). I did integration by parts once (literally, once) in class. That's about it. When dealing with probability and statistics, you will at some point do multiple integration and things like that.
Just to be clear, getting a degree in that field without becoming an actuary sounds like a huge waste of time. I'm guessing it'd cost you around \$60,000 to \$80,000 in tuition, over 2 years, to get such a degree, not to mention you probably won't be working so you lose any money you would be making otherwise. That's a total of \$150,000 to \$200,000 you lose to get a degree that you don't even know if you want to use or not. Can it be interesting stuff to learn, and could it be worthwhile to know even if you don't become an actuary? Sure, but not for that much money. Buy the books and read them yourself.
You need to make up your mind if you really want to do this before you make any decisions like this. Companies won't hire people who aren't really sure if they want to be an actuary or not. They have plenty of good candidates who are sure.