I have a background in theoretical physics, but no rigorous training in math besides the introductory coursework I took in college a while ago. I would like to become more proficient in math, though -- mostly for fun. I thought I'd start with real analysis because it seems so central. How accessible is the "baby Rudin" book, and would that be a good place to start? How would you recommend going about this? I should say I don't know much about mathematical formalism (mathematical logic, set theory, proofs, and so forth) but I'm very eager to learn. Life is short, and I want to develop this interest!
The process you should take depends on a few things. What's your background with proofs? If you're not very experienced in reading and writing proofs, starting with baby Rudin will be a bit difficult - you might be better served starting with an easier intro analysis text like Abbott or Ross. If you're comfortable with proofs, you could start with Rudin and see how it goes (but expect it to be difficult regardless). You should try to as many exercises as you can, and if at all possible find people to discuss with; analysis is difficult, and it helps to have somebody to run your proofs by and who you can talk with about the material.
With a background in theoretical physics, after learning some real and complex analysis you might be interested in learning more about differential geometry and functional analysis. Functional analysis helps to make rigorous a lot of quantum mechanics, and similarly the language of differential geometry comes up quite often in classical mechanics and relativity. Rudin has a good text on functional analysis as well, and I quite like John Lee's series on manifolds. With a bit of differential geometry behind you, you might also appreciate Spivak's text "Physics for Mathematicians: Mechanics I".
If you're just looking to learn math for its own sake, you need not limit yourself to analysis as a starting point! You can get into abstract algebra, elementary number theory, and logic/set theory without a background in analysis - the only limiting factor, again, is your comfort with proofs. (If you find yourself wanting to learn one of those topics instead I'd be happy to give more book recommendations.)
EDIT: In terms of starting out with proofs, Rosen's book "Discrete Mathematics" will give you the language you need to get started in terms of basic logic and set theory. You could also jump straight into analysis with Abbott - in my opinion it's friendly enough for beginners that you could learn proofs as you go.
For abstract algebra, Dummit & Foote's algebra text is terrific but definitely requires some fluency in reading and writing proofs (and should definitely be supplemented with another algebra text for a first read). After getting down the fundamentals of proof-writing you could start on that book; others to supplement with could be Carter's "Visual Group Theory", Artin, or Fraleigh (my undergrad course in algebra used Fraleigh, but it felt a little too gentle for my taste - but especially for a first introduction to advanced math, it might be worth checking out!).