Short version: Anyone have any favorite examples of "mathematically-motivated statistics," and possibly a textbook that would appeal to me?
I have a pretty reasonable mathematics background, specifically algebra. I've taken a few statistics courses, and generally found them bland, or unmotivated: the kind of stuff you would ask a computer to do because the computer doesn't care why it was asked to compute something, or how the original idea for the algorithm came into being.
I did, however, have one highlight from a recent stats course (at the senior undergrad level). We were discussing linear regression, and were told about Pearson's regression coefficient.
Having sufficient mathematical training, I realized that this coefficient was quite literally the cosine of the angle between two lines in some $n$-dimensional space (of course, since the class was for "applied" folks, this went unsaid). This was the first time that any topic in stats really grabbed my attention and spoke to my need for mathematical motivation. It then, in my eyes, became a valid technique for measuring how related two data sets are.
The longer version of the question would be as follows. Does anyone have any nice examples of a statistics topic, that may appear quite mysterious, has a nice, simple, mathematical explanation/motivation? Are there stats books that appeal to someone with my tastes (i.e., emphasizing why such-and-such a procedure was cooked up in the first place) or am I just reading stats books the wrong way, and all will be addressed with a nice theoretical stats book? If the latter is the case, what's a good choice?
I've stumbled upon two books by David Williams (Weighing the Odds and Probability with Martingales), but they're clearly probability-oriented and probably fairly limited in scope. I'm applying for a job where I think I'll need to use a good bit of stats, and wouldn't mind trying to (re)learn my way, in the hopes that I'll remember the material a bit better than algorithm memorization.
I realize reference requests have been made before. But hopefully asking for fun examples (Pearson's coefficient, in my case) will save this question from being a duplicate!