Short version: Anyone have any favorite examples of "mathematically-motivated statistics," and possibly a textbook that would appeal to me?

Long version:

I have a pretty reasonable mathematics background, specifically algebra. I've taken a few statistics courses, and generally found them bland, or unmotivated: the kind of stuff you would ask a computer to do because the computer doesn't care why it was asked to compute something, or how the original idea for the algorithm came into being.

I did, however, have one highlight from a recent stats course (at the senior undergrad level). We were discussing linear regression, and were told about Pearson's regression coefficient.

Having sufficient mathematical training, I realized that this coefficient was quite literally the cosine of the angle between two lines in some $n$-dimensional space (of course, since the class was for "applied" folks, this went unsaid). This was the first time that any topic in stats really grabbed my attention and spoke to my need for mathematical motivation. It then, in my eyes, became a valid technique for measuring how related two data sets are.

The longer version of the question would be as follows. Does anyone have any nice examples of a statistics topic, that may appear quite mysterious, has a nice, simple, mathematical explanation/motivation? Are there stats books that appeal to someone with my tastes (i.e., emphasizing why such-and-such a procedure was cooked up in the first place) or am I just reading stats books the wrong way, and all will be addressed with a nice theoretical stats book? If the latter is the case, what's a good choice?

I've stumbled upon two books by David Williams (Weighing the Odds and Probability with Martingales), but they're clearly probability-oriented and probably fairly limited in scope. I'm applying for a job where I think I'll need to use a good bit of stats, and wouldn't mind trying to (re)learn my way, in the hopes that I'll remember the material a bit better than algorithm memorization.

I realize reference requests have been made before. But hopefully asking for fun examples (Pearson's coefficient, in my case) will save this question from being a duplicate!

  • $\begingroup$ You could try Stigler's Statistics on the Table for a history of statistics. Or if you like finance, there are plenty of finance books combining story telling with some statistics. I liked Bernstein's Capital Ideas. $\endgroup$
    – TooTone
    Jan 25 '14 at 18:46
  • $\begingroup$ Thanks @TooTone, I do have "statistics for finance" on the mind. I don't have a particularly strong background in probability or statistics so I will certainly look into this! $\endgroup$
    – pjs36
    Jan 25 '14 at 19:06

Since you mention linear regression: A good mathematically-inclined introduction is George Seber & Alan Lee: Linear Regression Analysis, second edition (Wiley). This will teach you that linear regression can be seen as a projection operator, and a good deal of understanding come from that viewpoint.

Then you can look into modern multivariate analysis, which can be seen as applied linear algebra (and n-dimensional geometry). There are many good books, one classical is Mardia, Kent & Bibby: Multivariate analysis, one more modern is Gower & Hand: Biplots.

  • $\begingroup$ Thank you! I do love $n$-dimensional geometry as my example may have led you to believe. These both seem like great resources that I'll certainly look into. $\endgroup$
    – pjs36
    Jan 25 '14 at 19:09

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.