There aren't anywhere nearly as many good mathematical introductions on statistics as there are on it's parent subject probability. It's a hard subject to clearly write about because there's no generally accepted manner of approaching the subject. About all the experts agree on about it is that it involves using probability distribution functions to test hypotheses.
Trust me-my teacher,Stefan Ralescu,is an expert.
My favorite book on statistics is George Casella and Robert Berger's Statistical Inference. Not only is it wonderfully clear and rigorous, requiring only a good understanding of one and several variable calculus, it covers an enormous number of examples. Sadly,most books don't do this and it's really important with a subject this unusual. It's a bit pricey, but well worth the cost.