Most mathematics (specially in statistics) books just mention the formula, and they don't mention the derivation of the formulas. Is there any book which has the derivation of the formulas?
I recommend Mathematical Statistics, A Unified Introduction written by George R. Terrell. As the title suggest, the author never separates statistics from abstract models in probability theory.
- Statistics proceeds and motivates concepts in probability theory.
- Many large-sample results accessible to undergraduates.
- Since math and statistics are treated in a unified manner, all results are proven.
However, after reading this introductory book, if you want to learn more advanced statistics, you'll probably need to learn measure theory. I recommend A User's Guide to Measure Theoretic Probability by David Pollard since it's also a book in which motivation precedes abstraction.
What you want is a book on the subject of mathematical statistics, of which there are many. The book that was used in the course I took was "Introduction to Mathematical Statistics" by Hogg and Craig. I think it's a good book, but I don't have enough experience with other texts to make comparisons. Some math stat books use measure theory, others do not. Hogg and Craig do not, which is probably right for a beginner.