Mathematics Stack Exchange is a question and answer site for people studying math at any level and professionals in related fields. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

Is there a mathematically rigorous book giving an introduction to boosting, etc. A book that is rigorous like "A Course in Probability Theory" by Kai Lai Chung.

share|cite|improve this question
up vote 3 down vote accepted

Along with the text that @William proposed (it's a great reference), for bootstrapping it's hard to beat:

Efron 1987, The Jackknife, the Bootstrap, and Other Resampling Plans.

Efron and Tibshirani 1994, An Introduction to the Bootstrap.

Hall 1995, The Bootstrap and Edgeworth Expansion is more rigorous than the above. Upon a cursory glance, it appears to dive pretty deep into the theoretical details of the bootstrap.

Specifically for boosting, Robert Schapire has a list of references to read found here. And for bagging, Martin Sewell has a list of references here. His site has a bunch of reference lists for a number of machine learning topics (see here).

share|cite|improve this answer
Could you elaborate on which of these many sources are mathematically rigorous? I don't like it when I read about some results and don't understand every little detail about why it's true. – user782220 Mar 6 '12 at 2:13
@user782220 For the boosting list, I'd start with the 'overview' papers he lists. Schapire's papers seem to be on the more rigorous end of the spectrum. For bagging, Breiman's papers are the place to start. Also, the bagging list has a few Annals of Statistics papers. These are (obviously) from a statistical point of view, but are quite 'rigorous.' As for Efron's books, I don't know how 'rigorous' you're hoping to get, but you really can't go wrong with the 1994 text (IMO). Another good bootstrap reference is Davison & Hinkley. – Mike Wierzbicki Mar 6 '12 at 6:52
@user782220 If you're still not satisfied, the Stats SE has a good number of machine learning folks who may offer a different perspective. – Mike Wierzbicki Mar 6 '12 at 7:02
@user782220 Upon more searching, I would surmise that Peter Hall's bootstrap book is probably the most rigorous one out there. – Mike Wierzbicki Mar 6 '12 at 15:54

Elements of Statistical Learning is available online.

There is a chapter on Boosting (chapter 10).

Does it satisfies your definition of rigorous? If by "rigorous" you mean "measure theoretic," then no.

share|cite|improve this answer

Shapire's Boosting: Foundations and Algorithms is IMHO a very didatic and rigorous book about the subject.

share|cite|improve this answer

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.