What is a good book on the history of Fourier Analysis? I'm looking for a book which explains how it came to be and what the mathematicians (or physicists) were thinking when they came up with it. If the author has an engaging style, like the Richard Rhodes's "The Making of the Atomic Bomb", that would be great.
You might like The Evolution of Applied Harmonic Analysis: Models of the Real World by Prestini.
An old work on Fourier series is William Elwood Byerly's 1893 An Elementary Treatise on Fourier’s Series (cf. esp. its short Chapter 9, "Historical Summary").
Also, Fourier series date at least as far back as Ptolemy's epicyclic astronomy. Adding more eccentrics and epicycles, akin to adding more terms to a Fourier series, one can account for any continuous motion of an object in the sky. For how epicyclic astronomy can be re-expressed in the modern mathematical idiom of complex Fourier series, see this classic article:
David Bressoud's A Radical Approach To Real Analysis contains a wealth of information about the early breakthroughs in Fourier analysis and critical role it played in the rigorous formulation of the real numbers and calculus. T.W.Korner's Fourier Analysis also contains some very nice and informative historical notes about the early results.More important as a source is Norbert Wiener's classic article, The Historical Background of Harmonic Analysis, which can be found online here.For the more modern developments and the transition from concrete Fourier analysis to abstract topological groups,a very good source is the following article coauthored by Mark Karpovsky;Remarks on History of Abstract Harmonic Analysis
Addendum: I forgot one of the most important sources for those looking for an overview of harmomic analysis and it's history, there's the classic 1976 MAA Studies in Mathematics volume, containing classic review articles by Elias Stien,Richard A.Hunt and many others. It may be the single best overview of the entire field to that point (1976) you can find anywhere.
These should all give you a nice overview.
Another wonderful book to throw into the mix is T. W. Körner's Fourier Analysis (Cambridge University Press). He has a lot of history and interesting applications.
The initial sections (especially 7 and 8) of the paper below may be useful:
Mackey, George W.
Harmonic analysis as the exploitation of symmetry—a historical survey.
Bull. Amer. Math. Soc. (N.S.) 3 (1980), no. 1, part 1, 543–698. MR0571370 (81d:01017)