The presence of an inner product on a complete vector space is used in a lot of ways in number theory, though perhaps not on the straightest path: traditional number theory doesn't use them so much as some of the more modern ideas on how to think about the subject.
Let's start with the basics: in an area called algebraic combinatorics which is something of its own field in terms of materials, but is used in conjunction with hilbert space methods to prove results on primes in arithmetic progressions, which is the subject of a very important modern theorem due to Terence Tao and Ben Green. This school of thought treats groups like $\Bbb Z$ or $\Bbb Z/n\Bbb Z$, or vector spaces over finite fields as basic objects, and uses things like indicator functions for sets as basic objects, and performs harmonic analysis on them to prove inequalities on sizes of things like sets of small doubling, that is to say sets which have constants $k$ such that
One method in this is to note that $L^2$ functions are a hilbert space, and use the Cauchy-Schwarz inequality to prove bounds on the sets' sizes. It's not always the case that they use the completeness of the space, but this is also a more basic example.
Another major way where the methods are used are in the study of expander graphs which, oddly enough, are related to arithmetic lattices in a wonderful way. Here one uses the Fell topology on representations of a group, to show certain types of Cayley graphs are expanders iff the associated group--in the number theory applications, usually $SL_n(\Bbb Z)$ or $SL_n(\Bbb R)$--have certain algebraic properties. In this the analysis uses the inner-product on $L^2$ functions extensively. If I recall correctly, Terence Tao was thinking of blogging or writing something up on this exact topic some time ago, he may have completed that in the interim. The man who pioneered a lot of this material is Grigory Margulis, the genius who settled the Oppenheim Conjecture some time ago.
Indeed, Hilbert Spaces of Entire functions demonstrates deep connections with GRH and $L$-functions. This is one I'm not as heavily qualified to comment on, but it's the one where I think the functional analysis can most rightly be called "Hilbert space methods" in a context which is firmly number theory, and certainly supports the point from the book. This is the one I would say where you really need the full power of Hilbert spaces to prove the results, if any there are.
On a less serious note, harmonic analysis on groups has been a topic of study to number theorists for some time. The Fourier transform is extensively used in the field, and is a very popular means of discovering new theorems in analytic and algebraic number theory.
Now, with respect to the actual comment in your book, I would wager it's much more of a stretch than some of the other areas in that number theorists generally do not "need" heavy machinery like dual spaces, Riesz representation, and the higher-level tools of functional analysis, and indeed aside from the second to last example, I would not really claim I was using "Hilbert space methods" despite the fact that this is technically, literally true. This is of course because that machinery was not really developed specifically in the context of the Hilbert space axioms, et cetera.
Hopefully that helps clear things up a bit and gives you something to think about.
Overall I would say the methods which can rightly be called "Hilbert space methods" have not played anything I would call a "foundational" role. They simply aren't ubiquitous enough in the subject. Neither would I say they were just added later on, I would say they are important where they are used, but not strictly essential in the way I would use that word.
The Debranges spaces do need the Hilbert machinery most heavily, and depend upon them a lot, and are results I think the only known proofs for are via Hilbert space methods (because that's their natural home).