You must separate two approaches, of course interconnected but conceptually independent from each other : "rigorization" and "foundation".
With "rigorization" I mean the work of most of XIX century mathematician on calculus, aimed at removing the obscurity inherited from Newton's and Leibniz's discoveries. The work of Cauchy , Weierstrass , Bolzano, Cantor, etc. gave us the modern definition of limit, etc. based on real numbers.
This effort of rigorization was completed at the turn of the century by Dedekind, Frege, Peano and Hilbert, who achieved several big results :
Dedekind (1872 - Stetigkeit und irrationale Zahlen (Continuity and irrational numbers) : analysis of irrational numbers and construction of real numbers as Dedekind cuts , i.e. as non-geometrical entities in book
Frege (1879 : Begriffsschrift and 1884 : *strong text*Die Grundlagen der Arithmetik) : modern mathematical logic and philosophical anlysis on the nature of numbers;
Dedekind (1888 - Was sind und was sollen die Zahlen? (What are numbers and what should they be?) and Peano (1889 - The principles of arithmetic presented by a new method ) : axiomatisation of natural number, i.e. characterization of the mathematical structure that we refer to as "natural numbers" in terms of some basic properties (as you said : "agreed to be basic and reasonable") form which all known properties of natural numbers (their "behaviour") can be deduced in a rigorous way (see also Frege) .
Hilbert ( 1899 - Grundlagen der Geometrie): modern axiomatisation of geometry ;
Hilbert ( 1928 - with Wilhelm Ackermann, Grundzüge der theoretischen Logik) : foundations of mathematical logic (based on works of Peano, Frege and Russell)
Hilbert ( 1920s-30s - with Paul Bernays, Grundlagen der Mathematik, vol. 1 - 1934 and vol.2 - 1939) : mathematical investigations on formal systems.
These mathematicians (with Russell, Brouwer, Weyl) tried also to develop research programs (e.g. logicism, intuitionism, formalism) aimed at answering basic philosphical questions about the existence of mathematical objects (i.e.numbers), the way we can have knowledge of them, etc.
Those research programs was greatly propelled by the discovery of Paradoxes (Cantor's, Russell's), so they become known as "foundational" programs, aimed at find the basic principles that can "secure" our mathematical knowledge.
One of the most important result of this movement was Zermelo's axiomatisation of Set Theory (ZFC) : with this, mathematicians was able to find some basic axioms (but this time NOT all "agreed and reasonable") capable of "generating" (up to now without contradictions) all known properties of set AND capable also of building a proxy for other mathematical structures, like natural numbers. This means that Peano Axioms for numbers are now theorems of Set Theory.
Can we say that we have reduced numbers to set ?
From one point of view : YES. The language of sets is so basic that quite all mathematical concepts can be "described" with it and sets' axioms are so powerful that all properties of the defined mathematical concept can be proved starting from them.
Form another point of view (more philosophical ) : NO. In what sense we can say that our basic insight into existence of natural number and their properties are less "clear" or "certain" than our insight into the existence of sets (the cumulative hierarchy) and their properties (i.e.Axiom of Choice) ?