Since I am living in France now on exchange, I can tell you a tiny bit about the system here, and why I might recommend France.
Let me start by mentioning:
- The Tuition at some universities* is little to non-existent for undergraduate students. Even international students.
- It is one of the most prominent countries for mathematics over the last 200 years, and mathematics is one of the most respected subjects among many top French Universities. (At Polytechnique it is the most important subject, as there is the underlying ideology that all scientists and leaders should have a concrete understanding of mathematics) Also, France is second only to the US for number of Fields Medalists with a population which is a 5th of the US.
One important thing is that their system is very different from North America and the rest of Europe.
There are two main branches after highschool, University and "Les Classes Préparatoires" which leads to "Les Grandes Écoles." The second is considered more prestigious, and the most coveted institution in that line are École Normale Superieure and École Polytechnique. (For what it is worth, I believe that half of the CEO`s of the top 50 corporations in France, and 3 former presidents graduated from Polytechnique, while all 10 French Fields Medalists are alumni of École Normale Superieure)
I am not sure how admission to these schools works for international students, as taking "Les Classes Préparatoires" is probably not an option. This would be something to look into. (I can tell you about the Exchange program but that is probably of no help!)
For the other line, there are many very strong Universities as well. In particular Paris 6 and Paris-Sud 11 are probably the most notable for Mathematics as they are both excellent on the world stage. (Paris 6 is also known as Pierre and Marie Curie University, part of the Sorbonne) Often times for research Paris 6 is ranked as number 1 in France.
An enormous number of the French Universities are in Paris, which I think is a plus because Paris is a beautiful city. The only downside to this is that the rent is extremely high. However, fortunately the French government is quite left leaning and has set up programs which help students. (Yes even international students!) In particular the CAF pays part of the rent for students, the so called "aide de logement."
Lastly French is very useful language as far as it goes. There are quite a few countries which speak French, and many math papers are written in French.
In short, I suggest considering many universities in France, and actively looking for more details. In particular, if you are interested you should seek out all the information you can, when the deadlines are, how the process works for international students at that particular institution, what are the costs, do I need to write a particular test, do they require a particular French level on the TOEFL, etc...
Remark: Just worth thinking about the fact that the cost of living for 4 years is going to be a lot wherever you go.