Does it ever make sense NOT to go to the most prestigious graduate school you can get into? I'm a senior undergrad at a top-ish(say, top 15) math school. I'm a solid, not stellar, student. This year I'm taking the qualifying exam grad courses in algebra and analysis and have been taken aback by the "pressure cooker" atmosphere among grad students here. That is, even moreso than in the undergraduate program.
If I'm self driven, could going to a "less prestigious" school afford me more space(I mean in a psychological sense) to produce a more solid contribution to math? By "less prestigious", I mean a school "ranked" significantly lower than the range of schools that I could comfortably get into. For me, "less prestigious" would be ranked around 40-60 on, say, USNews or NRC.
My reasoning is that at such a school, I would be more able to learn the fundamentals at my own pace, as opposed to a pace dictated to me by the program. I know I want to do math, and I think my learning style may be better suited to going at my own pace. Thoughts?
 A: I think that there are several important points.
First, you will learn just as much from your fellow
graduate students, especially in the initial years, than
you will from your professors, and far more from your
fellow students if there is a good atmosphere for it. Thus,
I suggest that you try to land at a school with a
supportive and studious student environment. You can visit
the places to find out, and talk to graduate students
there. From this perspective, it is very helpful to be at a
place that has many talented graduate students---they will
help you rise to their level (or you will all rise
together). Since the top schools tend to have stronger
students, this can be a good reason to go to a top school
when it is possible.
Second, most of the pressure on graduate students is
self-imposed, instantiating their drive to do well
mathematically. Every school, including the top programs,
have some students that proceed at a different pace. So you
can often resist whatever external pressure you imagine is
there. (One important exception to this is at a school
where financial support might be withdrawn for slow
progress---so look into that at the places in which you are
interested.)
Third---and actually I find this to be the most important
point---you shouldn't look at the school only and make a
such an important life decision based only on mathematics
and prestige. Rather, look at all aspects of how the choice
of a school will affect your life. You must consider the
city and region as well. For example, do you prefer city
living or country living? If you like the mountains, hiking
and snow, then don't go to Florida, and similarly, if big
city living is your preference, then don't go to small
town. I find that this kind of consideration is oddly often
neglected among mathematics students.
A: There aren't anywhere near enough faculty positions for all the PhDs we churn out, not even close.  Academia might dramatically contract over the next couple decates even.
Industry knows almost nothing about your PhD subject, but they recruit heavily amongst the Ivy league graduates.  Also, one finds vastly more parochialism amongst the smaller schools.
Imho, you should select the best school possible because (a) it'll make you a broader more flexible mathematician and (b) it'll maximize your chances of doing the most interesting mathematics you can if/when you leave academia for industry. 
You should however realize that school rankings aren't the entire story.  There are schools that seriously over work their teaching assistants, like Purdue for example.  Avoid such places even if it costs you institutional prestige. 
A: There are circumstances that could lead one to be admitted to more prestigious graduate program X and less prestigious graduate program Y and choose to attend Y.  For instance, there was a graduate student at UGA a few years ago who was doing well in the program but at some point revealed to me that he had to care for a sick family member in a different state, so moved to that state and enrolled in a distinctly less prestigious program there.  Of course I don't disagree with his decision -- indeed I admire it -- but there was no question that this was being done for higher reasons than his mathematical well-being and future career.
Treating this question as advice, I would say No -- you should attend the most prestigious graduate program you can get into.  
Of course, graduate programs are not linearly ranked: i.e., holding up two graduate programs to each other, one may not be able to identify one as more prestigious than the other.  When this is possible, it is usually the case that one is significantly more prestigious.  So, to refine the advice of the previous paragraph: you should choose a maximal element of the set of graduate schools you get admitted to, partially ordered by prestige.
Included in this are considerations such as the following: you strongly believe that you want to work with a particular faculty member at a program which is overall less prestigious than some other.  This is a tricky decision that I don't want to enter into here in detail, except to say: if there is any doubt in your mind that you will complete a PhD under Professor X, be very skeptical about trading more than a little in the prestige of the school.
In the OP's case, s/he seems to want to choose a non-maximal element in order to take some of the pressure off.  Although I understand the sentiment, I advise strongly against this.  Two points:
1) Graduate school is a high-pressure, hard-slog situation for everyone.  It has to be.  If you're not fully prepared for that, perhaps you're not fully prepared for a PhD program: think about it.
2) Much of graduate school does proceed at your own pace.  However, the quality of the students and faculty in your program sets an example and a standard, in my experience the higher the better.  
A: Maybe this is just getting redundant, but I think this is a slightly different point from what people has said above:  while there do exist compelling reasons to go to grad schools that don't maximize prestige, it really doesn't sound to me like you have put forth any.  
I think on the whole grad students are pretty naive about how big an influence where they go to grad school has on their opportunities after graduating.  Of course, lots of people do manage to go to less fancy schools and go on to have good careers, but if you look at the schools in the 40-60 range you mention, most of their faculty went to more prestigious places.  For example, at Oregon where I'm a professor (US News rank #56), fully half the professors who got Ph.D.s in the US got them at Harvard, MIT, Berkeley or Stanford (all in the top 5 on the USN ranking) and most of the rest at institutions in the low 2 digits (Texas, Maryland, Wisconsin, Penn).  Some of this is for unfair reasons (bias, etc.), some of this is selection reasons, and some is because going to a better school give you more opportunities to be a better mathematician (more exposure to new ideas, more visitors, higher quality peers).  I wouldn't give up on those things because of a nebulous fear of not liking the environment.  Try it out; it's much easier to go to a Berkeley or Michigan for a year and then transfer somewhere else than the other way around.
EDIT: Just as an addendum, I thought I should emphasize here that like Pete said, prestige isn't really well ordered.  There are lots of pairs where it's hard to say really conclusively which is more prestigious, in which case you should go with your gut.  So I'm not talking about things like comparing Chicago and Columbia, and choosing Columbia because you'd rather be in New York, but rather the difference between going to graduate school in the 40-60 range (in USNWR or NRC), and something in, say, the top 20.
A: I really appreciate this question. 
Many people still hold the out-dated belief that you learn mathematics in the classroom. Sure, you may learn classical formulas, the statement of famous theorems and some folklore surrounding its solver or length of openness (if there is none), but you'll learn "how to be a mathematician" at home or in the library (with [insert your favorite book here]) or alone at the board wrestling with problems. This is the time when you develop your problem solving skills and gauge your limitations. 
What is nice about the better schools is having a (sound) sounding board for your ideas. If your colleagues are sharp, they will provide quick and clean jugular shots to your questions. However, sometimes it's better to think about questions on your own time and settle them yourself with no interference. This simple and time-honored exercise develops your "internal compass" much more than having access to slick solutions from others. 
Remember, it is sometimes (perhaps often) the case that we do not have the best vantage point to judge the right course of action in a given (perhaps crucial) point in our lives.
Here is what I think: Postpone your question for a little while. Go to the best school that you can get into, and find an advisor that suits you and your tempo. Be honest with him/her about your strengths and weaknesses. Develop an on-going dialogue with your colleagues. Work hard but don't go crazy doing so. Ask questions if you get stuck, but not too many. Be respectful. Listen to people that you hold in high regard (and some that you may not). Think for yourself. Find an interesting problem. Write a good thesis. Graduate. Return to the issue of "solid contribution" when you have a better view of the landscape of your chosen field.
A: Maybe you would like an answer from someone who is there. 
I went to undergrad at a "Top 100" school. I did extremely well there, but utterly failed to get into the level of grad schools I was aiming at, and am now a grad student at a "Top 50" school. 
I can only answer based on what I've seen at these schools, so maybe things are different elsewhere. But to me, the situation is kind of sad here. The students know so little and don't seem to really have the right attitude. I think I'm missing a big component of interacting with peers who are learning the same material. I can honestly say I've never had any sort of study session or even just a discussion about mathematics with fellow students that was productive for me. (Although this may more reflect my experience at my undergrad school... I mostly talked to the grad students there, and this was true there, but I'm still new at my graduate school and don't know many people, so maybe this will change.) 
I've never been to one of these prestigious schools, but seeing people online talk about them, it seems like a completely different world. 
On the one hand, it's nice because I can stand out here. But I can stand out here without even trying very hard. I think this leads to more of a complacent attitude on my part, which is not good. 
I wonder, what would happen if I were at a top 10 school? I honestly don't know. I might get massacred. I certainly have a much weaker background than entering students at those schools seem to have. 
There's also the career issue. As far as I can tell, very few students graduating here end up in a tenure track job at the kind of university I would like to be at. It seems like a big challenge to overcome. 
It's not all negative. I suppose it is nice to not have so much pressure. There are some strong professors who do interesting work I might like to work with. But if you're ambitious (and why get a PhD in math if you're not ambitious?), I wouldn't recommend coming to this type of school if you have other options. 
A: I happened to have gone to a toppish school (look up my profile if you really care which one). Even though I didn't realize it at the time, there are distinct advantages from the mathematical environment there. But I also think you shouldn't base your decision on graduate schools purely on artifical rankings of prestige. If you are deciding between schools, you have to visit them to get a sense of the environment there. 
Unlike the undergraduate degree, if you do a graduate degree at a reasonably strong place, you will be spending 4, 5, or maybe 6 years with a close-knit group of people that number from 20-30 to 100+. It is where you will get to know a lot of future colleagues in your field, it is where you pick up new ideas from passing discussions, and it is where you learn how to do research. And the most important factor that goes into your decision of which graduate school to go to is the people. 
You need to find out if you can work with the faculty there: is there anyone there in your field? Is his or her interest something you want to study? Are they taking students? How do they treat their students? All these are things you should talk to people, especially current students, about when you visit. 
You also need to find out if you have fellow students†: you don't want to be the only student in the department doing (for example) PDEs when everyone else does algebraic geometry. The students whose research interests are closely related to yours will be the ones you discuss most with (and see most of); they will be the ones you ask for help and they will be your future colleagues. Find out if they have informal student seminars. Arrange your trip so that you can sit in on some of them. Having a couple more senior students who are willing to give you sound advice when you are just starting out will be very useful. 
If you really must make a decision without first hand experience visiting, then so be it: higher ranked schools tend to have a larger, more diverse pool of faculty. They also tend to attract the better students. So that's where I will put my money were I to bet blindly. But I cannot stress enough the importance of finding out whether the school and environment matches you first hand. 

A little side remark about "more room psychologically": I would argue that that is actually counter-productive. When I was in graduate school there was significant amount of down time where the research just doesn't go well. Looking back, I am really glad to have had my large and rowdy office: at least one of us will have made some progress at any given time. And with seminars happening now and then, one can at least learn something useful through discussions when the thesis project is stuck. Were not for all these "distractions", I fear that the ennui caused by the glacial pace of my thesis research may have led to psychological problems. 

† Added Oct 2011, partly in response to Isomorphism's comment below: of course, fellow PhD students are not the only people you can talk to. They are often the ones you feel most comfortable approaching. But you should certainly also factor in PostDocs at the institution when you make your decision, as long as you be proactive about approaching them for discussions. 
A: It depends largely on what your career goals are.  If your goal is to be a professor at a top 100 research school, then you should certainly go to the most prestigious option available as explained by Pete and Ben (and with their caveats that "more" means "significantly more.")
However, if you have different goals (for example, you want a job at a more teaching oriented institution) then it seems less clear to me that the increase in prestige is always for the best.
While I was a graduate student a professor moved from a top 25 school to Berkeley bringing several of his students along with him.  Those students had the experience of having the same advisor at two different schools, and they found the experience at the two schools to be substantially different.  In particular, at the previous school the professor was not very busy and spent hours talking with his students.  Whereas at the new school there were lots and lots of visitors/postdocs/other profs around all the time and the prof had the opportunity to talk to those people rather than his students.  (He was still a responsible advisor who met regularly with his students.)  So, if having an advisor with lots of extra time who actually seeks you out to talk to you is really important to you then going to a lower ranked school actually can make a significant difference in experience.
A: There are many good comments above, I'd like to add a few other points. 
Prestige is a fairly abstract, borderline meaningless word.  It's meant to be an amalgamam of other, actually important ideas.  And those concepts are the things one really ought to pay attention to.
For example, when you go to grad school, one thing to keep in mind is "will university X give me the time it takes to do substantial research?".  It's a good thing to go to a school that can promise you long-term funding with enough teaching work so that you get familiar with the process, but not so much that it distracts substantially from your graduate studies.  Depending on how strictly you use this criterion, you'll find this may narrow down the list of potential graduate schools to something pretty close to the traditional "prestigious" university list.  The above is concerned with the component of prestige associated to "does the university have ample research funding for grad students?" Even among the "top" universities you'll find this varies pretty widely. 
Another thing to keep in mind is much of the point of going to grad school is to be exposed to as many ideas from diverse areas of mathematics as possible, as most undergraduate programs rarely get beyond the 19th century in mathematical understanding. Smaller universities tend to have less diverse faculty members, less grad students and postdocs, sometimes their teaching load is higher.  The net effect is that if you're at a smaller, more isolated place you're less likely to go to some wonderful talk on a random topic you didn't know anything about.  It's amazing how quickly one can pick up new ideas by going to good random talks, ideas that affect how you look at other subjects, the interconnections between fields, applications to other areas of life, an outsider's perspective on "your" field, and so on.  So going to a "big" place is very useful in that regard. So this is the component of prestige associated to being a large department/university. 
There are of course other components to prestige, but the above are two fairly substantial ones.
