How to tell if I'm good enough for graduate school? I'm concerned about my level of preparation for graduate school. I have decent grades (3.64 GPA, 4.0 in math), great recommendations lined up, and some OK research experience (no publications), but I really don't feel like I'm ready for the next step. 
I'm interested in applied math, and was a physics/computer science double major for most of undergrad, switching into math just last semester. I'm taking Analysis 1 now as a first semester senior, and it is the only pure math class I've ever taken (though I've taken a bunch of applied courses and semi-pure courses, like advanced udergrad ODEs). I'm doing very well in the course, and I'm really enjoying it, but I feel that, because I'm so late in joining mathematics, I'll be really behind everyone when I get to graduate school, if I do get in somewhere decent. My scores on practice tests for the math GRE are in the mid 600's, and that makes me feel like a phony as well (although I haven't really studied that hard yet, maybe they'll go up? Also, its a good thing many applied programs don't care about the subject test...). Sure, I have some skills from my background that other students don't have, such as non-trivial software development and an ability to solve difficult physics problems, but when I look at the profiles of the 'good' people on this website, or talk to my friends that have been doing math olympiads since they were 3, I feel like there is just no way I can be good enough for a career in mathematics given my 'late' start.
Is it possible to catch up with others at this stage? Will I be doomed to a life of constantly being behind those who are smarter than me? Has anyone ever built a strong career from my kind of situation before?
I just wish I had some more time...
 A: At my university we use baby Rudin, Dummit and Foote, and Munkres in the first year analysis, algebra, and topology courses (respectively), which is the average schedule for an incoming grad student.  You can also test out of these and move to the 2nd year if you happen to be good in one of those fields (or two, if you're really exceptional).  My opinion is that most math majors could handle this schedule given an appropriate amount of studying.  Worst case you can opt out at a master's if you decide you don't want to do research.
As far as being stupider than other people goes: of course you are.  Unless you're the smartest person on earth, there are smarter people, and if you spend your time sitting around thinking about who's smarter than who, they're all you're going to pay attention to. If you're getting a math degree, you're probably pretty smart, and that's good enough.
I embraced my own stupidity long ago, and in doing so, I gained the ability to ask questions without losing face.  Instead of worrying about other students knowing I'm not a genius, I can talk my problems out with them, gain their insight, and learn from my own mistakes.  For whatever reason, people in the mathematical field have this tendency to try to convince you that you're too dumb to do anything. Don't listen to them.  If I miss a proof, I'll get mad, reread the chapter, work book problems, harass the professor, and stop sleeping until I understand everything. Be stubborn, not smart. You've got to have grit.
A: I think you are over-worried, because most graduate schools require students to go over fundamental materials in the first year. It is true that some material will almost never be covered in such classes and assumed to be known by everyone, like implicit function theorem. But usually it does not take a lot of time and effort to catch up with other people on a specific area, unless that subject is extremely advanced or needs a lot of preparation by the student. However, most of the time students do not know what is their future area of study until second or third year of graduate school, and a lot of them even change thesis direction during the research process. I do not think there is a perfect map for success in graduate school, in the sense that if one entered into graduate school without knowing a certain subject he/she is expected to be in deep trouble; and students who know all these subjects are adequately prepared. 
However, it require a huge amount of personal work to push oneself beyond the amateur stage. I think you should have the confidence that you can compete with other people more advanced than you currently are not based on track record, but based on your interest in mathematics. If you believe you can contribute something unique belonging to yourself that has its importance to be justified in future, then you should focus your time and energy to build your career to expand these ideas into reality. On the other hand, if you found what you can do in mathematics is no different from other people, then maybe it is a wise choice to abandon it.  
I think only because we believe we can contribute something so creative that deserves to be immortal then shall we consider pursuing a career in mathematics. In this way, when Prof. Clark or someone boldly suggested you that the competition in job market is extremely keen and the chance to get tenure is even slimmer, you would be able to refute him or her by the fact you still managed to contribute something. Perelman considered himself an unsuccessful mathematican because he was having trouble to get a tenured professorship right after post doc. But to us his deep contribution is at a level much higher than that of most other experts in his research field. A very basic fact of a mathematican's career is that a proof is a proof, such that when it is finished and checked to be correct, one does not need other external incentive to judge its importance. 
Background: I am a second year graduate student in Binghamton University, ranked around 80-90 (when I applied here I did not even check this, as it is not important to me). Prof Clark specifically asked people to list it at here. So without disclosing further personal information this is my background. I just passed my quals and started research. I used to receive the same piece of advice as his from other people, and my old advisor (also Harvard graduated, has a respectable advisor, etc) suggested otherwise. There is obviously no right or wrong at here, but to me I really appreciate my advisor's support during a difficult time. 
(edit):
While having little time to respond to Prof. Clark's further detailed comments(it is 3am now), I wish to make the following (practical) suggestion to OP or anyone in OP's situation. I strongly recommend him or her to consider having an appointment with his/her advisor, and see what he/she says. In my own case, I was informed by my advisor that my mathematical caliber is enough for any top ranked graduated school, but my credentials(GPA, REU, etc) did not support that. And I should be prepared to enter a much lower ranked state university. While this was extremely disappointing I learned to prepare for my career early this way, and I am confident that I am the "best" graduate student in our department this year, even though I did not do very well as an undergraduate. 
I also want to suggest OP to consider staying for an extra year at his home university taking a few graduate level courses or considering applying for a master program catering his taste. Personally I have known quite a few fellow students followed this plan and done quite well(she was admitted by University of Chicago, Cornell, U of Michigan, Columbia, etc and chose Chicago in the end). I strongly recommend OP to try Cambridge's Part III program, which opens for students without a pure math background and do not require any GRE subject tests, but does care about GPA and reference letters. From what I heard of from friends, it is a great program. In case OP wants to apply PhD later, he/she would have improved his/her resume (and chance of grad school admittance) by showing he/she did well in Cambridge. My other friend who could only got into Arizona state university in the first year was accepted by NYU in the second year. 
I hope these suggestions might help, even though I have never been enrolled in any top graduate program(defined by rank) or served in its admission committee to make any suggestion with authority. 
A: I will take your question in reverse. 
We all wish we had enough time. I certainly wish that I could live to see the Starship Enterprise, and I would be happy to mop the floors there if I could be so fortunate.
Yes (emphatically) you will be doomed to a life of constantly being behind those who are smarter than you. You really need to accept this because it is reality. You may be suffering from a very common thought paradigm. Many people, especially when they are younger, base their own self worth on their ranking among their peers. The truth of the matter is that no matter how good, talented, or gifted you are, there is probably (degree of probability very near unity) someone that is much better than you. 
Stop competing with your peers. Take neither pleasure nor pain in how your performance compares to your known peers. Shut the thought of comparing your own performance with that of your known peers out of your mind forever, and take neither joy nor sadness in these sorts of results in the future. You need a real peer to compare your self worth and performance against. I can tell you from very deep experience that this peer is you. You can spend the rest of your life trying to outdo yourself, and you will actually succeed sometimes. After that success you will still have your biggest challenge in the mirror. If you want true joy out of success in life, then push your own envelope of ability, and when you "outdo" yourself, take pleasure there. You really are and will always be your own greatest competitor, take my sage advice here. 
I have a secret for you. I will share it here on stackexchange, but as it turns out, most people will not actually take advantage. By definition, most people are mediocre. This applies to the smartest people ever born. Einstein was probably to the right of 4 standard deviations in antimediocrity, who was known as a genius (that sucked at math). He actually was bright, but his true gift was that he shot for the stars. You can take a lesson from what I say, and take comfort in the fact that most people, even the brightest, are mediocre in life. All of the natural born talent in the world rarely results in an Einstein. The bigger factor usually breaks down to a willingness to do the hard work. You can top the brightest of minds just by shooting for the stars in your own effort. Attempt to outdo yourself and only yourself daily. 
GRE tests are crap. Shut that whole deal out of your mind. Most schools do not take them seriously, they just require them.
When you graduate, you will be ready for graduate school. By virtue of your diploma you have actually prepared for graduate school. Shut this malarkey about other people being better, or the things that you do not know out of your mind. 
That being said, you should do a few things to prepare, and you should start doing them tonight and for the rest of your life. 
Do not be a mediocre slackard like the average person. When given a choice, take the harder class and not floating in the swimming pool 101. 
When you are taking a math class (or any class) do not wait for a teacher to tell you what to learn and when. Come up with your own questions, and enjoy the struggle of discovery. Grab knowledge by the cohojones!
Do not be afraid of failure. Do be afraid of not doing your best.
You definitely need to do well in your classes when you are taking classes. After those classes, if you really want to get strong in math then get yourself into the habit of doing about 6 hours per day of just math, and plan to continue this for several years (until you decide to retire). Review your studies, explore new ideas, read, write, solve problems, and attempt the proofs. Euler did it until the day he died. We call him a genius, and I am absolutely convinced that Euler was one of the greatest mathematical minds in all of history, but I suspect that much of his success came from the fact that he spent all of his days doing the math with a spirited unquenchable passion for all things math that continued even through blindness and ended on a gentle summer day when he probably died in the midst of a great problem in applied mathematics. 
Physics and computer science are actually ideal, probably optimal skills for a degree at any level in applied mathematics. Capitalize on this. Programming skills are especially important, and you should not let them go away. You will definitely need them, so do not discount them. Computer science is really a domain of mathematicians that became its own offshoot. We developed the first computers, the first languages, and most of the greats of computer science were actually mathematicians. Just being able to program will give you a serious edge, so do not discount this.
Go to school anywhere they will take you my friend and don't look back. 
A: It feels good to be sunny and encouraging when giving advice.  But if one gives sunny, encouraging advice regardless of the situation then it seems to be of little use other than as a pep talk.  If one gives sunny, encouraging advice when the situation is actually rather bleak, then this could actually be irresponsible.
There is also the question of who is giving you the advice.  Anonymous answers to math questions are worth almost as much as any other kind of answer: you can assess for yourself whether the answer is correct.  Anonymous career advice seems to me to be worth little.  In this case, I am concerned that the OP is asking advice about a PhD program and a career in mathematics, and it seems to me that many of the answerers have not completed a PhD program and/or do not currently have a career in mathematics.
I got a PhD in mathematics at Harvard University in 2003, and I am currently a tenured, associate professor in the mathematics department at the University of Georgia.  I have already directed the PhD thesis of one student, and I have two students currently doing thesis work under me.  I have also been on the thesis committees of about a dozen more students over the last few years.  I tell you all this so you can weigh my credentials against the following pronouncement:

The academic job market in mathematics is currently extremely tight.

People usually like to say that the market is cyclical, meaning that if it is bad now one can wait a few years for it to get better.  I think it is true that there is a lot of variation in the market: when I was on the market for tenure track jobs in the 2005-2006 academic year, it sure seemed like there were more jobs available then than there had been only a few years before.  (In fact, in 2003 I went on the postdoctoral job market.  From all the jobs I originally applied to I got exactly one offer, and that came late enough for me to worry that I was not going to get anything whatsoever.  I reiterate that I was graduating from Harvard University, and my thesis adviser was famous and influential even by the standards of the faculty there.)  But proponents of the "cyclical" theory will usually not go so far as to suggest the period: in other words, so far as anyone knows, the job market will be as bad as it is now or worse for five years or more.  Most people seem to agree that a necessary condition for serious improvement is for the local, national and international economies to substantially improve: exactly when that will happen is anyone's guess.
When I started graduate school in mathematics I had a similar attitude to the OP: I wasn't looking farther ahead than getting a PhD in mathematics.  That seemed to me to be an extremely daunting goal: it seemed somehow presumptuous and silly to plan farther ahead than that.  But when I knew I was about to graduate and feared that I would have no academic job whatsoever I cringed at my previous naivete.  I remember well that my sister called me and tried to cheer me up: "Isn't it good enough for now just to have a PhD in mathematics?"  That was a good question.  I reflected upon it and the answer was a resounding no.  A PhD is a preprofessional degree par excellence: it prepares you for an academic research career.  If you can't continue on in that career, then having the PhD seems worthless.  Well, it might be worthwhile if you had a backup plan (or even a different plan A) as to what to do with your PhD in mathematics.  I most certainly did not.
Thus one thing I would tell the OP is that he is not asking exactly the right question.  As Betty Mock indicates, sure, there are places where a minimally competent person (and even the little information the OP provides convinces me that he is more than minimally competent; a score in the 600's on the GRE math is distinctly above average, and many of the weaker programs do not require the exam at all) can be assured of getting a PhD.  The question should be

Why do I want a PhD in mathematics?  What will I do with it afterwards?

If you have a specific professional career in mind for which a PhD is either required or very helpful: great.  Please keep that goal in mind at every step of the process, especially the application process.  If you are doing it because you want an academic career in mathematics...please think carefully about what I've said and how tight things are right now.  I know a lot of extremely bright people who played the academic game for as long as they could and eventually had to cut their losses.
If you run the numbers in American academia, you see that math departments everywhere are fighting tooth and nail to replace each departing faculty member's line.  On the other hand graduate enrollments are not significantly shrinking and many departments without PhD programs are trying to start up small PhD programs.  The arithmetic is therefore discouraging.
Although getting a PhD from a top program is definitely not a guarantee of future employment and getting a PhD from a lesser program is not a death sentence, statistical studies show that a much higher percentage of graduates from the top ten programs get desirable academic jobs than graduates from the next forty or so programs, and much beyond the top 50 programs it is possible but quite unusual for graduates to pursue academic careers (by which I mean involving some mixture of research and teaching).  So here is some less sunny, but I hope more responsible advice to the OP:

Apply to a wide range of schools.  Apply to some top ten programs -- why not?  Apply to some which are just outside of the top ten but still very good.  And so forth: I would say not to apply to schools which are not in the top hundred unless there are very exceptional circumstances.  Then based on where you get in, you can decide whether or not you are "good enough" for graduate school.
My graduate program is about the 50th best.  We get some very good students, students who could go on to serious academic careers if the market would bear it.  But we are having trouble placing our excellent graduates in academic jobs, and every year some people leave academia who didn't want to.  We have a few graduates, including recent ones, who are doing extremely well.  So getting a PhD from the 50th best program and then trying to make an academic career is a real struggle, but it can be done.  If I were you, I would think carefully about whether or not to enroll in a program like UGA's and try to figure out how to "beat the odds" from the very beginning.  If you can't get into a program as good as UGA's, then in my opinion it would be a better career move to spend a year improving your dossier and trying again to get into a more reputable place.

A: Impostor syndrome is not uncommon among people about transition to the "next step," or among people entering into or about to enter into new experiences that they perceive to be a result of skill.
Believe it or not, although graduate school is designed to be a "down-selection" of undergraduates, first and second year graduate students can be as clueless as anyone else. Don't believe me? Survey your professors.
Your abilities have come from a combination of skill, hard work, and desire. A major-specific 4.0 is not bad. Just because you haven't established a coursework background in certain subjects doesn't mean you cannot do so. Think about it like this: no matter what you do, pretty soon you're going to have to learn material outside the structured framework of a formal course of study.
Finally, practice GRE scores in the Math subject test are a poor discriminator. The test is so variable that scores can only be compared to other scores from the same test. A 600 on one test might be in the 95th percentile, while a 600 on another might only be the 75th.
Ultimately, regardless of what you do, you're going to have to trust your own abilities, and also trust in others' assessments of your skills. It's a very difficult thing to do, sometimes.
A: I think you are worrying too much about being "behind" or whether you are smart enough.  Everyone in mathematics is working at his/her level of incompetence.  That is, if you get good at something, you are instantly promoted to the next topic, or the next harder problem.
A great deal is thought and written about the effects of talent vs hard work.  You probably are not going to get better than someone who has an enormous talent and has been working his/her butt off since the 3rd grade.  But that is not relevant.  What's relevant is whether you can get good enough to be pleased with what you are able to accomplish.  
It is my opinion that a person who has done well in undergraduate math can always get to a very high level of competence.  It comes down to how hard you are willing to work.  If you work really hard you can learn anything -- you might learn more slowly than Joe down the hall, but so what?
Now some graduate schools are kind of a joke, and anyone can get through, (and unfortunately anyone does).  If you are serious you do not want to go to that kind of place.  If you end up in a strong school, you are going to have to work hard to catch up and to keep up.
So what you need to think about is:  am I a worker?  or could I be a worker? Do I care enough about math to do the kind of work it takes. These are not trivial questions.  You have to know yourself well; and you have to know what hard work means, which you may not at this point.  Are you willing to give up your weekend to spend 25 hours on one problem?  And then do all the other problems?  And keep it up week after week, month after month?  It may be that the only way to find out is to tackle a hard school and see what's needed to keep up. 
I suspect you can get into a good school.  A sincere interest in math and strong desire to learn is very attractive to anyone who loves the subject.  Go talk to the professors at some of the schools that interest you.  If one of them takes to you, you'll get in.
Best wishes. 
A: You just take courses and if they are too hard you take them again or take easier courses. I will suggest if you ask for math you can prepare yourself with taking this course it probably the best out there it wont make you Gaus but it will soften the conceptual barrier which is what you want to do.
The course have started but if you have some time you can go through it and also Kieth is very nice, your goal should be to be conceptually prepared, but a university basically is an institute that suppose to arrange you the condition to overcome that state in its essence: 
https://www.coursera.org/course/maththink
