People are going to tell you different things. I don't know any good universal definition of success. When have you succeeded? Some might not feel like a success before they have received the fields medal, for others success is just getting some random job. For some, being successful in mathematics isn't even about getting the right job or recognition, but about having a personal feeling of accomplished.
That said, it sounds like what you want to do is to pursue a Ph.D in mathematics.
From my experience, the biggest reason people fail in the graduate school is laziness. I have seen plenty of people who seem quite bright, but who are just not willing to invest the amount of work it takes to succeed. I have unfortunately seen plenty of really smart people who have "failed" just because they didn't do the work. For some this is of course not just related to laziness, but also to outside influences. Do you have a family that you are trying to support while in graduate school? Do you have any medical conditions that will prevent you from spending the required time? Granted, each of these things don't mean that you can't succeed. It might just make things harder.
The big question (IMO) then becomes: how do some people managed to work so hard? Where do they find the motivation to keep at it even when it sucks. How do some people get up at 5:30 in the morning so they can get a couple of hours of studying done before they go to class? Being dedicated to something can be hard.
IMO the key things that drives the engine powering your motivation should be a love of math. If you don't have some deeper of appreciation for math, if math is just work for you, if you are just wanted to do math to get girls (?), then I don't think you will succeed. Getting a Ph.D. in anything means hours and hours or hard work that sometimes seem to have been wasted. You might work on a problem for days, just to find out that it can't be solved (maybe there was a type in the problems statement). You have to have the right mindset to power through things despite this. (Not to sound to negative: Yes, there are/ will be a lot of wonderful moments when you have solved a hard problem or when you have finally understood something, and these moments can make it all worth it.)
So who can tell you if you have the right mindset/attitude/motivation? Well, probably not people on MSE. They don't know you. There are of course math tests you can take, but I only think they give you a partial answer as to whether or not you can/will succeed.
So can you do it? It sounds like you are "smart enough", and if you are willing to work hard and prioritize you studies, and if you have a love of the subject, then yes!