I've just finished high school but I don't feel my knowledge of mathematics is good enough. I'd like to start again from scratch, possibly adding a bit (a lot) of problem solving to it. What is the order in which I should study the various branches of mathematics? What are some good books to do so? Keep in mind that for 'starting from scratch' I mean starting from (possibly at a university level) set theory and the four operations. Thanks in advance for the answers!
If you want to read up on introductory maths at university level, I would suggest that you take a look at the first-year undergraduate maths curriculum of some university. Especially the "big" universities have websites containing detailed descriptions of their courses and curriculum. See which topics they cover and which books they use. Often you will also be able to find lecture notes etc. which are a good supplement to the suggested literature.
I had a similar situation some 2 years ago, and here are some quick points from my journey:
I feel like it depends on where you are headed. If you want to make mathematics you future profession, the way you take will be different from what say an engineer will take. For example, in my case I am engineering student and i got to study plenty of calculus, probability and much of the fancy stuff but by the end of the day i still felt my knowledge of maths to be unsatisfactory(that's why i am on his site by the way).
So on the way to achieving your goal, here is what i can tell depending on my experience.
If engineering is your way:
You have to work very much on problem solving. A possible way to approach the task, here it is. Start will "normal" calculus but now try to understand the concepts not just for computing answers but also try to understand what it means in real life. For example, say "limits". You must have studied those in high school. Understand carefully what it means. Try to find examples where this concept might fit. Here is an example: I am given a material whose 'flexibility' is modeled by a given function. And that function depends on temperature. Here limits may help you understand how the material behaves when the temperature tends towards a certain value. See ... Try to start thinking like that about concepts, not just solve some exercises - but don't get me wrong: exercises are of crucial importance in learning, but the difference between you and a maths software is that you must understand the why of every computation you are doing.
Now a possible road map:
From there you can go ahead and study other areas of interest mainly (i) Engineering optimization and numerical analysis (ii) Statistics and probability.
Those two because as an engineer the sooner you start producing results, the better off you are.
Starting with calculus is important because it has a lot of applications you can play with, it gives computational skills fast if you do exercises, has interesting concepts and forms the foundation of much mathematics engineers deal with.
So basically, it boils down to
If mathematics is your way:
Now if you want to make mathematics your profession, you will need a different frame of mind. First i am neither a professional mathematician nor have i reached a level where i can say that i am thinking like one. Yet that is my goal too. So i will share with you what i have learnt so far.
First, mathematicians, from i can tell so far, work differently from say physicists and engineers. When a you hit a theorem, don't go ahead and read the proof, first try to prove it yourself.
That will form the basis of the mathematician in you.
Here is the books i can advice to start with.
Once you are grounded in Set theory ( not too much though, whatever is provided by the two previous will be enough ) and proofs, continue with these:
Always try to prove theorems before reading the proof. Every time you read a mathematics book, usually graduate level ( don't be concerned about these for now ), and they tell you that a certain amount of mathematical maturity is expected from the reader, what that simply means is that they expect you to be able to prove the theorems or at least follow the logical arguments.
Also I highly advice like others that you try to read about mathematics in the general sense. Some books you may start with, here they are.
You may not be able to follow, the proofs in the two first books but nonetheless, you will enjoy the ride!!!
So that's the best i can do for my level and I wish you good luck and success!