Mathematics needed in the study of Quantum Physics As a 12th grade student , I'm currently acquainted with single variable calculus, algebra, and geometry, obviously on a high school level. I tried taking a Quantum Physics course on coursera.com, but I failed miserably, as I was not able to understand the mathematical operators (one being the Laplace operator).
I would really like your advice as to what precise part of mathematics I should teach myself, and some suggestions regarding proper textbooks, if possible.
Thanks a lot! :D
 A: I think that Higher Maths for Beginners – Zeldovich, Yaglom and Elements of Applied Mathematics books have the math you need for QM. They're written by Zeldovich, a co-father of Soviet nuclear bomb project. 
The latter book has a related volume called Elements of math physics. Noninteracting particles, unfortunately it's not translated to English. The Russian version used to be my favorite text for math physics.
One of my favorite math physics text was Methods of Theoretical Physics by Morse and Feshbach. It's huge, and is written like a handbook, no need to read the whole thing.
If you read German, then there's this crazy handbook The mathematical tools of a physicist by Erwin Madelung, I'm not sure if it's available in English, but I saw it in Russian.
A: I have taken two courses in QM. There were a range of different math courses that are extremely useful. I would recommend studying multi-variable calculus, linear algebra, partial differential equations and probability theory.
A: It depends a lot on the point of view you want to take on QM. For an experimental physicist's point of view, you will need real/complex analysis, linear algebra and probability theory. If you want the theoretical physicist's/mathematician's point of view, then add functional analysis (maybe with a focus on $C^*$-algebras/algebras of operators) and representation theory.
A: I hate to be that guy, but in order to properly understand Quantum Mechanics, you need to have a solid understanding of Newtonian Physics. Since you're already on Coursera, try and slow down a bit and take one or two introductory physics courses before moving on to quantum. For the first few weeks (or likely the entire first course), you won't be learning any new math since it's all basically single variable calculus, but as you progress, you'll see that physics courses generally bundle in the required math with them, though not necessarily to the same level of depth (for example, the Laplace operator is covered pretty extensively when discussing electricity, that way people who come to learn quantum mechanics are "already prepared").
Instead of diving in headfirst with some hard math, start by learning Newtonian physics until you are ready to progress into quantum mechanics - you'll have the proper skills to advance onwards to quantum once you're done, though it certainly won't hurt (it'd be quite fun really) to do a few math courses as you go along.
A: I am going to plug my undergraduate professor's book again, but it is honestly the best book I know to prepare oneself for the math involved in QM.  (I should know, as I experienced his course as a Math/Physics double major.)  The book is Applied Analysis by the Hilbert Space Method by Samuel S. Holland.  It is now available in paperback and relatively inexpensive.  This book is custom tailored for the math and physics student on the cusp of taking a first course in QM.  There's even a chapter on the Schroedinger equation, with the solution to the hydrogen atom worked out in detail.  I cannot recommend highly enough.
A: It depends on what type of QM course you want to take.  Courses in QM for engineers, undergraduate physics majors, graduate students in physics, and graduate students in mathematics are all pretty different.  I will assume you're seeking an "undergraduate physics major" understanding of QM.
I have two recommendations:


*

*In my opinion, the best Quantum Mechanics book for self-study is Shankar.  My main reason is that the first third or so of the book is a survey of the mathematics you'll need in the other two thirds, i.e., it answers precisely the question you've asked.  You'll need to know calculus first (including vector calculus), but from there Shankar will give you what you need.  I also like this book because it includes tons of fully worked out examples, and pages upon pages of "what does this mean" type exposition.  Usually, you'll see this book being used in graduate or advanced undergraduate quantum mech courses, but that is just because it is a very long book about a very involved subject, it doesn't mean it isn't accessible to the beginner.

*When I took quantum mech I read Lang's Linear Algebra concurrently, and it made everything so, so much easier.  You don't necessarily have to finish it, basically just get real comfortable with inner products and dual spaces, up through the spectral theorem.
Some additional remarks:


*

*I think it's an exaggeration to say a course in quantum mechanics requires functional analysis or operator theory, even though that is essentially what you're doing. The mathematically rigorous forms of those disciplines are pretty advanced, but you don't need to understand them at that level to do quantum mechanics.  You just need to be able to use them.  Shankar will teach you how to do that. (Of course, I wouldn't discourage you from learning them eventually, but they aren't strictly necessary for the purposes of QM.)  He should catch you up on the basics of probability theory, too.

*I would not say that about abstract linear algebra, however.  You will need to understand vector spaces, dual bases, inner products, eigenvalues, etc. on a rigorous level.

*Group theory and representation theory are definitely not necessary.  Although they are important to quantum mechanics, you will not see them until very advanced levels.
So, long story short: you need to know calculus up through vector calculus and linear algebra up through abstract linear algebra.
Addendum.
It's worth mentioning that, although what I've mentioned above would be sufficient for the mathematics side of things, it would be real good if you had seen mechanics, electricity and magnetism, and thermodynamics beyond an introductory level beforehand.  It's possible to do it without previous physics courses, but you'll find yourself saying "so what?" a lot, as the weirdness of quantum mech will not seem as jarring to you if you have not seen how things work at larger scales.
A: Just take the standard courses going forward:
-college (or AP) chemistry:  You don't technically need this, but it is good background to some aspects of physics, especially the baby P-chem that forms most of the course. 
-calc 3 (multivariable calculus)
-differential equations ("calc 4", needed for college level E&M and really key course to not freak out when you see the Schroedinger equation). 
-calc-based introductory physics (mechanics and E&M, 1 sem each).  Halliday and Resnick level.
-if you are a physics major, you may have a third semester, which is a survey on modern topics:  QM is touched on LIGHTLY here, using the Shroedinger equation
-junior (or sophomore) level courses in mechanics and E&M.  Yes, you do all over again what you just did at a survey level with H&R or Giancoli with more dedicated texts like Wangness for E&M.  Lot more vector math...and kind of toughens you up for the more abstractness of QM, which can be a hassle (no pulleys or ice skaters or rockets...boo!)
-as a junior or so, if you are a physics major, you can take a 1-2 semester class that goes into more detail on QM.  This is the MEDIUM touch. [if you are a chemist, it is covered in junior p-chem, with emphasis on solving the hydrogen atom only (and students are led through the steps).  Material scientists usually get exposed in dept courses that are a bit more gentle than physics.  Same for EE.] 
-During junior year (sophomore if calc 1 and 2 done in high school), you should pick up a course in "math methods for physicists" or "engineering mathematics"  (this is basically a survey of select chapters from a book like Arfken or Kreyszig:  mostly classic PDEs and getting to see Yo-Bessel and Jo-Bessel, but a little linear algebra is thrown in also if you don't have requirement for a dedicated course).  [This is "calc 5".  A quick look at Kreyszig or Arfken will show you that there's enough in there to make a calc 6, 7, 8, etc.  But you don't need all that to start really getting into QM.  If you go deeper, you can always pick up the parts of those books that are needed or even go into semester long courses on some topic at a deeper level than they do.  But you DON'T need a math minor before starting to get familiar with QM to at least the "medium" level.]  
-If you go on to Ph.D. in physics, QM will also be revisited even HARDER in graduate level courses.
My advice to you is to not try to learn QM "perfect" the first time.  By that I mean learn it rigorously.  You will get more out of learning it a couple times at different levels of sophistication.  Also, there are definite benefits to having dedicated math classes and math preparation, but there is also benefit to having the science classes at the same time (or shortly after) learning the relevant math.  They sort of reinforce each other better...and even some more advanced math classes tend to have example problems that use simple problems of mechanics or heat transfer or the like.  
It's like strength training and gymnastics:  you won't go far if you try to become insanely strong for years (math) before learning gym tricks (physics).  But there are some tricks you can't do without some strength.  Just take things in the normal order and they will reinforce each other.  
For that matter, also be open to other things.  You are in 12th grade:  you don't know if you want to be a Ph.D. physicist (and even there there are many, many flavors...geeks who do math and manly men who drill into concrete floors to vibration mount equipment, and make illicit taps into high voltage buswork to power their gear.)  You might go into mechanical engineering or econ or chemistry or EE or do a B.S. in physics and join the nuclear navy.  If you run more of a standard general course, you preserve the option of trying different things as you learn more about them. 
P.s.  You don't need real analysis or topology or abstract algebra or some of the more extreme math recommendations here.  I get the impression some people don't read the whole question and consider even what you wrote about your background.  
