I'm quite knowledgeable when it comes to Mathematica but I'm by no means a real mathematician. Here are some tips that might give you a start. First off, we do have a Mathematica Stack Exchange site, and we are (I'm proud to say this) very friendly and helpful to anyone with interest. Our chat is quite active, and you are welcomed to drive by if you hit any rocks on the road to learning the language.
As a long-time user of Mathematica I can tell you that one of its great strengths are the vast amount of possibilities you have to visualize abstract concepts. In the past, we have for instance created highly pleasing analytical surfaces
We have used it to create perspective projections of QR codes to make them readable from impossible angles
or we have simulated mirror cubes
and those are only a few examples where the core is applied mathematics.
One of the things you probably shouldn't start with is integrals or transformations on complicated or hard analytical expressions. Although Mathematica is really powerful with these, when you hit a dead end you often cannot gain insight, why Mathematica cannot solve it. We have many "Why can't Mathematica solve this integral" questions and more than once we could not tell why it did not work. On the other hand, if you are a mathematician chances are good that you can simplify your problem and can use Mathematica to explore the difficulties.
It is a completely different situation when it comes to numerical computations. Most of the algorithms for integrating, finding minimums or solving PDEs are well documented and they usually work out of the box. For linear algebra, both the numerical and the analytical algorithms work very well as far as I have used them.
Finally, Mathematica comes with support for many different areas like neuronal networks, image processing, graphs, statistics or modeling to name a few. Here, it really depends on what you are interested in.
My advice is the following: Get used to the basic language. You could for instance start here or maybe the fast introduction for programmers fits you better. After that, my advice is not to do a "freestyle exploration" but to take a topic that interests you and start working on a small problem. IMO working on real problems works better if you want to learn a language.
If you elaborate a bit on what you are doing (and I'm somewhat familiar with it), I might be able to give a more definite suggestion.
During a chat with @shardulc it became clear that he is interested in various topics and in general very open minded.
Although he seems to have an interest in artificial intelligence, I did not point him to (deep) neuronal networks because this works out-of the box in Mathematica. I was afraid that it would probably be more an experience of using built-in functions instead of learning to program.
Additionally, the algorithms of current trends in natural language processing didn't seem to be a good start for learning a programming language as many algorithms are quite involved.
Instead, we discussed several other topics that are both, mathematically quite involved and visually extremely pleasing. Here, I had several ideas basing on books I own and topics I already touched. One book instantly crossed my mind:
Indra's Pearls is unusual because it aims to give the reader a sense of the development of a real-life mathematical investigation, rather than just a formal presentation of the final results. It covers a broad span of topics, showing interconnections among geometry, number theory, abstract algebra and computer graphics. It shows how computers are used by contemporary mathematicians. It uses computer graphics, diagrams and cartoons to enhance its written explanations.
In addition to that, I pointed him to the topic of rewriting systems that are used to create plant-like structures and fractals, and reaction-diffusion systems that can be used to re-create patterns of seashells:
Last but not least, I pointed him to the web-page of Paul Nylander that covers so many different areas that it can hardly be expressed in one paragraph. Paul has many high-quality images and for most of them he provides Mathematica code