I was planning to mention Arnold's book, which was mentioned by @Frank Science. This is a wonderful book but requires a little mathematical maturity to follow. I am trying to write this as a reply of the last line of your question.
Can anyone suggest a mathematical book which is dedicated to teaching all the math that is used in physics?
It is impossible to know every aspect of maths which has some use in physics. Moreover, mathematics books are generally written in a certain way which someone from a physics background may find strange. (Converse is also true. Mathematics students find lack of mathematical rigor in Physics texts as very disturbing.) It is better to know and understand the mathematical aspects used in the 'area which you are studying'. This will give you mare insight about the physical problems.
For your current problems with linear algebra, I suggest you look at Hoffman & Kunze. You can also see the algebra of Artin. I found this book very lucid and written in a way which is easy to apply for practical problems. A book of similar nature is Rudin, which you must read for analysis.
After reading Rudin (& gaining some more mathematical and physical understanding) you can try Methods of Modern Mathematical Physics by Michael Reed and Barry Simon (four volumes). This book does not cover all mathematical aspects a physicist may require, but it definitely contain a large section of useful ones. I like the end notes of each chapter where the background physical motivations are explained. These are not very easy books.
The only missing areas are (differential) geometry. I guess somebody else can tell about this better than me. I generally look at Nash and Sen.
All the best for your work.