I think Electromagnetic Theory and Computation: A Topological Approach by Gross and Kotiuga might be just what you're looking for. However, it does assume that you know some general and algebraic topology to start with.
I would recommend that you read John Lee's Topological Manifolds first. The text covers what you would expect in a typical topology book, but focusing primarily on manifolds, which are the physicist's preferred sort of spaces. However, it can be a bit difficult for beginners, since it assumes mathematical maturity, so you may want to keep a more elementary reference like Munkres handy for when you get stuck.
Alternatively, you could read a more physicist-oriented introduction to topology like Nakahara's Geometry, Topology, and Physics. I have not personally read it, but it seems like it should be accessible for you. There is also Gauge Fields, Knots, and Gravity by Baez and Munian, which is a very well-written book that provides good intuition, but is more of a survey than a textbook for learning the details.