Since you're a graduate student and sets are not totally new to you, I suggest Enderton's Elements of Set Theory.
It starts with the very basics of set theory, without overlooking the axioms of ZFC (some axioms are introduced later in the book as they're not useful in the beginning). It even teaches you to justify that a given collection of elements is indeed a set (as there are collections that are not sets, for example the collection of all sets), so it does things in the rigour that you're looking for.
The book covers the topics you mentioned. It also explains the definition given by your professor: actually, the book gives a definition of finite sets, then defines infinite sets to be sets that are not finite. Then it gives the following in page 157:
Corollary 6P A set is infinite iff it is equinumerous to a proper subset of itself.
Hence, you'll understand why one can define an infinite set as your professor defined it, and then finite sets to be sets that are not infinite.
The book contains a lot of exercises varying in difficulty. You'll be able to cover all the topics you mentioned by reading chapters 1-6 (knowing that chapter 1 is a discussion about set theory, and chapter 5 is not required for any chapter in the book).
I find that the author explains really well. He also presents important notions like induction and recursive definition in a rigorous way, so it doesn't lack the pure mathematics viewpoint. The author also discusses sometimes the history of set theory and talks about the foundational issues related to it.
The book also covers in good rigour cardinal numbers (the size of sets, even infinite ones) and ordinal numbers (which represent the different well orderings you can have)in chapters 7 and 8. Ordinal numbers have nice applications like transfinite induction, which is more general than induction on $\mathbb N$. It can be used to solve some problems in topology and measure theory for example. Ordinal numbers and cardinal numbers are also important in studying infinite discrete structures. They're definitely crucial if you want to study infinite combinatorics and infinite graphs for example.
Finally, if you want more problem solving, you can work on additional exercises in Komjáth and Totik's Problems and Theorems in Classical Set Theory.