How do we know that topological spaces and groups are essentially different? As we study mathematics, we come across objects like vector spaces, groups, rings, fields, metric spaces and topological spaces. We may see many seemingly different examples of the objects, but we can classify them up to isomorphism/homeomorphism. Then I have the following question in mind:

How do we know that two classes of objects, for example, groups and topological spaces, are essentially different, that there does not exist an "isomorphism" between them?

This seems to be related to category theory, but I have never taken any course on that.
 A: The right notion of sameness between categories is arguably equivalence which in anyway is implied by "isomorphy".
Well, to see that two things aren't the same you must show that one thing has a property that the other thing doesn't have.
In this case we of course only consider properties that are preserved by equivalence. These are basically all "categorical properties" you would care about (this excludes things like the size of the category: this is because we want isomorphic objects in a category to be considered the same, so we shouldn't care about how many "same" objects there are).
What kind of properties we specifically should be looking for kind of depends on the actual question.
In your example: if you have any category of algebraic objects (including groups, vector spaces, rings, etc.) then in this category regular epis are stable under pullbacks. This is untrue for the category of topological spaces (see the counterexample on this page of the nLab if you want). 
Since this property (regular epis are stable under pullbacks) is preserved by equivalence, the category of topological spaces is not equivalent to any category of algebras (therefore it can't be isomorphic to one of them either).
