The duality is that between extremal monomorphisms and extremal epimorphisms. A monomorphism is the right generalization of injective function to situations with more structure, such as topological spaces with continuous functions. We say $f:X\to Y$ is a monomorphism if, whenever $g,h:Z\to X$ are such that $f\circ g=f\circ h$, we can infer $g=h$. Monomorphisms and epimorphisms of topological spaces are just injective and surjective continuous functions, respectively, so aren't too exciting.
But topological spaces have some relatively unusual properties-not shared by algebraic things like groups or vector spaces (or sets-)summed up by the slogan that the first isomorphism theorem does not hold. In particular, injective maps aren't necessarily homeomorphisms onto their image. The latter are clearly interesting, so we look for some kind of abstract characterization of why they're interesting, and the answer I'm offering is that they're extremal. The general definition is that $f:X\to Y$ is an extremal monomorphism if, whenever $f=g\circ e$ for $e:X\to Z,g:Z\to Y$ with $e$ an epimorphism, $e$ is actually an isomorphism (a general word which means bijection when interpreted in sets, homeomorphism when interpreted in topological spaces.)
If you think about it for a minute, you'll find that this definition reads strangely when $X,Y,Z$ are sets and $f,g,e$ arbitrary functions: $f$ is an injection and $e$ a surjection, so that certainly $e$ is a bijection. And indeed all monomorphisms of sets are extremal, but this is not so for topological spaces! It's a good exercise to prove that the extremal monomorphisms where $X,Y,Z$ are topological spaces and $f,g,e$ continuous functions are exactly the embeddings. Use the fact that the identity map from $(X,\tau)$ to $(X,\tau')$ is an epimorphism but not a homeomorphism whenever $\tau'$ is strictly coarser than $\tau$.
It's a general principle that whenever you have a notion defined entirely in terms of arrows, like monomorphism or extremal monomorphism, you have a dual property gotten by turning all the arrows around: this is the basic duality of category theory. When we turn the arrows around in the definition of monomorphism, we get the notion of epimorphism, and when we do the same for extremal monomorphisms, we get the notion of extremal epimorphism: $f:X\to Y$ is an extremal epimorphism if whenever $f=g\circ e$ and $g$ is a monomorphism, then $g$ is an isomorphism. And in topological spaces, this notion is exactly that of quotient map!
So the upshot is that we like injections and surjections, and have various generalizations of them, many of which end up being interesting in any new context we begin to study. And there's always a duality between such generalizations extending the duality between monomorphisms and epimorphisms. So as soon as you notice embeddings are interesting, you can immediately begin searching for quotient maps! Of course, that's a ludicrously ahistorical description, but it's conceptually not entirely useless.
For completeness' sake, let me mention that I could have also told this whole story with regular monomorphisms and epimorphisms, which are the analogue of normal subgroups (and quotient groups) from group theory. In general these are much stronger than extremal mono/epis, but in the case of spaces they coincide.