What makes the popular categories so popular? If you know about finite categories, you know that there are a lot categories out there.  There are many, many categories.  Oddly, though, mathematics as a whole seems to be interested in just a few of these.  Sets, groups, manifolds, vector spaces etc.  Now, obviously there aren't infinitely many mathematicians working for infinitely long to probe out every category.  Still, is there a way of specifying why the few categories that show up in literature have been chosen in the first place?
The guess I have worked on for a long time had to do with locally finitely presentable categories.  My first guess was that the compact categories in CAT were the familiar ones, then I was thinking that it was the colimits over compact categories, then both.  It didn't get me anywhere.
 A: I would guess it is more about the popularity of the objects of the categories you cite than about the categories themselves. Sets, groups, manifolds, vector spaces, etc. are here for a very long time in mathematics (even if not formalized as such in the early years), hence the understanding of those objects is something mathematicians can get a grasp on. Hence it is very natural for them to turn to those objects when trying to develop new ideas in new formalism. At the risk of sounding caricatural, remember when you first learned about rings: would you then have preferred to have $\mathbb Z,\mathbb Q,\mathbb R$,etc. as examples or the Haar-integrable functions on a locally compact Hausdorff topological group?
Also keep in mind that your view of what is popular is de facto subjective. For example, in my lab, I would have more chances selling some categorical property based on $\mathsf{Rel}$ rather than on $\mathsf{Vect}_k$, or on some abstract elementary topos rather than on $G$-sets for a group $G$. I don't even want to think of the uncanny face expressions I would get trying to explain something by interpreting it in the category of Lie groups. Everyday mathematics is very very different from one mathematician to another.
