When does $|\mathrm{Hom}(A,B)| = |\mathrm{Hom}(B,A)|$ imply $A \simeq B$?

In which categories does $|\text{Hom}(A,B)| = |\text{Hom}(B,A)| \neq 0$ and $|\text{Hom}(A,A)| = |\text{Hom}(B,B)|$ imply that $A$ and $B$ are isomorphic?

-
Is there some intuition which leads you to ask this question? It's not true in some of the easy categories one normally thinks of: in the category of sets, take $A = 4$ and $B = 2$, and in the category of groups, take $A = \mathbb{Z}/2\mathbb{Z}$ and $B = \mathbb{Z}/3\mathbb{Z}$. It is true in preorder categories though. – Zhen Lin Oct 30 '11 at 17:18
@Zhen Lin: Indeed, I had in mind as an intuition the category of sets, but your example shows that the preconditions are too weak. For my intuitions's sake it's OK to strengthen it by $|\text{Hom}(A,A)| = |\text{Hom}(B,B)|$ (as I did above). – Hans Stricker Oct 31 '11 at 7:34
Category theory is not enriched set theory. Consider the category which has two objects $A$ and $B$, each with a single absorbing non-identity morphism $0$ and morphisms $A \to B$ and $B \to A$ whose composition either way is $0$. By construction there are no non-trivial isomorphisms, but all your conditions are satisfied. – Zhen Lin Oct 31 '11 at 9:01
OK, this is a counter-example, but I asked for categories where the implication does hold. – Hans Stricker Oct 31 '11 at 10:33
I'm not even convinced it holds for infinite sets. Asaf probably knows. If you have natural isomorphisms $\textrm{Hom}(A, -) \cong \textrm{Hom}(B, -)$ then $A \cong B$ by Yoneda's lemma. I think that's the best one could hope for. – Zhen Lin Oct 31 '11 at 14:11

I am not expert in category theory, so I don't know if your question as a good answer or it is too general. Also I have some doubt that the notation $|\cdot|$ makes sense in general, because the class of morphismsmight not be a set. But maybe the following observation can help. In some cases you need an abstract version of Cantor-Bernstein theorem. For instance, take the category of sets with injections. In this case it suffices that the two sets $Hom(A,B)$ and $Hom(B,A)$ are both not empty to conclude, thanks to Cantor-Bernstein theorem, that $A$ and $B$ are isomorphic.
Here, instead, a counterexample: take the category of groups with monomorphisms: then the free group on two generators $\mathbb F_2$ embeds into the one on three generators $\mathbb F_3$ and then $Hom(\mathbb F_2,\mathbb F_3)\neq\emptyset$. Now, it's a classical result that also $Hom(\mathbb F_3,\mathbb F_2)\neq\emptyset$ but $\mathbb F_2$ is not isomorphic to $\mathbb F_3$.
[at]Valerio Capraro: the "union" $\amalg_{a,b}\hom(a,b)$ of all the morphisms in the category might not be (and usually isn't) a set; but in any sensible (i.e. useful) definition of a category, for any couple of objects $a,b$ the collection of morphisms $\hom(a,b)$ is a set (I'm not an expert in these subtleties too, but I would say that this is true at least in any 1-category). – Fosco Loregian Oct 31 '11 at 10:25
This condition holds for preorder seen as categories: that's because if there's a morphism $A \stackrel{f}{\to} B$ and a morphisms $B \stackrel{g}{\to} A$ then $g\circ f = 1_A$ and $f \circ g = 1_B$. I'm not aware if this condition holds for more general categories, but it seems a too weak condition so I don't think it's worth to considered so general categories (but maybe I'm just ignorant) .