Are groups with all the same Hom sets already isomorphic? I was thinking about the following: Say we got two groups $A$ and $B$, and we know that for any group C, there is a bijection
$$Hom(A,C) \to Hom(B,C).$$
Are $A$ and $B$ already isomorphic?
If the family of bijections was natural in $C$, this follows from Yoneda's lemma. But does it hold anyway without naturality, just from the cardinalities? I would suspect no but in simple cases (finitely generated). I'd like to hear any thoughts on how to come up with a proof or counterexample.
Also, does the answer change when we consider the contravariant case with an identification $Hom(C,A) \to Hom(C,B)$?
 A: If there is a surjective homomorphism $\rho:A \rightarrow B$ then, for any group $C$,  $\alpha \mapsto \alpha \circ \rho$ is an injective map ${\rm Hom}(B,C) \to {\rm Hom}(A,C)$.
So, for any pair of groups $A$ and $B$ which are isomorphic to homomorphic images of each other, we have $|{\rm Hom}(A,C)| = |{\rm Hom}(B,C)|$.
There are examples of pairs of non-isomorphic finitely generated groups with this property. An example is given in Theorem 3 of the well-known paper "Some two-generator one-relator non-Hopfian groups" by Baumslag and Solitar
$$A = \langle a,b \mid a^{-1}b^2a=b^3 \rangle,$$
$$B = \langle c,d \mid c^{-1}d^2c=d^3, ([c,d]^2c^{-1})^2=1 \rangle.$$
A: Let $A$ be the free group with countably many generators, and let $B$ be group with countably many generators of which one has the relation $g^2=e$.
These are not isomorphic, because a free group is torsion-free and $B$ isn't.
If $C$ is any group with (finite or infinite) cardinality $\lambda$ of which $\kappa$ have order 1 or 2, then we have
$$ \begin{align}
|\operatorname{Hom}(A,C)| &= \lambda^{\aleph_0} \\
|\operatorname{Hom}(B,C)| &= \lambda^{\aleph_0}\cdot\kappa \end{align} $$
which are equal because $1\le \kappa \le \lambda $.

For the contravariant case, we can dualize Derek's idea: If there are injective homomorphisms $A\to B$ and $B\to A$, then $|\operatorname{Hom}(C,A)|=|\operatorname{Hom}(C,B)|$.
In that case we can take $A$ to be $\mathbb Q^\omega$ and $B=\mathbb Z\times\mathbb Q^\omega$. Then $A$ and $B$ are not isomorphic (because $B$ has an element that is not twice anything), but they easily inject into each other.
(As Derek notes in the comments you can also take $A$ and $B$ to be free groups of different finite ranks $\ge2$, for a finitely generated example. For example, if $A$ is the free group on $\{a,b\}$ and $B$ is the free group on $\{a,b,c\}$, then $A$ injects natively into $B$, and $\{a\mapsto ab, b\mapsto a^2b^2, c\mapsto a^3b^3\}$ generates an injective homomorphism $B\to A$).
