The number of the group homomorphsim , $\phi : S_3 \to S_3$ Surely, $S_3 = \langle(1,2), (1,2,3)\rangle$ 
And Put $\alpha = (1,2)$ and $\beta = (1,2,3)$ for homomorphism, $\phi : S_3 \to S_3$
Say $\phi(\alpha) =f(\in S_3)$ and $\phi(\beta) =r(\in S_3)$
Then, $\vert f \vert $ and $\vert r \vert$ should be divisor of the $2$ and $3$ respectively. 
Hence there are 4 cases like the below.
1) $\vert f \vert  =  1$ and $\vert r \vert  =  1$ (The number of the $\phi$ is $1$)
2) $\vert f \vert  =  1$ and $\vert r \vert  =  3$  (The number of the $\phi$ is $2$)
3) $\vert f \vert  =  2$ and $\vert r \vert  =  1$(The number of the $\phi$ is $3$)
4) $\vert f \vert  =  2$ and $\vert r \vert  =  3$(The number of the $\phi$ is $6$)
So my answer is $12$.
But the answer of that was $10$ not $12$. 
Which point I was wrong? Any advice would be appreciated. 
 A: Just because a group $G$ is generated by elements $g_1$ and $g_2$, this does not mean that, whenever $h_1$ and $h_2$ are elements of $G$ with orders dividing those of $g_1$ and $g_2$, respectively, there is an endomorphism of $G$ mapping $g_i$ to $h_i$.
For example, your case 2) is never attainable: Suppose $\phi(1,2)=1$. Then
\begin{align*}
\phi(1,3)&=\phi((2,3)^{-1}(1,2)(2,3))\\
&=\phi(2,3)^{-1}\phi(1,2)\phi(2,3)\\
&=\phi(2,3)^{-1}\phi(2,3)\\
&=1
\end{align*}
But $(1,2)$ and $(1,3)$ generate $S_3$, so $\phi=1$, the trivial endomorphism.

So the argument should be a little more refined, using the group structure of $S_3$.
As you already notices, $|\phi(a)|$ always divides $|a|$. Let us look at a few cases:


*

*The computation above shows that if $\phi(1,2)=1$, then $\phi$ is the trivial endomorphism.


So suppose $\phi(1,2)\neq 1$. Then $\phi(1,2)$ is a $2$-cycle, say $\phi(1,2)=(a,b)$. We have two further possibilities:


*$|\phi(1,2,3)|=1$:In this case, the image of $\phi$ will be contained in $\langle \phi(1,2),\phi(1,2,3)\rangle=\left\{(a,b),1\right\}$, which is the group with two elements. In this case, $\phi$ will just be a realization of the "parity" homomorphism: $$\phi(x)=\begin{cases}
(a,b)&\text{, if }x\text{ is odd}\\
1&\text{ if }x\text{ is even}
\end{cases}$$
For these cases, $\phi$ is completely determined by the choice of $a$ and $b$ (and the order does not matter): there are $3\times 2/2=3$ choices for $a$ and $b$ in this case.

*$|\phi(1,2,3)|=3$.
So up to changing the order of the $a$ and $b$ (because $(a,b)=(b,a)$), we can write $\phi(1,2,3)=(a,b,c)$.
So we see that $\phi$ is just "swapping $1$ with $a$, $2$ with $b$ and $3$ with $c$".
More formally, let $\tau\in S_3$ be defined as $\tau(a)=1$, $\tau(b)=2$ and $\tau(c)=3$. Then $\phi(x)=\tau x\tau^{-1}$ whenever $x$ is any of the generators $(1,2)$ or $(1,2,3)$. Since the "conjugation by $\tau$", $c_\tau:x\mapsto \tau x\tau^{-1}$ is an endomorphism, we actually have $\phi=c_\tau$ on all of $S_3$.

This means that is $|\phi(1,2)|=2$ and $|\phi(1,2,3)|=3$, then $\phi$ is just conjugating with some element of $S_3$. There are $6$ elements in $S_3$, and all the conjugation endomorphisms (the endomorphisms of the form $c_\tau$ for some $\tau\in S_3$) are different. Since $S_3$ has $6$ elements, there are $6$ such "conjugation morphisms".
In summary, we have
- the trivial morphism (if $|\phi(1,2)|=1$);
- 3 morphisms if $|\phi(1,2)|=2$ and $|\phi(1,2,3)|=1$;
- 6 morphisms if $|\phi(1,2)|=2$ and $|\phi(1,2,3)|=3$;
so in total, 10 endomorphisms of $S_3$.
A: $|f|=1$ and $|r|=3$ is not possible. If an element of order $2$ is in the kernel then the homomorphism is trivial because no proper normal subgroup contains an element of order $2$.
A: When classifying homomorphisms of a group, generally one cannot prescribe the orders of images of generators independently.
Indeed: If $\operatorname{ord}(\phi((12))) = 1$, then $\phi((12)) = 1$ (the identity permutation). But in a symmetric group any conjugacy class consists precisely of all permutations of a given cycle type, so there is some element $g \in S_3$ such that $$g (12) g^{-1} = (23) .$$ (I encourage you to verify this claim in this case by finding such an element $g$ explicitly.)

But since $\phi$ is a homomorphism, $$\phi((23)) = \phi(g (12) g^{-1}) = \phi(g) \phi((12)) \phi(g^{-1}) = \phi(g) \phi(g)^{-1} = 1$$ and so $$\phi((123)) = \phi((12)(23)) = \phi((12)) \phi((23)) = 1 \cdot 1 = 1.$$ In summary, the condition $\phi((12)) = 1$ in fact forces $\phi$ to be trivial, so there are no homomorphisms $S_3 \to S_3$ satisfying case (2). The remaining cases add up to $10$ distinct homomorphisms, but our observation in this answer shows that more is required to verify that all all of the remaining cases really are realizable.

Remark More generally, if a homomorphism $\psi : S_n \to S_n$ maps any transposition to $1$, then since all transpositions are conjugate, the above argument shows that $\psi$ maps all transpositions to $1$. But $S_n$ is generated by its transpositions, so any such $\phi$ is actually the trivial homomorphism.
This gives another illustration of the first statement in this answer. Another generating set for $S_3$ is $\{(12), (13)\}$, but since both of these generators have the same cycle type, their images under a homomorphism $S_3 \to S_3$ necessarily coincide.
