You have already found that: $\forall \ n, a_n+b_n+c_n=1$
Without loss of generality, suppose that: $a_{n}\leq b_{n}\leq c_{n}$. Then:
$a_{n+1}=a_{n}^{2}+2b_{n}c_{n}=a_{n}^{2}+b_{n}c_{n}+b_{n}c_{n}\leq a_{n}c_{n}+b_{n}c_{n}+c_{n}c_{n}=c_{n}(a_{n}+b_{n}+c_{n})=c_{n}$.
Similary,
$b_{n+1}=b_{n}^{2}+2a_{n}c_{n}\leq b_{n}c_{n}+a_{n}c_{n}+c_{n}^{2}=c_{n}(a_{n}+b_{n}+c_{n})=c_{n}$
And, $c_{n+1}=c_{n}^{2}+2a_{n}b_{n}\leq c_{n}^{2}+a_{n}c_{n}+a_{n}b_{n}=c_{n}$.
So: $\max{\{a_{n+1},b_{n+1},c_{n+1}\}}\leq c_{n}$
Had we supposed that $a_{n}$ and $c_{n}$ were bounded above by $b_{n}$ or $b_{n}$ and $c_{n}$ were bounded above by $a_{n}$ we would have gotten respectively:
$\max{\{a_{n+1},b_{n+1},c_{n+1}\}}\leq b_{n}$ or $\max{\{a_{n+1},b_{n+1},c_{n+1}\}}\leq a_{n}$
So, finally:
$\max{\{a_{n+1},b_{n+1},c_{n+1}\}}\leq \max{\{a_{n},b_{n},c_{n}\}}$
A similar argument can be used for the smallest term:
supposing again that: $a_{n}\leq b_{n}\leq c_{n}$, we have:
$a_{n+1}=a_{n}^{2}+2b_{n}c_{n}\geq a_{n}^{2}+b_{n}a_{n}+c_{n}a_{n}= a_{n}$ and so on.
So we conculde that the largest $n$-th term forms a monotonic decreasing sequence which is bounded below while the smallest $n$-th term forms an increasing sequence bounded above.
This answer is in the link below where they go further to find the limit of each sequence.
Source:
This problem is from Putnam Competition 1947
https://mks.mff.cuni.cz/kalva/putnam/psoln/psol475.html