If $\cos3A + \cos3B + \cos3C = 1$ in a triangle, find one of its length I would like to solve the following problem.

In $\triangle ABC, AC = 10, BC = 13$. If $\cos3A + \cos3B + \cos3C = 1$, compute the length of $AB$.

I thought that I could apply the Law of Cosines. Using the fact that $A+B+C=\pi$, I attempted to build the equation up from there.
What I got was that $$\cos3A+\cos3B-\cos(3A+3B)=1$$
Expanding, I got $$\cos3A+\cos3B-\cos3A\cos3B-\sin3A\sin3B=1$$
Now, it's possible that I'd be able to factor it somehow by rewriting it all in terms of cosine and arrive at the answer, but is there a better way to solve the problem? Thanks!
 A: $\mathbf {Hint...}$
$$\cos {3A}+\cos {3B}+ \cos{3C}=1$$
$$\Rightarrow 4 \sin {\frac {3C}{2}} .\sin {\frac{3B}{2}} .\sin{\frac{3A}{2}}=0$$
Hence the largest angle of triangle is $\frac{2\pi}{3}$ which can be either angle $C$ or angle $A$. By applying cosine rule in each of these cases we get the value of $AB$ as $\sqrt {399}$ or $\sqrt {94}-5$ respectively.
Note: 
$$\cos {3A}+\cos {3B}+ \cos{3C}=1$$
$$\Rightarrow -2\cos {\frac {3(A-B)}{2}}\sin {\frac {3C}{2}}- 2\left(\sin  {\frac {3C}{2}}\right)^2=0$$
$$=\sin\frac{3C}{2}\left(\cos\frac{3(A-B)}{2}+ \sin\frac{3C}{2}\right)=0$$ 
$$\sin\frac{3C}{2}\left(\cos\frac{3(A-B)}{2}-\cos\frac{3(A+B)}{2}\right)=0$$
$$\sin\frac{3C}{2}\sin\frac{3B}{2}\sin\frac{3A}{2}=0.$$
A: We have $$2\cos\frac{3A+3B}{2}\cos\frac{3A-3B}{2}-2\sin^2\frac{3C}{2}=0$$ or
$$\cos\left(\frac{3\pi}{2}-\frac{3C}{2}\right)\cos\frac{3A-3B}{2}-\sin^2\frac{3C}{2}=0$$ or
$$\sin\frac{3C}{2}\left(-\cos\frac{3A-3B}{2}-\sin\frac{3C}{2}\right)=0$$ or
$$\sin\frac{3C}{2}\left(-\cos\frac{3A-3B}{2}+\cos\frac{3A+3B}{2}\right)=0$$ or
$$\sin\frac{3C}{2}\sin\frac{3C}{2}\sin\frac{3C}{2}=0$$ or
$$\prod_{cyc}\left(3\sin\frac{A}{2}-4\sin^3\frac{A}{2}\right)=0$$ or
$$\prod_{cyc}\left(3-4\sin^2\frac{A}{2}\right)=0$$ or
$$\prod_{cyc}\left(3-2(1-\cos{A})\right)=0$$ or
$$\prod_{cyc}\left(1+\frac{b^2+c^2-a^2}{bc}\right)=0$$ or
$$\prod_{cyc}(b^2+c^2+ab-a^2)=0.$$
Can you end now?
A: Using the identity
\begin{align} 
\cos3\alpha+\cos3\beta+\cos3\gamma
&=
1-\frac{r\,(3\,\rho^2-(r+3\,R)^2)}{R^3}
\tag{1}\label{1}
,
\end{align}
where $r$ is the inradius, $R$ is the circumradius
and $\rho$ is the semiperimeter 
of the triangle, 
condition 
\begin{align} 
\cos3\alpha+\cos3\beta+\cos3\gamma&=1
\end{align}
results in
\begin{align} 
3\,\rho^2-(r+3\,R)^2&=0
,\\
3\,\rho^2-\left(\frac{S}{\rho}+3\,\frac{abc}{4S}\right)^2&=0
,\\
3\,\rho^2
-\frac{S^2}{\rho^2}
-\frac{3abc}{2\rho}
-\tfrac{9}{16}\frac{(abc)^2}{S^2}&=0
\tag{2}\label{2}
,\\
\end{align}
where $S$ is the area of triangle.
With the help of substitution
\begin{align}
\rho&=\tfrac12(a+b+c)
,\\
S^2&=\tfrac1{16}(a+b+c)(-a+b+c)(a-b+c)(a+b-c)
,\\
a&=13
,\quad
b=10
\end{align}  
equation \eqref{2} becomes
\begin{align}
\frac{(c^2-399)(c^2+13c+69)(c^2+10c-69)}{(c^2-9)(c^2-23^2)}
&=0
,
\end{align}
with just two suitable roots,
\begin{align}
c_1&=\sqrt{399}
,\\
c_2&=\sqrt{94}-5
.
\end{align}

A: $$\cos {3A}+\cos {3B}+ \cos{3C}=1$$
$$\Rightarrow 4 \sin {\frac {3C}{2}} .\sin {\frac{3B}{2}} .\sin{\frac{3A}{2}}=0$$
Hence the largest angle of triangle is $\frac{2\pi}{3}$ which can be either angle $C$ or angle $A$. By applying cosine rule in each of these cases we get the value of $AB$ as $\sqrt {399}$ or $\sqrt {94}-5$ respectively.
