I need help with solving this geometry problem.

Please, help me with this problem. I've been trying to solve it for more than week and I am not even sure it is correct.

Given an obtuse triangle $\triangle ABC$ with obtuse angle $\angle BAC$. The internal bisector of angle $\angle B$ intersects the side $AC$ at point $E$ and the external bisector of angle $\angle A$ at point $O$.

If the center of the circumcircle of $\triangle AEO$ lies on $AB$ and $AE=2$, $EC=3$, find the area of triangle $\triangle ABC$.

• What have you figured out so far? – Ron Gordon Feb 11 '13 at 10:36
• It is beyond me. Maybe the problem is not correct. We have only one side given. It seems impossible to find the area. – user61810 Feb 11 '13 at 10:38
• I think it should state "If the center of the circumcircle of AOE lies on AB and AE=2, EC-3, find the area of triangle ABC". But even in this case it seems unsolvable. – user61810 Feb 11 '13 at 10:44
• I don't get it: "...and the external bisector of angle $\,\angle A\,$ at $\,O\,$"...**what** "at $\,O\,$?? The external bisector of A meets AC at A, of course...or am I missing something? – DonAntonio Feb 11 '13 at 12:37
• @DonAntonio: I take that sentence to say (1) $E$ is the intersection of the internal bisector and side $AC$; and (2) $O$ is the intersection of the internal bisector and the external bisector. – Blue Feb 11 '13 at 12:55

Let $K$ (on an extension of $AB$) be the center of the circumcircle of $\triangle OEA$, and let the circumradius be $r$. Let $\alpha = \angle CAO = \angle KAO$ be the external half-angle at $A$. Note that these two parameters are dependent: $\triangle KAE$ is isosceles with legs $r$, base $2$, and base angles $2\alpha$; thus, $r\cos 2\alpha = 1$.

Since $\triangle AKO$ is isosceles with $\angle KOA = \angle KAO = \alpha = \angle CAO$, we have $OK \parallel CA$. This guarantees $\triangle OKB \sim \triangle EAB$, and we see

$$\frac{|OK|}{2} = \frac{|BK|}{|AB|} \implies \frac{r}{2} = \frac{|AB|+r}{|AB|} \implies |AB|= \frac{2 r}{r-2}$$

(Observe that this requires $r > 2$.) By the Angle Bisector Theorem in $\triangle ABC$,

$$\frac{|AE|}{|AB|} = \frac{|CE|}{|CB|} \implies \frac{2}{|AB|}=\frac{3}{|CB|} \implies |CB| = \frac{3r}{r-2}$$

Via the Law of Cosines at (internal) $\angle A$ in $\triangle ABC$,

$$\cos\left(\pi-2\alpha\right) = \frac{|AC|^2+|AB|^2-|BC|^2}{2|AC||AB|} = \frac{r^2-5r+5}{r(r-2)}$$

Recalling that $r\cos 2\alpha=1$ (and, of course, that $\cos(\pi-2\alpha)=-\cos 2\alpha)$), we have

$$-\frac{1}{r} = \frac{r^2-5r+5}{r(r-2)} \implies (r-1)(r-3) = 0 \implies r = 3 \quad \text{(since r>2)}$$

Therefore,

$$\cos 2\alpha = \frac{1}{3} \implies \sin 2\alpha = \frac{2\sqrt{2}}{3}$$ $$|AB| = 6$$

so that

$$|\triangle ABC| = \frac{1}{2} |AB||AC| \sin 2\alpha = 10 \sqrt{2}$$