I will just give a sketch for now how to solve it, I can fill in more details if necessary.
First define vectors $v_i = (x_i-a,y_i-b),\, i = 1,2,3$. Then we can rewrite the equations as
\begin{align}
\langle v_1, v_2\rangle = a_{12},\ \langle v_2, v_3\rangle = a_{23},\ \langle v_3, v_1\rangle = a_{31},\\
\det[v_1^t\ v_2^t] = b_{12},\ \det[v_2^t\ v_3^t] = b_{23},\ \det[v_3^t\ v_1^t] = b_{31},\\
\end{align} where I changed dot's and det's to a's and b's.
Denote by $\vartheta_{ij}$ the (oriented) angle between vectors $v_i$ and $v_j$. Remember that dot product in $\mathbb R^2$ is given by formula $\|v\|\|w\|\cos\vartheta$ and that determinants above measure (signed) area of parallelogram formed by the vectors, which can also be expressed as $\|v\|\|w\|\sin\vartheta$, and thus the system becomes
\begin{align}
\|v_1\|\|v_2\|\cos\vartheta_{12} = a_{12},\ \|v_2\|\|v_3\|\cos\vartheta_{23} = a_{23},\ \|v_3\|\|v_1\|\cos\vartheta_{31} = a_{31},\\
\|v_1\|\|v_2\|\sin\vartheta_{12} = b_{12},\ \|v_2\|\|v_3\|\sin\vartheta_{23} = b_{23},\ \|v_3\|\|v_1\|\sin\vartheta_{31} = b_{31}.\\
\end{align}
Then, you can find the lengths $\|v_i\|$ by looking at the system $$\|v_1\|\|v_2\| = c_{12},\ \|v_2\|\|v_3\| = c_{23},\ \|v_3\|\|v_1\| = c_{31},$$ where $c_{ij} = \sqrt{a_{ij}^2+b_{ij}^2}$ obtained by squaring the above equations and using $\sin^2t+\cos^2t = 1$. Solving it gives you $$\|v_1\|=\sqrt{\frac{c_{12}c_{31}}{c_{23}}},\ \|v_2\|=\sqrt{\frac{c_{12}c_{23}}{c_{31}}},\ \|v_3\|=\sqrt{\frac{c_{23}c_{31}}{c_{12}}}.$$
The whole thing is, from geometric perspective, obviously rotationally invariant (rotating all of the vectors won't change the angles between them or the areas), so fix some angle $\vartheta$. Then, the solution can be represented as complex numbers as
$$v_1 = \|v_1\|e^{i\vartheta},\ v_2 = \|v_2\|e^{i(\vartheta +\vartheta_{12})},\ v_3 = \|v_3\|e^{i(\vartheta-\vartheta_{31})}.$$
More explicitly, $e^{i\vartheta_{12}} = \cos\vartheta_{12} + i\sin\vartheta_{12} = \frac{a_{12}}{c_{12}}+\frac{b_{12}}{c_{12}}i$ and $e^{i\vartheta_{31}} = \cos\vartheta_{31} + i\sin\vartheta_{31} = \frac{a_{31}}{c_{31}}+\frac{b_{31}}{c_{31}}i$, so we have
\begin{align}v_1 &= \sqrt{\frac{c_{12}c_{31}}{c_{23}}}(\cos\vartheta + i\sin\vartheta),\\
v_2 &= \sqrt{\frac{c_{12}c_{23}}{c_{31}}}(\cos\vartheta + i\sin\vartheta)(\frac{a_{12}}{c_{12}}+i\frac{b_{12}}{c_{12}}),\\
v_3 &= \sqrt{\frac{c_{23}c_{31}}{c_{12}}}(\cos\vartheta + i\sin\vartheta)(\frac{a_{31}}{c_{31}}-i\frac{b_{31}}{c_{31}}),\ \vartheta\in\mathbb R.\end{align}
All you have to do now is expand and $x$'s will be the real parts, while $y$'s the imaginary parts of the above complex number representation.
Note, however, that the system is overdetermined since knowing the angles between $v_1$ and $v_2$ and $v_2$ and $v_3$ will also give you the angle between $v_1$ and $v_3$.
