According to maple, $a=-3/4, b=21/4$ is one solution, and if $r$ is a solution of $2x^2-12x+15=0$ then $a=r/2, b=12-(11/2)r$ is another solution. As the quadratic here has two real zeros, there are three pairs $(a,b)$ of real numbers for your system.
Of course this is not an elegant solution, and even worse it relies on maple. But it seemed odd to me since when the equations are subtracted and the result solved for $b$, we get
$b=(26a^2-40a+21)/(8-6a)$, which may then be put into the first equation, so I would have expected a fourth degree equation for a.