Eliminate $\theta$ from $4x=5\cos\theta -\cos 5\theta$ and $4y=5\sin\theta -\sin 5\theta$ Eliminate $\theta$ from the equations.
$$4x=5\cos\theta -\cos 5\theta$$
$$4y=5\sin\theta -\sin 5\theta$$
Alternative forms are
$$x=5\cos^3\theta-4\cos^5\theta$$
$$y=5\sin^3\theta-4\sin^5\theta$$
and $$x=\cos^3\theta(3-2\cos2\theta)$$
$$y=\sin^3\theta(3+2\cos2\theta)$$
 A: These are the parametric equations of an epicycloid, where the large circle has radius $R=1$, and the smaller exterior rolling circle has radius $r=\frac{1}{4}$.
A possible simplification is writing
$$4a=4(x+i y) = 5 z - z^ 5\\
4b=4(x-i y) = \frac{5}{z} - \frac{1}{z^5}$$
where $z = \cos \theta + i \sin \theta$
and get a relation between $a$, $b$, that implies a relation between $x$ and $y$, see link. Or, we can just give the result provided by WA :
$$-81 - 45 x^2 + 365 x^4 - 15 x^6 - 480 x^8 + 256 x^{10} - 45 y^2 - 2395 x^2 y^2 - 45 x^4 y^2 - 1920 x^6 y^2 + 1280 x^8 y^2 + 365 y^4 - 45 x^2 y^4 - 2880 x^4 y^4 + 2560 x^6 y^4 - 15 y^6 - 1920 x^2 y^6 + 2560 x^4 y^6 - 480 y^8 + 1280 x^2 y^8 + 256 y^{10}=0$$
a curve of degree $10$
$4$ leaves" />
We can simplify the defining polynomial by writing it as a polynomial in $x^2 + y^2$ and $x^2 y^2$.
$\bf{Added:}$ Following the solution of @ACB: we have the polar representation ($x = r \cos \phi$, $y=r \sin \phi$)
$$\cos^2 2 \phi = \frac{(9 - 4 r^2)(16 r^4+ 3 r^2 + 6)^2}{3125 r^4}= f(r)$$
The function in $r>0$ on RHS is strictly decreasing on $(0, \infty)$. Indeed, we have
$$f'(r) = -24\, \frac{(-1 + r^2)^2 (9 + 16 r^2) (6 + 3 r^2 + 16 r^4)}{3125\, r^5}$$
Now, $f(1) = 1$, and $f(\frac{3}{2})=0$. We see that our curve is situated between the circles of radius $1$, and $\frac{3}{2} = 1 + 2\cdot \frac{1}{4}$, as it should.
So now we can write the curve in polar coordinates
$$r = f^{-1}(\cos^2 2\phi)$$
where $f^{-1}\colon [0,1]\to [1, \frac{3}{2}]$ is the inverse function of $f$.
A: Let $c=\cos2\theta$.
$$4x=5\cos\theta -\cos 5\theta\tag1$$
$$4y=5\sin\theta -\sin 5\theta\tag2$$
Squaring and adding ,$$16(x^2+y^2)=25+1-10\cos4\theta$$ $$8(x^2+y^2)=13-5(2c^2-1)$$ $$5c^2=9-4(x^2+y^2)=k$$
Squaring and subtracting, $$16(x^2-y^2)=25\cos2\theta+\cos10\theta-10\cos6\theta$$ $$16(x^2-y^2)=25c+(16c^5-20c^3+5c)-10(4c^3-3c)$$ $$4(x^2-y^2)=c(4c^4-15c^2+15)$$
Substituting $k$,
$$100(x^2-y^2)=c(4k^2-75k+375)$$
Further simplifying leads to, $$25(x^2-y^2)=c(16\lambda^2+3\lambda+6)$$ where $\lambda=(x^2+y^2)$.
Squaring again and replacing all values finally we get, $$3125(x^2-y^2)^2=[9-4(x^2+y^2)][16(x^2+y^2)^2+3(x^2+y^2)+6]^2$$
And this is that epicycloid, as being described in @orangeskid's answer!
