Hint: Instead of trying to find an equation for your locus right away, it is much easier to find a parametric representation of it.
Let $A(t)=(x,y)=(R\cos(t), R\sin(t))$ and then compute where the corresponding intersection between your $BC$ and $AO$ is, as a function of $t$. You can work in terms of $x$ and $y$ rather than $\cos$ and $\sin$ most of the way.
Once you have done this, it should be easy to recognize that you have arrived at a parametric representation of what you also already know as the locus of $x^2+y^2=(R/3)^2$.
(Note that when $A$ lies on one of the axes, the two lines you're intersecting coincide, so the description of the construction in the problem does not really make sense there. From the known solution it seems that the problem setter is blithely ignoring that; you can probably get away with doing likewise).