If you just want a way to give all possible natural numbers $a,y,c,p$ satisfying your equation, then the only assumption you need is that $y>a$. In this case your equation is equivalent to
We can now assume that $y,a$ are fixed natural numbers with $y>a$. Write the fraction $(y^2-a^2)/y$ in lowest terms as $m/n$ (where $\gcd(m,n)=1$). Then the values of $p,c$ are common multiples of $m,n$, namely there is positive integer $k$ with $p=mk,q=nk$. This describes all the solutions $a,y,c,p$ in positive integers of your equation.
It seems likely you meant to ask something more particular about your equation.
EDIT: the questioner has now put it that $p$ is to be a fixed prime number. That is, the equation is now
ADDED NOTE: From the original equation, one can get to $ca^2=cy^2-py$, and then on multiplying by $4c$ and completing the square, to $(2ca)^2=(2cy-p)^2-p^2.$ So from your equation in integers we get pythagorean triples $[p,2ca,2cy-p].$ However it's unclear to me how to "work backward" from pythagorean triples and obtain all solutions of your equation, since one must have the entries of the triple in the special forms $[p,2ca,2cy-p].$ Still it seems interesting how the equation relates to pythagorean triples!
RE-EDIT. The questioner has now allowed $p$ composite also, but still the idea is to have $p$ fixed, and look for the $a,y,c$ making the equation hold. So now it looks even more complex to find all solutions!
This seems more interesting, and can be divided into cases as to whether $p$ divides $c$,in which case putting $c=pk$ makes the equation $y=k(y^2-a^2)$, or $p$ does not divide $k$, in which case $p$ must divide one of $y-a,y+a$. This breakdown may be a start.