Find all prime solutions of equation $5x^2-7x+1=y^2.$ Find all prime solutions of the equation $5x^2-7x+1=y^2.$
It is easy to see that 
$y^2+2x^2=1 \mod 7.$ Since $\mod 7$-residues are $1,2,4$ it follows that $y^2=4 \mod 7$, $x^2=2 \mod 7$  or $y=2,5 \mod 7$ and $x=3,4 \mod 7.$ 
In the same way from $y^2+2x=1 \mod 5$ we have that $y^2=1 \mod 5$ and $x=0 \mod 5$ or $y^2=-1 \mod 5$ and $x=4 \mod 5.$
How put together the two cases?
Computer find two prime solutions $(3,5)$ and $(11,23).$
 A: Since $(0,1)$ is a solution to $5x^2-7x+1=y^2$, it  can be  used  to parametrize all rational solutions to $5x^2-7x+1=y^2$. That will give us:
$$x:=\frac{-2ab - 7b^2}{a^2 - 5b^2}$$
and
$$y:=\frac{a}{b}x+1$$
where $a,b\in \mathbb{Z}$, $b\neq 0$.
Since $x$ is prime, it follows that either $b=1$ or $x=b$.

*

*case $b=1$
This gives us $x:=(-2a - 7)/(a^2 - 5)$. Since $x\in \mathbb{Z}$,
we get  $a=\pm 2$, and then $x=11$ or $x=3$.
Those will give $y=23$ or $y=5$, repectively.


*case $x=b$.
We have  that $\frac{-2ab - 7b^2}{a^2 - 5b^2}=b$ gives
$$(*) \hspace{2cm} a(a + 2) =5b^2 - 7b.$$
Since $y=a+1$ and $x=b$ are prime, and  $x=2$ or $y=2$ do not give solutions to $5x^2-7x+1=y^2$, we conclude  that  $y=a+1$ and $x=b$ are ODD primes. In particular, $a(a + 2)\equiv 0 \mod 4$.
Now reducing (*) $\mod 4$, contradicts the fact that $b$ is odd. Thus, the case $x=b$ does not occur.
Therefore  $(x,y)= (11,23)$ and  $(x,y)=(3,5)$ are the only solutions
where both coordinates are prime numbers.
A: Try working mod $3$ and mod $8$. Assuming $x, y>3$, we have $x,y = \pm 1$ mod $3$. Since $x, y$ are odd we have $x^2, y^2=1$ mod $8$, so
$$x^2, y^2 = 1 \text{ mod } 24.$$
Substituting in the equation gives $$x = 24k+11 $$ for some integer $k$.
Rearranging the original equation we get
$$x(5x-7)=(y-1)(y+1), \tag{1}$$
therefore $x |y-1$ or $x|y+1$, since $x$ is a prime number.
Solving for $x$ gives
 $$ x = \frac{17}{10} + \frac{1}{10}\sqrt{20y^2+29}>\frac{1}{3}(y+1).$$
Note that $x$ is odd and $y \pm 1$ is even, so $x \ne y\pm1$. This forces $x = \frac{1}{2} (y \pm1)$, or 
$$y = 2x \pm 1 = 48k + 22 \pm 1 \Rightarrow y = 48k+23.$$
$48k+21$ is rejected being divisible by 3.
Plugging these in $(1)$ gives the solution $k=0$ or
$$x = 11, \space y = 23.$$
A: Completing the square and dividing by $5$, we have 
$$ (10 x - 7)^2 - 20 y^2 = 29$$
Thus $z = 10 x - 7$ and $w = 2 y$ are solutions of the Pell-type equation
$$ z^2 - 5 w^2 = 29$$
The positive integer solutions of this can be written as
$$\pmatrix{z\cr w\cr} = \pmatrix{9 & 20\cr 4 & 9\cr}^n \pmatrix{7\cr 2\cr} \ \text{or}\ \pmatrix{9 & 20\cr 4 & 9\cr}^n \pmatrix{23\cr 10\cr}$$
for nonnegative integers $n$.
Now you want $w$ to be even and $z \equiv 3 \mod 10$.  All the solutions will have $w$ even, and $z$ altermately $\equiv \pm 3 \mod 10$.  Thus for $n$ even you get integers for $x,y$ with
$$ \pmatrix{z_n\cr w_n\cr} = \pmatrix{9 & 20\cr 4 & 9\cr}^n \pmatrix{23\cr 10\cr}$$
and for $n$ odd, 
$$ \pmatrix{z_n\cr w_n\cr} = \pmatrix{9 & 20\cr 4 & 9\cr}^n \pmatrix{7\cr 2\cr}$$
You do get primes for $n=0$ ($z_0 =  23, w_0 = 10, x_0 = 3, y_0 = 5$)
and $n=1$ ($z_1 = 103, w_1 = 46, x_1 = 11, y_1 = 23$).  In general,


*

*$x_n \equiv 0 \mod 3$ for $n \equiv 0$ or $3 \mod 4$. 

*$x_n \equiv 0 \mod 17$ for $n \equiv 2$ or $3 \mod 6$.  

*$x_n \equiv 0 \mod 5$ or $y_n \equiv 0 \mod 5$ for $n \equiv 0,3, 4, 5, 6, 9 \mod 10$. 

*$x_n \equiv 0 \mod 11$ for $n \equiv 1, 8 \mod 10$.

*$x_n \equiv 0 \mod 13$ for $n \equiv 3, 5, 6, 7, 8, 10 \mod 14$.

*$y_n \equiv 0 \mod 23$ for $n \equiv 1, 2, 5, 6 \mod 8$.


And every integer $n$ is in at least one of these classes.  We conclude there are no other prime solutions. 
A: Can we just charge straight at this?
$y$ is odd. $x=2 \ (\Rightarrow y^2=7)$ is not a solution, so $x$ is an odd prime.
$x(5x-7) = (y-1)(y+1)$, so $x \mid (y-1) $ or $x \mid (y+1)$  ($x$ is prime)
 so $kx=y\pm1$, $k$ even
$k\ge4$ is too large: $(kx\pm1)^2\ge (4x-1)^2 $ $= 16x^2-8x+1$ $>5x^2-7x+1$.  So only $k=2$, that is $x=\frac 12(y\pm1)$, makes the equality feasible.
Considering the two cases:  


*

*(1) $x=\frac 12(y+1)$, $y=2x-1$:
$x(5x-7) = 4x(x-1) \implies x = 3, y=5$

*(2) $x=\frac 12(y-1)$, $y=2x+1$:
$x(5x-7) = 4x(x+1) \implies x = 11, y=23$
Note that I didn't constrain $y$ at any point - the two solutions just happened to have $y$ prime.
A: If there is a solution it has more than 4000 digits, which makes me think there is no solution beyond the one two already mentioned.
In Mathematica,
i=1;
ans=Solve[5 x ^2-7x+1==y^2&&x>0&&y>0,{x,y},Integers];
ans2=ans/.C[1]->i//RootReduce
Dynamic@i
Dynamic[IntegerLength@ans2[[All,All,2]]]

and then run
    While[FreeQ[PrimeQ[ans2[[All,All,2]]],{True,True}],i++;ans2=ans/.C[1]->i//RootReduce]

Setting i=0 will stop the loop with the two known solutions:
{{x->3,y->5},{x->11,y->23}}

but there appear to be no easily found solutions beyond i=1
