Finding value of 1 variable in a 3-variable $2^{nd}$ degree equation The question is: If $a,b,\space (a^2+b^2)/(ab-1)=q$ are positive integers, then prove that $q=5$. Also prove that for $q=5$ there are infinitely many solutions in $\mathbf N$ for $a$ and $b$. I simplified the equation as follows:-$$\frac {a^2+b^2}{ab-1}=q\\\begin{align}\\&=>\frac {2a^2+2b^2}{ab-1}=2q\\&=>\frac{a^2+b^2+2ab+a^2+b^2-2ab}{ab-1}=2q\\&=>(a+b)^2+(a-b)^2=2q(ab-1)\\&=>2(a+b)^2+2(a-b)^2=q(4ab-4)\\&=>2(a+b)^2+2(a-b)^2=q((a+b)^2-(a-b)^2-4)\end{align}$$Substituting $a+b=X$ and $a-b=Y$, we get $$2X^2+2Y^2=q(X^2-Y^2-4)\\\begin{align}&=>(q-2)X^2=(q+2)Y^2+4q\end{align}$$Now using the quadratic residues modulo $5$, I know that $X^2,Y^2\equiv0, \pm1(mod\space 5)$. But using this directly doesn't give the answer. So what to do after this? An answer without the use of co-ordinate geometry would be greatly appreciated as it seems there is a very good resemblance of the equation to a pair of hyperbolas which are symmetric with respect to the line $y=x$ but I don't understand co-ordinate geometry very well. 
 A: I take it that you have looked into the details of Vieta Jumping wiki link, so if we know the curve $x^2+y^2 - qxy + q = 0$ has a single integer solution then there are infinitely many, generated by the pair of iterations:
$$(x,y) \mapsto (qx-y,x)$$ $$(x,y) \mapsto (y,x)$$
on the graph of the curve.
Further observe that for a lattice point $(x,y)$ with $y > x >0$, the orbit of the iteration $(x,y) \mapsto (qx-y,x)$ does not change the branch of hyperbola, i.e., remains in the branch in the first quadrant.
Geometrically this is equivalent to moving from a lattice point $(x_0,y_0)$ to the point $(x_0,x_0)$ on the line $y = x$ and then join and extend the line joining $(x_0,x_0)$ and $(y_0,x_0)$ to meet the curve again at $(qx_0 - y_0,x_0)$.A sequence of iterations that will look like: 

Now, $x^2+y^2 - qxy + q = 0$ has no solution in natural numbers if $q = 1,2$
since,$x^2+y^2 - qxy + q > 0$ for $q = 1,2$.Thus, $q \ge 3$.
We note that the vertex of the branch in concern $\color{blue}{A} = \left(\sqrt{\frac{q}{q-2}},\sqrt{\frac{q}{q-2}}\right)$ lies on the line joining $\color{black}{J} = (1,1)$ and $\color{black}{H} = (2,2)$ for $q \ge 3$.
If $I = (x_i,y_i)$ is the lattice point in the orbit of the iteration closest to the vertex $\color{blue}{A}$, then the next point in the orbit $K$ no longer stays in the upper half (region $y>x$) part of the hyperbola, $K = (x_k,y_k)$ must lie in the lower half region ($x > y$).
But in order to go from $I$ to $K$ we find that the point on the line $x = y$, i.e., $(y_k,y_k)$ must be a lattice point as well. The only lattice point on this line between the origin and vertex $\color{blue}{A}$ is $J = (1,1)$, hence $y_k = 1$.
Therefore, both the roots of the quadratic $x^2+1 - qx + q = 0$ must be positive integers, viz $x_k$ and $qx_k - 1$.
The discriminant $\Delta = q^2 - 4(q+1)$ must be a perfect square. But $\Delta$ differs from $(q-2)^2$ by $8$, and the only perfect squares that differ by $8$ are $1$ & $9$.
$$\implies (q-2)^2 = 9 \implies  q = 5 \textrm{ since } q \ge 3$$
A: For such equations:   
$$\frac{x^2+y^2}{xy-1}=-t^2$$    
Using the solutions of the Pell equation.  $$p^2-(t^4-4)s^2=1$$    
You can write the solution.    
$$x=-4tps$$  $$y=t(p^2+2t^2ps+(t^4-4)s^2)$$  
It all comes down to the Pell equation - as I said.  Considering specifically the equation:  
$$\frac{x^2+y^2}{xy-1}=5$$  
Decisions are determined such consistency.  Where the next value is determined using the previous one.  
$$p_2=55p_1+252s_1$$  $$s_2=12p_1+55s_1$$  
You start with numbers.  $(p_1;s_1) - (55 ; 12)$  
Using these numbers, the solution can be written according to a formula. 
$$y=p^2+2ps+21s^2$$  $$x=3p^2+26ps+63s^2$$  
If you use an initial $(p_1 ; s_1) - (1 ; 1)$
Then the solutions are and are determined by formula.
$$y=s$$
$$x=\frac{p+5s}{2}$$
As the sequence it is possible to write endlessly. Then the solutions of the equation, too, can be infinite.
