Let $a,b$ be positive integers.
When $$k = \frac{a^2 + b^2}{ab+1}$$ is an integer, it is a square.
Proof 1: (Ngô Bảo Châu): Rearrange to get $a^2-akb+b^2-k=0$, as a quadratic in $a$ this has two values: $a$ and $kb - a = (b^2-k)/a$. (The second root is determined in two different ways from the expansion $(x-r_1)(x-r_2) = x^2 - (r_1 + r_2)x + r_1 r_2$.)
Now suppose we have $a,b$ such that $k$ is an integer but not a square, by the investigation about roots we have that the second root is a nonzero integer since $k,b,a$ are integers and $k \not = b^2$, futhermore it is positive which is easily seen from its defining equation.
WLOG assume $a \ge b$ so that the second root is strictly smaller than $a$. This leads to a decent, replacing $a$ with the second root.
Proof 2 (Don Zagier): Apply reduction theory (specifically, Sätze 1 and 2 of Section 13 of my book on quadratic fields) to the quadratic form $x^2 + kxy + y^2$, which is the unique reduced quadratic form in its equivalence class.
Note that Proof 2 is pretty much the same as Proof 1 when written out in explicit detail, but I could not read Zagier's book because I cannot read German.
I would like to know more approaches to this and other alternative proofs of this result if possible! Thanks in advance.
I would also be interested in related problems (especially easier ones of a similar nature) and texts which cover the reduction theory in English.
