Finding positive integer solutions to $n = ax^2 +by^2 - cxy$ How can I find the positive integer solutions to $x$ and $y$, given that $n$, $a$, $b$ and $c$ are all positive integers, in an equation of the form:
$$n = ax^2 + by^2 - cxy.$$
Specifically, I want to find the positive integer solutions to the following equation, given $n$:
$$n=3 x^2+20 y^2-16 x y.$$
 A: $$ n = (3x - 10y)(x-2y)  {}{}  $$
Alright, changing variables with
$$  u = x - 4 y, \; v = x - 3 y  $$ so that
$$  x = 4 v - 3 u, \; y = v - u,  $$ we have
$$ 3 x^2 - 16 x y + 20 y^2 = -u^2 + 4 v^2.   $$
So, factoring $n,$ we find all possible ways to write
$$  n = -u^2 + 4 v^2 = (2 v + u )(2 v - u)   $$ 
as an even square minus an odd square, definitely including both $u,-u$ for each success, also both $v,-v.$ Then, for each success (finitely many) we switch back to the original variables with $  x = 4 v - 3 u, \; y = v - u,  $ and choose the solutions that include your conditions on positivity, whatever you might have meant by that.
Note that there are no solutions if $n \equiv 1,2 \pmod 4.$ When $n<0,$ there are no solutions when $ |n| \equiv 2,3 \pmod 4. $ 
EDIT, Wednesday morning, before Gerry wakes up in Australia, another completely cosmetic change: switch to $-n$ and find all solutions to
$$  -n = s \, (s + 4 t)   $$
which is just to find ALL ways of writing $$  -n = FG  $$ such that $F$ and $G$ differ by a multiple of $4.$ Then use
$$ x = -3s - 2 t, \; y = -s-t,   $$ from
$$ s = -x+2y, \; t = x-3y.     $$ 
So There. 
Well, if $n=0$ I guess there really are infinitely many solutions. But, as soon as $n \neq 0,$ we get finitely many solutions because
$$   |s| \leq |n|, \; \; |s+4t| \leq |n|.     $$
A: $ax^2+by^2-cxy=n$
Expressing as a quadratic equation of x, $ax^2-x(cy)+by^2-n=0$
As x is positive  integer, $=>(cy)^2-4.a(by^2-n)=(c^2-4ab)y^2 +4an$ must be perfect square.
$c=16, a=3, b=20 =>16y^2+12n=d^2(say)$=>d is even=2e(say)
$=>4y^2+3n=e^2$=>e is odd $>e^2≡1(mod\ 8)$
$=>e^2-4y^2 \equiv 1 \pmod4$ => $3n \equiv 1 \pmod 4$=>$n\equiv -1 \pmod 4$ to admit solution.
So, n=4m-1 for some integer m.
Applying the the approach on y, $(c^2-4ab)x^2 +4bn$ must be perfect square,
or, $16x^2+80n=f^2$=>f must even=2g(say),
$=>4x^2+20n=g^2$=>g is even=2h(say),
$=>x^2+5n=h^2$
If h=5s for integer s, $x^2≡0(mod\ 5)$=>x=5t for some integer t.
If h=5s±1 for integer s, $x^2≡1(mod\ 5)$=>x=5t±1 for some integer t.
If h=5s±2 for integer s, $x^2≡4(mod\ 5)$=>x=5t±2 for some integer t.
n will be $\frac{h^2-x^2}{5}$
$=>x^2+5(4m-1)=h^2$
If x,h are both odd or both even, $5(4m-1)=h^2-x^2≡0(mod\ 4)$  which is impossible.
If x is even, $h^2=x^2+5(4m-1)≡-1(mod\ 4)$ which is impossible.
=> x must be odd and h must be even to admit solution.
y can also calculated from the given once x is known.
