Non-collinear integral coordinates in a plane. Given that A, B, and C are noncollinear points in the plane with integer coordinates such that the distances AB, AC, and BC are integers, what is the smallest possible value of AB?
What's the best approach to solving such problems?
 A: We are given three noncollinear points $\,A, B, C\,$ with integer coordinates in a plane and integer distances. We are trying to show that $|AB|>2$. By using rigid motions preserving the integer lattice (that is, translations and horizontal and vertical reflections), and if we assume $|AB|<3$, then without loss of generality, the coordinates of the points can be written as (proof omitted):
$$ A = (0,0),\quad B = (0,t),\quad C = (x,y) \tag{1}$$
where $\,t,x,y\,$ are all positive integers. The
positive integer distances are
$$ t := |AB|,\quad u := |AC| = \sqrt{x^2+y^2},
\quad v := |BC| = \sqrt{x^2+(y-t)^2}. \tag{2} $$
The triangle inequality for the triangle $ABC$ states that
$$ |AC| \!<\! |AB| \!+\! |BC| \;\; \text{ or } \;\; u \!<\! t \!+\! v
\;\; \text{ which implies } \;\; u-v < t. \tag{3} $$
Next, to upper bound $\ u-v,\ $ we compute the difference of squares
$$ (u-v)(u+v) = u^2-v^2 = (x^2+y^2) - (x^2+(y-t)^2) = t(2y-t). \tag{4}$$
Now we consider cases. Since $\ t\ge1\ $ the minimal case is $\ t=1.\ $
If $\ t=1\ $then by $\ u-v<t\ $ we have $\ u\le v.\ $ Given that $\ y>0,\ $ then $\ 2y-t\ge 1,\ $ and by equation $(4)$ we have $\ u>v \ $ which contradicts $\ u\le v.\ $
The next case is $\ t=2.\ $ If $\ t=2\ $ then by
$\ u-v<t\ $ we have $\ u\le v+1.\ $ If $\ u=v+1\ $ then by equation $(4)$ we get $\ u^2-v^2 = 2v+1 = t(2y-t)\ $ but $\ t=2\ $ is even and odd equals even is a contradiction. So assume $\ u=v\ $ and by equation $(4)$ this implies $\ y=1\ $ but now $\ ABC\ $ is an isoceles triangle and when bisected leads to the previous $\ t=1\ $ case which is excluded. This leaves assuming $\ u<v\ $ and by equation $(4)$ the left side is negative but $\ t>0\ $ and $\ 2y-t\ge 0\ $ implies the right side is non-negative which is a contradiction.
We have now proved that $\ t\ge 3\ $ and the case
$\ t=3,\ x=4,\ y=0\ $ is the minimal $3,4,5$ right triangle.
There are an infinite number of such triangles with minimum side $3$ given by the OEIS sequence A072221 and similarly with minimum side $4$ given by the OEIS sequence A016064.
A: We can assume that, up to a translation, $A$ is placed in $(0;0)$.
Case $AB=3$ is possible ; it suffices to consider $A(0;0), B(3;0), C(0;4)$, the famous egyptian (right) triangle with sides 3-4-5.
Can we have $AB=2$ or $AB=1$ ? We are going to see that it is impossible due to a property of the so-called "Heronian triangles" (see below):
Case $AB=2$ : Let us assume that such a triangle exists. There are 4 points of $\mathbb{Z}^2$ belonging to the circle with center $A$ and radius $2$ : $(\pm 2;0),(0;\pm 2)$ ; we can, up to a symmetry, assume that $B$ is situated in $(2;0)$. The area of triangle $ABC$ is 
$$\frac12 AB \times h=h\tag{1}$$
where $h$ is the length of the altitude dropped from $C$ onto basis $AB$ ; this altitude is clearly an integer (absolute value of the ordinate of $C$). A triangle with integer sides and integer area is called a Heronian triangle see there. In this article is mentionned the fact that 
$$\text{no Heronian triangle with side 2, neither with side 1, exists} \tag{2}$$
This is lemma 3 of this document associated with a proof using the famous Heron formula ; I found the reference to this document in the Wikipedia article cited above. 
Application of (2) rules out case $AB=2$.
Case $AB=1$ ; Let us assume again that such a triangle exists. The same reasoning as before shows that we can take $B$ in $(1,0)$. The area of $ABC$ is now :
$$\frac12 AB \times h=\frac{h}{2}\tag{2}$$
i.e., either an integer or a half integer. Multiply all coordinates of $A,B,C$ by $2$, yielding a triangle $A'B'C'$ whose area is $4$ times the area of triangle $ABC$, thus an integer in both cases, with integer sidelengths, i.e., a Heronian triangle with one of its sidelengths equal to $2$ : impossible once again.   
