I proved via pigeonhole principle that $5$ is enough is if we're taking mean of two points. Define a function which maps the lattice point to its modular class in $\mathbb Z_2\times \mathbb Z_2$, then atleast one of the classes will have 2 points and hence we are done.
I am looking for a similar proof in which we preferably only apply the pigeonhole principle once. I know $13$ works by repeated applications of pigeonhole, but I don't know if $12$ doesn't work out.
The proof for $13$ goes as follows. Define $f:\mathbb N\times\mathbb N$ as $f(x,y)=x(mod \,\,3)$. Then atleast one of the classes will have $5$ elements. Then consider those $5$ points only. We are distribution the $5$ points in $3$ classes (based on second coordinate) thus either all classes have atleast $1$ element or one class has $3$ elements and hence we done.