Confusion regarding the completeness axiom

Why would we use square root of 2 in our example to show that the rationals don't have a supremum when square root of 2 is not an element of the rationals? Wouldnt the supremum be the element of the rationals that is greater square root 2 or less than square root 2?

We can still talk about the set of rationals less than $$\sqrt 2$$ without making direct reference to $$\sqrt 2.$$ Write it as $$\{q\in\mathbb Q: \mbox{q^2<2 or q<0}\}.$$
As to your second sentence, I'm not sure I understand... any element of the rationals is greater than or less than $$\sqrt{2}.$$ Maybe you meant greater than or equal to and less than or equal to $$\sqrt 2$$? But this just means it is equal to $$\sqrt{2},$$ and the usual proof that $$\sqrt{2}$$ is irrational shows that this is impossible.
Showing the set has no least upper bound in the rationals requires one more step than showing there is no rational $$q$$ with $$q^2 =2.$$ We also use the fact that the rationals are dense. We know that any upper bound must have $$q^2 > 2,$$ and then we can show that there is a slightly smaller rational that still squares to greater than two.
You don't use the square root of $$2$$ directly, rather you look at the set of rational numbers $$q$$ such that $$q^2 < 2$$, so the definition of that set does not refer to anything other than rational numbers and multiplication of them.