Suppose I write down a system of axioms, like so:
- $\forall x. \forall y. (x y) z = x (y z)$
- $\exists e. \forall x. e x = x e = e$
- $\forall x. \exists y. x y = y x = e$
This system of axioms is incomplete, because for instance it does not allow us to determine whether the following statement is true or false:
- $\forall x. \forall y. x y = y x$.
Why? Well, if you consider axioms (1) - (3), there is more than one mathematical object satisfying them. One object which satisfies them is the multiplicative group $\mathbb{R}^\times$ of real numbers. Another is the group $\operatorname{GL}_2(\mathbb{R})$ of $2 \times 2$ real matrices. In $\mathbb{R}^\times$ axiom (4) is true; in $\operatorname{GL}_2(\mathbb{R})$ it is false.
Moreover, any time a system of axioms is incomplete it is for precisely this reason: it describes more than one mathematical object and those objects have different properties.
Now if we're trying to describe $\mathbb{R}^\times$ with our axioms and we want to rule out $\operatorname{GL}_2(\mathbb{R})$, we could add (4) to our list of axioms. But they'd still be incomplete. (Exercise: check this!)
Now, maybe if you have a particular mathematical object in mind you can just keep adding true statements about it and at some point your axioms will completely describe it (up to elementary equivalence). At that point your axioms will be complete, since they can determine if any first-order statement about the object is true or false.
But there's no guarantee you'd ever finish. To draw an analogy, consider an arbitrary real number, say
$r = 2,345,098.231456981324509813245098123409123409123049\ldots$
If there's no useful "pattern" to the number, it's possible that we can't describe it in a finite amount of space. To make this precise, we say that a number $r$ is computable if there exists a Turing machine that, given a positive integer $n$ as input, prints out the first $n$ decimal digits of $r$. Since there are countably many Turing machines and uncountably many real numbers, almost all real numbers are uncomputable.
It's often the case that a mathematical object requires infinitely many axioms to completely describe it (up to elementary equivalence). (It's always possible, because you could simply take your list of axioms to be every single true statement about the object.) However, such a list might be more akin to
$\pi = 3.14159265358\ldots$
-- that is, computable -- or it may be more akin to the uncomputable number $r$ above.
What Godel's incompleteness theorem says is that if you're trying to describe the natural numbers, every complete set of axioms describing them is uncomputable.