I guess Goldbach's conjecture is a good example of a short open problem in number theory, and "Goodstein sequences reach 1" is a good example of a statement undecidable from first-order Peano arithmetic.
How about statements that are short in the language/alphabet of ZFC, though? Or in this alphabet extended by the standard abbreviations for null set, union, intersection, complement, and power set? My experience is that most statements that are short to write using arithmetic symbols +, $\cdot,$ ^ etc. become rather lengthy if written out without these abbreviations.
The axiom of choice is of course undecidable in ZF, and the continuum hypothesis/generalized continuum hypothesis is, in turn, undecidable in ZFC. When large cardinal axioms are written out, are they relatively short statements? And how about open problems? Do any of these refer (more or less) to finite sets?