The title is slightly misleading, since the theory I'm looking at is an extension of PA; however, the question is "morally" about very weak arithmetic.
Specifically, consider the following theory PA':
The language of PA' consists of the usual language of PA - namely, $+,\times,0,1$ - together with a new unary predicate symbol $C$.
The axioms of PA' consist of the usual PA axioms together with "$C$ is downwards closed, contains $0$, and is closed under successor." Crucially, we do not extend the induction scheme to formulas involving the new symbol "$C$."
A model of PA' consists then of a model $N$ of PA together with an initial segment $C$ closed under successor; this $C$ is in general very poorly behaved, and in particular even though $N$ itself satisfies a very strong arithmetic we can't even show that $C$ is closed under addition! Note that this relies on the fact that we didn't extend the induction scheme to formulas involving $C$.
Nonetheless, there is a sense in which - working within any model of PA' whatsoever - we can find "well-behaved" cuts inside $C$ (this is due to Nelson):
We can find a definable initial segment of $C$ which is closed under successor and addition.
We can even find a definable initial segment of $C$ which is closed under successor and addition and multiplication.
However, the proofs of the above facts which I know rely on associativity; they thus fail when we try to move to exponentiation. My question is whether this is in fact unavoidable:
Question. Does every model of PA' necessarily have a definable initial segment of its $C$ which is closed under successor, addition, multiplication, and exponentiation?
Note that while exponentiation isn't in the language of PA, PA is strong enough to define it and prove basic facts about it, so this is fine. Moreover, note that the above is a bit redundant - from closure under successor and exponentiation we get closure under addition and multiplication - but meh.
See this earlier question of mine for proofs of the above facts; I'm omitting them since Eric Wofsey showed that not only do their proofs not generalize to exponentiation, the construction involved can't even work here, so they really are irrelevant.