What are the smallest possible (elementary) toposes, in terms of objects and arrows, that still meet the definitional requirements of a (elementary) topos? Let me give you a flavor of what I'm looking for.
The smallest topos is just the terminal object $1$ with its identity arrow. Its finite products are iso to itself. Same with the exponent. And if we choose for a subobject classifier $\Omega$ to have $true \circ\ ! = id_\Omega$, we get $\Omega$ iso to $1$ (We already have $! \circ true = id_1$ since $1$ is terminal, and all arrows from $1$ to $1$ are equal).
That's pretty boring, so what structure do we get if we choose to have our only two guaranteed arrows not invert? That is, we can choose $true \circ\ ! = a \neq id_\Omega$. One of the properties we can get right away is that $a \circ a = a$, so $a$ is sorta idempotent. Then there's $2^n$ product arrows from $\Omega^n$ to itself. If we were to read arrow tuples like binary, composition looks like an OR logic function. That's about as far as I got.
Going like this, we keep choosing things to have as little possibility of being different as possible. That is, when we can choose iso objects, we will. Is there anybody out there that has developed topos theory in this manner?
I would especially appreciate references. I'm looking for connections between topos theory and finite model theory, and although I know the relation between quantifiers and functor adjoints, I don't know how much of this stuff holds for a finite topos. I don't have any graduate experience in mathematics, but I can (kinda) read research stuff.
EDIT: FinSet is a pretty small topos. FinOrd is even smaller! But is that last example equivalent to it? As in, could we construct FinOrd just by saying "these two arrows are not iso arrows"? Or is there at least one topos smaller than FinOrd?