Although every non-simple group can be described as a group extension, it is well-known that not every non-simple group can be expressed as a semidirect product of smaller groups.
For example, a cyclic group of order $p^2$ is an extension of $\mathbb{Z}_p$ by $\mathbb{Z}_p$, but cannot be written as a semidirect product. The quaternion group $Q_8$ is also not a semidirect product. Indeed, this seems to be very common property for $p$-groups.
So my question is:
What's the smallest non-simple group $G$ such that:
$G$ isn't a $p$-group, and
$G$ can't be expressed as a semidirect product of smaller groups?