This question was inspired by this question. It is clear that all abelian groups satisfy this property (let us call it the K-property). Also if two groups $G$ and $H$ satisfy the K-property then so does $G \times H$. All dihedral groups also possess this property, as do the symmetric groups $S_3, S_4,$ and $S_5$. From $S_6$ on this is not the case anymore. An interesting note is that if $G$ has the $K$-property then $G/K$ is the direct product of copies of $C_2$. So candidates are to be sought among the extensions of such direct products but with what? Some simple groups work like $C_2 \ltimes PSL(3,2)$, produced with the automorphism of matrices $(M^T)^{-1}$. Is there a way to classify those groups?
Edit
Thanks to a comment from j.p. we can add the groups of odd order to the list. Moreover they possess the stronger property that $K = G$. Indeed, let $x \in G$ then $x$ has odd order $p$ so we have for $y = x^{\frac{p+1}{2}}$ that $y^2 = x$.