The topological statement is "There is a relative homotopy of the three-dimensional situation where the vest is worn to the situation where the vest is removed, relative to the body and the jacket."
As mentioned in the comments, the vest may need to be sufficiently elastic. Also, popular demonstrations of the property cheat by moving the body and jacket during the procedure.
Interestingly, many popular explanation about what topology is are about homotopy rather than homeomorphism. Often one even reads that a coffee cup and a donut are homeomorhic ecaxtly because(!) they can be transformed into each other without tearing or glueing when made out of rubber. Of course this property is much stronger than homeomorphism, it is about homotopy of embeddings into $\mathbb R^3$.