Lie group is a group equipped with a smooth manifold structure such that multiplication and inverse are smooth maps. What I don't understand is why we require multiplication and inverse to be smooth maps? I know it has something to do with compatibility of group structure and manifold structure but I am not able to reason it. Hope for a clear explanation and an example which shows what goes wrong if those two maps are not smooth.
The reason is simply because if the group structure and the manifold structure were not compatible, then there would be no connection between them and therefore the group-theoretical properties and the differential geometry properties would have to be studied separately. Here's an example: every connected Lie group is generated by any neighborhood of the identity element. There would be no reason for this to be true if the group structure was not compatible with the manifold structure.