First of all it is not really historically meaningful to phrase the question "To what extent.." and "aware of lack of rigour". To really know the answer we would surely need a time machine, which we do not have (at least I don't.) Additionally, the question also deals with awareness which is a delicate and debatable subject even in its own field, namely psychology. So let's assume you are aware of your own thoughts and feeling and your own knowledge. But are you aware of thoughts and of feelings of someone else? How can you be sure you are? And even more, How can you be aware of the things you do not know?
History and the Scientific process should be understood in the context they occurred. The mathematicians of previous times where trying to lay their grasp of the world mathematically with the syntactical tools they had or could invent. Understandably, there are a lot of tools and concepts that where developed after their time which they could not use in their research.
So, removing the psychological notion which is only can be answered with conjectures and guessworks. It is logical to say that their proofs and mathematical language and deduction was just rigour enough to be accepted by their contemporaries, and than re-examined and change like in other scientific areas Like claimed by Popper and then Kuhn.
Wikipedia has what seems to be a really good, yet short, description of the term rigour in the context of mathematics and its historical context. The key thing you can take from this answer is that historical events and peoples thoughts (and feelings) are heavily influenced by their time and contextual frame. I hope this answer explain and give you some of the necessary information and a viable direction to understand these concepts in clear way.