How to come up with a 200-page proof? I have often seen mathematicians write a 200-page proof or even longer for a certain problem. Now, my question is how does one even come up with such a long proof. I've not seen such long problems ever in my curriculum, how are these mathematicians sure that they're headed down the right path, how can they have such foresight because you can often meet dead ends when approaching even a textbook problem in a certain way.
It's like you can see one or two steps ahead in a chess game, but how do mathematicians see like 200 page ahead in a method of proof for a certain problem and quite a few mathematicians are capable of such a feat.
 A: Few notes:


*

*Published proofs, by definition, exhibit "survivor bias". Your implicit assumption seems to be that mathematicians only commit to writing when the "path is clear". And that is far from true! In generally even for short papers there are tons of false starts and mistakes in the process, and there are certainly many cases where one ends up 50 or 100 pages into a proof before one sees an obstacle that voids the whole enterprise. 
In other words, the word you are looking for is probably more "perseverance" and not "foresight". 

*Mathematicians, unlike French authors of a certain period, don't start out trying to write enormous 200-page-long arguments. Aesthetics and laziness combine to make us prefer shorter proofs when possible. As Lord Shark says, oftentimes the end product of a 200 page paper is exactly what happens when one starts out writing a 50 page paper and gets stuck proving one technical lemma, which turns out to require the other 150 pages to prove. 

*That said, experience often allows one to construct proofs from the top down. There was a study done (I can't track down the source now, sorry!) where they examined the problem solving processes of professional mathematicians against that of the mathematical novice (undergraduate students etc.) The main difference is that the novice approaches the problem as a linear thing: 


*

*start trying something 

*work until a wall is hit

*start over with something else. 


The professional's thought processes are more nonlinear. For example, one strategy is 


*

*start trying something

*work until a wall is hit

*Peek beyond the wall: assume the wall can be surmounted, work a bit more to see if there are other problems. If it gets thornier, possibly start over. If not, finish the problem and go back to breaking down the wall or working around it. 


Another is


*

*Start by breaking down the problem into a series of milestones to be reached (this requires experience, foresight, and a lot of guessing). The though process goes something like: "I can probably prove the result I want if I can prove this other fact." Rinse and repeat. 

*For the smaller goals work until a wall is hit. The goals don't have to be approached in order. 

*When a wall is hit, allow oneself the freedom to move the goal post (with suitable adjustments to the rest of the proof-outline). 


