How to learn from proofs? Recently I finished my 4-year undergraduate studies in mathematics. During the four years, I met all kinds of proofs. Some of them are friendly: they either show you a basic skill in one field or give you a better understanding of concepts and theorems. 
However, there are many proofs which seem not so friendly: the only feeling I have after reading them is "how can one come up with that", "how can such a long proof be constructed" or "why does it look so confusing". What's worse, most of those hard proofs are of those important or famous theorems. All that I can do with these hard proofs is work hard on reciting them, leading me to forget them after exams and learn nothing from them. This makes me very frustrated. 
After failing to find the methodology behind those proofs, I thought, "OK, I may still apply the same skill to other problems." But again, I failed. Those skills look so complicated and sometimes they look problem-specific. And there are so many of them. I just don't know when to apply which one. Also, I simply can't remember all of them. 
So my questions are: How to learn from those hard proofs? What can we learn from them? What if the skill is problem-specific? (How do I find the methodology behind them?)
I need your advice. Thank you!
P.S. Threre are a lot of examples. I list only four below.
Proof of Sylow Theorem in Algebra
Proof of Theroem 3.4 in Stein's Real Analysis.

Theorem 3.4 If $F$ is of bounded variation on $[a,b]$, then $F$ is differentiable almost everywhere.

Proof of Schauder fixed point theorem in functional analysis.
Proof of open mapping theorem in functional analysis.
 A: Quite often there is intuition to be taken away from a result even when the proof is completely opaque. I'm not sure which proofs of the Sylow theorems you've seen, but I'll illustrate with the proof of the first Sylow theorem that I first encountered;


*

*If $p^k$ divides $|G|$ then $G$ has a subgroup of order $p^k$.


The proof I encountered first is the one on Wikipedia -- let $G$ act on the subsets of $G$ of size $p^k$ by (left or right) multiplication, and then the stabiliser of such a subset is a subgroup of order $p^k$.
It's perhaps easy to say "Well, this is unenlightening, all we've done is direct calculation." But I think this is interesting in itself; what I first took away from this proof is its simplicity. The hardest idea applied here is the orbit-stabiliser theorem, but what it gives us is a huge constraint on the structure of all finite groups. It suggests something very fundamental is going on.
If instead you were just asking "What can I learn from this proof, which I can take away and apply to other proofs in finite group theory?" then the answer is: Lots of results are elementary (even if they are somewhat longer to prove), and that if you come across an interesting question involving subgroup structure, you should not immediately give up on solving it with basic techniques alone.
We can also interpret the simple proof as a hint that (the prime-factorisation of) the order of a finite group very strongly controls its structure. And indeed this turns out to be the case, e.g. as well as the other two Sylow theorems, which allow us to classify large numbers of groups (e.g. those of order $pq$), we get Burnside's $p^{a}q^{b}$ Theorem and the Feit-Thompson Theorem, which tell us that every group of order $p^{a}q^{b}$ for primes $p$, $q$, and every group of odd order is soluble. So even if you find the proof of the first Sylow theorem unhelpful, there is definitely a lot of intuition to be taken away from the result itself, which can be applied to other proofs.
A: It is meaningless and useless, I think, if we just want to recite the proofs. According my experience, I always closed the book and try to think and draw a picture in my brain. For every author of his theorem, when he  proof the theorem, the plan as a picture is first in him.
A: I like your characterization of friendly proofs-- many well-written proofs do provide a basic insight or helpful perspective that will illuminate your study of that entire field of math; or they demonstrate a technique that you'll use over and over again in various forms.
I would say that the long, complex, messy proofs you're asking about are not all that different from these friendly proofs-- it just takes a lot more effort and familiarity before you can consider them old friends.  
The mathematicians who use those ideas frequently really do hold such proofs in their heads, at least in outline form, and can pick out or rearrange the key steps to attack new problems.  I find this is true even at the frontiers of research-- experts can take a paper that would baffle nearly all their colleagues and quickly identify the key ideas on pages 2 and 13, can skim over the routine but messy calculations that fill the 10 pages in between, and can identify a few open problems that could be attacked with the new ideas.  
This doesn't mean you need to reach this familiarity with every challenging proof you encounter.  The Sylow theorems are basic tools in algebra, but I would guess many mathematicians who don't use them don't remember the proofs.  The effort you've spent learning messy proofs the first time is still worthwhile-- you have at least some sense of what's out there, and as you see some of these proofs and ideas again, they'll come to you more quickly and will feel more and more like old friends.
A: The complicated proofs usually don't arise out of nothingness. People don't come up with proofs the same way they write them. They look at objects and observe their properties, until they see more and more, and then they try to somehow catch the essence of their observation, and the reasons for it: that is a theorem and its proof.
The steps in the proof often flow naturally from the observations that motivate the theorem itself. You should notice that complicated proofs often use many tools and lemmas which, while often not very general, are themselves of interest. I don't think that many mathematicians not dealing with geometry or group theory would easily come up with a proof of Sylow's theorem without knowing it beforehand (or even the exact formulation).
The general principles, ideas used in theorems tend to be relatively simple, and the proofs come from applying and combining them in the right way. What you can gain from a proof (beyond the theorem itself) is the intuition to tell you the ways you do these things.
If you want more insight, you should try to prove the theorem yourself, perhaps using the given proof as a source of hints, rather than the entire solution. When you see exactly what difficulties are there, you will appreciate the work done by the author of the proof and see why you should do some things and not the others. Or perhaps you will come with a completely different proof, in which case you will see two ways to look at the same problem, which also can be quite enlightening.
But you cannot expect to be able to come up with all kinds of hard proofs just like that. Tough proofs take time to develop. A radical example would be the classification theorem for finite simple groups (some history on Wikipedia).
A: I find it very helpful to revisit proofs after some time. I find there's two levels of understanding a proof:
--> line-by-line: you find the proof written in a particular textbook. for each sentence in the proof, you understand how it follows from the prior one.
--> conceptually: you can reproduce the proof because at each step it's clear to you what to do. (maybe there's a calculation or two that involves a random trick that you have to look up.)
Obviously the goal is to reach the second arrow.
Reading a proof over and over until you have reached the first arrow is sort of like doing an exercise, and, like all exercises, it helps acclimate your mind to the ideas at hand, and, in this way, it helps you get to the second arrow. But there's also an essential ingredient of time. Come back in three months, see how you feel.
You might think the second arrow entails the first, but actually it can still be hard to read an author's particular way of writing a proof, even when you understand the proof of a theorem. For instance, I think I could write down the proof of the Baire category theorem (don't quiz me), but if I were to go read a proof in a book, I'd probably get confused about the particulars of the notation without spending a little time, or, worse, convince myself that I was following. "You just do the obvious thing right? Yeah, ok, so wait, why is he saying $x_n \in B_n$, wait, what's $B_n$, oh, whatever, I know this." 
A: One important thing about proofs is that you will never be able to appreciate them, and therefore to learn from them, if you are not capable of reading the statement to be proved with a sceptical attitude, and to try to imagine it is untrue:


*

*What's this nonsense they are claiming, it cannot be true!


*Certainly it must be possible to satisfy the hypotheses without being obliged to accept the conclusion!

Once you have some mental idea of what a counterexample to the statement would look like, you can interpret the proof as an argument that systematically talks this idea out of your head, convincing you that it really is not possible to ever come up with such a counterexample.
Then you will have acquired a feeling of what the proof is really about, and you will be far more likely to retain it, and to come up with similar arguments when you need to prove something yourself.
But if you take a docile attitude and accept the statement to be proved from the onset, you will never be able to understand what all this reasoning was needed for in the first place.
