Local vs. global in the definition of a sheaf Apologies in advance that this question is inescapably soft. What I am stuck on is squishy; I have the feeling that if I could even make it precise, I'd already be satisfied.
To what extent is a sheaf determined by its stalks? Is there global information about the sheaf hiding in the stalks?
Here is the story of what is bugging me. One of the first things I learned about sheaves was the sheafification of a presheaf. How I think about the sheafification is that it is the unique sheaf with the stalks dictated by the presehaf.  Thus early on I began to think of a sheaf as an object that is entirely determined by local information. There is the caveat that sections of the sheaf must come from gluing germs together "compatibly," which implies that nearby stalks are talking to each other to decide what the sheaf is, but "nearby-ness" can be as small as it wants to in this context, so the sheaf still feels basically like a local object. (This paragraph is very vague; see below for some clarification if needed.)
Now, more recently I have been studying locally free sheaves on schemes. What is bugging me is that locally free sheaves of a given rank cannot be distinguished by looking at the stalks, at least up to algebraic isomorphism. For example if $\mathscr{L}$ is an invertible sheaf, aka line bundle, on a scheme $X$, then $\mathscr{L}_p = \mathcal{O}_{X,p}$ for every $p\in X$, so different invertible sheaves on $X$ are not distinguished by their stalks. They are distinguished by the ways that the patches of a local trivialization of the bundle are glued together, which is a global consideration.

In light of the last paragraph, how is this happening?
Is it that the requirement that "nearby germs be glued together compatibly" to make sections allows all this global information to be stored in a local way? If so, how? Or is the argument in the previous paragraph that sheaves are determined by local information fundamentally wrong, in which case what am I missing?

Anything you could tell me to help me think clearly about this situation would be much appreciated.
Appendices (you can ignore, but thoughts are welcome)
Clarification: Let me make explicit what I'm referring to in the paragraph above about sheafification. My definition of the sheafification is coming from Hartshorne: let $\mathscr{F}$ be a presheaf on a topological space $X$ and let $U$ be an open subset of $X$. The sections of the sheafification of $\mathscr{F}$ over $U$ are functions $f:U\rightarrow\coprod_{p\in U}\mathscr{F}_p$ such that for all $p\in U$, (1) $f(p)\in\mathscr{F}_p$ and (2) there exists open $V\subset U$ containing $p$ and $g\in \mathscr{F}(V)$ such that $f(q)$ equals $g$'s germ in $\mathscr{F}_q$ for all $q\in V$. Condition (2), about the existence of $V$, is what I meant by "sections come from gluing germs together compatibly." What I meant by "nearby-ness can be as small as it wants to" was the fact that $V$ is an arbitrary (i.e. arbitrarily small) open neighborhood of $p$.
More thoughts:
The above discussion brought to mind the monodromy theorem from complex analysis: the germ of an analytic function at a single point in a simply connected region of $\mathbb{C}$ determines the function on the whole region. However, with more thought, I feel like it cannot be something like this that is going on: let $X$ be a topological manifold and let $S,T$ be two fiber bundles on $X$ with individual fibers homeomorphic but that are not isomorphic globally.  Then the sheaves of sections should have isomorphic stalks, right? But now there is no "rigidity" result analogous to the monodromy theorem or anything that would happen with an algebraic vector bundle: if $p\in X$ and $U$ is any open neighborhood of $p$ no matter how small, the germ of a section at $p$ will not determine the section anywhere outside $U$. This seems to me to free the stalks of the sheaves of sections from enmeshment with any but their closest neighbors. Yet still, the bundles will be distinguished by these sheaves! So the global information contained in the sheaves cannot be coming from some sort of rigidity theorem like the monodromy theorem or some sort of algebraic rigidity.
A final thought: I was moved to post this question because I kept getting stuck at step (2) while giving myself the following exercise: (1) realize $\mathcal{O}(1)$ as the sheaf of sections of an algebraic vector bundle, described explicitly with a gluing construction, as I did in this previous question; (2) use this explicit description to compute an equally explicit description of its dual sheaf. I keep getting stuck because I haven't been able to see how to use the gluing information when writing down the dual. I am hoping that answers to this question will lead to a general clarification of my thinking that will allow me to get past this specific stuck point for myself, but if they don't, I may ask a followup question.
 A: I don't understand the question. Consider locally constant sheaves, which are sheaves of sections of covering spaces. With a nice path-connected base, these correspond to group actions of the fundamental group of the base. The stalks of this sheaf give you very little information: if the covering space is also path-connected, you have a transitive group action, and looking at stalks only tells you its cardinality. To get the rest of the data you need to look at monodromy, which isn't a local thing. 
A: Sheaves are given by local data, yes, but this local data is not only encoded in the stalks. As already indicated in the comment by Jared and the answer by Qiaochu, even in the question (that section about line bundles), stalks are not enough to understand a sheaf. Beyond examples, there are several ways to make the difference explicit:
One of the classical definitions of a sheaf is via étale spaces. If $X$ is a topological space, then an étale space over $X$ is just a continuous map $p : Y \to X$ which is a local homeomorphism. The corresponding sheaf in the usual sense is the sheaf of sections of $p$. The stalk at a point $x \in X$ is just the fiber $p^{-1}(x)$. The fibers of $p$ determine the set $Y$, but not its topology.
If $F,G$ are sheaves on a space $X$ (or equivalently étale spaces if you like), then a family of maps of stalks $F_x \to G_x$ ($x \in X$) comes from a (unique!) homomorphism of sheaves $F \to G$ iff the following continuity condition holds: For every section $s \in F(U)$ there is some open covering $U=\cup_i U_i$ such that on each $U_i$ there is some section $t \in G(U_i)$ such that $F_x \to G_x$ maps $s_x \mapsto t_x$ for all $x \in U_i$.
