Why do people care about principal bundles? I've started to learn a little about principal bundles (in the smooth category) and while I see how notions like connections and curvature are abstracted from the setting of vector bundles and brought into the principal bundles world, I feel highly unmotivated regarding  to why this is done in the first place. 
So,


*

*How do principal bundles appear "naturally"? 

*Why did people started thinking about connections, curvature, parallel transport on principal bundles? Why would one want to put connections on them? Study their curvature? Even determine whether they are flat or not?

*Do we gain something by considering the frame bundle associated to a vector bundle and studying its connection and curvature and not working directly with the vector bundle itself?

*What kind of problems the machinery of principal bundles helps to solve? What notions does it clarifies?


Feel free to provide examples of results or ideas from any field, including physics. Explicit geometric examples are especially welcomed.
 A: In string theory, principal bundles can arise quite naturally when considering the compactification of extra dimensions. 
T-Duality in string theory is a duality which relates string theory on one spacetime to string theory on another spacetime. The simplest example of T-duality says that bosonic string theory on $\mathbb{R}^{25} \times S_{R}^1$ is equivalent (in terms of the physical observables of the theory) to bosonic string theory on $\mathbb{R}^{25} \times S_{1/R}^1$. 
(The subscript on the $S^1$ refers to the radius of the compactified dimension.)
A natural generalisation of T-duality considers a spacetime with a free circle action whose orbit space is the uncompactified spacetime manifold $M$. Locally, spacetime is a product $M \times S^1$, but not globally.
One is led to consider a principal $S^1$ bundle $\pi : E \to M$, where $M$ is the uncompactified part of your spacetime, and $E$ is your total spacetime. T-duality then refers to the construction of another principal bundle $\hat{\pi}: \hat{E} \to M$ on which you have an equivalent string theory.
A: For 1, given any free action of a compact Lie group $G$ on a manifold $M$ (or a free proper action of a noncompact Lie group), the orbit space $M/G$ naturally has the structure of a smooth manifold such that the projection $\pi:M\rightarrow M/G$ is a smooth submersion.  (Free means the only group element which fixes at least one point is the identity).
It turns out that $\pi$ is actually a $G$-principal fiber bundle.  So, if you care about group actions on manifolds, principal bundles arise naturally.
For 3, one of the main uses of the frame bundle I know of is the following:  Suppose a compact Lie group $G$ acts effectively on a Riemannian manifold $M$.  (Effective means the only group element which fixes all points is the identity).  Since the action is not free, the orbit space $M/G$ isn't a manifold in any kind of natural way (though it is still not so bad as a topological space!).
On the other hand, the action induces an action on the tangent bundle $TM$ (which still might not be free), and induces and action on the frame bundle $FM$.  This induced action on the frame bundle is free, so the quotient $FM/G$ is a manifold, so all the tools of differential geometry can be used to study $FM/G$, which in turn can give information about $M$.
A: For me, principal bundles can be seen like very natural objects to study fibre bundles of your interest.
Given any G-bundle $E\to M$ with standard fibre $S$, the associated principal $G$-bundle $P$ is, in a sense, a generalized frame bundle for the original bundle. To see this, observe that the principal $G$-bundle can be constructed in the following way. Given a trivializing atlas $\{(U_{\alpha},\phi_{\alpha})\}$ of $E$, we can define for every $m\in M$ the set
$$
P_m:=\{\tilde{\phi}_{\alpha}(m) \circ g: \alpha \text{ such that }m\in U_{\alpha}, g\in G\}
$$
Their elements are the different ways of express the fibre $E_m$ (maybe an "uncomfortable" space) in terms of the standard fibre $S$ (a "comfortable space), "preserving" the structure provided by $G$. You can think of $E_m$ being a vector space (in the case of a vector bundle or $GL(n)$-bundle) and $S=\mathbb R^n$, for example.
We can call the elements of $P_m$ the $G$-bases for $E_m$ (when $E_m$ is a vector space, $S$ is $\mathbb{R}^n$ and $G=GL(n)$ the set $P_m$ is made of the different basis of $E_m$ . Maybe this note can help you.
Observe that for $m\in U_{\alpha\beta}$ it could happen
$$
\tilde{\phi}_{\alpha}(m) \circ g=\tilde{\phi}_{\beta}(m) \circ g'\in P_m
$$
and then must be $g=g_{\alpha\beta}(m) g'.$
So if we consider the space
$$
P=\bigsqcup_{m\in M} P_m
$$
we can provide maps
$$
\begin{array}{cccc}
\psi_{\alpha}:&U_{\alpha}\times G & \longrightarrow & P\\
& (m,g) &\longmapsto & \tilde{\phi}_{\alpha}(m) \circ g
\end{array}
$$
such that it can be proven that $P$ is a principal $G$-bundle with atlas $\{(U_{\alpha},\Psi_{\alpha})\}$.
(It can be proven with the final topology and the fact of $G$ being a Lie group...)
Components of an "element"
Thus, given a G-bundle and its associated principal $G$-bundle constructed in this way, we can define the components of $v\in E_m$ with respect to any $\phi \in P_m$:
$$
\phi^{-1}(v)\in S.
$$
Then, any local section of $P$ can be called a $G$-frame (that is, a different $G$-basis in every point), and therefore every local trivialization $(U,\psi)$ is equivalent to a distinguished $G$-frame:
$$
p(m)=\psi(m,e), \text{ for } m\in U
$$
and $e$ being the identity in $G$. And conversely, every local section of $P$ provides a local trivialization for $P$ and also for $E$.
By the way, in some context the sections of a principal bundle are called moving frames or choice of a gauge. In case that the G-bundle $E$ is the tangent bundle of the manifold, the concept of $G$-frame is the usual frame, and the associated principal bundle is the frame bundle.
A: Off the top of my head I can think of a few reasons:


*

*For vector bundles there is the useful notion of a connection 1-form.  This is a little messy since it is dependent on a frame.  For principal bundles however, a connection 1-form is a global, well-defined object.

*They are useful in defining bundles.  For example if you have a principal-$G$ bundle $P$ over $M$ and a representation $V$ of $G$, you get a vector bundle over $M$ and a connection on the principal bundle induces a connection on the vector bundle.  These sorts of vector bundles are often easier to work with because their sections are just $G$ invariant functions from $P \to V$.  These constructions are vital in things like spin geometry.

*It is also useful from an algebraic topological point of view.  General characteristic classes can be defined for a group $G$ which then give characteristic classes for any vector bundle associated to a principal $G$-bundle.  

*Geometric structures on a vector bundle can be thought of as reductions of the frame bundle.  For example a Riemannian metric is a reduction to $O(n)$, a complex structure is a reduction to $GL(n,\mathbb C)$, etc.


EDIT:
Here's a nice concrete example combining my point 2 with Jason's answer.  Let $G$ be a compact semisimple Lie group and $T$ a maximal torus.  An irreducible representation of $G$ is determined by it's highest weight $\mu$ which determines a map $e^\mu: T \to \mathbb C^\times$, i.e. a one-dimensional complex representation of $T$.  By Jason's point $G \to G/T$ is a principal $T$-bundle so by my point 2, you can form the vector bundle associated to $e^\mu$.  The group $G$ acts on sections of this vector bundle and the Borel-Weil theorem says that the space of holomorphic sections is the representation of $G$ with highest weight $\mu$.  So in using the principal bundle $G \to G/T$, we can geometrically construct all of the representations of $G$ explicitly.
