As mentioned on MO, this http://www.math.wayne.edu/~isaksen/Expository/carrying.pdf gives a wonderful elementary viewpoint.
I have another, simple explanation using matrices that explicitly shows why cocycles arise.
Suppose we consider representations of a group $G$ over a field $K$. Suppose we have two one-dimensional representations $\chi_1,\chi_2$ (also known as "characters"). We'd like to find extensions of $\chi_2$ by $\chi_1$, i.e. representations $V$ such that there is an exact sequence $$1 \to \chi_1 \to V \to \chi_2 \to 1.$$
Now $V$ will automatically be two-dimensional (over $K$), so we can write the representation as $\rho:G \to \mathrm{GL}_2(K)$. Choose a basis adapted to the subspace $\chi_1$, so that $\rho(g)$ for $g \in G$ has a matrix representation
$$
\begin{pmatrix}
\chi_1(g) & f(g)\\
0 & \chi_2(g)
\end{pmatrix}$$
where $f$ is some function $G \to K$. Notice that if $f$ is always zero, then $V$ is the trivial extension (i.e. the product of the two one-dimensional representations), and conversely, if $V$ is trivial, then choosing an appropriate basis makes the function $f$ equal to $0$. In other words, this upper right matrix entry governs the non-triviality of the extension.
Now, the fact that $\rho(gh)=\rho(g)\rho(h)$ implies that $$
\begin{pmatrix}
\chi_1(gh) & f(gh)\\
0 & \chi_2(gh)
\end{pmatrix}
=
\begin{pmatrix}
\chi_1(g) & f(g)\\
0 & \chi_2(g)
\end{pmatrix}
\begin{pmatrix}
\chi_1(h) & f(h)\\
0 & \chi_2(h)
\end{pmatrix}
=
\begin{pmatrix}
\chi_1(gh) & \chi_1(g)f(h)+\chi_2(h)f(g)\\
0 & \chi_2(gh)
\end{pmatrix}
$$
Therefore, $f(gh)= \chi_1(g)f(h)+\chi_2(h)f(g)$. One can (and should) think of this as a cocycle condition for this map $G \to K$.
In particular, if $\chi_1=\chi_2=1$, then $f$ is a homomorphism $G \to K$.
Intuitively, this explains why representations of a finite group over a field of characteristic prime to the order of the finite group should be semisimple; the only homomorphisms from that finite group to the additive group of the field are trivial.
Exercise: Think about how an isomorphism between two extensions gives a coboundary making the two cocycles cohomologous.
Note that one can cook up cocycles for arbitrary finite-dimensional representations; the block structure would just look a bit different. If we examine $\mathrm{Ext}(K,M)$, where $K$ is the trivial one-dimensional representation and $M$ has degree $m$, then an extension will be a representation of $G$ into $(m+1) \times (m+1)$ matrices, with a block of $m$ in the upper left corresponding to $M$. The "upper right" will be a column of length $m$. This makes sense - the target of the mapping out of $G$ should be into $M$. In fact, this is a cocycle for the first group cohomology of $M$. More generally, a cocycle for an element of $\mathrm{Ext}^1(N,M)$ is a map from $G$ into $\mathrm{Hom}_K(N,M)$. This makes sense, for if $N$ has degree $n$, then the upper-right entry should be an $m \times n$ matrix, which should represent an element of $\mathrm{Hom}_K(N,M)$.
