What is meant by a "structure map"? The title is the question. Somehow I should know the answer, but I am by no means sure what is meant exactly by it. Perhaps it doesn't have a definite meaning and only in context, could someone provide examples? For instance is the following use of the term acceptable: the structure maps of a direct limit/fibred product/whatever are the morphisms that form part of the direct limit/fibred product/whatever, i.e. which are "part of the structure of the direct limit" (I mean by definition a direct limit is an ordered pair consisting of an object and a family of morphisms having some properties). See also https://mathoverflow.net/questions/42241/errata-for-atiyah-macdonald, the answer by 
Georges Elencwajg.
I am aware of only one "official use" (abuse of the word): in EGA when defining the category of $S$-objects: the $S$-objects are morphisms $X\rightarrow S$ and informally one refers to $X$ as a $S$-object and the morphism $X\rightarrow S$ as "the structure map of $X$".
 A: A structure map is a map which is being specified as part of a structure! 
For example, as you say, a scheme over a scheme $S$ is a scheme $X$ equipped with a map $X \to S$; that map specifies the structure of being a scheme over $S$ and so is the structure map. Similarly, a limit or colimit is an object equipped with maps such that etc., and those maps specify the structure of being a limit or colimit and so are the structure maps. 
A: From any category $\mathcal C$ and any object $c$ of $\mathcal C$, you can construct a new category $\mathcal C_{ / c}$, called the slice category over $c$ or the category of objects over $c$, whose


*

*objects are the morphisms $p \colon c' \to c$ of $\mathcal C$ with codomain $c$,

*morphisms $(c' \overset p \to c) \to (c'' \overset q \to c)$ are those arrow $r \colon c' \to c''$ of $\mathcal C$ such that commutes
$$ \begin{matrix}
c' & \overset r \longrightarrow & c'' \\
\searrow \!\!\!\!\!\!\!\!\!\!\! & & \!\!\!\!\!\!\!\!\!\!\!\!\!\! \swarrow\\
& c, &
\end{matrix}$$

*composition is induced by the one in $\mathcal C$.


For any object $c' \overset p \to c$ of $\mathcal C_{/c}$, one can call $p$ a structure map for $c'$.

With this definition, an $S$-scheme $X$ is an object $X\overset p \to S$ of the category $\mathsf{Sch}_{/S}$ ($\mathsf{Sch}$ being the category of scheme).
For a small category $\mathcal I$ and any category $\mathcal C$, the category of diagrams of shape $\mathcal I$ in $\mathcal C$ is the category $\mathcal C^{\mathcal I}$ whose objects are the functor $\mathcal I \to \mathcal C$ and the morphisms are the natural transformations between such functors. Take a diagram $D \colon \mathcal I \to \mathcal C$ and consider the full subcategory $\mathrm{Cone}(D)$ of $(\mathcal C^\mathcal I)_{/D}$ whose objects are those $D' \to D$ with 
$$ D'(i) = D'(j) \forall i,j\in\mathcal I \quad\text{and}\quad D'(i\to j)=\mathrm{id}_{D'(i)} \forall (i\to j) \in D' .$$
Then, by definition, the limit of $D$, if it exists, is the final object of $\mathrm{Cone}(D)$. It is an object $\lim_\mathcal I D \overset p \to D$ and p is the structure map of this limit.
You can formalize as well the structure map of a colimit by dualizing all the notion (coslice category ${}_{c\backslash}\mathcal C$, cocone, etc.).
