What does "Arrows are more important than objects" really mean? I am a final year undergraduate student and I am trying to learn category theory. I am familiar with the basic notions. I am reading Pareigis's notes, http://www.mathematik.uni-muenchen.de/~pareigis/Vorlesungen/04SS/Cats1.pdf. 
In his introduction, he states " Category theory proves that all information about a mathematical object can also be drawn from the knowledge of all structure preserving maps into this object. The knowledge of the maps is equivalent to the knowledge of the interior structure of an object. “Functions are everywhere!” "
I am not sure what he means exactly by this. I have seen before sentences like "It is the arrows that matter most". But I don't see the big picture I guess. Can someone give me some example of how the arrows, in some circumstance, give all the information about the object in a category? (It might be easier for me to understand examples coming from group theory or topology.)
 A: As @Adeel mentions in the comments, learn the Yoneda Lemma, which is the actual answer to your question.
For an example from topology, consider the quotient space $X/\sim$ where $\sim$ is an equivalence relation. Do you want to think about the topology on that?
If you're like me, then no, you don't, but the good news is that there's no need to -- you can just use that a continuous map out of $X/\sim$ is the same as a map out of $X$ with $f(x) = f(y)$ whenever $x \sim y$. Yoneda's lemma is what guarantees you that you won't miss out on anything about $X/\sim$ by thinking of it this way (you still need the topological construction to know that it exists, but if you trust your textbook, you don't need to use that construction).
(You could say something similar about quotient groups -- if you like, you can think of them in terms of cosets, but you never have to do this to prove anything interesting about them.)
I've motivated the "arrows are more important than objects" viewpoint from the perspective of laziness, but I should mention two caveats:


*

*Somehow the "right"/"most elegant" definition of, say, a quotient object is the functorial one, and the particular construction in a given category is not so important. 

*For more complicated objects like moduli spaces, where the construction might take a whole book, this stops being a question of laziness and starts being a question of practicality.
A: There are many different answer to this question I'm gonna write some of the them.
For start we can observe that when dealing with structured set (for instance groups or topological spaces) lot's of the set theoretical structure can be recovered respectively via group homomorphisms and contiuous functions.
For instance there's a one-on-one correspondence between the elements of  (the underlying set of) a group $G$ and the homomorphisms of the type $\mathbb Z \to G$.
Similarly there's a one-on-one correspondence between the points of a space $X$ (i.e. the elements of the underlying set) and continuous function of the form $\{*\} \to X$ (where $\{*\}$ is the pointed space with the obvious topology).
So we can indentify elements of the underlying sets of these structure with the some arrows in their respective categories, meaning that we can recover the element-structure from arrows.
Another example (which is still a topological one is the following): consider the topological space $\{0,1\}$ with the topology $\{\emptyset,\{0,1\},\{1\}\}$ and let's call this space $\Omega$. There's a one-on-one correspondence between continuous functions of type $X \to \Omega$ and open sets of $X$ (for every $f \colon X \to \Omega$ we have the open set $f^{-1}(\{1\})$ and for every $A \subseteq X$ open we have the continuous function $f_A \colon X \to \Omega$ sending every $x \in A$ in $1$ and the other points in $0$).
So again we could recover some internal structure of the topological space (the open sets) in an arrow theoretic fashion.
Finally there's the above mentioned yoneda lemma which proves that morphisms/arrows are what really matters. 
In every category $\mathbf C$ for every $c \in \mathbf C$ there is a canoical functor $\mathbf C(-,c) \colon \mathbf C^\text{op} \to \mathbf {Set}$.
This functor associate to every $x \in \mathbf C$ the set of morphism $\mathbf C(x,c)$. Yoneda lemma states that for every other $c' \in \mathbf C$  the two functor $\mathbf C(-,c)$ and $\mathbf C(-,c')$ are naturally isomorphic (i.e. there's a family of natural bijections $\langle \mathbf C(x,c) \to \mathbf C(x,c')\rangle_{x \in \mathbf C}$)
if and only if $c$ and $c'$ are isomorphic as objects of $\mathbf C$.
This says that the structure (up to isomorphism) of the object of a category can be recovered by the families of sets $\langle \mathbf C(x,c)\rangle_{x \in \mathbf C}$, so by the arrows on that objects.
Of course there are other good example of how to recover information of an object from the arrows but I hope this ones could be sufficient to motivate the motto morphisms are what matters the most.
A: Well, objects without morphisms are essentially boring. Think of finite-dimensional vector spaces over a field $K$. They are all essentially just $K^n$. So why study them at all? Well, the linear maps between them are interesting! One would like to classify them.
Morphisms serve as "communication" between objects. The theme "arrows are more important than objects" is best illustrated in full generality by the Yoneda Lemma. But actually there are a lot of familiar and specific examples in mathematics.
Algebra: The fundamental theorem on homomorphisms is the tool to work with quotient groups (as well as quotient rings, etc.). It is not really important that they are made up out of cosets, but rather that homomorphisms $G/N \to H$ correspond to homomorphisms $G \to H$ which "kill" $N$. Actually this is the whole idea of this construction: We want to kill elements.
Algebraic geometry: Let $f \in \mathbb{Z}[x,y]$ be a polynomial in two variables, for example $f(x,y) = x^2 - 2 y^2$. Then the solutions of $f$ in a commutative ring $R$ are exactly the homomorphisms of rings $\mathbb{Z}[x,y]/(f) \to R$.
Differential geometry: One tries to understand a manifold $M$ by means of its vector bundles, which are maps $V \to M$ (with extra structure).
Algebraic topology: One tries to understand a nice space $X$ by means of its homotopy groups, which are made up out of maps $S^n \to X$ in the homotopy category. These are groups because $S^n$ carries a natural cogroup structure.
Combinatorics: A coloring of a set $X$ with $n$ colors is just a map $X \to \{1,2,\dotsc,n\}$.
Representation theory: We try to understand a group $G$ by its $K$-linear representations, which are homomorphisms $G \to \mathrm{GL}_n(K)$.
The list is endless, therefore I will stop here. It is very easy to find examples, because almost every modern mathematical publication promotes the theme.
You might be also interested in the theme of "categorical characterizations", started by Freyd, Bergman and others (see for instance here or there or here).
