Compatible family in a presheaf Let C be a category. Let c be an object in C. One definition I know for a compatible family on c in a presheaf F is that it is a natural transformation from R to F where R is a sieve on c. Another definition which I know, is a family $(x_f|f\in R)$ where R is a sieve on c such that for all appropriate g, we have $x_{fg}=F(g)(x_f)$. I was trying to look at $(x_f|f\in R)$ as a set which I came up with the question that what these really are as a set, where they exist. Since if we look at them as natural transformations, if they be equal, we get that the sieves which they have been defined on them, are equal, I guess as a set they should be defined in a way that also we can get this.
 A: Since this is a question about why two definitions are essentially the same thing, it is important to be precise about what these definitions are precisely. Throughout we fix a small category $\mathcal{C}$. If we want to be really precise we also have to adopt the convention that hom-sets are disjoint (i.e. domain and codomain are part of the data of an arrow).

Definition. A sieve on an object $C$ in $\mathcal{C}$ is a set $R$ of arrows in $\mathcal{C}$ with codomain $C$. Such that if $f: D \to C$ is an arrow in $R$ and $g: E \to D$ is any arrow in $\mathcal{C}$, then $fg \in R$.

Recall that a subobject $G$ of a presheaf $F$ in $\mathbf{Set}^{\mathcal{C}^\text{op}}$ consists of a subset $G(C) \subseteq F(C)$ for each object $C$ in $\mathcal{C}$. Such that for any arrow $f: C \to D$ and $x \in G(D)$ we have that $F(f)(x) \in G(C)$. Of course, technically speaking anything isomorphic to such $G$ would be a subobject, but any subobject can be canonically represented as just described. So it is easier to think of them in this way.
Let $y: \mathcal{C} \to \mathbf{Set}^{\mathcal{C}^\text{op}}$ denote the Yoneda-embedding and write $y_C$ and $y_f$ for the embedding of $C$ and $f$ respectively. The point is that a sieve on $C$ is precisely the data of a subobject of $y_C$ in $\mathbf{Set}^{\mathcal{C}^\text{op}}$. That is, given a sieve $R$ on $C$, we can define a presheaf $G_R$ by setting
$$
G_R(D) = \{ f \in R : \operatorname{dom}(f) = D \}.
$$
Conversely a subobject $G$ of $y_C$ yields a sieve $R_G$ by setting
$$
R_G = \bigcup_{D \text{ an object in } \mathcal{C}} G(D).
$$
One easily checks that these operations are indeed well-defined. In particular, we see that the requirement $f \in R \implies fg \in R$ is precisely saying that $f \in G_R(D) \implies y_C(g)(f) \in G_R(E)$ (for any $g: E \to D$). And the same when we start with $G$ and build $R_G$.
So a sieve is a set of arrows in $\mathcal{C}$, and what I denoted by $G_R$ is a subobject in $\mathbf{Set}^{\mathcal{C}^\text{op}}$ and is thus in particular a functor (however you want to encode that). Of course this is only a question about encoding these things, because the above shows that they are essentially the same thing.

The link to compatible families is that a compatible family in some presheaf $F$ for a sieve $R$ is the same thing as a natural transformation $G_R \to F$. That is, given a compatible family $(x_f : f \in R)$ we can define a natural transformation $\xi: G_R \to F$ by letting $\xi_D: G_R(D) \to F(D)$ be given by
$$
\xi_D(f) = x_f.
$$
Conversely, given a natural transformation $\gamma: G_R \to F$ we get a compatible family $(y_f : f \in R)$ by setting
$$
y_f = \gamma_{\operatorname{dom}(f)}(f).
$$

So yes, it only makes sense for two natural transformations to be the same, if their domains are the same. So if we view compatible families $(x_f : f \in R)$ and $(x'_f : f \in R')$ as natural transformations $\xi: G_R \to F$ and $\xi': G_{R'} \to F$, then they can only be the same if $G_R = G_{R'}$, i.e. $R = R'$. This makes sense, because two compatible families are the same if they are pointwise equal, and how would we do that if their indexing sets are different?

Edit. Part of the confusion seemed to come from the notation $(x_f \mid f \in R)$. As explained in the comments: this just means we have a set indexed by $R$. Put differently, it is a function $R \to \bigcup_{C \text{ an object in } \mathcal{C}} F(C)$ sending $f$ to $x_f$. Note the similarity with the natural transformation $\xi$ above, which comes down to breaking this function up for each object of $\mathcal{C}$.
