What does it mean for points of the modular curve $X(N)$ to be "defined over $\mathbb{F}_p$"? I'm trying to study a collection of elliptic curves over some fixed finite field $\mathbb{F}_p$. By browsing the literature and discussing with my supervisor, it seems like it will be fruitful to study some sort of a modular curve.
I am familiar with the classic modular curves $X(N),X_0(N),X_1(N)$, etc. through the first three chapters of Diamond and Shurman's A First Course In Modular Forms. I understand that these curves are Riemann surfaces and form a moduli space for isomorphism classes of elliptic curves over $\mathbb{C}$.
In this paper that I am reading, the authors speak of points of $X_1(N)$ that are defined over $\mathbb{F}_p$ and they use the notation $X_1(N)(\mathbb{F}_p)$ which makes it seem like they're probably using the language of schemes.
I am aware of the books (or long papers):

*

*Les schemas de modules de courbes elliptiques by Deligne-Rapoport (1973)

*Arithmetic Moduli of Elliptic Curves by Katz-Mazur (1985)

I know the basic definitions of schemes from Hartshorne's Algebraic Geometry but I do not know many results beyond their first properties. Both of these books address the topic of reduction mod $p$ at some point and I believe this is exactly what I'm looking for. However, as I'm just beginning my study on this topic, I'm finding it hard to digest the discussions, or to even locate the relevant discussions, in either book, especially the first one, which is written in French, and seems to be the canonical reference on this subject by my search online.
Any of the following responses will be helpful to me:

*

*A rough sketch motivating what it means to have points of $X(N),X_0(N),X_1(N)$ defined over $\mathbb{F}_p$.

*Reference to a specific page or section in Deligne-Rapoport or Katz-Mazur addressing my first point.

*Reference to a source outside of Deligne-Rapoport and Katz-Mazur addressing my first point. Ideally, the source should be in English.

 A: There is an excellent answer above, so I'm not going to address the general situation - instead I'll provide a different (much more explicit) description when thinking about $X_1(N)$ and discuss the case when $N = 5$ which (in my opinion) gives a bit more intuition that grappling with the general setup in the (pretty difficult) material of KM and DR.
Recall that the moduli problem is set up as follows (at least in moral terms) for $Y_1(N)$:

Does there exist a curve, $X_1(N)$, over $\mathbb{Q}$ (can we make sure it's defined over $\mathbb{Q}$ with integer coefficients and make sure bad stuff doesn't happen when reducing mod $p$ for $p$ not dividing $N$?) such that for each $K/\mathbb{Q}$, each point $Q \in X_1(N)(K)$ corresponds to a pair $(E, P)$ where $E/K$ is an elliptic curve and $P$ is a $K$-rational point on $E$ of exact order $N$?

The answer to the above question is "yes when $N > 3$" (and note that when you take $K = \mathbb{C}$ this is exactly what you know!). With $X_0$ and $X$ you need to replace $K$-rational with "galois stable cyclic subgroup (or isogeny over $K$)" and "$K$-rational basis $\{P_1,P_2\}$ and some stuff with the Weil pairing" respectively.
Now for something explicit. Lets compute! $X_1(5)$ (well, of course this is just $\mathbb{P}^1$, but the moduli interpretation will also come).
I'll follow Silverman's The Arithmetic of Elliptic Curves exercise $8.13$. Take a pair $(E, P)$ where $E/K$ is an elliptic curve ($K$ has characteristic $0$) and $P \in E(K)$ has order $5$. We can assume that $E$ is given by a Weierstrass equation
$$E : y^2 + u xy + vy = x^3 + vx^2$$
and $P = (0,0)$ (prove this for general $N \geq 4$ - what about the characteristic). Then for $5P = O$ we must have $3P = -2P$. In particular we must have
$$(-u + 1 : u - v - 1 : 1) = (-v : 0 : 1) $$
i.e., $P$ has order $5$ if and only if $v = u-1$. So we have $X_1(5) \cong \mathbb{P}^1$ and the moduli interpretation takes
$$ [u:1] \mapsto (E_u, (0,0) )$$
where $E_u : y^2 + uxy + (u-1)y = x^3 + (u-1)x^2$.
So now: (1) can you mimic this to work over $\mathbb{F}_p$ where $p \neq 5$?, (2) can you find the hole when $p = 5$?
A bit harder is to use this exact construction to show that when $\operatorname{K}$ has characteristic not diving $N$ there is a curve $X_1(N)$ which works. A bit harder than that is to show that the "same" curve works for each characteristic (this is more or less "the moduli problem is representable over $\mathbb{Z}[1/N]$).
The moral is just that this is not so scary as long as $K$ does not have characteristic $p$!
A: Let me start with a disclaimer that my answer might or might not be what you are looking for/need. But since you mentioned the two references, KM and DR, I will try to digest some important points for your purposes. Furthermore I will do everything for $Y_i(N)$ instead of $X_i(N)$, i.e. I will ignore the issue of compactification. While it certainly deserves attention, I think at first it is easier to blend it out and not worry about it.
As you mentioned, the point of view one ultimately wants to take is that of schemes. But to do that, let us first start with the complex picture. In that case we can choose a congruence subgroup and a moduli problem:

*

*$\Gamma(N)$: In this case $\mathcal{H}/\Gamma(N)$ classifies $\{E,\alpha:(\mathbb{Z}/N)^2\cong E[N]\}$ an elliptic curve over $\mathbb{C}$ with an explicit choice of an isomorphism trivializing the $N$-torsion group of $E$

*$\Gamma_1(N)$: In this case $\mathcal{H}/\Gamma_1(N)$ classifies $\{E,Q\in E[N] \textrm{ of exact order } N\}$

*$\Gamma_0(N)$: In this case $\mathcal{H}/\Gamma_0(N)$ classifies $\{E,C\subset E[N] \textrm{ a cyclic subgroup of rank } N\}$
Now the very first question to ask here is: Are these spaces $\mathcal{H}/\Gamma_i(N)$ algebraic? This means: Are there schemes $Y_i(N)/\mathbb{C}$ such that $Y_i(N)(\mathbb{C})\cong \mathcal{H}/\Gamma_i(N)$? Or even better a scheme which lives over a number field $K$?
The good news is that this works in a great generality. The bad news is that the $Y_i(N)$ might be not representing a moduli problem.
Before continuing to talk about moduli problems, let me justify a bit, why all this trouble will be necessary to define modular curves over finite fields. So far we are considering two objects: Modular curves $Y_i(N)$ over $\mathbb{C}$ and modular curves $Y_i(N)_{\mathbb{F}_p}$ over a finite field. But we have no way of comparing these two objects, since they live in fields over different characteristic. What does it mean that $Y_i(N)_{\mathbb{F}_p}$ is a modular curve? The way to make sense of this is via integral models. Put short you will need a ring $R$ with maps to both the complex numbers and the finite field in question (for us this will be something like $\mathbb{Z}[\frac{1}{n}]$). Then a scheme $Z/R$ is called an integral model of $Y_i(N)$, if $Z\times_R \mathbb{C}$ is isomorphic to $Y_i(N)$ as varieties over $\mathbb{C}$. In that situation you call $Z\times_R \mathbb{F}_p$ the modular curve over the finite field. Now let me explain how to get such an integral model via moduli problems.
First let us take a closer look at the moduli problems we are interested in. We can consider the following functors {Schemes} $\to$ {Sets}

*

*$F:S\mapsto \{E/S,\alpha:(\underline{\mathbb{Z}/N})_S\cong E[N] \textrm{ an isomorphism of group schemes}\}/\sim$ ($\sim$ means up to isomorphism respecting the extra structure)

*$F_1:S\mapsto \{E/S, Q\in E[N](S) \textrm{ a point of exact order }N\}/\sim$

*$F_0:S\mapsto \{E/S, C\subset E[N] \textrm{ cyclic subgroup scheme of exact order }N \}/\sim$
Note that these descriptions use the theory of finite flat group schemes, but that is not necessary to understand to get the big picture. Now are these functors representable? That is, is there a scheme $Z$ such that $F_i(S)\cong \textrm{Hom}(S,Z)$? If so, we would have found our integral model, since by definition $F_i(\mathbb{C})\cong Y_i(N)(\mathbb{C})$ (the actual argument is slightly more subtle, but that is the essential idea).
In full generality this will not work, a reason being that finite flat group schemes of order $p^n$ behave rather bad mod $p$. The important thing is that everything works out if you look at schemes over $\mathbb{Z}[\frac{1}{N}]$. To be more precise let me state the following Theorem:

Theorem: Let $N\geq 3$ (resp. $N\geq 5$), then $F$ (resp. $F_1$) is representable by an affine smooth scheme of dimension $1$ over $\mathbb{Z}[\frac{1}{n}]$.

The $F_0$ case is more involved. If you are interested in this, you can consult Brian Conrad's notes, Section 4.2.5-4.2.7 (I recommend skimming it at first read). I know that KM also address this theorem, at least partially in Chapter 4.
And finally, let me now say a few words about the special fiber (i.e. the objects over finite fields). Here you can now use adjectives like supersingular and ordinary and apply them to our moduli problems. This is, partially, what DR studies, yielding interesting and clearly understood relationships between certain moduli problems/modular curves over finite fields. This is addressed in 4.2.8 in Conrad's notes.
