Question: "Two questions: Is this formula even true if the field is not closed (and/or not perfect)? If 1. is true, then what is a (citable) reference that states this formula for curves over arbitrary fields? I will accept an answer that posts nothing but a citable source (maybe including the page) which contains the statement for arbitrary fields."
Answer: You also find a discussion of the arithmetic genus for a projective scheme of finite type over any field in Hartshorne, Ex.III.5.3. In ChI Ex.7.2, the arithmetic genus is defined using the Hilbert polynomial. In ChIII it is defined using the Euler characteristic $\chi(\mathcal{O}_X):=\sum_i h^i(X, \mathcal{O}_X)$ and the exercise proves that this equals the definition in ChI. Using the Hilbert polynomial it is not clear that the arithmetic genus is independent of choice of embedding, but the formula
$$p_a(X)=(-1)^n(\chi(\mathcal{O}_X)-1)$$
proves independence - the structure sheaf is an intrinsic invariant and does not depende on an embedding.
Note: The constructions of chapter I in Hartshorne are valid over an algebraically closed field $k$. The arithmetic genus $p_a(Y)$ is defined in Ex.I.7.2 for a projective variety $Y \subseteq \mathbb{P}^n_k$, hence $Y$ is by definition irreducible. By definition $p_a(Y):=(-1)^r(P_Y(0)-1)$ where $P_Y(t)$ is the Hilbert polynomial of the graded coordinate ring $S(Y)$ of $Y$ wrto the embedding $Y \subseteq \mathbb{P}^n_k$, hence with this definition it is not clear if $p_a(Y)$ is independent of choice of embedding: The coordinate ring $S(Y)$ is not an invariant of $Y$.
In chapter III they give a general definition of the arithmetic genus $p_a(Y)$ for any closed subscheme $Y \subseteq \mathbb{P}^n_k$ over any field. Hence if you study a projective scheme over a non-algebraically closed field you must use the definition in III.Ex.5.3. For a curve $C \subseteq \mathbb{P}^2_k$ you get by definition
$$p_a(C):=1-h^0(C, \mathcal{O}_C)+h^1(C,\mathcal{O}_C),$$
and there are several methods to calculate this. You must somehow calculate $h^i(C,\mathcal{O}_C)$ and you find methods in the chapter on Cech cohomology. Here is a similar question with some suggestions:
Arithmetic genus of curves
Note (dec 2022) : This is also stated as an exercise in Hartshorne (HH) (Ex.III.4.7) in the chapter on cohomology. You must write down the Cech complex and calculate the dimensions explicitly. There is no condition on the base field or if the polynomial is irreducible. I do not think there is a "quick and easy method" as someone claims. The Hartshorne exercise reduces the case to covering the curve $C$ with two affine open subsets.
Here is a calculation using Ex.III.4.1 in HH: By Serre duality and Ex.III.4.1 in HH the following holds: Let $i:C \rightarrow S:=\mathbb{P}^2_k$ be the embedding into the projective plane. Since $i$ is an affine morphism it follows
$$H^i(C, \mathcal{O}_C) \cong H^i(S, i_*\mathcal{O}_C)$$
and
$$H^2(S, \mathcal{O}_S(-d)) \cong H^0(S, \mathcal{O}_S(d-3))^*$$
hence $h^2(S, \mathcal{O}_S(-d))=\frac{(d-1)(d-2)}{2}$ and $h^0(S, \mathcal{O}_S)=1$. Taking the long exact cohomology sequence of the sequence
$$ 0 \rightarrow \mathcal{O}_S(-d) \rightarrow \mathcal{O}_S \rightarrow i_*\mathcal{O}_C \rightarrow 0$$
you get the sequence
$$0 \rightarrow H^0(S, \mathcal{O}(-d)) \rightarrow H^0(S, \mathcal{O}_S) \rightarrow H^0(C, \mathcal{O}_C) \rightarrow $$
$$H^1(S, \mathcal{O}(-d)) \rightarrow H^1(S, \mathcal{O}_S) \rightarrow H^1(C, \mathcal{O}_C) \rightarrow $$
$$H^2(S, \mathcal{O}(-d)) \rightarrow H^2(S, \mathcal{O}_S) \rightarrow H^2(C, \mathcal{O}_C)=0.$$
It follows
$$H^0(C, \mathcal{O}_C) \cong H^0(S, \mathcal{O}_S) \cong k$$
and
$$H^1(C, \mathcal{O}_C) \cong H^2(S, \mathcal{O}(-d)) \cong H^0(S, \mathcal{O}(d-3))^*$$
hence
$$p_a(C)=1-h^0(C,\mathcal{O}_C)+h^1(C, \mathcal{O}_C)=$$
$$ 1-1+h^1(S,i_*\mathcal{O}_C) = h^2(S, \mathcal{O}(-d))-h^2(S, \mathcal{O}_S)=h^2(S, \mathcal{O}(-d))=h^0(S, \mathcal{O}(d-3)).$$
You get the formula
$$p_a(C)=\frac{(d-1)(d-2)}{2}.$$
Hence the result also follows from Ex.III.4.1 and III.5.1 in Hartshorne. In general a curve cannot be embedded into the projective plane, and for such curves the above does not apply.
