Question about intersecting pullback of invertible sheaves by a projective morphism. This is exercise 20.1.J in Vakil's Foundations of Algebraic Geometry. $\def\supp{\operatorname{supp}} \def\int{\operatorname{int}}$
Let $X$ be a projective scheme over a field $k$, and $F$ be a coherent sheaf with $\supp(F)$ proper and $\dim(\supp(F)) \leq n$. Let $L_1, \ldots, L_n$ be invertible sheaves on $X$. We define $\int_X(L_1, \ldots, L_n, F) = \sum_{I \subseteq \{1, \ldots, n\}} (-1)^{|I|} \chi(X, \otimes_{i \in I} L_i^\vee \otimes F) $, where $\chi$ is Euler characteristic.
Now suppose $f : X \rightarrow Y$ is a morphism of integral projective schemes, both having dimension $n$. Let $L_1, \ldots, L_n$ be invertible sheaves on $Y$. The question is to prove $\int_Y(f^* L_1, \ldots, f^* L_n, O_Y) = \int_X(L_1, \ldots, L_n, O_X) * \deg(f)$.
I'm stuck trying to solve this. First, I reduced to the case where $L_i$ are all very ample. Then, the hint given is to find a dense open set where $f_* O_X$ is locally free of rank $\deg(f)$. I did this, but I'm not sure why the rank of $f_* O_X$ is relevant.
I also tried to use $L_i = O(D)$ for some effective Cartier divisor $i : D \rightarrow Y$. I then tried to use induction, but $D$ might not be irreducible so I can't use the induction hypothesis directly. I then tried to use $L_n$ to embed $Y$ into projective space. Then, find $n$ hyperplanes that intersect with $Y$ in a finite number of points (this is 11.3.C in Vakil). This corresponds to global sections of $L_n$ that vanishes at a finite number of points. But I'm not sure what to do next.
The thing that I'm stuck on is why does "$f_* O_X$ is locally free of rank $n$ on a dense open subset" help solve the problem?
 A: Here's the solution I came up with while reading this section of Vakil's book.
We can assume $k$ is infinite. This is because, by base change with the field extension $k(t)/k$, where $t$ is transcendental, the degree of morphisms is preserved, $X_1, X_2$ remain integral schemes, and intersection numbers are also preserved, by Exercise 20.1.D. Also, by linearity of the intersection form we can assume all the $\mathscr{L}_i$ are very ample (by Exercise 16.6.F)
Case 1: $\deg(\pi)>0$. $\pi$ is dominant.
Using the hint given in the exercise, we can find a dense open set $U\subset X_2$ over which the morphism is finite, and $\pi_*\mathscr{O}_{X_1}$ is free of rank $d=\deg{\pi}$. Let $Z$ be the complement of $U$, with the reduced scheme structure. We have $\dim(Z)<\dim(X_2)=n$.
Pick an embedding $X_2\to \mathbb{P}^N$ using $\mathscr{L}_1$. We can find a hyperplane $H$ in $\mathbb{P}^N$ missing all of the associated points of $X_2$, the associated points of $Z$, and the images of the associated points of $X_1$ by $X_1\to X_2\to \mathbb{P}^N$. Then $D_1=H\cap X_2$ is a cartier divisor, with $\mathscr{L}_1\cong \mathscr{O}(D_1)$.
We now inductively choose $D_i$ as follows:

*

*Embed $X_2\to \mathbb{P}^N$ using $\mathscr{L}_i$.

*Pick a hyperplane not containing the images of the associated points of the following schemes:
$$X_2, X_1$$
$$D_1\cap \cdots \cap D_{i-1}$$
$$ Z\cap D_1 \cap \cdots \cap D_{i-1}$$
$$ \pi^*D_1\cap \pi^*D_2\cap \cdots \cap \pi^*D_{i-1}$$
(where $\pi^*D$ is the pullback of $D$ to $X_1$, a Cartier divisor).

*Now set $D_i=H\cap X_2$,

We then have by construction that $D_i$ misses the associated points of the schemes
$$D_1\cap \cdots \cap D_{i-1},\ Z\cap D_1\cap \cdots \cap D_{i-1}$$
and $\pi^*D_i$ misses the associated points of $\pi^*D_1\cap \cdots \cap \pi^*D_{i-1}$.
We thus have
$$\dim(Z\cap D_1\cap \cdots \cap D_n)<\dim(Z\cap D_1\cap\cdots \cap D_{n-1})<\cdots<\dim(Z)<n$$
with $n+1$ terms, which proves that $\dim(Z\cap D_1\cap\cdots \cap D_n)\leq -1$, in other words that intersection with $Z$ is empty. Similarly, using
$$\dim(D_1\cap \cdots \cap D_n)<\dim(D_1\cap\cdots \cap D_{n-1})<\cdots<\dim(D_1)<n$$
we have $\dim(D_1\cap \cdots \cap D_n)\leq 0$, meaning the intersection has dimension zero or is empty. In any case, it has no higher cohomology.
Now, using exercise 20.1.C iteratively, we can write
$$(\mathscr{L}_1\cdot \mathscr{L_2}\cdots \mathscr{L}_n)=\chi(D_1\cap D_2\cap \cdots \cap D_n)=h^0(D_1\cap D_2\cap \cdots \cap D_n)$$
This intersection is entirely contained in $U$ (the intersection with $Z$ is empty). We can repeat this process on $X_1$: we have
$$\dim(\pi^*D_1\cap \cdots \cap \pi^*D_n)<\dim(\pi^*D_1\cap\cdots \cap \pi^*D_{n-1})<\cdots<\dim(\pi^*D_1)<n$$
so $\pi^*D_1\cap \cdots \cap \pi^*D_n$ has dimension $0$ or is empty. In addition, as $\pi^*\mathscr{L}_i\cong \mathscr{O}(\pi^*D_i)$, we have
$$(\pi^*\mathscr{L}_1\cdot \pi^*\mathscr{L_2}\cdots \pi^*\mathscr{L}_n)=h^0(\pi^*D_1\cap \pi^*D_2\cap \cdots \cap \pi^*D_n)$$
where the intersection is contained in $\pi^{-1}(U)$.
Now, $D_1\cap D_2\cap \cdots \cap D_n$ consists of a finite set of possibly nonreduced closed points $p_1, ..., p_m$. Then
$$h^0(D_1\cap D_2\cap \cdots \cap D_n)=\sum_{i=1}^m h^0(p_i)$$
where each $p_i$ is considered as a scheme.
The pullback $\pi^*D_1\cap \pi^*D_2\cap \cdots \cap \pi^*D_n$ is the disjoint union of the fibers $\pi^{-1}p_i$, considered as closed subschemes of $X_1$. From this
$$h^0(\pi^*D_1\cap \pi^*D_2\cap \cdots \cap \pi^*D_n)=\sum_{i=1}^m h^0(\pi^{-1}p_i)$$
Thus we only need to prove that for a closed subscheme $p$ of $U$ consisting of a single closed point, we have
$$h^0(\pi^{-1}p)=d\cdot h^0(p)$$.
Pick an open affine neighborhood $\text{Spec}\ B\subset U$ of $p$. Let $p = \text{Spec}\ A$. Let $\text{Spec}\ C=\pi^{-1}(\text{Spec}\ B)$, so that $C\cong B^{\oplus d}$ as $B$-modules.
Then
$$\pi^{-1}p\cong \text{Spec}\ A\otimes_B C$$
We want to compute the dimension as a vector space over $k$ of this ring. So we can forget about the ring structure. As a $B$-module (and in particular as vector spaces), we have an isomorphism
$$A\otimes_B C\cong A\otimes_B B^{\oplus d}\cong A^{\oplus d}$$
This obviously has dimension $d\cdot h^0(\text{Spec}\ A)$, so we're done.
Case 2: $\deg{\pi}=0$, meaning $\pi$ is not dominant.
$\pi$ is a closed morphism, as a map between projective varieties. Let $Z$ be the scheme-theoretic image of $X_1$ in $X_2$. It is irreducible (the image of an irreducible scheme), and reduced (because its sheaf of ideals is defined locally on $\text{Spec} A\subset X_2$ by the kernel of
$$A\to \mathscr{O}_{X_1}(\pi^{-1}(\text{Spec} A))$$
which is a radical ideal, as both rings are integral domains). Then $\dim(Z)<\dim(X_2)$, and $X_2\to Z$ is dominant.
Repeat the construction of the $D_i$ in the previous part, so that $\mathscr{L}_i\cong \mathscr{O}(D_i)$, $D_i$ does not contain an associated point of $Z\cap D_1\cap \cdots \cap D_{i-1}$, and $\pi^*D_i$ does not contain an associated point of $\pi^*D_1 \cap \cdots \cap \pi^*D_{i-1}$.
Now, for each $i=1, 2, \dots , n$ we have
$$\dim(Z\cap D_1\cap D_2\cap \cdots \cap D_i)<\dim(Z\cap D_1\cap \cdots D_{i-1})$$
as $D_i$ was chosen not to contain associated points of $Z\cap D_1\cap \cdots \cap D_{i-1}$. Becuase $\dim(Z)<n$, and each time the dimension decreases, the intersection $Z\cap D_1\cap \cdots \cap D_n$ must be empty.
By a similar reasoning to the previous case,
$$(\pi^*\mathscr{L}_1\cdot \pi^*\mathscr{L_2}\cdots \pi^*\mathscr{L}_n)=h^0(\pi^*D_1\cap \pi^*D_2\cap \cdots \cap \pi^*D_n)$$
but the last intersection is the pullback of the empty scheme, so empty. Thus the intersection number is $0$ as expected. $\blacksquare$
A: Question: "The thing that I'm stuck on is why does "f∗OX is locally free of rank n on a dense open subset" help solve the problem?"
Answer: Let $U \subseteq Y$ be an open set where $f_*(\mathcal{O}_X)$ is locally trivial of rank $n$ and let $Z:=Y-U$ with $dim(Z)<dim(X)$. If your "Euler characteristic" satisfies "excision"
it follows
$$\chi(Y,F)=\chi(Z,F_Z)+\chi(U,F_U)$$
where $F_Z, F_U$ is the restriction of $F$ to $Z$ and $U$. This depends on what type of "Euler characteristic" you are studying. Maybe the book uses excision without mentioning it.
Whenever you have a "cohomology theory" $H^*(-)$ of $k$-vector spaces with $k$ a field, defined for quasi coherent sheaves you may define the $H$-Euler characteristic
$$\chi_H(X,F):= \sum_{i\geq 0}  (-1)^i(dim_k(H^i(X,F)).$$
This may not be well defined if the dimensions are infinite.
The properties of $\chi(X,F)$ will depend on your cohomology theory $H$.
If you study cohomology of coherent sheaves, it follows your "Euler characteristic" is additive on short exact sequences: Whenever
$$0 \rightarrow F'' \rightarrow F \rightarrow F' \rightarrow 0$$
is exact, it follows
$$\chi(F)=\chi(F'')+\chi(F').$$
This is proved in Hartshorne, Ex.III.5.1.
