Hilbert scheme of quasi-projective variety

Suppose $X$ is a projective scheme over an algebraically closed field $k$, denote its Hilbert scheme with Hilbert polynomial $p$ by $\text{Hilb}^p_X$, then from section 1.1 of Nakajima's book, Lectures on Hilbert Schemes of Points on Surfaces, it claims that

" Moreover, if we have an open subscheme $Y$ of $X$, then we have the corresponding open subscheme $\text{Hilb}^p_Y$ of $\text{Hilb}^p_X$ parametrizing subschemes in $Y$. In particular, $\text{Hilb}^p_Y$ is defined for a quasi-projective scheme $Y$."

I could see this is true for the Hilbert scheme of points, i.e. $p$ is a constant polynomial. Is it true generally? One basic problem is could the Hilbert polynomial of a quasi-projective variety be defined?

• Not sure if this is good enough for you but there is some discussion in section 9 here – Billy O. Mar 14 '18 at 7:51

In the sentence you quoted, "parametrizing subschemes in $Y$" should be interpreted as "parametrizing closed subschemes of $X$ which are entirely contained in $Y$". So these subschemes are still projective, and their Hilbert polynomial is well-defined.
So, for example, if $Y$ is affine, then $Hilb_Y^p=\emptyset$ whenever $p$ has positive degree, as $Y$ does not contain any projective scheme of positive dimension.
Note also that a family of such subschemes in $Y$ over a base $S$ can be defined intrinsecally (without the need of the open immersion $Y\hookrightarrow X$) as a closed subscheme $Z\subset Y\times S$ which is flat and projective over $S$.
• Well, probaly I am wrong, but as I understand, for Hilbert schemes, Lemma 6.2 is saying that every closed subscheme of $Y$ can be prolonged to a closed subscheme of $X$, and Theorem 6.3 says that $Hilb_Y^p$ is actually "parametrizing closed subschemes of $Y$", not "closed subschemes of $X$ which are entirely contained in $Y$" (Regarding OP's question, Lemma 6.2 is telling you how to compute the Hilbert polynomial of a quaisprojective variety). – MonLau Mar 30 at 11:46
• I agree with the first part of your sentence (of course that is the same as taking the closure in $X$), but not with the second: look at how the Hilbert scheme is defined between pages 3 and 4 of the notes you linked (in particular page 4, line 5), that's essentially the last sentence of my answer. – Andrea Mar 30 at 16:01