From where does the dualizing sheaf come from? I have been going through basic Algebraic Geometry, and although I have been using Serre duality for quite a lot of time, I still do not understand how one came up with the dualizing sheaf for a Projective scheme $X \hookrightarrow \mathbb{P}^N.$ The way we prove Serre duality in Hartshorne is like this: (I) First prove it for Projective Space
(II) Define dualizing sheaf, and then prove that for a projective scheme $\mathcal{E}xt(i_*\mathcal{O}_X,\omega_{\mathbb{P}^N})$ is a dualizing sheaf.
(III) Then for the duality we need ample $\mathcal{O}(1)$ and Cohen-Macaulayness.
The proof went smooth, but I did not understand how Hartshorne came up with that dualizing sheaf. I know how in derived categories we look for the right adjoint of $f_*$ to prove Serre duality, but how does that influence the definition of the dualizing sheaf in Hartshorne is still not clear to me. Any comments are welcome.
I am aware of almost all the questions on dualizing sheaf and Serre duality on StackExchange, but none of them seemed to clear my confusion. Thank you for your time.
 A: Actually, we've already seen a sheaf on projective space which satisfies some of the properties we're after! That is, we've seen a sheaf $\mathcal{E}$ on $\Bbb P^n_k$ and a morphism $t:H^n(\Bbb P^n_k,\mathcal{E})\to k$ which for some class of coherent sheaves $\mathcal{F}$ on $\Bbb P^n$ satisfies the properties of a dualizing sheaf, namely that $$\operatorname{Hom}(\mathcal{F},\mathcal{E})\times H^n(\Bbb P^n_k,\mathcal{F})\to H^n(\Bbb P^n_k,\mathcal{E})$$ followed by $t:H^n(\Bbb P^n_k,\mathcal{E})\to k$ gives an isomorphism $\operatorname{Hom}(\mathcal{F},\mathcal{E})\to H^n(\Bbb P^n_k,\mathcal{F})'$.
In the computation of the cohomology of sheaves of the form $\mathcal{O}(d)$ on projective space $\Bbb P^n_A$ (theorem III.5.1), we showed that the natural map $$H^0(\Bbb P^n_A,\mathcal{O}(d))\times H^n(\Bbb P^n_A,\mathcal{O}(-n-d-1))\to H^n(\Bbb P^n_A,\mathcal{O}(-n-1))\cong A$$ is a perfect pairing of finitely generated free $A$-modules. Reindexing slightly, we can rewrite this as $$H^0(\Bbb P^n_A,\mathcal{O}(-n-d-1))\times H^n(\Bbb P^n_k,\mathcal{O}(d))$$ and recognizing that $\operatorname{Hom}(\mathcal{O}(d),\mathcal{O}(-n-1))\cong \operatorname{Hom}(\mathcal{O},\mathcal{O}(-n-d-1))\cong H^0(\Bbb P^n_A,\mathcal{O}(-n-d-1))$, we've exactly found that $\mathcal{E}=\mathcal{O}(-n-1)$ and the isomorphism $H^n(\Bbb P^n_k,\mathcal{O}(-n-1))\cong k$ satisfy what we ask of a dualizing sheaf when $\mathcal{F}$ is among the sheaves of the form $\mathcal{O}(d)$. As $\mathcal{O}(-n-1)\cong\bigwedge^n\Omega_{\Bbb P^n_k/k}$ is the canonical sheaf of projective space, this would be a sign that we're on the right track if we're trying to invent this theory.

There's also good intuition coming from the manifold case: remember, Serre duality is supposed to be kind of like an analogue of Poincare duality. How does Poincare duality work? Cap with the fundamental class! What is the fundamental class? It's an element of the top-dimensional homology which tracks orientations. Orientations are described by an ordered basis of the tangent space at each point up to equivalence, which can be measured by top-dimensional differential forms. So poking around top-dimensional differential forms to try and invent a duality theory is a natural avenue for experimentation.

Finally, let me point out that I've never actually asked Serre, Hartshorne, or any of the other mathematicians who developed this theory about how they got to the idea or read accounts of their thought processes, so I can't be sure that what I've related here was actually a factor in how they got to these concepts. But I think this material is certainly at least suggestive of a direction to explore - once you have the motivation of "can I prove some sort of duality for varieties?" and some of this evidence in the form of calculations showing some preliminary success, it would be natural to investigate further.
