First and Second Quantization of the Hamiltonian - Quantum Mechanics

I am trying to show the relation between the first and second quantization in Quantum Mechanics. I have been told that the general relationship that holds is that in the first quantization, we can write the Hamiltonian as:

$$\hat{\mathcal{H}}_1=\sum_n \bigg[ \frac{\hat{p}^2_n}{2m}+V(\hat{x}_n) \bigg]+ \frac{1}{2} \sum_{n \neq m} W(\hat{x}_n-\hat{x}_m)$$

and in second quantisation: $$\hat{\mathcal{H}}_2= \int dx\, \hat{\Phi}^{\dagger}(x) \Big[ \frac{- \hbar^2}{2m} \partial_x^2+V(x)\Big]\hat{\Phi}(x)+\frac{1}{2}\int dx\, dx'\; W(x-x')\hat{\Phi}^{\dagger}(x)\hat{\Phi}^{\dagger}(x')\hat{\Phi}(x')\hat{\Phi}(x),$$

and then it follows that: $$\langle x_1 \dots x_N |\hat{\mathcal{H}}_2|\phi \rangle=\mathcal{H}_1 \phi(x_1,\dots,x_N),$$

where $$\hat{\Phi}$$ is the Field operator and $$W$$ is the interaction between particles.

Now I just want to show this for a single particle, where the interaction terms disappear.

My attempt was this: I will use that $$\hat{\Phi}(x_1) \hat{\Phi}^{\dagger}(x) = \big[\hat{\Phi}(x_1),\hat{\Phi}^{\dagger}(x)\big]+\hat{\Phi}^{\dagger}(x)\hat{\Phi}(x_1)$$ (Field operator for bosons).

Then;

\begin{align*} &\langle 0| \hat{\Phi}(x_1) \int dx\, \hat{\Phi}^{\dagger}(x) \Big[ \frac{- \hbar^2}{2m} \partial_x^2+V(x)\Big]\hat{\Phi}(x)|\phi \rangle \\ =& \langle 0| \int dx\, \hat{\Phi}(x_1) \hat{\Phi}^{\dagger}(x) \Big[ \frac{- \hbar^2}{2m} \partial_x^2+V(x)\Big]\hat{\Phi}(x)|\phi \rangle \\ =& \langle 0| \int dx \, \bigg\{\big[\hat{\Phi}(x_1),\hat{\Phi}^{\dagger}(x)\big]+\hat{\Phi}^{\dagger}(x)\hat{\Phi}(x_1)\bigg\} \Big[ \frac{- \hbar^2}{2m} \partial_x^2+V(x)\Big]\hat{\Phi}(x)|\phi \rangle \\ =& \langle 0| \int dx\, \bigg\{ \delta(x_1-x)\Big[ \frac{- \hbar^2}{2m} \partial_x^2+V(x)\Big]\hat{\Phi}(x) \bigg\} |\phi \rangle\\ =& \langle 0|\Big[ \frac{- \hbar^2}{2m} \partial_{x_1}^2+V(x_1)\Big]\hat{\Phi}(x_1) |\phi \rangle\\ \end{align*}

Now if I can get the Field operator to commute with the differential operator and potential bracket, I will have exactly what I desire. I feel like this is wrong to do and believe I have made a mistake above, most likely with using the delta function, but I cannot identify it myself.