We will use
$$
\int_{-\infty}^\infty x^{2k}e^{-\frac{x^2}{2}}\,\mathrm{d}x=(2k-1)!!\sqrt\pi\tag{1}
$$
If we take the principal value, we get the convergent series
$$
\begin{align}
&\mathrm{PV}\frac1{\sqrt{2\pi}\sigma}\int_{-\infty}^\infty\frac1xe^{\large-\frac{(x-\mu)^2}{2\sigma^2}}\,\mathrm{d}x\\
&=\frac1{\sqrt{2\pi}\sigma}\int_0^\infty\frac1x\left(e^{\large-\frac{(\mu-x)^2}{2\sigma^2}}-e^{\large-\frac{(\mu+x)^2}{2\sigma^2}}\right)\,\mathrm{d}x\\
&=\frac1{\sqrt{2\pi}\sigma}\int_{-\infty}^\infty\frac1x\sinh\left(\frac{\mu x}{\sigma^2}\right)e^{\large-\frac{\mu^2+x^2}{2\sigma^2}}\,\mathrm{d}x\\
&=\frac1{\sqrt{2\pi}\sigma}e^{\large-\frac{\mu^2}{2\sigma^2}}\int_{-\infty}^\infty\frac1x\sinh\left(\frac\mu\sigma x\right)e^{\large-\frac{x^2}{2}}\,\mathrm{d}x\tag{2}\\
&=e^{\large-\frac{\mu^2}{2\sigma^2}}\sum_{k=0}^\infty\frac{(2k-1)!!}{(2k+1)!}\frac{\mu^{2k+1}}{\sigma^{2k+2}}\tag{3}
\end{align}
$$
We can also get an asymptotic expansion from $(2)$ using stationary phase:
$$
\begin{align}
&\frac1{\sqrt{2\pi}\sigma}e^{\large-\frac{\mu^2}{2\sigma^2}}\int_{-\infty}^\infty\frac1x\sinh\left(\frac\mu\sigma x\right)e^{\large-\frac{x^2}{2}}\,\mathrm{d}x\\
&=\mathrm{PV}\frac1{\sqrt{2\pi}\sigma}\int_{-\infty}^\infty\frac12\left(\frac1{\frac\mu\sigma+x}+\frac1{\frac\mu\sigma-x}\right)e^{\large-\frac{x^2}{2}}\,\mathrm{d}x\\
&=\mathrm{PV}\frac1{\sqrt{2\pi}\mu}\int_{-\infty}^\infty\frac12\left(\frac1{1+\frac\sigma\mu x}+\frac1{1-\frac\sigma\mu x}\right)e^{\large-\frac{x^2}{2}}\,\mathrm{d}x\\
&\sim\frac1\mu\sum_{k=0}^\infty(2k-1)!!\frac{\sigma^{2k}}{\mu^{2k}}\tag{4}
\end{align}
$$
We can get a "closed form" in terms of $\mathrm{erfi}$ from $(2)$
$$
\begin{align}
&\frac{\mathrm{d}}{\mathrm{d}\alpha}\frac1{\sqrt{2\pi}}\int_{-\infty}^\infty\frac1x\sinh\left(\alpha x\right)\,e^{\large-\frac{x^2}{2}}\,\mathrm{d}x\\
&=\frac1{\sqrt{2\pi}}\int_{-\infty}^\infty\cosh\left(\alpha x\right)\,e^{\large-\frac{x^2}{2}}\,\mathrm{d}x\\
&=e^{\large\frac{\alpha^2}{2}}\tag{5}
\end{align}
$$
Therefore,
$$
\begin{align}
\frac1{\sqrt{2\pi}}\int_{-\infty}^\infty\frac1x\sinh\left(\alpha x\right)\,e^{\large-\frac{x^2}{2}}\,\mathrm{d}x
&=\int_0^\alpha e^{\large\frac{t^2}{2}}\,\mathrm{d}t\\
&=\frac{\sqrt2}i\int_0^{i\alpha/\sqrt2} e^{\large-t^2}\,\mathrm{d}t\\
&=\frac{\sqrt{2\pi}}{2}\,\mathrm{erfi}(\alpha/\sqrt2)\tag{6}
\end{align}
$$
and plugging $(6)$ into $(2)$ yields
$$
\mathrm{PV}\frac1{\sqrt{2\pi}\sigma}\int_{-\infty}^\infty\frac1xe^{\large-\frac{(x-\mu)^2}{2\sigma^2}}\,\mathrm{d}x
=\frac{\sqrt{2\pi}}{2\sigma}e^{\large-\frac{\mu^2}{2\sigma^2}}\mathrm{erfi}\left(\frac{\mu}{\sigma\sqrt2}\right)\tag{7}
$$
Extended precision is not enough
It is mentioned in a comment that $x<0$ was rejected. This poses a theoretical problem. The computations above are carried out in principal value, that means that a small interval $[-\delta,\delta]$ is rejected, where $\delta\to0$. However, if $x\lt\delta$ is rejected, then, as $\delta\to0$, the the contribution to expected value from the singularity grows like
$$
-\frac{\log(\delta)}{\sqrt{2\pi}}e^{-50}\tag{8}
$$
Even using extended precision, where $\delta=2^{-16382}$, $(8)$ amounts to about $8.74\times10^{-19}$ which is pretty insignificant. However, as $\delta\to0$, $(8)\to\infty$.
Therefore, even extended precision arithmetic is insufficient to expose the problems with a simulation where $x\lt0$ is rejected.