Yau gives some quick explanation of this in his scholaropedia page for CY manifolds (which is really a great read if you're interested in such things). In particular, speaking about spacetime manifolds $\mathbb{R}^{3,1}\times X$ (with standard Minkowski metric on the first part), he says "for the most basic product models $N=1$ supersymmetry, the space $X$ must be a Calabi-Yau manifold of complex dimension $3$".