How is it that the required sample size for a specified error and confidence is not dependent on population size?

When calculating confidence intervals for population parameters, the population size is never a factor, rather sample size and the estimated parameter are used.

It seems to me very counter-intuitive that to assert with a certain confidence that a certain view has certain probability, one requires the same sample size regardless of whether the population is 2K persons or 2M persons.

Why is the confidence of estimated parameters of normal distributions independent of the population size?

-

I might be wrong, but I think sample size is a function of population, but the formula is typically given in the limit as $pop\rightarrow\infty$.