Likelihood function for continuous densities When doing ML-estimates for discrete distributions, the definition of likelihood makes perfect sense
$ \ L(x,\theta) = \Pi_{1:n}\ P(X_i=x_i|\theta)$
Since there is a non-infinitesimal probability that $X=x$. But why does this work for continuous distributions? Isn't $P(X=x)$ always infinitesimal? When doing excercises, everything works out fine. But I don't understand how. Why doesn't the likelihood function require intervals for $P(X)$?
(Found a related thread Why is likelihood not always 0 in continuous case? but it's more about explaining why $L(x,\theta)$ can be greater than one. I still don't get how $P(X=x) \neq 0$ for continuous distributions.)
 A: You have come upon on a little-discussed aspect of MLE in continuous probability spaces: the typical formulas for MLE for continuous formulas are actually not strictly likelihoods, since, as you have pointed out, they are ratios of densities, not probabilities. A great discussion of just this point (and several other commonly accpted, but not exactly accurate notions) is given by JK Lindsey, a well-known and published statistician specializing in likelihood inference. See pages 9- 11 for a direct discussion of your problem.
What is going on here? Basically, when using continuous models and likelihood as density ratios, there is an implied assumption of infinite precision:
The true likelihood is $L(\theta, \mathbf{x}) = \prod\limits_{x_i}\int\limits_{x_i-\Delta}^{x_i+\Delta} f(x_i;\theta)dx_i$, with $\Delta$ being the "precision" of our measurement of the $x_i$. This is actually what should be done, but either the precision is not known, or assumed to be 0. Either way, we run into problems with the direct application of likelihood principles, so we resort to numerical approximations based on limiting assumptions:
If we let $\Delta \rightarrow 0$ then, as you pointed out, the likelihood will be 0 for all values of $\theta$, to get around this, we examine an analytic approximation to the likelihood:
$\lim\limits_{\Delta \rightarrow 0} \prod\limits_{x_i}\int\limits_{x_i-\Delta}^{x_i+\Delta} f(x_i;\theta) dx_i = \prod\limits_{x_i} f(x_i;\theta) dx_i$ Which is just the continuous approximation you noted above. 
Again, this is truly a numerical approximation. There will always be actual limits to precision. Also, even if there were not, the likelihood concept would not apply, so the density definition would be an extension of the classical notion of likelihood.
