# Maximum Likelihood Estimator for Non Absolutely Continuous Distributions

Let $$\theta\in[0,1]$$ be a parameter. Suppose that $$Y$$ is a random variable that takes value $$\theta$$ with probability $$1/2$$ and is uniformly distributed on $$[0,1]$$ with probability $$1/2$$. What is the maximum likelihood estimator of $$\theta$$ given independent observations $$y_1,...,y_n$$?

Stepping back from formalism, it seems very clear how to guess $$\theta$$ given a list of observations of $$Y$$. However, I don't see how a likelihood function can be defined for this family of distributions.

• What is the meaning of '$Y$ is uniformly distributed on $[0,1]$ with probability $1/2$'? How is $\theta$ involved here? – StubbornAtom Apr 18 at 8:41
• @StubbornAtom If you want a more technical description, take $X$ to be Bernoulli with $p=1/2$. Then define $Y_\theta = \theta X + (1 - X)U$ where $U$ is independent and uniformly distributed on $[0,1]$. – user3281410 Apr 18 at 18:13
• Thank you...... – StubbornAtom Apr 18 at 20:03

Let's start by stating the intuitive answer:

Clearly the observation value which appears most often is the maximum likelihood estimate for $$\theta$$. There will be such a value with probability $$1-\frac{n+1}{2^n}$$ if $$n \gt 1$$ and with probability $$1$$ if $$n=1$$; otherwise, each observation value will appears only once with probability $$\frac{n+1}{2^n}$$ if $$n \gt 1$$ and they are each equally likely to be $$\theta$$ even though there is a positive probability that none of them are

Now let's try to express this in likelihood terms, noting that the probability a single observation is $$\theta$$ is $$\frac12$$ and the density for any observation is not $$\theta$$ is $$\frac12$$, though there is a fundamental distinction between probabilities and densities:

In a sense the likelihood is proportional to the product of the $$n$$ probability densities and probabilities associated with the observations. But providing that none of these are zero, any non-zero probability should conceptually lead to a higher likelihood than a probability density, e.g. combining $$k$$ probabilities and $$n-k$$ probability densities has an intrinsically higher likelihood than $$k-1$$ probabilities and $$n-k+1$$ probability densities. You cannot quite say that a probability corresponds to an infinite density or that a finite density corresponds to a zero probability, since you may have several of each, but this does provide a weak analogy to how probabilities dominate densities

So if you have a value $$x_m$$ which appears $$k \gt 1$$ times together $$n-k$$ other distinct observations, if $$\theta = x_m$$ then the likelihood is proportional to $$\frac1{2^n}$$ with $$k$$ probabilities and $$n-k$$ densities while if $$\theta \not = x_m$$ then the likelihood is proportional to $$\frac1{2^n}$$ with $$1$$ or $$0$$ probabilities and $$n-1$$ or $$n$$ densities. The $$\theta = x_m$$ case then has a higher intrinsic likelihood than any $$\theta \not = x_m$$, making $$x_m$$ the maximum likelihood estimator

On the other hand if you have $$n$$ distinct observations $$\{x_i\}$$ then if any one of them is $$\theta$$ then the likelihood is proportional to $$\frac1{2^n}$$ with $$1$$ probability and $$n-1$$ densities while if $$\theta \not \in \{x_i\}$$ then the likelihood is proportional to $$\frac1{2^n}$$ with $$0$$ probabilities and $$n$$ densities. The former cases then have a higher intrinsic likelihood than the latter cases making each of the $$\{x_i\}$$ maximum likelihood estimators

• I appreciate the response, but I am mostly interested in rigorous definitions that allow the MLE to be defined in terms of a solution to an optimization problem. We can't define a likelihood function because there is no dominating measure for this family of distributions. I believe you are saying that the MLE is the function $f(y_1,...,y_n)$ that selects whatever coordinate is equal to others the most times and returns an arbitrary coordinate if there are no repetitions. However, I think it would be hard to modify the same reasoning if the problem was more complicated. – user3281410 Apr 18 at 18:26