# Maximum Likelihood Estimate for an Unknown Distribution

Suppose that $X_1,\dots,X_n$ are i.i.d. random variables having a CDF (cumulative distribution function) $F$. For each fixed $x$, I am asked to determine the maximum likelihood estimate of $F(x)$.

I am having difficulty understanding what the question is asking. For one, I do not see how to recover a probability density (or mass) function from $F$, let alone how to maximize the likelihood function without a given parameter space.

If anyone could shed some light on this it would be greatly appreciated.

Edit

I now understand that I should take the MLE will be the empirical distribution. However, I am still having difficulty proving this directly from the definition.

## 2 Answers

Here the CDF is the thing you are estimating. You can think of its values as an infinite number of parameters (in a constrained space that says they need to comprise a right-continuous, nondecreasing function, between zero and yada yada yada).

Let's say we get $X_1=3$ and $X_2=4.$ We need to find the CDF that maximizes the probability of this data. It's pretty clear that anything other than an atom at $3$ and an atom at $4$ is a waste of real estate. Let $p$ be the mass at $3$ and $(1-p)$ the mass at $4$. Then we want to maximize $p(1-p),$ so we get $p=1/2$ (What else could it have been?)

This generalizes to putting $1/n$ mass at each of the points $X_1,\ldots, X_n.$

• How does this relate to $F(x)$? – Quoka Mar 17 '18 at 1:30
• @MathUser_NotPrime The estimate of the CDF will be flat almost everywhere, but have a jump of size $1/n$ at $X_1,\ldots X_n.$ (Or if two of the $X_i$ are the same the jump there will be of size $2/n,$ etc.) I was talking in terms of the distribution more generally, but any precise description of a distribution maps to a CDF. – spaceisdarkgreen Mar 17 '18 at 1:34
• I must be missing something. I thought I was supposed to estimate the CDF at the point $x$ and not on the whole line? How does F(x) relate to a Bernoulli? – rolandcyp Mar 17 '18 at 1:44
• @rolandcyp If $X_1=3$ and $X_2=4$ then the estimate of the whole CDF is that of a distribution where $P(X=4)=P(X=3)=1/2.$ So the estimate of $F(x)$ takes the value $0$ for $x\in (-\infty,3),$ $1/2$ for $x\in [3,4)$ and $1$ for $x\in [4,\infty).$ In other words, the MLE for $F(x)$ is the value of the empirical CDF en.wikipedia.org/wiki/Empirical_distribution_function – spaceisdarkgreen Mar 17 '18 at 1:50

$F(x) = P(X \leq x) = p_x$. The problem asks you to find a MLE for $p_x$ regardless of what $x$ is.

You know how to find MLEs for the $p$ of a Bernoulli distribution, correct? Can you find a sequence of Bernoulli random variables, $I_1, \ldots, I_n$, from $X_1, \ldots, X_n$? Sure you can, use an indicator function:

$$I_j = \begin{cases} 1 & \text{ if } X_j \leq x \\ 0 & \text{ otherwise}\end{cases}$$

So then what is $P(I_j = 1)$? How does it relate to $F(x)$? Use $I_1, \ldots, I_n$ to get the MLE of $p_x$.

• I see that $P(I_j = 1)$ is precisely $F(x)$. However, I don't see how to use the $I_1,\dots,I_n$. Could you elaborate on this? – rolandcyp Mar 17 '18 at 1:03
• $I_1, \ldots, I_n$ are iid and follow a Bernoulli distribution with parameter $p_x$. Find the MLE for $p_x$. How does it relate to $F(x)$? – cgmil Mar 17 '18 at 7:12