# Maximum Likelihood Estimate for an Unknown Distribution

Suppose that $X_1,\dots,X_n$ are i.i.d. random variables having a CDF (cumulative distribution function) $F$. For each fixed $x$, I am asked to determine the maximum likelihood estimate of $F(x)$.

I am having difficulty understanding what the question is asking. For one, I do not see how to recover a probability density (or mass) function from $F$, let alone how to maximize the likelihood function without a given parameter space.

If anyone could shed some light on this it would be greatly appreciated.

Edit

I now understand that I should take the MLE will be the empirical distribution. However, I am still having difficulty proving this directly from the definition.

Here the CDF is the thing you are estimating. You can think of its values as an infinite number of parameters (in a constrained space that says they need to comprise a right-continuous, nondecreasing function, between zero and yada yada yada).

Let's say we get $X_1=3$ and $X_2=4.$ We need to find the CDF that maximizes the probability of this data. It's pretty clear that anything other than an atom at $3$ and an atom at $4$ is a waste of real estate. Let $p$ be the mass at $3$ and $(1-p)$ the mass at $4$. Then we want to maximize $p(1-p),$ so we get $p=1/2$ (What else could it have been?)

This generalizes to putting $1/n$ mass at each of the points $X_1,\ldots, X_n.$

• How does this relate to $F(x)$? – Quoka Mar 17 '18 at 1:30
• @MathUser_NotPrime The estimate of the CDF will be flat almost everywhere, but have a jump of size $1/n$ at $X_1,\ldots X_n.$ (Or if two of the $X_i$ are the same the jump there will be of size $2/n,$ etc.) I was talking in terms of the distribution more generally, but any precise description of a distribution maps to a CDF. – spaceisdarkgreen Mar 17 '18 at 1:34
• I must be missing something. I thought I was supposed to estimate the CDF at the point $x$ and not on the whole line? How does F(x) relate to a Bernoulli? – rolandcyp Mar 17 '18 at 1:44
• @rolandcyp If $X_1=3$ and $X_2=4$ then the estimate of the whole CDF is that of a distribution where $P(X=4)=P(X=3)=1/2.$ So the estimate of $F(x)$ takes the value $0$ for $x\in (-\infty,3),$ $1/2$ for $x\in [3,4)$ and $1$ for $x\in [4,\infty).$ In other words, the MLE for $F(x)$ is the value of the empirical CDF en.wikipedia.org/wiki/Empirical_distribution_function – spaceisdarkgreen Mar 17 '18 at 1:50

$F(x) = P(X \leq x) = p_x$. The problem asks you to find a MLE for $p_x$ regardless of what $x$ is.

You know how to find MLEs for the $p$ of a Bernoulli distribution, correct? Can you find a sequence of Bernoulli random variables, $I_1, \ldots, I_n$, from $X_1, \ldots, X_n$? Sure you can, use an indicator function:

$$I_j = \begin{cases} 1 & \text{ if } X_j \leq x \\ 0 & \text{ otherwise}\end{cases}$$

So then what is $P(I_j = 1)$? How does it relate to $F(x)$? Use $I_1, \ldots, I_n$ to get the MLE of $p_x$.

• I see that $P(I_j = 1)$ is precisely $F(x)$. However, I don't see how to use the $I_1,\dots,I_n$. Could you elaborate on this? – rolandcyp Mar 17 '18 at 1:03
• $I_1, \ldots, I_n$ are iid and follow a Bernoulli distribution with parameter $p_x$. Find the MLE for $p_x$. How does it relate to $F(x)$? – cgmil Mar 17 '18 at 7:12