The problem:

The title sums up the problem pretty well, we have a sample from an independent idential (uniform) distribution (i.i.d.):

$X_1, ..., X_n \thicksim Unif[0,1]$,

and we only know one exact value from this: someone told us that "one of the values is equal to $x$":

$X_k^* = x$ for an unknown $k \in \{1,\dots,n\}$, where $X_1^*\le ...\le X_n^*$ is the ordered $X_1, ..., X_n$ sample.

Give a Maximum Likelihood estimation for $k$, using the value of $x$.

Here is what I've done so far:

Let's order the sample into $X_1^*, \dots, X_n^*$. Now the density function of $X_k^*$ follows a Beta-distribution of $X_k^* \thicksim Beta(k,n-k+1)$:

$$\lim_{\varepsilon \rightarrow 0}\Bbb{P}(x - \varepsilon < X_k^* < x + \varepsilon) = f_{X_k^*}(x) = \\ = \frac{\Gamma(n+1)}{\Gamma(k)\Gamma(n-k+1)}x^{k-1}(1-x)^{n-k} = \\ = \frac{n!}{(k-1)!(n-k)!}x^{k-1}(1-x)^{n-k} = \\ = k\binom{n}{k}x^{k-1}(1-x)^{n-k}$$

When given a sample of one $X_k^*$, the Likelihood function is the same as the density function:

$$L_{n,k}(x) = k\binom{n}{k}x^{k-1}(1-x)^{n-k}$$

I set up the log-Likelihood equation:

$$\ln L_{n,k}(x) = \\ = \ln(k) + \ln\left(\binom{n}{k}\right) + (k-1)\ln(x) + (n-k)\ln(1-x)$$

And differentiate it in terms of $k$, because I want to find its maximum in $k$:

$$\frac{d}{dk}\ln L_{n,k}(x) = \\ = \frac{1}{k} + \frac{d}{dk}\ln\left(\binom{n}{k}\right) + \ln(x) - \ln(1-x)$$

I used Wolfram Alpha to calculate $\frac{d}{dk}\ln\left(\binom{n}{k}\right)$, and it turns out to be:

$$\frac{d}{dk}\ln\left(\binom{n}{k}\right) = H_{n-k} - H_k$$

where $H_n$ is the $n$th harmonic number:

$$ H_n = \sum_{i=1}^n \frac{1}{i}$$

So going back to the original equation:

$$\frac{d}{dk}\ln L_{n,k}(x) = \\ = \frac{1}{k} + H_{n-k} - H_k + \ln(x) - \ln(1-x) = \\ = H_{n-k} - H_{k-1} + \ln(x) - \ln(1-x) \stackrel{?}{=} 0$$

Now this is where I'm stuck. I have no clue how to order this equation to $k$ on one side, everything else on the other side; because that is how I would get a maximal value for $k$.

An obvious answer:

The solution would be something like:

$$k = nx$$

Since if we have a sample of size $n$, and $x \in [0,1]$, we'd expect the place of $x$ to fall close to its size (factored up by $n$).

Or to be more specific, $k$ could be:

$$k = (n-1)x+1$$

Since this formula runs from $1$ to $n$, not $0$ to $n$.

But I cannot figure out how we arrive at this. Any help would be appreciated!


You want to find integer value of $k$ that maximizes $L_{n,k}(x)$. To do it, it is sufficient to consider $$\frac{L_{n,k+1}(x)}{L_{n,k}(x)}=\frac{x}{1-x}\cdot\frac{n-k}{k}$$ and compare it with $1$.

We get: $L_{n,k+1}(x)\geq L_{n,k}(x)$ for $k \leq nx$ and $L_{n,k+1}(x)\leq L_{n,k}(x)$ for $k \geq nx$.

So, for $nx\in\mathbb Z$, we have two values of $k$ where $L_{n,k}(x)$ attaines maximum: $k=nx$ and $k=nx+1$.

For $nx\not\in\mathbb Z$, $k=[nx]+1$ is MLE.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.