# Convergence in probability and asymptotic distribution of MLE for Uniform $(-\theta, \theta)$

Problem.

I am having difficulties with asymptotic results of the maximum likelihood estimator (MLE) of the parameter $$\theta$$ for the Uniform $$(-\theta, \theta)$$ distribution, given an IID sample $$X_1, X_2,..., X_n$$. Namely showing explicitly that:

$$\widehat{\theta}_n \overset{p}{\rightarrow} \theta$$

and finding the limiting distribution

$$n(\theta - \widehat{\theta}_n)$$

Where $$\widehat{\theta}_n$$ denotes the maximum-likelihood estimator.

I am aware that if certain regularity conditions are satisfied, the MLE is a consistent estimator of a parameter of a distribution; so this question concerns showing it for a particular form of the Uniform distribution. I would have liked this question to be more concise, but it is not so because the issue lies in the fact that I am making reasoning errors which I am unable to discern.

My attempt.

Convergence in probability

I first computed the likelihood function $$L(\theta)$$, and found that the MLE of $$\theta$$ can be expressed as $$\widehat{\theta}_n = \max \{ -X_{(1)}, X_{(n)} \}$$, where $$X_{(1)}$$ and $$X_{(n)}$$ are the 1st and nth order statistics. Now in order to show consistency, I began by considering the following, with a view to showing that it converges to 0 as $$n \rightarrow \infty$$:

$$P(|\widehat{\theta}_n - \theta | \geq \epsilon) = \underbrace{P(\widehat{\theta}_n \geq \theta + \epsilon)}_{=0} + P(\widehat{\theta}_n \leq \theta - \epsilon) = P(\widehat{\theta}_n \leq \theta - \epsilon)$$

I then proceeded by evaluating the RHS and this is where things start to become uncertain for me:

\begin{align} P(\widehat{\theta}_n \leq \theta - \epsilon) &= P(\max \{ -X_{(1)}, X_{(n)} \} \leq \theta - \epsilon) \\ &= P \left( \left\{ -X_{(1)} \leq \theta - \epsilon \right \} \cap \left \{ X_{(n)} \leq \theta - \epsilon \right \} \right) \\ &= P(-X_{(1)} \leq \theta - \epsilon) P(X_{(n)} \leq \theta - \epsilon) \end{align}

1. In going from the 1st to the 2nd equality, I reasoned that in order for the maximum of $$-X_{(1)}$$ and $$X_{(n)}$$ to be less than $$\theta - \epsilon$$, both $$-X_{(1)}$$ and $$X_{(n)}$$ must be less that $$\theta - \epsilon$$. However, as it is a maximum of order statistics, the nesting is confusing me, and I'm starting to experience doubt as to whether this is a valid justification.

2. In going from the 2nd to the 3rd equality, I am fairly certain that is appropriate as the $$X_i$$ are independent, and hence their order statistics are independent.

I computed the CDF of the Uniform $$(-\theta, \theta)$$ to get:

$$F_{X_i}(x_i) = \begin{cases} 0 \quad &x_i \leq -\theta \\ \frac{x_i + \theta}{2\theta} \quad &-\theta \leq x_i \leq \theta \\ 1 \quad &x_i \geq \theta \\ \end{cases}$$

Evaluating the probability on the nth order statistic:

$$P(X_{(n)} \leq \theta - \epsilon) = P \left(\bigcap^n_{i=1} X_i \leq \theta - \epsilon \right) = \prod^n_{i=1} P (X_i \leq \theta - \epsilon) = \left(1 - \frac{\epsilon}{2 \theta} \right)^n$$

Evaluating the probability on the 1st order statistic:

$$P(-X_{(1)} \leq \theta - \epsilon) = P(X_{(1)} \geq - \theta + \epsilon) = 1 - P \left(X_{(1)} \leq -\theta + \epsilon \right) = 1 - P \left( \bigcap^n_{i=1} X_i \leq - \theta + \epsilon \right) = \left( 1 - \frac{\epsilon}{2\theta} \right)^n$$

Setting aside my reservation about how the above two calculations are combined , the above two individual probability calculations seem to accord with my intuitions, in that they are the same due to symmetry arguments. Which yields:

$$P(|\widehat{\theta}_n - \theta | \geq \epsilon) = \left( 1 - \frac{\epsilon}{2\theta} \right)^{2n} = \left(\frac{2 \theta - \epsilon}{2\theta} \right)^{2n}$$

Which converges to 0 as $$n \rightarrow \infty$$ for all $$\epsilon > 0$$.

Convergence in distribution

From a previous example in my notes, I know that for IID $$X_1, ... ,X_n \sim \text{Uniform}(0,1)$$, the limiting distribution of the nth order statistic is an $$\text{Exponential}(1)$$ distribution; that is:

$$X_{(n)} \overset{d}{\rightarrow} \text{Exponential}(1)$$

On this basis, I guessed that the solution would have a similar flavour.

Invoking the arguments I made previously, I found that:

\begin{align} P(n(\theta - \widehat{\theta}_n) \leq t) &= 1 - P \left(\widehat{\theta}_n \leq \theta - \frac{t}{n} \right) \\ &= 1 - P\left( \left\{-X_{(1)} \leq \theta - \frac{t}{n} \right \} \cap \left \{ X_{(n)} \leq \theta -\frac{t}{n} \right \} \right) \\ &= 1 - \left( 1 - \frac{t}{2n \theta} \right)^{2n} \\ \end{align}

Now I am left with something that looks very similar to an exponential CDF, and as a guess I suspect it might be an $$\text{Exponential}(1 / \theta)$$; and that the appearance of the factor of 2 is erroneous (due to an issue in the previous arguments which I am unable to discern).

Another attempt.

After reading through some previous posts on here, I decided to try reformulating the MLE estimator to something equivalent, that is by setting $$\widehat{\theta}_n = \max\{|X_i|\} \space \forall \space i = 1, ... , n$$. I found that this skirts around the issue of not being sure about point 1. Focusing on the convergence in distribution argument (as the convergence in probability argument is a stepping stone) I found that:

\begin{align} P(n(\theta - \widehat{\theta}_n) \leq t) = 1 - P \left(\widehat{\theta}_n \leq \theta - \frac{t}{n} \right) &= 1 - P \left( \max \{ |X_i | \} \leq \theta - \frac{t}{n} \space \forall \space i = 1, ... , n \right) \\ &= 1 - P \left( \bigcap^n_{i=1} |X_i| \leq \theta - \frac{t}{n} \right) \\ &= 1 - \prod^n_{i=1} \left( P(X_i \leq \theta - \frac{t}{n}) + P(-X_i \leq \theta - \frac{t}{n} ) \right) \\ &= 1 - \prod^n_{i=1} \left( P(X_i \leq \theta - \frac{t}{n}) + P(X_i \geq -\theta + \frac{t}{n}) \right) \\ &= 1 - \prod^n_{i=1} F_{X_i}\left( \theta - \frac{t}{n} \right) \left[1 - F_{X_i}\left(-\theta + \frac{t}{n} \right) \right] \\ &= 1 - \prod^n_{i=1} \left( \frac{2 \theta - t/n}{2 \theta} + 1 - \frac{t/n}{2 \theta} \right) \\ &= 1 - \left( 2 - \frac{t}{2n \theta} \right)^n \end{align}

So I know that as I am not getting the same results, there are errors of reasoning being made. However, I am having trouble discerning where these lie.

I would appreciate if members of this community were to assist me.

• Since $|X_i|\sim U(0,\theta)$, see math.stackexchange.com/q/2484567/321264, math.stackexchange.com/q/2950994/321264. Nov 10, 2020 at 19:07
• @StubbornAtom Using the observation you made, and the latter post you linked, I managed to get find that the limiting distribution of the MLE $\widehat{\theta}_n$ is $\text{Exponential}(1 / \theta)$. Is that correct? If so, I am unable to see where it is that I am erring in the calculations I have above. Nov 10, 2020 at 20:53
• Just commenting on your second statement: "In going from the 2nd to the 3rd equality, I am fairly certain that is appropriate as the $X_i$ are independent, and hence their order statistics are independent." If I tell you that the $n$th order statistic is 0, does that give you information about the other order statistics? Nov 11, 2020 at 0:49
• @Yacob Kureh. Yes, if you were to tell me that the $n$th order statistic is 0, then that provides information about the $(n-1)$ other order statistics. On the basis of the data being generated according to a $U(-\theta, \theta)$ distribution, then I can infer that the $(n-1)$ other order statistics, i.e. the rest of the data must be less than 0. Meaning that I forgot that order statistics, and statistics more broadly, are functions of the data. And also that the independence assumption I've made is faulty. Thank you for such an excellent pedagogical prompt. Will now review what I've written. Nov 11, 2020 at 1:41

In case anyone else struggles with these questions, the following is a verbose solution that I transcribed from my write-up, following helpful suggestions from StubbornAtom and Math Helper in the comments.

Showing $$\hat{\theta}_n \overset{p}{\rightarrow} \theta$$.

In order to explicitly show that the maximum likelihood estimator $$\hat{\theta}_n = {\max}_i \{ \lvert X_i \rvert \}$$ converges in probability to the 'true' parameter $$\theta$$ of the $$\text{Uniform}(-\theta, \theta)$$ distribution, i.e. that $$\hat{\theta}_n$$ is a consistent estimator, we opt for directly showing that as $$n \rightarrow \infty$$, then

$$P(\lvert \hat{\theta}_n - \theta \rvert > \epsilon) \longrightarrow 0$$

for all $$\epsilon > 0$$.

Notice that we can simplify the left hand side of the above to get

\begin{align*} P(\lvert \hat{\theta}_n - \theta \rvert > \epsilon) &= P \left(\{ \hat{\theta}_n - \theta > \epsilon\} \cup \{ -(\hat{\theta}_n - \theta) > \epsilon\}\right) \\ &= P(\hat{\theta}_n > \theta + \epsilon) + P(\hat{\theta}_n < \theta - \epsilon) \\ &= P(\hat{\theta}_n < \theta - \epsilon) \end{align*}

because

$$X_i \sim \text{Uniform}(-\theta, \theta) \implies \underset{i}{\max} \{ \lvert X_i \rvert \} \leq \theta \implies P(\hat{\theta}_n > \theta + \epsilon) = 0$$

To simplify this further, notice that

\begin{align*} P(\lvert \hat{\theta}_n - \theta \rvert > \epsilon) &= P(\hat{\theta}_n < \theta - \epsilon) \\ &= P(\underset{i}{\max} \{ \lvert X_i \rvert \} < \theta - \epsilon) \\ &= P \left( \bigcap^n_{i=1} \{ \lvert X_i \rvert < \theta - \epsilon \}\right) \\ &= \prod^n_{i=1} P(\lvert X_i \rvert < \theta - \epsilon) \\ \end{align*}

Using the symmetry of the $$\text{Uniform}(-\theta, \theta)$$ about $$0$$, the probability mass contained within the interval $$[-(\theta - \epsilon), \theta - \epsilon]$$ consists of a rectangle of height $$1 / 2\theta$$ and length $$2(\theta - \epsilon)$$, meaning that

\begin{align*} P(\lvert \hat{\theta}_n - \theta \rvert > \epsilon) &= \prod^n_{i=1} P(\lvert X_i \rvert < \theta - \epsilon) \\ &= \left(\frac{\theta - \epsilon}{\theta} \right)^n \longrightarrow 0 \\ \end{align*}

as $$n \rightarrow \infty$$ for all $$0 < \epsilon < \theta$$, which is a consequence of the fact that $$\frac{\theta - \epsilon}{\theta} < 1$$

In the case that $$\epsilon \geq \theta$$, then $$P(\lvert X_i \rvert < \theta - \epsilon) = 0$$ because the absolute value function is, by definition, non-negative. Hence as $$n \rightarrow \infty$$, we have that $$P(\lvert \hat{\theta}_n - \theta \rvert > \epsilon) \longrightarrow 0$$ for all $$\epsilon > 0$$, and hence $$\hat{\theta}_n \overset{p}{\rightarrow} \theta$$.

Limiting distribution of $$n(\theta - \hat{\theta}_n)$$.

To find the limiting distribution of $$n(\theta - \hat{\theta}_n)$$, we begin by considering

\begin{align*} P(n(\theta - \hat{\theta}_n) \leq t) &= P \left( \hat{\theta}_n \geq \theta - \frac{t}{n} \right) \\ &= P \left( \underset{i}{\max} \{ \lvert X_i \rvert \} \geq \theta - \frac{t}{n} \right) \\ &= 1 - P \left( \underset{i}{\max} \{ \lvert X_i \rvert \} \leq \theta - \frac{t}{n} \right) \\ &= 1 - P \left( \bigcap^n_{i=1} \left\{ \lvert X_i \rvert \leq \theta - \frac{t}{n} \right\} \right) \\ &= 1 - \prod^n_{i=1} P \left( \lvert X_i \rvert \leq \theta - \frac{t}{n} \right) \\ \end{align*}

In order to evaluate the term within the product operator, we compute the CDF of a transformed $$Y = r(X) = \lvert X \rvert$$ where $$X \sim \text{Uniform}(-\theta, \theta)$$. To find the CDF, we need to find the set $$A_y = \{x : \lvert x \rvert \leq y \}$$ for all $$y$$, which will yield

\begin{align*} F_Y(y) &= P(Y \leq y ) \\ &= P( \{x : \lvert x \rvert \leq y \}) \\ &= \int_{A_y} f_X(x) dx \end{align*}

Noting that the transformed $$y \in [0, \theta]$$, the set $$A_y$$ constitutes a line segment of length $$y$$, so that $$A_y = y$$. Meaning that

\begin{align*} \int_{A_y} f_X(x) dx = \int_{A_y} \frac{1}{\theta} dx = \int^y_{0} \frac{1}{\theta} dx = \frac{y}{\theta} \end{align*}

The CDF of the transformed $$Y = \lvert X \rvert$$ is therefore

$$F_Y(y) = \begin{cases} 0 \quad & y < 0\\ \frac{y}{\theta} \quad & 0 \leq y \leq \theta \\ 1 \quad &y > \theta \\ \end{cases}$$

which is the CDF for a $$Y \sim \text{Uniform}(0, \theta)$$ distribution. Returning to our previous expression, we have that

\begin{align*} P(n(\theta - \hat{\theta}_n) \leq t) &= 1 - \prod^n_{i=1} P \left( \lvert X_i \rvert \leq \theta - \frac{t}{n} \right) \\ &= 1 - \prod^n_{i=1} F_Y \left(\frac{\theta - t/n}{\theta} \right) \\ &= 1 - \left( 1 - \frac{t}{n \theta} \right)^n \\ \end{align*}

Using the power series representation of the exponential function together with the Binomial theorem, the exponential function has the following limit representation:

$$\exp(t) = \lim_{n \rightarrow \infty} \left(1 + \frac{t}{n} \right)^n$$

Consequently, we have that

$$\exp \left( -\frac{1}{\theta}t \right) = \lim_{n \rightarrow \infty} \left(1 - \frac{t}{n \theta} \right)^n$$

Considering the limit as $$n \rightarrow \infty$$ in the previous expression, we have that

$$\lim_{n \rightarrow \infty} P(n(\theta - \hat{\theta}_n) \leq t) = 1 - \lim_{n \rightarrow \infty} \left(1 - \frac{t}{n \theta} \right)^n = 1 - \exp \left(-\frac{1}{\theta}t \right)$$

Hence we have that

$$n(\theta - \hat{\theta}_n) \overset{d}{\longrightarrow} \text{Exponential}\left( \frac{1}{\theta} \right)$$

i.e. the limiting distribution of $$n(\theta - \hat{\theta}_n)$$ is an exponential distribution with parameter $$1 / \theta$$.