Finding the maximum likelihood estimator

Find the MLE of the unknown parameter $\theta$ when $X_1,X_2,...,X_n$ is a sample from the distribution whose density function is: $$f_X(x) = \frac12e^{-|x-\theta|}, -\infty<x<\infty$$ What I did so far: $$\text{maximize }(\frac12)^ne^{-|x_1-\theta|-|x_2-\theta|-...-|x_n-\theta|} \\ \text{thus we want to minimize }|x_1-\theta|+|x_2-\theta|+...+|x_n-\theta|$$ Since $|x_i-\theta|$ is positive for all $i$, we want to solve for $\theta$ so: $$|x_1-\theta|+|x_2-\theta|+...+|x_n-\theta|=0$$ What do I do from here?

• What is name of this distribution ? Nov 18, 2019 at 20:10
• @Kutsit Laplace distribution Aug 16, 2021 at 21:27

$$f_X(x) = \frac12e^{-|x-\theta|}, -\infty

is a special case of the Laplace distribution given as follows:

$$f_X(x|\mu,\sigma)=\frac{1}{\sqrt{2}\sigma}e^{-\frac{\sqrt{2}|x-\mu|}{\sigma}},x\in\mathbb{R}$$

for $$\sigma=\sqrt{2}$$ and $$\mu:=\theta$$. To be more general, lets consider the Laplace distribution with parameters $$(\mu,\sigma)$$.

Consider the likelihood function for $$N$$ data samples:

$$L(\mu,\sigma;x)=\prod_{t=1}^N \frac{1}{\sqrt{2}\sigma}e^{-\frac{\sqrt{2}|x_t-\mu|}{\sigma}}=(\sqrt{2}\sigma)^{-N}e^{\frac{-\sqrt{2}}{\sigma}\sum_{t=1}^N |x_t-\mu|}$$

Take the log likelihood funtion as $$l(\mu,\sigma;x)=\log(L(\mu,\sigma;x))$$ and we get $$l(\mu,\sigma;x)=-N\ln (\sqrt{2}\sigma)-\frac{\sqrt{2}}{\sigma}\sum_{t=1}^N |x_t-\mu|$$ Take the derivative with respect to the parameter $$\mu$$ $$\frac{\partial l}{\partial \mu}=-\frac{\sqrt{2}}{\sigma}\sum_{t=1}^N \frac{\partial|x_t-\mu|}{\partial\mu}$$ which is equal to $$=\frac{\sqrt{2}}{\sigma}\sum_{t=1}^N\mbox{sgn}(x_t-\mu)$$ using the identity $$\frac{\partial |x|}{\partial x}=\frac{\partial \sqrt{x^2}}{\partial x}=x(x^2)^{-1/2}=\frac{x}{|x|}=\mbox{sgn(x)}$$ To maximize the likelihood function we need to solve
$$=\frac{\sqrt{2}}{\sigma}\sum_{t=1}^N\mbox{sgn}(x_t-\mu)=0 \quad\quad (1)$$ For which we have two cases; $$N$$ is even or odd.

If $$N$$ is odd and we choose $$\hat{\mu}=\mbox{median}(x_1,\ldots ,x_N)$$, then there are $$\frac{N-1}{2}$$ cases where $$x_t<\mu$$ and for the other $$\frac{N-1}{2}$$ cases $$x_t>\mu$$, therefore $$\hat{\mu}$$ satisfies ($$1$$) and is the Maximum likelihood estimator for the parameter $$\mu$$

If $$N$$ is even, we can not simply choose one $$x_t$$ which will satisfy ($$1$$), however we can still minimize it through ranking the observations as $$x_1\leq x_2\leq \ldots,x_N$$ and then choosing either $$x_{N/2}$$ or $$x_{(N+1)/2}$$

In summary $$\hat{\mu}=\mbox{median}(x_1,\ldots ,x_N)$$ is the maximum likelihood estimator for any $$N$$

• You are minimizing a nondifferentiable function by equating the derivative to $0$. for the odd case the function is not even differentiable at its minimizer. You should better elaborate more. Sep 27, 2020 at 16:34
• You mean non-differentiable functions at $0$s right? Because the function is differentiable at every other point. "The real absolute value function is an example of a continuous function that achieves a global minimum where the derivative does not exist." From Wiki. So I can in fact take the derivative and make it equal to zero although the function is no differentiable at $0$. Sep 28, 2020 at 19:45
• No, you can't differentiate a function at a point where it is not differentiable. Apr 27, 2022 at 18:29
• @Ramanujan I didn’t say I can differentiate. Apr 27, 2022 at 18:43
• But you did. The formula for the derivative with respect to $\mu$ is only correct if $x_k \ne \mu$ for all $k \in \{ 1, \ldots, N \}$. Apr 27, 2022 at 20:12

Another approach is using absolute value inequality.

Since $$|a| + |b| \ge |a-b|$$, and the equal sign holds only when $$a \cdot b \leq 0$$,

Let $$x_i$$ has an order from smallest $$x_i$$ to largest $$x_n$$. If $$n$$ is an odd number, denote $$m=\frac{n+1}{2}$$. \begin{align*} \sum_{i=1}^{n} |x_i - \theta| & \ge |x_n - \theta - (x_1 - \theta)| + |x_{n-1} - \theta - (x_2 - \theta)| \\ & \qquad+ \cdots +|x_{m+1} - \theta - (x_{m-1} - \theta)| + |x_m - \theta| \\ & = \sum_{i=1}^{m-1} |x_{n+1-i} - x_{i}| + |x_m - \theta| \end{align*} The condition that holds the equal sign is $$(x_{n+1-i} - \theta) \cdot (x_i - \theta) \le 0$$ for all $$i = 1, \cdots, m-1$$.

So $$x_{m-1} \le \theta \le x_{m+1}$$, and in order to achieve the minimum of the last term in the inequality we set $$\theta = x_m$$.

The case when $$n$$ is an even number is similar, and $$\theta$$ can be any number between $$x_m$$ and $$x_{m+1}$$ where $$m = \frac{n}{2}$$.

If you look here, the estimator of $\theta$ is the median of $x_1,...,x_n$. This is standard because you are actually minimizing the sum of absolute deviations.

• Without knowing beforehand that the MLE of $\theta$ is the median, how would you go about figuring that out? Nov 19, 2012 at 10:29