Im trying to figure out the ML estimator of $$f_X(x)=\frac{1}{2\beta}\exp\left(-\frac{|x|}{\beta}\right)$$ as well as the variance of this estimator. So far I have $$L(\beta;x)=\prod_{i=1}^n\frac{1}{2\beta}\exp\left(-\frac{|x_i|}{\beta}\right),$$ $$\ln[L(\beta;x)]=\sum_{i=1}^n\left[-\ln(2\beta)-\frac{|x_i|}{\beta}\right] =-n\ln(2\beta)-\frac{1}{\beta}\sum_{i=1}^n|x_i|,$$ $$\frac{\partial \ln[L(\beta;x)]}{\partial\beta}=-n\beta+\frac{1}{\beta^2}\sum_{i=1}^n|x_i|=0.$$ So I get $$\hat\beta=\frac{1}{n}\sum_{i=1}^n|X|$$ and with a hint in my textbook $\text{Var}(|X|)=\beta^2$ $$\text{Var}(\hat\beta)=\frac{1}{n^2}\sum_{i=1}^n \text{Var}(|X|)=\frac{\beta^2}{n}$$ Unfortunately this variance does not correspond with the variance in my distribution summary for an double exponential distribution, which is $2\beta^2$.

I think my mistake arises from the absolute value. I hope someone can figure it out. Thanks!

  • 1
    $\begingroup$ There is a mistake in your derivative, but you still get the correct result after. Your $|X|$ has a so called Laplace distribution, which has variance $2\beta^2$, not $\beta^2$. Finally there is a difference between the variance of your estimator and the variance of the distribution of one data variable. $\endgroup$ – Marc Dec 9 '16 at 10:44
  • $\begingroup$ @Marc: Right, except for a quibble: It is $X,$ not $|X|$ that has the Laplace (double-exponential) distribution. $\endgroup$ – BruceET Jan 10 '17 at 7:32

You have shown that $\hat \beta = \frac{1}{n}\sum_{i=1}^n |X_i|$ is the MLE of $\beta$ in a Laplace (double-exponential) distribution with median $0$ and scale parameter $\beta$ according to the given PDF. (See Wikipedia on 'Laplace distribution'.)

One way to generate $X,$ having this distribution with $\beta = 1,$ is to let $X = Y_1 - Y_2,$ where $Y_i$ are independently exponential with rate 1. In R statistical software below I generate $m = 100,000$ samples of size $n = 50$ from this distribution, find $\hat \beta$ for each sample, and simulate its expectation $E(\hat \beta)$ by averaging these 100,000 values of $\hat \beta.$ The result is consistent with $E(\hat \beta) = 1.$ As you remarked above, the variance of this Laplace distribution is $\sigma^2 = 2\beta^2 = 2.$ With $m = 100,000$ samples, results should be accurate to two (maybe three) decimal places. [For comparison, I also found the medians of the $m$ samples, which are consistent with population median $0,$ and their variances are consistent with $\sigma^2 = 2.$ Finally, the variance of the 100,000 $\hat \beta$s is consistent with $Var(\hat \beta) = \beta^2/n = 1/50.$]

m = 10^5;  n = 50;  x = rexp(m*n) - rexp(m*n)
DTA = matrix(x, nrow=m)    # m x n matrix, each row a sample of n
h = apply(DTA, 1, median)  # vector of m sample medians
b = rowMeans(abs(DTA))     # vector of m beta-hats
v = apply(DTA, 1, var)     # vector of m sample variances
## -9.330546e-05           # consistent with pop median = 0
## 0.9999066               # consistent with pop beta = 1
## 1.998009                # consistent with pop var = 2
var(b);  1/n
## 0.02005612              # consistent with Var(beta hat) = 1/50
## 0.02

I hope this demonstration helps you identify the role of the estimator $\hat \beta$ and its relationship to the population variance.

Below is are plots of original Laplace data x, sample medians h, sample $\hat \beta$s b, and sample variances v.

enter image description here


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.