2
$\begingroup$

Consider the regression model: $y_i = bx_i + e_i,\quad 1 ≤ i ≤ n$,

Suppose that $x_i$’s take values −1 or +1 and $e_i$’s have density $f(t) ={\frac{1}{2}}e^{−|t|}, t \in \mathbb{R}$.

Find the maximum likelihood estimator of $b$.

Therefore $\; y_i-bx_i \sim \epsilon \quad \text{,which follows}\quad f(t) ={\frac{1}{2}}e^{−|t|}\\ \therefore f(y,b,x_i)= {\frac{1}{2}}e^{−|y_i-bx_i|}\\ \Rightarrow L(y,x_i,b) = {\frac{1}{2}}^n e^{−\sum|y_i-bx_i|}\\ \Rightarrow \frac{\partial\log L(y,x_i,b)}{\partial b} = -\frac{\partial{\sum |y_i-bx_i|}}{\partial b} $

Any ideas about how to proceed??

$\endgroup$
3
  • $\begingroup$ Try searching for "least absolute deviations" $\endgroup$
    – bitesizebo
    Mar 31, 2018 at 10:07
  • $\begingroup$ @Z.Xie did not help $\endgroup$
    – DRPR
    Mar 31, 2018 at 13:57
  • $\begingroup$ The MLE of the expected value of Laplace distribution is the median of $Y_1,..,Y_n$. Not sure how to generalize it for the regression model with an explained variable. $\endgroup$
    – V. Vancak
    Mar 31, 2018 at 15:03

2 Answers 2

1
$\begingroup$

In this case your log-likelihood function is:

$$\ell (b) = - \sum_{i=1}^n | y_i - b x_i |.$$

Since your $x_i$ values are either negative or positive one, we can let $\mathscr{X} \equiv \{ i = 1,...,n | x_i =1 \}$ be the set of indices for which this explanatory variable is one. We then have:

$$\ell (b) = - \sum_{i \in \mathscr{X}} | y_i - b | - \sum_{i \notin \mathscr{X}} | y_i + b |.$$

Taking $\tfrac{d}{dx}|x| = \text{sgn}(x)$ the score function is:

$$\begin{equation} \begin{aligned} \frac{d\ell}{db} (b) &= - \sum_{i \in \mathscr{X}} \text{sgn}( y_i - b ) - \sum_{i \notin \mathscr{X}} \text{sgn}( y_i + b ) \\[6pt] &= \sum_{i \in \mathscr{X}} \text{sgn}( b - y_i ) - \sum_{i \notin \mathscr{X}} \text{sgn}( b + y_i ). \\[6pt] \end{aligned} \end{equation}$$

The score equation is:

$$\sum_{i \in \mathscr{X}} \text{sgn}( \hat{b} - y_i ) = \sum_{i \notin \mathscr{X}} \text{sgn}( \hat{b} + y_i ).$$

The MLE $\hat{b}$ is any value that balances these signs. It is likely that this will be an interval of values, and so in this case you will need to choose a value in the interval arbitrarily (e.g., the midpoint). It should be possible to solve this equation numerically to obtain $\hat{b}$ by checking values over intervals between the appropriate $y_i$ values.

$\endgroup$
0
$\begingroup$

A friend of mine who is not in MSE deserves credit for this answer.

The likelihood function of b is :-$$ L(b,y_i,x_i)\,\,\,=\,{\frac{1}{2}}^n e^{−\sum|y_i-bx_i|}\\ \Rightarrow log \, L(b,y_i,x_i)= c - {\sum |y_i-bx_i|} $$ Now need to maximize the loglikelihood function wrt b, ie minimize ${\sum |y_i-bx_i|}$. Note that $x_i $ can only take values of $\pm 1$.

$\therefore |x_i|=1$, Using this we can write $${\sum |y_i-bx_i|} \Rightarrow ({\sum |y_i-bx_i|})\frac{1}{|x_i|} \Rightarrow ({\sum |\frac{y_i}{x_i}-b|})$$

Here I am going to use an interesting result ${\sum |t_i-b|}$ is minimized if $b=median(t_1,t_2,t_3,....t_n)=t_m$. Using this result , we can say that $${\sum |y_i-bx_i|} \text{ is minimized at }z_m\, \text{where }z_m=median(\frac{y_1}{x_1},\frac{y_2}{x_2},\frac{y_3}{x_3},...\frac{y_n}{x_n})$$

$$\Rightarrow L(b,y_i,x_i)= c - {\sum |y_i-bx_i|} \text {is maximized at }z_m $$

$\therefore$ the MLE of b is $ median(\frac{y_1}{x_1},\frac{y_2}{x_2},\frac{y_3}{x_3},...\frac{y_n}{x_n})$

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.