Take the 2-minute tour ×
Mathematics Stack Exchange is a question and answer site for people studying math at any level and professionals in related fields. It's 100% free, no registration required.

Let $X_1, ..., X_n$ be independent observations of a random variable $X$ that has with probability $p$ the distribution $\mathcal{U}[0,a]$ and with probability $1-p$ the distrbiution $\mathcal{U}[0,b]$ where $a$ and $b$ are known with $a<b$.

  1. What is the maximum-likelihood estimator for $p$?

  2. Is that estimator efficient? (This means, is the Cramér–Rao bound achieved?)

I'm having quite some trouble with this task for several hours now. I think the density of $X$ is given by $$f_X(x) = p \frac{1}{a}1_{[0,a]}(x) + (1-p) \frac{1}{b}1_{[0,b]}(x).$$

For the trivial case $n=1$ this gives $\hat{p}=1_{[0,a]}(x_1)$ where the expected value of $X_1$ is $a/b$. So the estimator is in fact biased. So this doesn't really help with the second task.

I can't even figure out the general likelihood function. I think it should be something like $L(p) = (p \frac{1}{a})^k ... + ((1-p) \frac{1}{b})^m ...$ where $k$ denotes the number of values from the sample $x_1, ..., x_n$ where $x_i\le a$ and $m=n-k$. But I don't understand how to deal with the indicator functions and if maybe I have to add a factor like with the binomial distribution.

However, I took the raw form $L(p) = (p \frac{1}{a})^k + ((1-p) \frac{1}{b})^{(n-k)}$, $k$ like explained above and tried to maximize by using the log-likelihood. But from that I got $\hat p = \frac{an-bN}{an-bn}$ which doesn't strike me as correct as because of $a<b$ it's negative for small $N$. Nonetheless, I tried to calculate the Fisher information and compared it's inverse to the variance of $\hat p$ - I wasn't very surprised not to get an equality...

Could be that I messed up some calculations, too - dealing with the derivatives of the log-likelihood was a little messy. Now I'm rather frustrated.

Can anyone show me how to solve this?

share|improve this question

1 Answer 1

up vote 1 down vote accepted

If the indicators functions confuse you, don't use them. Write instead

$$f_X(x)=\begin{cases} \begin{array}{lc} {p \over a} + \frac{1-p}{b} & x\in (0,a)\\ \frac{1-p}{b} & x\in (a,b) \end{array}\\ \end{cases}$$

which, letting $A=1/a$ $B=1/b$ can also be written as

$$f_X(x)=\begin{cases} \begin{array}{lc} p\, (A-B) +B & x\in (0,a)\\ (1-p) \, B & x\in (a,b) \end{array}\\ \end{cases}$$

If you have $n$ samples, of which $k$ fall in $(0,a)$, then the likelihood is

$$L = \left(p\, (A-B) +B\right)^k \left((1-p)B\right)^{n-k}$$

The derivative of the log-likelihood is then and $$ \frac{k (A-B)}{p\, (A-B) +B} - \frac{(n-k)B}{(1-p)B}$$

which gives me

$$p_0=\frac{k}{n}-\left(1-\frac{k}{n}\right)\frac{a}{b-a}=\frac{1}{b-a}\left(\frac{k}{n}b-a\right)$$

Not however that this is just the critical point, which does not necessarily coincide with the ML estimator. ACtually, that is valid only if $k/n > a/b$, and

$$\hat p_{ML}= \begin{cases} \begin{array}{lc} \frac{1}{b-a}\left(\frac{k}{n}b-a\right) & k \ge n \frac{a}{b}\\ 0 & elsewhere \end{array}\\ \end{cases}$$

It doesn't seem to be consistent, as, when $n\to \infty$ $k/n \to p$ (in prob.), hence $\hat p_{ML}$ won't converge to $p$. Edited: sorry for the errors; actually $n/k \to p+(1-p)a/b = [p(b-a)+a]/b$ and hence the estimator should be at least asymtotically unbiased.

share|improve this answer
    
Thanks for the input. Is there any more agreeable way to check for efficiency than to actually compute the fisher information and compare it's inverse to $Var(\hat p)$? –  Amarus Jun 13 '13 at 1:48
    
I'd say that this estimator is biased (even asymtotically), so it cannot be efficient (actually, the Cramer Rao bound applies only to unbiased estimators –  leonbloy Jun 13 '13 at 2:20
    
BTW, you might have better answers at stats.stackexchange.com –  leonbloy Jun 13 '13 at 2:20
    
Why would $\frac{k}{n} \to p$ for $n \to \infty$? Values from $U[0,b]$ have lie within $[0,a]$ with a probability of $\frac{a}{b}$, i.e. $\mathbb{P}(X \leq a) = p + (1-p)\frac{a}{b}$ and thus $\frac{k}{n} \to p + (1-p)\frac{a}{b}$, no? –  fgp Jun 13 '13 at 8:38
    
I don't understand why biasedness is a problem. We introduced the CR bound in a general form that makes no use of unbiasedness: $$Var_\theta(T) \ge (\partial/\partial \theta E_\theta(T))^2 / I(\theta),$$ where $I(\theta)$ is the Fisher information. The usual CR bound follows from this in the unbiased case. With this, we can define efficiency independent of unbiasedness, I guess? –  Amarus Jun 13 '13 at 9:06

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.