I've tried to search for an answer to this question but did not manage to find one.

I have an assignment where we were given some data of a secret distribution, we are supposed to test these 3 models for the data.

enter image description here

According to instructions we're supposed to use the point estimate of the mean & standard deviation to estimate the parameters a and b.

Now for my question: I've calculated the standard mean & deviation, but how can I estimate the parameters a and b using this information?

I don't see how I can use those to estimate a and b.


  • $\begingroup$ First derive the expression for MLE estimates of $a$ and $b$. $\endgroup$ – Dhruv Kohli - expiTTp1z0 Nov 25 '16 at 13:34
  • $\begingroup$ Which point estimate? As @expiTT--I--1z0 says, MLE for $a$ and $b$ might be a good idea. $\endgroup$ – Shraddheya Shendre Nov 25 '16 at 14:14
  • $\begingroup$ There are some different possible answers to this question depending on your "fitting metric", but from context it sounds like they're actually intending for you to use a "method of moments estimator". That is, solve for the mean and variance as a function of $a,b$, set them equal to your sample mean and variance, then solve for $a,b$. $\endgroup$ – Ian Nov 25 '16 at 17:20
  1. $f(x) = Ke^{-ax}, \, x \geq 0$

First, compute $K$ so that $f(x)$ is a probability distribution.

$$\int\limits_0^\infty Ke^{-ax} \, dx=1 \Rightarrow \frac{K}{a} = 1 \Rightarrow K = a$$

Likelihood expression will be,

$$L(a) = \prod_{i=1}^n f(x_i) = \prod_{i=1}^n ae^{-ax_i} = a^ne^{-a \sum\limits_{i=1}^n x_i}$$

$$\log(L(a)) = n\log(a) -a\sum_{i=1}^n x_i$$

Now, you would like to maximize the likelihood (which is equivalent to maximizing log likelihood) by computing the best possible value of $a$ (Maximum Likelihood Estimate (MLE) of $a$). Take the derivative of log likelihood and equate it to zero.

$$\frac{n}{a} - \sum_{i=1}^n x_i = 0 \Rightarrow a = \frac n {\sum\limits_{i=1}^n x_i}$$

Therefore, $a_\text{MLE} = \frac{n}{\sum\limits_{i=1}^n x_i} = \frac 1 {\hat{\mu}}$, where $\hat{\mu}$ is the sample mean.

  1. $f(x) = Kxe^{-ax}$

Find K.

$$\int\limits_0^\infty Kxe^{-ax} \, dx = 1 \Rightarrow 0 + \frac{K}{a}\int\limits_0^\infty e^{-ax} \, dx = 1 \Rightarrow \frac{K}{a^2} = 1 \Rightarrow K = a^2$$

Now, the expression of likelihood,

$$L(a) = \prod_{i=1}^n a^2x_i e^{-ax_i} = (a^2)^n \, e^{-a\sum\limits_{i=1}^n x_i} \prod_{i=1}^n x_i$$

$$\log(L(a)) = 2n\log(a) -a\sum\limits_{i=1}^n x_i + \sum_{i=1}^n \log(x_i)$$

Take derivative with respect to $a$ and equate to zero.

$$\frac{2n} a - \sum_{i=1}^{n}x_i = 0 \Rightarrow \frac 2 a - \hat{\mu} = 0 \Rightarrow a = \frac{\hat{\mu}} 2 $$

Now, I'll leave the third part to you.


Presumably you have calculated the sample mean - usually called $\bar x$. This can be used as an unbiased estimate of the population mean. Let's call this value $\hat \mu$

Presumably you have also calculated the sample standard deviation. This is trickier. If you have used $n$ as the divisor, then what you have is a biased estimate of the standard deviation of the population. If you have used $n-1$ as the divisor, then what you have is an unbiased estimate of the population standard deviation. Let's call this $\hat \sigma$.

Now to consider the population mean $\mu$. This can be found from the pdf.

$\mu=\int_0^{\infty}xe^{-ax}dx$ - use integration by parts. This will give you an expression for $\mu$ in terms of $a$. You can equate this expression to $\hat \mu$ and rearrange to make $a$ the subject - there's your estimate of $a$.

  • $\begingroup$ Thank you for the helpful answer! Somehow forgot you can calculate the mean by integrating the pdf. $\endgroup$ – user3284549 Nov 25 '16 at 15:05
  • $\begingroup$ Although the mean of the exponential distribution can be written in the form of the parameter $a$ but this might not be true for other distributions. The right way is to first compute the MLE of $a$ and $b$. $\endgroup$ – Dhruv Kohli - expiTTp1z0 Nov 25 '16 at 15:25
  • $\begingroup$ Minor point: with $n-1$ you have an unbiased estimate of the variance but not of the standard deviation. $\endgroup$ – Ian Nov 25 '16 at 17:17
  • $\begingroup$ @expiTT--I--1z0 You can definitely get the mean and variance explicitly, they are just the usual gamma function integrals with integer exponents. $\endgroup$ – Ian Nov 25 '16 at 22:33
  • $\begingroup$ Yes you are correct. $\endgroup$ – Dhruv Kohli - expiTTp1z0 Nov 25 '16 at 22:59

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.