What do we know about the pdf of $\bar{X}$

Question

We have $n$ independent random variables $X_i$ all with mean $\theta$ and variance $\sigma^2$. The sample mean is given by $$\bar{X} = \frac{1}{n} \sum\limits_i^n X_i$$ and the means square error is $$MSE=E\left[(\bar{X}-\theta)^2 \right]$$

The question is:

1. What do you know about the underlying probability density function of $\bar{X}$, and how can you use that knowledge to estimate $\theta$ ?

It should be very straitght forward... But thats whats confusing me i guess?

So should I use the central limit theorem? Or ... Uhm.. So yeah.

Answer 1

Since you are summing indpendent random variables with finite means and variances, the CLT will be useful to approximate the distribution of the sample mean statistic. The degree of inaccuarcy from using the CLT can estimated if you also happen to know the third absolute moment of the random variables. If you know that, then use the Berry-Eseen theorem to see how far off your probability estimates could be.

For the sake of this question, you can use the CLT to approximate the sampling distribution of $\bar X$ as $\mathcal{N}(\theta, \frac{\sigma}{\small\sqrt{n}})$. Since you don't know $\theta, \sigma^2$ then you will need to form a confidence interval using the t-distribution with n-1 df. The t-distribution assumes your estimator is normally distributed, which will be approximately the case here (to varying degrees depending on how skewed your actual distributions are). I think the t-distribution confidence interval is what you are looking for.

Note that the MSE = Sample Variance so you have all the info you need to get the t-interval.

Answer 2

The expectation value of $\overline{X}$ is $\theta$. So $\overline{X}$ is a good estimate of $\theta$. Can you figure out the variance of $\overline{X}$?