Here's the problem statement: Let $X_1$, . . . , $X_n$ be independent random variables with common mean $\mu$ and variances $σ_i^2$ . To estimate $μ$, we use the weighted average $T_n$ = $\sum_{i=1}^n w_iX_i$

with weights $w_i$ = $\frac{\sigma_i^{−2}}{\sum_{i=1}^n \sigma_j^{-2}}$

Show that the estimate $T_n$ is consistent (in probability) if $\sum_{i=1}^n \sigma_j^{-2} \Rightarrow \infty$. End Problem.

So I know that a consistent estimator is one that asymptotically approaches the parameter as the sample size goes to infinity. So by the problem statement, if the denominator of each of the weightings goes to infinity, then each of the $w_iX_i$ would just go to 0, yes? I'm a little confused on what this problem is trying to show. Thanks for any help!


Yes, $w_{i}X_{i}$ go to 0, but you are interested in the behaviour of $T_{n}$ . Convergence in probability means .

\begin{equation} \lim_{n \rightarrow \infty }P(|T_{n} - \mu | \ge \epsilon) = 0 , \forall \epsilon > 0 \end{equation}

Try using some concentration inequality involving the variance .

  • $\begingroup$ It seems that the Bernstein inequality you want to use require finite moments of any order, which is not specified in the question and not needed to reach the conclusion with the assumptions of the problem. $\endgroup$ – Davide Giraudo Feb 1 at 14:25
  • $\begingroup$ You are right , I see the variables can be unbounded .I will edit the answare . Thanks !. $\endgroup$ – Popescu Claudiu Feb 1 at 17:42

Here are some steps:

  1. Without loss of generality, $\mu=0$.
  2. Compute $\operatorname{Var}\left(T_n\right)$. Using independence this reduces to $\sum_{i=1}^n\operatorname{Var}\left(\omega_iX_i\right)$. Since $$\operatorname{Var}\left(\omega_iX_i\right)=\omega_i^2\sigma_i^2,$$ it follows that $$ \operatorname{Var}\left(T_n\right)=\sum_{i=1}^n\left(\frac{\sigma_i^{-2}}{\sum_{j=1}^n\sigma_j^{-2}}\right)^2\sigma_i^2=\sum_{i=1}^n\sigma_i^{-2}\frac 1{\left(\sum_{j=1}^n\sigma_j^{-2}\right)^2}=\frac{1}{\sum_{j=1}^n\sigma_j^{-2}}. $$
  3. Since $\mathbb E\left[T_n\right]=\mu$, we get that $\mathbb E\left[\left(T_n-\mu\right)^2\right]\to 0$ from which the convergence in probability follows.
  • $\begingroup$ I'm a little confused by how setting $\mu$ = 0 helps. When I calculate Var($T_n$), I get the following: Var($T_n$) = $\sum w_i Var(\sum(X_i)) = \sum w_i \sum Var((X_i)) = \sum \frac{\sigma_i^{-2}}{\sum \sigma_j^{-2}}\sum \sigma_i^2$, and I'm not sure how this variance shows that the sum $T_n$ converges to the mean. $\endgroup$ – psun Feb 1 at 20:53
  • $\begingroup$ It have edited. Actually this computation works also if $\mu\neq 0$. $\endgroup$ – Davide Giraudo Feb 1 at 21:25
  • $\begingroup$ So by showing this is the variance of $T_n$, the variance of this estimate goes to 0. That much I understand. I guess what I don't quite get is how this necessarily shows that $T_n$ converges to the population mean $\mu$. $\endgroup$ – psun Feb 1 at 23:05
  • $\begingroup$ @psun I have edited. $\endgroup$ – Davide Giraudo Feb 2 at 18:27
  • $\begingroup$ Thanks for everything! This question is clear once you see all the moving parts :) $\endgroup$ – psun Feb 4 at 17:37

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.