2
$\begingroup$

In the proof for the Cramer Rao Inequality, my book writes:

$$E[\hat{\theta}] = \int{\hat{\theta}(\textbf{x})L(\theta;\textbf{x})d\textbf{x}=\theta}$$

Then differentiating both sides of this equation with respect to $\theta$, and interchanging the order of integration and differentiation, gives

$$\int\hat{\theta}\frac{\delta L}{\delta \theta}d \textbf{x}=1$$

This is because $\hat{\theta}$ does not depend on $\theta$.

But why does $\hat{\theta}$ not depend on $\theta$? Because i think $\hat{\theta}$ depends on the random sample which depends on $\theta$

$\endgroup$
1
$\begingroup$

The estimator is a function of the random variables $x_i$, which means the estimator is itself a random variable. In a frequentist framework the parameter, however, is not a random variable; it is just a fixed but unknown constant.

It is in this sense that $\hat{\theta}$ is not a function of $\theta$ (even though you are right that each $x_i$ “depends on” $\theta$ in the sense that the $x_i$ are drawn from a distribution with parameter $\theta$).

For the sake of concreteness, say $X$ has a normal distribution with parameters $\mu$ and $\sigma$. Say we draw an iid sample of size $n$ to estimate $\mu$. Then our estimator, the sample mean $\bar{x}=1/n\Sigma x_i$, is a function of the $x_i$, which are each identically distributed $N(\mu, \sigma^2)$ distributions, but it is not a function of $\mu$ or $\sigma$.

This can all be made absolutely precise in the formalism of measure theory, where the estimator is a measurable function, but the parameters describe the underlying probability measure itself.

$\endgroup$
0
$\begingroup$

It's because $\partial _x$ denotes "explicit" differentiation. For example, Say you have a function $f=n^2$. Then $\partial _n f = 2n$, even if $n=z^2$. In that case, $\partial _z f$ would still yield $0$. $D_x$ and $\partial_x$ do not mean the same thing.

One thing to notice. Although the derivative on the left hand side might be a total derivative ($D_x$, in this case), by application of the Leibniz Integral Rule, the differentiation becomes partial inside the integral sign.

$\endgroup$
  • $\begingroup$ This is not really the reason. It is true the $x$ depends on the parameters, but you are conflating a random variable with a probability measure. $\endgroup$ – symplectomorphic Feb 25 at 7:49
0
$\begingroup$

The estimator $\hat\theta$ is first of all a statistic, which by definition, is a function of the sample observations $X_1,X_2,\ldots,X_n$ independent of the parameter of interest ($\theta$ in this case). How can an estimator of an unknown parameter be dependent on the very parameter you are estimating?

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.