Formulation of a sentence from the estimator of a likelihood function

Question

The likelihood $L$ for a parameter $\theta$, given the outcome $x$ of an experiment is : $L(\theta ; x)=P(x ; \theta)$

where $P(x ; \theta)$ is the probability to have $x$ if the parameter is $\theta$.

We construct the distribution obtained for $L$ as a function of $\theta$.

Let's asssume that for $\theta=10$, the likelihood is the maximum, and that for $\theta=1$, the likelihood is very low.

Should I say :

a) "the value of parameter $\theta=10$ is very likely, and value of parameter $\theta=1$ is very unlikely" or should I say that

b) "obtaining from an experiment the estimator $\hat{\theta}=10$ is very likely, and obtaining from an experiment the estimator $\hat{\theta}=1$ is very unlikely"

Is (a) correct ? Is (b) correct ? Are (a) and (b) correct ?

Answer 1

TL;DR The statements (a) and (b) are not so much "wrong" as they are poorly worded and misleading. (The question seems to adopt a non-Bayesian framework, so my answer will do likewise; i.e., it will not consider parameter $\theta$ as having a probability distribution, except for the final note.)

For a "likelihoodist", Maximum Likelihood Estimation is based on the principle that the parameter value that maximizes the likelihood function -- i.e. assigns highest probability to the data actually observed -- is the parameter value most strongly supported by that data. The degree to which the data $x$ supports $\theta=\theta_1$ relative to $\theta=\theta_2$ is measured by the likelihood ratio $\Lambda(\theta_1:\theta_2;x)={L(\theta_1;x)\over L(\theta_2;x)}={P(x;\theta_1)\over P(x;\theta_2)}.$ (This line of reasoning is an example of what the statistician J. Savage called "making the Bayesian omelette without breaking the Bayesian eggs.") The likelihood ratio also appears naturally in Bayesian methods as the Bayes factor in posterior odds-ratios, as well as in other non-Bayesian methods (e.g., Likelihood Ratio Tests), where it can be justified on the grounds of its large-sample properties.

Thus, in the example, we have the likelihood ratio $$\Lambda(\theta=10:\theta=1;x)={L(10;x)\over L(1;x)}={P(x;\theta=10)\over P(x;\theta=1)},$$

and two things may be said -- both of which are correct under the assumed data-generation model $P(x;\theta)$:

(A) The parameter value $\theta=10$ is $\Lambda$ times more "likely" than is $\theta=1$, given the generated data $x$.

(B) The generated data $x$ is $\Lambda$ times more probable if $\theta=10$ than it is if $\theta=1$.

These correspond to your (a) and (b), respectively, but note how (a) and (b) use the term "likely" in two different ways without comment: In (a) (and my (A)), "likely" refers to "likelihood" (the technical term), not probability -- $\theta$ has no probability distribution; whereas in (b), "likely" does mean probable -- referring to $P(x;\theta)$.

NB: Of course, in a Bayesian approach, the likelihood function is only part of the picture, as $\theta$ is assigned a prior probability distribution that gets updated by the data $x$ -- via the likelihood function -- to give a posterior distribution. For certain so-called "non-informative" priors, a posterior odds-ratio will be the same as the corresponding likelihood ratio, in which case the term "likely" appearing in (a) and (A) would also mean "probable" with respect to the posterior distribution.

Answer 2

I think both statements are not correct. Recall the basic idea of Maximum likelihood estimation(MLE) is: what you can observe empirically has higher chance to happen than what you can not observe. The goal of MLE is to find distribution of random variable X (characterized by vector parameter $\theta$ (in your case, $\theta$ is one dimension)) such that empirical observed sample data have highest chance to happen.

So the suitable statement in this case should be: When $\theta=10^\circ$, the likelihood that outcome $x$ of the experiment happen is maximum.

Answer 3

I don't really see any significant difference between (a) and (b). If you want to conclude something based on your measurement, I'd recommend doing some additional analysis (e.g. calculateing the standard error) and if everything is in-line, you can arrive to the conclusion.