Rigorous definition of the Maximum likelihood estimator

Question

I want to get into the habit of correctly using rigorous notation where possible. I have tried to do this as best as possible with the maximum likelihood estimator, but I am having some slight confusion as a lot of books will reference $\hat{l}(\theta;x_1,...,x_n)$.

I have been stating the following when talking about the MLE of variable $\theta$:

$\{\hat{\theta}_{MLE} \} \subseteq \{\text{argmax}_{\theta \in\Theta}L(\theta;x_1,...x_n)\}$

Where $L(\theta;x_1,...x_n)$ is the Likelihood function, $x_1,...x_n$ are the sample of iid observations from the distribution, and $\Theta$ refers to the whole parameter space.

Does this look adequate, to me it makes sense, but again I don't know if my notation is off. Any confirmation would be appreciated, thanks!

Answer 1

Since you want to be rigorous, I think your definition might be slightly incorrect. In order to see this, you should think about the definition of an estimator. If $\mathcal{X}$ is the sample space and $\Theta$ is the parameter space, then an estimator, $\hat{\theta}$, is a function from $\mathcal{X}$ to $\Theta$. That is, $\hat{\theta}: \mathcal{X} \rightarrow \Theta$. Your definition doesn't make this clear.

The maximum likelihood estimator is a particular type of estimator. For each $\theta \in \Theta$, let's say that the data has density $f(\mathbb{x}|\theta)$. Now, for each $\mathbb{x} \in \mathcal{X}$, the likelihood function for that point, $L_{\mathbb{x}}$, is a function from $\Theta$ to $\mathbb{R}^{+}$ such that $L_{\mathbb{x}}(\theta)=f(\mathbb{x}|\theta)$. Finally, the maximum likelihood estimator, $\hat{\theta}_{ML}$, is an estimator such that, for every $\mathbb{x} \in \mathcal{X}$ \begin{align*} \hat{\theta}_{ML}(\mathbb{x}) &\in \arg\max_{\theta \in \Theta} L_{\mathbb{x}}(\theta) \end{align*} Of course, my definition also has a flaw from an ultra-rigorous perspective. One might ask how to obtain $f(\mathbb{x}|\theta)$. However, the answer to this question might be too technical for the scope of your question.