2
$\begingroup$

Let $X_1,\cdots,X_n$ be random variables defined on $\left(\Omega,\mathcal{F},\mathbb{P}\right)$. Let $f : \mathbb{R}^n \to \mathbb{R}$ be a measurable function with a "parameter" $\theta$, where $\theta$ is a real number. Define

$$\hat{\theta} := \text{argmax}_\theta f\left(X_1,\cdots,X_n; \theta\right)$$

When is $\hat{\theta}$ a random variable (on $\Omega$) an how do we prove it? It is clear to me that if maximizer can be expressed in a simple and explicit way (e.g. $\hat{\theta}= X_1 X_2 + X_3 - X_4 +\max(X_5,\cdots,X_n)$), then $\hat{\theta}$ is indeed a r.v. on $\Omega$. But $\text{argmax}$ is rather abstract, so I don't know if the claim always holds?

$\endgroup$
2
  • 1
    $\begingroup$ In general, the argmax need not even be single-valued. $\endgroup$ Dec 29 '17 at 18:46
  • 1
    $\begingroup$ ... and may not exist at all. $\endgroup$ Dec 29 '17 at 19:12
1
$\begingroup$

For concreteness and visualization, just assume $f$ is a measurable function on $\mathbb{R}^2$ to $\mathbb{R}$. You now want to ask whether $g(x)=\arg \max_y f(x,y)$ is measurable. The problem is that measurability on $\mathbb{R}^2$ can ignore bad behavior on "one dimensional" sets, but taking the argmax may "zoom in" on these sets. In particular, you can have a case where the argmax always occurs for one of two values of $y$ and switches between them depending on whether $x$ is in some nonmeasurable set $N$. For example, with

$$f(x,y)=\begin{cases} 1 & (x,y) \in N \times \{ 0 \} \cup N^c \times \{ 1 \} \\ 0 & \text{otherwise} \end{cases}$$

you have

$$g(x)=\begin{cases} 1 & x \in N^c \\ 0 & x \in N \end{cases}.$$

Now $N \times \{ 0 \}$ and $N^c \times \{ 1 \}$ are measurable subsets of $\mathbb{R}^2$ because they are null, even though their $x$-sections are not measurable.

This example has a very bad property: even though $f(x,y)$ is a measurable function on $\mathbb{R}^2$, $f(\cdot,0)$ and $f(\cdot,1)$ are not measurable functions of their arguments. In light of that, you might ask a related question: if $f(x,y)$ has the property that $f(\cdot,y)$ is measurable for all fixed $y$, is the argmax a measurable function of the data?

$\endgroup$
1
  • $\begingroup$ If f(x,y) has the property that f(⋅,y) is measurable for all fixed y, is the argmax a measurable function of the data? $\endgroup$
    – jlewk
    Oct 13 '18 at 19:08
1
$\begingroup$

Assuming $\text{argmax}_\theta f(x_1,\ldots, x_n;\theta)$ exists and is unique for all $x_1, \ldots, x_n$, it still may not be measurable. Let's take $n=1$, $X = X_1$ uniform on $[0,1]$, $B$ a nonmeasurable subset of $[0,1]$, $$f(x;\theta) = \cases{1 & if $x = \theta \in B$\cr 1 & if $x=\theta-1 \notin B$ \cr 0 & otherwise\cr}$$ This is a (Lebesgue) measurable function of $x \in [0,1]$ and $\theta \in [0,2]$, because $f(x;\theta)= 0$ a.e., and for every $\theta$ it is a measurable function of $x$. But $$\text{argmax}_\theta f(x;\theta) = \cases{x & if $x \in B$\cr 1+x & otherwise}$$ is not measurable.

$\endgroup$
0
$\begingroup$

I assume that the context of your question is maximum likelihood estimation. As such, likelihood function $\mathcal{L}(\theta|X_1,...,X_n)$ is a function (not a density function) of the parameter $\theta$ over the parametric space $\Theta$ given the data $(X_1=x_1,...,X_n=x_n)$. Namely, the argmax is generally well defined (generally, because there may occur "issues" with MLE on the boundary of $\Theta$, when you have to take $\sup \mathcal{L}(\theta|X)$ instead of argmax, but this is not related to your question).

EDIT

I think that I misinterpreted your question. Anyway, the argmax may be a degenerate random variable. Take for example a $\mathcal{B}ernoulli (p)$ random variable. Then the MLE is clearly the sample mean. However, if I restrict the parametric space to $[1,2]$ then your MLE is always $1$.

In the case that you meant a non-measurable function, then I guess if you restrict the parametric space to some non-measurable set, then the MLE will not be a r.v. at all.

$\endgroup$
1
  • $\begingroup$ I think the question is whether the argmax, if it is single-valued, is measurable as a function of $\omega$. $\endgroup$
    – Ian
    Dec 29 '17 at 18:51

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.