Is $\text{argmax}_\theta f\left(X_1,\cdots,X_n\right)$ a random variable?

Let $X_1,\cdots,X_n$ be random variables defined on $\left(\Omega,\mathcal{F},\mathbb{P}\right)$. Let $f : \mathbb{R}^n \to \mathbb{R}$ be a measurable function with a "parameter" $\theta$, where $\theta$ is a real number. Define

$$\hat{\theta} := \text{argmax}_\theta f\left(X_1,\cdots,X_n; \theta\right)$$

When is $\hat{\theta}$ a random variable (on $\Omega$) an how do we prove it? It is clear to me that if maximizer can be expressed in a simple and explicit way (e.g. $\hat{\theta}= X_1 X_2 + X_3 - X_4 +\max(X_5,\cdots,X_n)$), then $\hat{\theta}$ is indeed a r.v. on $\Omega$. But $\text{argmax}$ is rather abstract, so I don't know if the claim always holds?

• In general, the argmax need not even be single-valued. Dec 29 '17 at 18:46
• ... and may not exist at all. Dec 29 '17 at 19:12

For concreteness and visualization, just assume $f$ is a measurable function on $\mathbb{R}^2$ to $\mathbb{R}$. You now want to ask whether $g(x)=\arg \max_y f(x,y)$ is measurable. The problem is that measurability on $\mathbb{R}^2$ can ignore bad behavior on "one dimensional" sets, but taking the argmax may "zoom in" on these sets. In particular, you can have a case where the argmax always occurs for one of two values of $y$ and switches between them depending on whether $x$ is in some nonmeasurable set $N$. For example, with

$$f(x,y)=\begin{cases} 1 & (x,y) \in N \times \{ 0 \} \cup N^c \times \{ 1 \} \\ 0 & \text{otherwise} \end{cases}$$

you have

$$g(x)=\begin{cases} 1 & x \in N^c \\ 0 & x \in N \end{cases}.$$

Now $N \times \{ 0 \}$ and $N^c \times \{ 1 \}$ are measurable subsets of $\mathbb{R}^2$ because they are null, even though their $x$-sections are not measurable.

This example has a very bad property: even though $f(x,y)$ is a measurable function on $\mathbb{R}^2$, $f(\cdot,0)$ and $f(\cdot,1)$ are not measurable functions of their arguments. In light of that, you might ask a related question: if $f(x,y)$ has the property that $f(\cdot,y)$ is measurable for all fixed $y$, is the argmax a measurable function of the data?

• If f(x,y) has the property that f(⋅,y) is measurable for all fixed y, is the argmax a measurable function of the data? Oct 13 '18 at 19:08

Assuming $\text{argmax}_\theta f(x_1,\ldots, x_n;\theta)$ exists and is unique for all $x_1, \ldots, x_n$, it still may not be measurable. Let's take $n=1$, $X = X_1$ uniform on $[0,1]$, $B$ a nonmeasurable subset of $[0,1]$, $$f(x;\theta) = \cases{1 & if x = \theta \in B\cr 1 & if x=\theta-1 \notin B \cr 0 & otherwise\cr}$$ This is a (Lebesgue) measurable function of $x \in [0,1]$ and $\theta \in [0,2]$, because $f(x;\theta)= 0$ a.e., and for every $\theta$ it is a measurable function of $x$. But $$\text{argmax}_\theta f(x;\theta) = \cases{x & if x \in B\cr 1+x & otherwise}$$ is not measurable.

I assume that the context of your question is maximum likelihood estimation. As such, likelihood function $\mathcal{L}(\theta|X_1,...,X_n)$ is a function (not a density function) of the parameter $\theta$ over the parametric space $\Theta$ given the data $(X_1=x_1,...,X_n=x_n)$. Namely, the argmax is generally well defined (generally, because there may occur "issues" with MLE on the boundary of $\Theta$, when you have to take $\sup \mathcal{L}(\theta|X)$ instead of argmax, but this is not related to your question).

EDIT

I think that I misinterpreted your question. Anyway, the argmax may be a degenerate random variable. Take for example a $\mathcal{B}ernoulli (p)$ random variable. Then the MLE is clearly the sample mean. However, if I restrict the parametric space to $[1,2]$ then your MLE is always $1$.

In the case that you meant a non-measurable function, then I guess if you restrict the parametric space to some non-measurable set, then the MLE will not be a r.v. at all.

• I think the question is whether the argmax, if it is single-valued, is measurable as a function of $\omega$.
– Ian
Dec 29 '17 at 18:51