# How can the following $\arg \max$ function be reduce?

Let $\theta \sim \mathcal{U}$ and $A(x)$ be defined by the random variable $x$ by:

$$A(x)=x-\lfloor x\rfloor$$

Calculate the MAP estimator $\hat \theta_{MAP}$ for the inputs $A(x)\in [0,0.001]$

$$f(x|\theta)=\frac{2(|\theta|+1)}{\sqrt{2\pi}}e^{-2(x-\theta)^2(|\theta|+1)^2}$$

This is my work so far:

$$\hat \theta_{MAP}=\arg{\max_\theta{P\left(\theta |A(x)\right)}}=\arg{\max_\theta{\frac{P\left( A(x)|\theta\right)P(\theta)}{P\left( A(x)\right)}}}=\arg{\max_\theta{P\left( A(x)|\theta\right)}}$$ This is because $P\left( A(x)\right)$ is irrelevant for the $\arg{\max}$ function and $\theta \sim \mathcal{U}$ so all $P(\theta)$ are equal so it is also irrelevant.

And from here I am not sure on how to continue. By looking at $x\in[i-0.001,i+0.001] i=-k,\ldots,0,1,\ldots,k$. Is the following true?

$$\hat \theta_{MAP}=\arg{\max_\theta{\sum_i P\left( A(x)|\theta,x_i\right)P(x_i|\theta)}}=\arg{\max_\theta{\sum_i P\left( A(x)+i|\theta\right)}}$$

• How do I reduce the $\arg{\max}$ here?
• Is it correct to say that $\hat \theta_{MAP}=k$? If so why?
• There is something missing from your question: What is the distribution of $x$ given $\theta$? Commented Nov 26, 2016 at 15:45
• @passerby51 you are most certainly write. a second glance in my assignment sheet reveled it was not defined there to so I sent an email and will update the post as soon as I know. Commented Nov 26, 2016 at 16:00
• OK, sounds good. Commented Nov 26, 2016 at 16:02
• @passerby51 I have added some information on $f(x|\theta)$. Is that of any help? Commented Nov 27, 2016 at 9:52
• Yes, it helps. Are you sure about the form of $f(x|\theta)$? Is this a homework problem? You can try to plot the function you are trying to maximize and see how it looks like. (Hint: look at the behavior as $\theta \to \infty$.) Commented Nov 27, 2016 at 19:05

Assume that distribution of $x$ given $\theta$ has density $p_\theta(\cdot)$. Then, we have $$\mathbb P(A(x) \in (z,z+dz)) = \sum_{i \in \mathbb Z} p_\theta(z+i)dz$$ so that $A(x)$ will have the density $z \mapsto \sum_{i \in \mathbb Z} p_\theta(z+i)$ on $(0,1)$, assuming that the sum is convergent (which is the case if $p_\theta$ has compact support for example).

Consider the likelihood given $A(x)$, i.e., $$F(\theta;z) := \sum_{i \in \mathbb Z} p_\theta(z+i).$$ The MAP estimator is the same as the MLE in this case. Note that since the prior is supported on $[0,1]$, the posterior is also supported on $[0,1]$. The posterior will be proportional to $F(\theta;z)$ on $[0,1]$ and zero elsewhere. We have: $$\hat \theta := \arg \max_{\theta \in [0,1]} F(\theta;A(x))$$ which can't be simplified further without knowing more about $p_\theta$. We can find the maximum by plotting $F(\theta;A(x))$ as a function of $\theta$.

For the case where $x$ given $\theta$ is Gaussian with mean $\theta$ and variance $\frac1{4(1+|\theta|)^2}$:

The functions $\theta \mapsto f(z+i|\theta)$ are centered around $z+i$ and die down very quickly when moving away from the center. They also become taller for increasing $i$.

Since we are only concerned with $\theta \in [0,1]$, a plot helps showing that $F(\theta,0.001)$ is maximized at $\hat \theta = 1$ over $[0,1]$. In fact there seems to be a threshold $\tau$ such that as long as $A(x) \in [0,\tau]$ we have $\hat \theta = 1$, and $\tau \in [0.22,0.23]$ it seems.

(As another example, if $A(x) = 0.5$, we have nontrivial $\hat\theta \approx 0.56$.)

The hint in the problem is perhaps trying to say this: $$F(\theta;z) \approx \sum_{|i| \le k} p_\theta(z+i), \quad \text{for}\; \theta \in [0,1]$$ where $k$ even as small as $2$ or $3$ seems to work.

• Considering I an looking at $f(x+i|\theta)$ and the biggest $i$ possible is $k$, can I say that the $\arg {\max{f(x+i|\theta)}}$ functions is reduced to the element $f(x+k|\theta)$ and therefor $\hat {\theta}_{MAP}=k$? Did I get it? Commented Nov 28, 2016 at 6:47
• @havakok, There is no biggest $i$ (no restriction on $i$) as far as I understand. You have to look at $f(x+i|\theta)$ as a function of $\theta$. Also, you need to sum over $i$ before looking at the maximum. What I wrote might be a bit misleading. For simplicity assume $x = 0$ (or close to zero, it doesn't matter much). $f(x+i|\theta)$ as a function of $\theta$ is a bump around $x + i \approx i$ that dies down very quickly when you move away. You have one for each integer $i$ and they are mostly nonoverlapping except around $\theta =0$. Moving away from zero, these bumps become taller... Commented Nov 28, 2016 at 20:22
• @havakok, in short, what you have is not correct. (What is $k$ even?) Commented Nov 28, 2016 at 20:24
• @havakok, here is a plot. Commented Nov 28, 2016 at 20:40
• According to this plot as $\theta \to \infty$ also $f(x+i|\theta)\to \infty$. I am at a loss here. and k is as defined in my question $i= -k,\ldots , 0,1,\ldots ,k$, I am assuming that the biggest argument $k$, will yield the maximal $f(x+i|\theta)$ therefore the $\arg {\max{f(x+i|\theta)}}$ will be reduced to $k$ (be it $\infty$ or lower). Commented Nov 28, 2016 at 22:37