# Proving stochastic boundedness in rate of contraction posterior distribution

Consider a family of probability distributions $P_\theta$ indexed by $\theta \in \Theta$. The parameter space is endowed with some metric $d$. We assume that there is a true parameter $\theta_0$, and we are interested of the convergence of the Bayesian posterior distribution, given a prior $\Pi$. We denote the posterior distribution after $n$ samples by $\Pi_n(\cdot|X^{(n)})$.

The posterior distribution is said to contract at rate $\epsilon_n \to 0$ at $\theta_0$ if $\Pi_n(\theta:d(\theta,\theta_0) > M_n\epsilon_n | X^{(n)}) \to 0$ in $P_{\theta_0}^{(n)}$ probability, for every $M_n\to \infty$ as $n\to\infty$ (i.e., regardless of how slow $M$ goes to infinity).

Now I have to proof the following proposition:

Suppose that the posterior distribution $Pi(\cdot|X^{(n)})$ contracts at rate $\epsilon_n$ at $\theta_0$. Then $\hat{\theta}_n$, defined as the center of a (nearly) samllest ball that contains posterior mass at least $\frac{1}{2}$, satisfies $d(\hat\theta_n,\theta_0) = O_P(\epsilon_n)$ under $P_{\theta_0}^{(n)}$.

$O_P(\epsilon_n)$ is not defined but I suppose it denotes stochastic boundedness (see wikipedia), that is, for $\tilde\epsilon >0$, there are $M, N \in \mathbb{N}$ such that for $n>N$, we have $P_{\theta_0}^{(n)} ( d(\theta_0,\hat\theta_n)/\epsilon_n > M) ) < \tilde\epsilon$.

The notes state the proof is very similar to a previous proof, which brought me to the following:

Let $B(\theta,r)$ be the closed ball of radius $r$ centered at $\theta$. Define $\hat{r}_n(\theta) = \inf\{r:\Pi_n(B(\theta,r)|X^{(n)}) \geq \frac{1}{2}\}$.

Then $\hat{r}_n(\theta_0) < M_n\epsilon_n$ with a probability tending to 1, with $M_n$ an arbitrary sequence with $M_n \to \infty$.

As we have chosen $\hat\theta_n$ as a nearly smallest ball, certainly $\hat{r}_n(\hat\theta_n) \leq \hat{r}_n(\theta_0) + \frac{1}{n} < M_n\epsilon_n + \frac{1}{n}$.

Now, $B(\theta_0,M_n\epsilon_n)$ and $B(\hat\theta_n,\hat{r}_n(\hat\theta_n))$ are not disjoint with probability tending to 1, since they would have a joined probability tending to 1.5. So:

$d(\theta_0,\hat\theta_n) < 2M_n\epsilon_n + \frac{1}{n} \implies d(\theta_0,\hat\theta_n)/\epsilon_n < 2M_n + \frac{1}{\epsilon_n n}$,

with probability tending to one. In other words, for $\tilde\epsilon >0$ there is some $N$ such that for $n>N$,

$P_{\theta_0}( d(\theta_0,\hat\theta_n)/\epsilon_n \geq 2M_n + \frac{1}{\epsilon_n n} ) < \tilde\epsilon$.

However, if I understand stochastic boundedness correctly I need to somehow replace $2M_n + \frac{1}{\epsilon_n n}$ with a fixed $M$. But since $M_n \to \infty$, I don't see how this is possible. Any ideas on what I could do?

lecture notes

I think you're almost there. $$X_n=O_p(a_n)$$ when $$P( | \frac{X_n}{a_n} | \ge M_{\delta} ) < \delta$$. $$M_{\delta}$$ is affected by the choice of $$\delta$$. In other notations you can also write $$M_{\delta}$$ as $$M_{n}$$ where $$n > N(\delta)$$ s.t $$\ni P( |\frac{X_n}{a_n}| \ge M_n ) < \delta$$.
Bottom line is that in the definition of contraction rate, you could choose your $$M_n$$ as you please.