Consider a family of probability distributions $P_\theta$ indexed by $\theta \in \Theta$. The parameter space is endowed with some metric $d$. We assume that there is a true parameter $\theta_0$, and we are interested of the convergence of the Bayesian posterior distribution, given a prior $\Pi$. We denote the posterior distribution after $n$ samples by $\Pi_n(\cdot|X^{(n)})$.

The posterior distribution is said to contract at rate $\epsilon_n \to 0$ at $\theta_0$ if $\Pi_n(\theta:d(\theta,\theta_0) > M_n\epsilon_n | X^{(n)}) \to 0$ in $P_{\theta_0}^{(n)}$ probability, for every $M_n\to \infty$ as $n\to\infty$ (i.e., regardless of how slow $M$ goes to infinity).

Now I have to proof the following proposition:

Suppose that the posterior distribution $Pi(\cdot|X^{(n)})$ contracts at rate $\epsilon_n$ at $\theta_0$. Then $\hat{\theta}_n$, defined as the center of a (nearly) samllest ball that contains posterior mass at least $\frac{1}{2}$, satisfies $d(\hat\theta_n,\theta_0) = O_P(\epsilon_n)$ under $P_{\theta_0}^{(n)}$.

$O_P(\epsilon_n)$ is not defined but I suppose it denotes stochastic boundedness (see wikipedia), that is, for $\tilde\epsilon >0$, there are $M, N \in \mathbb{N}$ such that for $n>N$, we have $P_{\theta_0}^{(n)} ( d(\theta_0,\hat\theta_n)/\epsilon_n > M) ) < \tilde\epsilon$.

The notes state the proof is very similar to a previous proof, which brought me to the following:

Let $B(\theta,r)$ be the closed ball of radius $r$ centered at $\theta$. Define $\hat{r}_n(\theta) = \inf\{r:\Pi_n(B(\theta,r)|X^{(n)}) \geq \frac{1}{2}\}$.

Then $\hat{r}_n(\theta_0) < M_n\epsilon_n$ with a probability tending to 1, with $M_n$ an arbitrary sequence with $M_n \to \infty$.

As we have chosen $\hat\theta_n$ as a nearly smallest ball, certainly $\hat{r}_n(\hat\theta_n) \leq \hat{r}_n(\theta_0) + \frac{1}{n} < M_n\epsilon_n + \frac{1}{n}$.

Now, $B(\theta_0,M_n\epsilon_n)$ and $B(\hat\theta_n,\hat{r}_n(\hat\theta_n))$ are not disjoint with probability tending to 1, since they would have a joined probability tending to 1.5. So:

$d(\theta_0,\hat\theta_n) < 2M_n\epsilon_n + \frac{1}{n} \implies d(\theta_0,\hat\theta_n)/\epsilon_n < 2M_n + \frac{1}{\epsilon_n n}$,

with probability tending to one. In other words, for $\tilde\epsilon >0$ there is some $N$ such that for $n>N$,

$P_{\theta_0}( d(\theta_0,\hat\theta_n)/\epsilon_n \geq 2M_n + \frac{1}{\epsilon_n n} ) < \tilde\epsilon$.

However, if I understand stochastic boundedness correctly I need to somehow replace $2M_n + \frac{1}{\epsilon_n n}$ with a fixed $M$. But since $M_n \to \infty$, I don't see how this is possible. Any ideas on what I could do?

lecture notes


I think you're almost there. $X_n=O_p(a_n)$ when $P( | \frac{X_n}{a_n} | \ge M_{\delta} ) < \delta$. $M_{\delta}$ is affected by the choice of $\delta$. In other notations you can also write $M_{\delta}$ as $M_{n}$ where $n > N(\delta)$ s.t $\ni P( |\frac{X_n}{a_n}| \ge M_n ) < \delta$.

Bottom line is that in the definition of contraction rate, you could choose your $M_n$ as you please.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.