Is the square of a sample mean stochastically bounded of order square root of 1/n? Let $X_{1,}...,X_{n}$ be an i.i.d. sample from a Gaussian $N(\theta,\sigma^{2})$.
Denote by $\bar{X}_{n}=\frac{1}{n}\sum_{i=1}^{n}X_{i}$ the sample
mean. Then of course $\bar{X}_{n}\sim N(\theta,\sigma^{2}/n)$ and
$\bar{X}_{n}-\theta=O_{p}\left(n^{-1/2}\right).$ Now, what about
$\bar{X}_{n}^{2}-\theta^{2}$? Can we also show that $\bar{X}_{n}^{2}-\theta^{2}=O_{p}\left(n^{-1/2}\right)$?
 A: Notice that for any $M>0$:
$$
P\left(\sqrt{n}\left|X_{n}^{2}-\theta^{2}\right|>M\right)=
$$
$$
=P\left(\sqrt{n}\left|X_{n}^{2}-\theta^{2}\right|>M,\ X_{n}<\theta\right)+
$$
$$
+P\left(\sqrt{n}\left|X_{n}^{2}-\theta^{2}\right|>M,\ X_{n}>\theta\right)=
$$
$$
=2P\left(\sqrt{n}\left|X_{n}^{2}-\theta^{2}\right|>M,\ X_{n}<\theta\right)\leq
$$
$$
\leq2P\left(\sqrt{n}\left(X_{n}-\theta\right)^{2}>M,\ X_{n}<\theta\right)\leq
$$
$$
\leq2P\left(\sqrt{n}\left(X_{n}-\theta\right)^{2}>M\right).
$$
Now: fix $\varepsilon>0$. Since $X_{n}-\theta=O_{p}(n^{-1/2})$ then
there exists $M_{\varepsilon/2}>0$ s.t. $P\left(\sqrt{n}\left|X_{n}-\theta\right|>M_{\varepsilon/2}\right)<\varepsilon/2$,
$\forall n\in\mathbb{N}$. But the event $\left\{ \left|X_{n}-\theta\right|>M_{\varepsilon/2}\right\} $
is equal to the event $\left\{ \left|X_{n}-\theta\right|^{2}>M_{\varepsilon/2}^{2}\right\} $.
Hence plugging $M_{\varepsilon/2}^{2}$ in place of $M$ in the above
expression, we obtain $P\left(\sqrt{n}\left|X_{n}^{2}-\theta^{2}\right|>M_{\varepsilon/2}^{2}\right)<\varepsilon$,
$\forall n\in\mathbb{N}$. This is exactly what was to be proved.
