Question about convergence in probability (topic confusion) I'm taking second year stats and was introduced the below concepts

For the third one, we use that to estimate the mean squared error in the case where the estimator is a nonlinear function of the sample mean. However, I can't find it anywhere in mathematical statistics textbooks (don't you hate it when that happens?). Where can I find out more about the above topics?
 A: The concept is called "Convergence in Probability".  Although for a sequence of numbers there's only one way of interpreting $x_n\to x$ as $n\to\infty$ for functions or random variables there are several different ideas about convergence.
If $X_n$ are random variables then $X_n$ converges to $x$ in probability if
$$\mathbb P\left[ \|X_n - x\| <\varepsilon\right] \to 1 \text{ as } n\to \infty$$for every $\varepsilon>0$. 
This is quite close to the definition for real numbers, notice that it's not the same thing as $$\mathbb P\left[X_n\to x \text{ as } n\to\infty\right]=1$$
The second statement is much stronger and is known as "Almost Sure Convergence". There are lots of other important ones. I mentioned almost sure convergence to try and convince you there's a point to having lots of different ones.  It's a good exercise to try and come up with an example of a sequence of random variables that converge in probability but not almost surely.
The results of your statements follow from the definition.


*

*Firstly if $\mathbb E(X_n)\equiv c$ and $\mathop{SD}(X_n)\to 0$ then by Chebyshev's inequality we have $$\mathbb P\left[\|X_n - c\|>\varepsilon\right] \leq \frac{\mathop{SD}(X_n)}{\varepsilon^2} \to 0$$ as $n\to\infty$.

*Next if $\lim_{x\to c}g(x) = \ell$ then for every $\varepsilon>0$ there exists some $\delta>0$ such that $\|x-c\|<\delta\Rightarrow\|g(x)-\ell\|<\varepsilon$ hence
$$
\mathbb P\left[\left|g(X_n) - \ell\right| <\varepsilon\right] \geq 
\mathbb P\left[\left|X_n - c\right| <\delta\right]\to 1
$$as $n\to\infty$.

*The final statement follows directly from the second by noting that if $g$ is differentiable as $c$ then 
$$\lim_{x\to c} \frac{g(x)-g(c)}{x-c} =  g'(c).$$

