$\overline{\theta}$ the maximum likelihood estimator of $\theta \implies$? I can't understand how the following statement holds without any extra conditions on the function $g$:
$\overline{\lambda}$ the maximum likelihood estimator of parameter $\lambda \implies g(\overline{\lambda})$ the maximum likelihood estimator of $g(\lambda)$
Intuitively it would seem that we would need $g$ to be monotionic for this to be true. How can it be shown that this statement holds for an arbitrary function $g$?
 A: It is true for any function $g$, as can be seen in many introductory book on mathematical statistics. This is called the invariance property of the MLE (or sometimes Zehna's theorem).
The idea of the proof is that the transformation $g$ induces a likelihood on its target, and the maximizer of this induced likelihood is the maximum likelihood estimator.
Giving some more details: suppose $g:\Lambda\to\Theta$ is a transformation of your parameter space. For $\theta\in \Theta$, one can define 
$$\Lambda_\theta = \{\lambda \in\Lambda: g(\lambda)=\theta\}.$$
The induced likelihood I'm referring to above is given by
$$M(\theta) = \sup_{\lambda\in\Lambda_\theta}L(\lambda),$$
where $L$ is the likelihood function on the original parameter space. 
A: Suppose the likelihood function is $\alpha\mapsto L(\alpha)$.
Let $\beta=g(\alpha)$. I will assume $g$ is one-to-one.
The likelihood as a function of $\beta$ is not the same function as the likelihood as a function of $\alpha$.
The value of $\alpha$ determines the probability distribution of the data, so that $L(\alpha_0) = \Pr(X=x\mid\alpha=\alpha_0)$, where $x$ is the data actually observed (or if $X$ is a continuous variable, then $L(\alpha_0)=f(x\mid\alpha=\alpha_0)$, where $f$ is the probability density function).
I will give to the likelihood as a function of $\beta$ the name $K$.  Thus
$$K(\beta_0) = \Pr(X=x\mid \beta=\beta_0) = \Pr(X=x\mid g(\alpha)=\beta_0).$$
If $L(\alpha_1)<L(\alpha_2)$ then $\Pr(X=x\mid\alpha=\alpha_1)<\Pr(X=x\mid\alpha=\alpha_2)$, so $\Pr(X=x\mid g(\alpha)=g(\alpha_1))<\Pr(X=x\mid g(\alpha)=g(\alpha_2))$, and so $K(g(\alpha_1))<K(g(\alpha_2))$.
Hence if $\alpha_2$ is a value for which $L(\alpha_1)<L(\alpha_2)$ for EVERY value of $\alpha_1$, then $g(\alpha_2)$ is a value for which $K(g(\alpha_1))<K(g(\alpha_2))$ for EVERY value of $g(\alpha_1)$.
