Calculating method of moments estimators for exponential random variables I'm trying to find the method of moment estimators for $\sigma$ and $\tau$. I have the i.i.d. exponential random variables $X_1, \dots, X_n$ with the density functions
$$f(x; \sigma, \tau)=
\begin{cases}
 \dfrac{1}{\sigma} e^{-(x - \tau)/\sigma} &\text{if}\, x\geq \tau\\
      0 &\text{otherwise}
\end{cases}$$
I know that $E(X)=\sigma + \tau$ and $E(X^2)=2\sigma^2+2\sigma \tau + \tau^2$. Then, for method of moments, we set $\frac{1}{n}\sum_{i=1}^{n}X_i=\hat{\sigma}+\hat{\tau}$ and $\frac{1}{n}\sum_{i=1}^nX_i^2=2\hat{\sigma}^2+2\hat{\sigma}\hat{\tau}+\hat{\tau}^2$. But I don't understand how to proceed from here to get the estimators for $\sigma$ and $\tau$, and I don't really understand how $E(X)=\sigma + \tau$ and $E(X^2)=2\sigma^2+2\sigma \tau + \tau^2$ were calculated. For $E(X)=\sigma + \tau$, I tried
$$E(X) = \int_\tau^\infty \dfrac{1}{\sigma} e^{-(x - \tau)/\sigma} \ dx,$$
but this results in $1$, so I don't think it's correct.
So how do proceed from here to get the estimators for $\sigma$ and $\tau$? And how were $E(X)$ and $E(X^2)$ calculated?
 A: Since $X$ is a shifted exponential with location $\tau$ and scale $\sigma$, it follows that $X - \tau$ is exponential with scale $\sigma$, hence $\operatorname{E}[X - \tau] = \sigma$.  This implies $$\operatorname{E}[X] = \sigma + \tau.$$  Moreover, the variance of $X$ is unchanged by a location shift; i.e., $\operatorname{Var}[X] = \sigma^2$, so $$\operatorname{E}[X^2] = \operatorname{Var}[X] + \operatorname{E}[X]^2 = \sigma^2 + (\sigma + \tau)^2 = 2\sigma^2 + 2 \sigma \tau + \tau^2.$$  Of course, we can compute these via integration:  $$\operatorname{E}[X] = \int_{x=\tau}^\infty x \frac{1}{\sigma} e^{-(x - \tau)/\sigma} \, dx, \quad \operatorname{E}[X^2] = \int_{x=\tau}^\infty x^2 \frac{1}{\sigma} e^{-(x - \tau)/\sigma} \, dx;$$ however, knowing the properties of the exponential distribution is preferable.
As for the method of moments estimation, we match on the first two raw moments to obtain the system
$$\begin{align}
\bar X &= \operatorname{E}[X] = \sigma + \tau, \\
\bar X_2 &= \operatorname{E}[X^2] = \sigma^2 + (\sigma + \tau)^2,
\end{align}$$
where $\bar X_2 = \frac{1}{n} \sum_{i=1}^n X_i^2$ is the second raw sample moment.  Note that the way the second equation is written allows us to perform a substitution: $$\sigma^2 = \bar X_2 - \bar X^2.$$  Therefore, $$\sigma = (\bar X_2 - \bar X^2)^{1/2}$$ and $$\tau = \bar X - \sigma = \bar X - (\bar X_2 - \bar X^2)^{1/2}.$$  So the desired method of moments estimators are $$\hat \tau = \bar X - (\bar X_2 - \bar X^2)^{1/2}, \\ \hat \sigma = (\bar X_2 - \bar X^2)^{1/2}.$$
Why does this look different?  Well, notice
$$\begin{align}
\bar X_2 &= \frac{1}{n} \sum_{i=1}^n X_i^2 \\
&= \frac{1}{n} \sum_{i=1}^n (X_i - \bar X + \bar X)^2 \\
&= \frac{1}{n} \sum_{i=1}^n \left( (X_i - \bar X)^2 + 2(X_i - \bar X)\bar X + \bar X^2 \right) \\
&= \frac{1}{n} \sum_{i=1}^n (X_i - \bar X)^2 + \frac{2 \bar X}{n} \sum_{i=1}^n (X_i - \bar X) + \frac{1}{n} (n \bar X^2) \\
&= \frac{1}{n} \sum_{i=1}^n (X_i - \bar X)^2 + 0 + \bar X^2,
\end{align}$$
where in the last step we have used the fact that $\sum_{i=1}^n X_i = n \bar X$, thus the second term is zero.  Hence
$$\bar X_2 - \bar X^2 = \frac{1}{n} \sum_{i=1}^n (X_i - \bar X)^2,$$ and this leads to the alternative formula you referred to.  Note that in the linked question, $\theta = 1/\sigma$; i.e., $\theta$ is a rate parameter, not scale.
A: $P(\tau \leq x < \infty ) = \int_{\tau}^{\infty} f(x) dx = 1$
$E[X] = \int_{\tau}^{\infty} xf(x) dx = \sigma + \tau$, because this is simply an exponential distribution that has been shifted.
$E[X+c] = E[X] + E[c] = E[X] + c$
Variance $V[X+c] = V[X]$ because adding a constant everywhere does not change the shape of the distribution at all.
$E[X^2] = V[X] + (E[X])^2 = \sigma^2 + (\sigma + \tau)^2$
In your problem, you have two equations in two unknowns, $\hat \sigma$ and $ \hat \tau$. Solve for one (usually in the simpler equation) and substitute into the other.
In writing it all out, I'm going to use some new symbols to save on typing:  let
$$s = \hat \sigma, t = \hat \tau, y = \frac{1}{n}\sum X_i, z =\frac{1}{n}\sum X_i^2  $$ so our equations are $$y = s+t, z = 2s^2 + 2st + t^2$$ and the goal is to solve for $s$ and $t$ in terms of $y$ and $z$.
$$ s = y-t$$
$$ z = 2(y-t)^2 + 2(y-t)t + t^2$$
$$ z = 2y^2 - 4yt + 2t^2 + 2yt - 2t^2 + t^2$$
$$ z = t^2 - 2yt + 2y^2$$
$$ 0 = t^2 - 2yt + 2y^2-z$$ so
$$t = \frac{2y \pm \sqrt{4y^2 - 4\cdot1\cdot(2y^2 - z)} }{2}$$
$$t = y \pm \sqrt{y^2 - (2y^2 - z)}$$
$$t=y\pm\sqrt{z-y^2}$$
$$s = \sqrt{z-y^2}$$
so switching notation back, we get
$$\hat \sigma = \sqrt{\frac{1}{n}\sum X_i^2 - (\frac{1}{n}\sum X_i)^2}$$
$$\hat \tau = \frac{1}{n}\sum X_i \pm \sqrt{\frac{1}{n}\sum X_i^2 - (\frac{1}{n}\sum X_i)^2}$$
and as the other answer explained, this rearranges in a few more lines of algebra to the answer you gave.
