Why are second order-stationary processes also covariance stationary? The way I understand it is that second order stationary processes are characterized by their 1st and 2th moment being time-invariant. That is,
$E[Z_{t}] = \tau_{1}$ and $E[Z_{t}^2] = \tau_{2}$, where $\tau_{1}$, $\tau_{2}$ are time-independent. According to Wikipedia, covariance stationary is the same as second-order stationary, so this should imply that $E[Z_{t}Z_{t+k}] = \tau_{3}$, where $\tau_{3}$ is time-independent.
I don't understand why this is the case. Have I misunderstood something? Thanks. 
 A: Here are the definitions i'm using:


*

*First-order stationary (i.e. mean stationary): E$[Z_t] = \mu<\infty$ is
constant.

*Second-moment stationary: E$[Z_t] = \mu<\infty$ and E$[Z_t^2] = \tau<\infty$.

*Covariance/weak stationary: E$[Z_t] = \mu<\infty$ and E$[Z_t^2] = \tau<\infty$ and $\text{Cov}[Z_{t+h}, Z_t] = \gamma_h$.
Note that Covariance stationary implies Second-moment stationary:
$$
\gamma_0 = \text{Cov}(Z_t, Z_t) = \text{Var}(Z_t) = \mu^2 - E[Z_t^2]
$$
But second-moment stationary does not imply Covariance stationary. 
Let $X_0,X_1,X_2,\dots$ be i.i.d. with mean $0$ and variance $\sigma^2$. Define 
$$
Z_t := \dfrac{1}{\sqrt{t+1}} \sum_{k=0}^t X_k.
$$
Then E$[Z_t] = 0$ and Var$(Z_t) = \sigma^2$ for all $t$, so E[$Z_t^2] = \sigma^2$ for all $t$. However,
\begin{align*}
\text{Cov}(Z_{t+h},Z_t) &= \text{Cov} \left( \dfrac{1}{\sqrt{t+h+1}} \sum_{k=0}^{t+h} X_k,\dfrac{1}{\sqrt{t+1}} \sum_{j=0}^t X_j \right) \\
&= \dfrac{1}{\sqrt{t+h+1} \sqrt{t+1}} \sum_{k=0}^{t+h} \sum_{j=0}^t \text{Cov}(X_k,X_j) \\
&=  \dfrac{(t+1) \sigma^2}{\sqrt{t+h+1} \sqrt{t+1}},
\end{align*}
which clearly depends on $t$.
When I think of second-order stationary (in the context of time-series), I think of weak-stationary i.e. covariance stationary. The following link also might be of use.
https://stats.stackexchange.com/questions/65353/what-is-a-second-order-stationary-process
