# Autocorrelation of Binary Random Variable

Let $$X_i$$ be a random variable that takes either the value $$+1$$ or $$-1$$. Suppose that if the previous value $$X_{i-1}$$ is $$x\in \left\{ -1, 1\right\}$$, then $$X_i$$ also takes value $$x$$ with probability $$p$$, and it takes the value $$-x$$ with probability $$q=1-p$$.

Question: Can we calculate the autocorrelation function of $$X_i$$ at a given lag $$k\in \mathbb N$$?

My approach: We want to calculate $$\operatorname{corr}(X_i, X_{i+k})$$. For simplicity, let's first consider $$k=1$$.

We have that $$\operatorname{corr}(X_i, X_{i+1}) = \frac{\operatorname{cov}(X_i, X_{i+1})}{\sqrt{\operatorname{var}(X_i) \operatorname{var}(X_{i+1})}}$$ so let us begin by trying to calculate the quantity: $$\operatorname{cov}(X_i, X_{i+1}) = \operatorname{E}\left[ X_i X_{i+1} \right] - \operatorname{E}\left[ X_i \right] \operatorname{E}\left[ X_{i+1} \right].$$

We have $$X_i X_{i+1} = \begin{cases} X_i^2 & \text{ with probability } p \\ -X_i^2 & \text{ with probability } q \end{cases}$$

and therefore: $$\operatorname{E}\left[ X_i X_{i+1} | X_i\right] = pX_i^2 - qX_i^2 = p - q.$$

Moreover, it holds that $$\operatorname{E}\left[ X_i\right] = \operatorname{E}\left[ X_{i+1}\right] = p - q.$$

And therefore, by plugging into the earlier equation, we should have: $$\operatorname{cov}(X_i, X_{i+1}) = (p - q) - (p - q)^2$$

Next, let's compute the variance: $$\operatorname{var}\left[ X_{i+1}\right] = \operatorname{var}\left[ X_i\right] = \operatorname{E}\left[ X_{i}^2\right] - \operatorname{E}\left[ X_{i}\right]^2 = 1 - (p-q)^2.$$

If all of that is correct, then we should have: $$\operatorname{corr}(X_i, X_{i+1}) = \frac{\operatorname{cov}(X_i, X_{i+1})}{\sqrt{\operatorname{var}(X_i) \operatorname{var}(X_{i+1})}} = \frac{(p - q) - (p - q)^2}{1 - (p-q)^2}$$

But this is impossible because it gives values $$< -1$$, for example with $$p=0.1$$. Therefore, something above must be wrong. Please help me correct my calculation :-)

• You don't have sufficient information to calculate the variance.
– Paul
Commented Jul 15 at 14:10

Your error is when you say

Moreover, it holds that $$\operatorname{E}\left[ X_i\right] = \operatorname{E}\left[ X_{i+1}\right] = p - q.$$

Assuming $$X_0=+1$$ or $$-1$$ with equal probability, you have $$\operatorname{E}\left[ X_i\right] = \operatorname{E}\left[ X_{i+1}\right] = 0.$$ Even if there were a different distribution for $$X_0$$, the distribution for $$X_i$$ would converge towards $$+1$$ and $$-1$$ with equal probability as $$i$$ increases and thus $$\operatorname{E}\left[ X_i\right]\to 0$$, so long as $$0.

$$\operatorname{E}\left[ X_i\right] = \operatorname{E}\left[ X_{i+1}\right] = 0$$ would make $$\operatorname{var}\left[ X_{i+1}\right] = \operatorname{var}\left[ X_i\right] =1$$, and $$\operatorname{cov}(X_i, X_{i+1}) = p - q$$ and $$\operatorname{corr}(X_i, X_{i+1}) = p - q,$$ which will be between $$-1$$ and $$1$$ as you might hope.

For longer lags, you should get $$\operatorname{cov}(X_i, X_{i+k}) = \operatorname{corr}(X_i, X_{i+k}) = (p - q)^k,$$ which converges towards $$0$$ as $$k$$ increases, so long as $$0.

• Nothing in the problem implies that a large number limit is appropriate.
– Paul
Commented Jul 15 at 14:08
• @Paul: $\operatorname{E}\left[ X_i\right] = \operatorname{E}\left[ X_{i+1}\right]$ would imply that both are $0$ (or that $p=1$). Commented Jul 15 at 15:20
• sure, but there's no reason to expect that.
– Paul
Commented Jul 15 at 15:23

Let random variables $$Z_1,Z_2,...$$ be i.i.d. $$Z$$, such that $$P(Z=1)=p, P(Z=-1)=q=1-p,$$ so we have $$X_{i+1}=Z_{i+1}\,X_i$$, and \begin{align}X_{i+k}&=(Z_{i+k}\cdot\cdot\cdot Z_{i+1}) X_i=(Z_{i+k}\cdot\cdot\cdot Z_1)X_0, \quad k=1,2,3,...\end{align}

If the $$Z_i$$ are also independent of the $$X_i$$, then we have \begin{align}E(X_{i+k})&=E(Z)^k\,E(X_i)=(p-q)^kE(X_i)\\ E(X_iX_{i+k})&=E(Z)^k\,E(X_i^2)=(p-q)^k\end{align}

yielding

\begin{align} \text{corr}(X_i,X_{i+k})&={(p-q)^k - (p-q)^kE(X_i)^2 \over \sqrt{1-E(X_i)^2} \sqrt{1-E(X_{i+k})^2}}\\[2ex] &=(p-q)^k\,\sqrt{{ 1-E(X_i)^2\over 1-E(X_{i+k})^2}}\\[2ex] &=(p-q)^k\,\sqrt{{ 1-(p-q)^iE(X_0)^2\over 1-(p-q)^{i+k}E(X_{0})^2}}\\[2ex] \end{align}