I am a software engineer. I am self studying Time Series by reading a book by Chatfield. I was able to derive the following expression below:

$r_k = \sum\cos wt \cos w(t+k) / \sum \cos^2 wt$

Note for the above we have constant $w$ such that $w \in (0,\pi)$. Please note that the summation is for $t \in (1,N)$.

I want to show the above expression can further be shown to converge such that I desire to show that $r_k\rightarrow \cos kw$ as $N \rightarrow \infty$.

I am however stuck at this point. I don't need the entire answer as I still want to learn, but I have been unable to find a way to rewrite this expression to what I need.


I have two questions. Please read below.

Question One:

My text gave a hint that $\sum \cos^2wt = N/2$ for suitable large $N$. In other words, I take that as $\lim_{t \to \infty} \cos^2 wt = N/2$ However, it unfortunately lacked a proof and I would very much like to understand why this would be true.

Question Two:

How can I simplify further to show $r_k \rightarrow \cos kw$?

The textbook hinted we use the expression $2 \cos A \cos B = \cos(A+B) + \cos(A - B)$. So taking the hint I gave in question 1 and this identity. I rewrote $r_k$ as follows:

$r_k = [lim_{t \to \infty} \sum(\cos (wt + w(t+k)) +\cos (wt - w(t+k))] \times 2/N$

where by the identity I used $A = wt$ and $B = w(t+k)$. The $2/N$ part comes from the hint I was given. I can see $\cos (wt - w(t+k)) = \cos (-kw)$. Which I believe from the identity table means, $\cos (-kw) = \cos kw$. Right? So I get completely stuck here:

$r_k = [lim_{t \to \infty} \sum(\cos (2wt + kw) +\cos (kw)] \times 2/N$

This is the part I completely stall out on. This problem isn't necessary for me to understand the point of the chapter. It's that Correlogram converge to this kind of fluctuation for large values, but I want to understand why.

My humble thanks and offers of patience to a software engineer. Thank you.


I think the key trig formula here is the angle addition formula:

$$\cos(\theta + \phi) = \cos \theta\cos\phi - \sin\theta\sin\phi$$

and its consequence, the cosine double-angle formula: $$\cos2\theta = 2\cos^2\theta -1$$ which can be recast as $$\cos^2\theta = \frac{\cos2\theta + 1}2$$ Sine has its own double angle formula: $$\sin 2\theta = 2\sin\theta\cos\theta$$

Question 1

It is certainly not the case that $\lim_{t \to \infty} \cos^2 \omega t = N/2$. The limit and function on the left have nothing to do with $N$, so they can hardly be expected to converge to $N/2$ when $N$ is arbitrary.

First, use the double angle formula to rewrite the sum in terms of a straight cosine (I'm dropping using $t$ as an discrete index - not only is $t$ conventionally a continuous variable, choosing it to be an index is particularly inappropriate when time is involved.)

$$\begin{align}\sum_{n=1}^{N}\cos^2n\omega &= \sum_{n=1}^{N} \frac{\cos2n\omega + 1}2 \\&= \frac12\left(\sum_{n=1}^{N} \cos2n\omega + \sum_{n=1}^{N} 1\right) \\&= \frac12\sum_{n=1}^{N} \cos2n\omega + \frac N2\end{align}$$

Now, $\sum_{n=1}^{N} \cos2n\omega$ is bounded. That is, there is some value $M$, which does not depend on $N$, such that $\left|\sum_{n=1}^{N} \cos2n\omega\right| < M$ for all $N$. Intuitively (not a proof), the reason for this is that when $N$ is large the various values for $n \le N$ of $2n\omega \mod 2\pi$ are fairly equally spread all over the interval $[0,2\pi]$ and since cosine is negative over half of it, the negative and positive values cancel out, leaving the sum near $0$. One problem with this idea is that you can get (depending on the value of $\omega$) several positive values in a row before you get into the negative region and the sum goes back down. And because cosine is periodic, this continues to happen no matter how far out you go. This is why you can only say that $\sum_{n=1}^{N} \cos2n\omega$ is bounded, not that it approximates $0$.

So, if you choose $N$ to be orders of magnitude larger than $M$, then $N\pm M \approx N$, which means that $$\sum_{n=1}^{N}\cos^2n\omega \approx \frac {\pm M}2 + \frac N2 \approx \frac N2$$

Question 2

I don't find that hint very helpful myself. Maybe there is some way to make it work, but I prefer the angle addition formula here:

$$\begin{align}r_k &= \frac{\sum_{n=1}^{N}\cos n\omega\cos(n+k)\omega}{\sum_{n=1}^{N}\cos^2n\omega} \\&=\frac{\sum_{n=1}^{N}\cos n\omega(\cos n\omega\cos k\omega - \sin n\omega\sin k\omega)}{\sum_{n=1}^{N}\cos^2n\omega} \\&=\frac{\sum_{n=1}^{N}(\cos^2 n\omega\cos k\omega - \cos n\omega\sin n\omega\sin k\omega)}{\sum_{n=1}^{N}\cos^2n\omega} \\&=\cos k\omega\frac{\sum_{n=1}^{N}\cos^2 n\omega}{\sum_{n=1}^{N}\cos^2n\omega} - \sin k\omega\frac{\sum_{n=1}^{N}\cos n\omega\sin n\omega}{\sum_{n=1}^{N}\cos^2n\omega} \\&=\cos k\omega - \sin k\omega\frac{\sum_{n=1}^{N}\sin 2n\omega}{2\sum_{n=1}^{N}\cos^2n\omega}\end{align}$$

In the last expression, $\sum_{n=1}^{N}\sin 2n\omega$ is bounded by some $M'$ for the same reasons I gave above for $\sum_{n=1}^{N}\cos 2n\omega$. So for large $N$, we have that $$r_k \approx \cos k\omega - \sin k\omega\frac{M'}{N}$$

Since $M'$ is fixed, as $N \to \infty$, the second term goes to $0$, and $r_k \to \cos k\omega$.

  • $\begingroup$ This was beautiful. I really need to brush up on my arguments and trigonometry (and identities). Do you have any references to suggest? I would like to argue my case mathematically just like you did. $\endgroup$ – hlyates Mar 27 '16 at 13:07
  • $\begingroup$ One small clarifying question, you stated $\sum 1 = N/2$, wouldn't it be $\sum 1 = N(N+1)/2$? $\endgroup$ – hlyates Mar 27 '16 at 13:10
  • $\begingroup$ No. I stated $\frac12\sum_{n=1}^N 1 = N/2$. Those parentheses are not just there to make things pretty. And how do you get that $N$ copies of $1$ would add up to anything other than $N$? $\endgroup$ – Paul Sinclair Mar 27 '16 at 21:50
  • $\begingroup$ Because I had $\sum i = N(N+1)/2$ stuck in my head. You answer is perfectly valid. If possible, do you have any reference material to suggest for me to study to improve my proofs and identities. Practice makes better. $\endgroup$ – hlyates Mar 28 '16 at 23:56
  • 1
    $\begingroup$ I'm sorry, but all my references are 30+ years out-of-date. Any good trig book can give you the formulas and practice on them. I recommend really getting to know the cosine and sine angle sum and difference formulas. They are the fountainhead for just about everything one does with trig and calculus. If you deal with trig functions regularly, knowing these 4 formulas by heart and how to handle them is of immeasurable value. $\endgroup$ – Paul Sinclair Mar 29 '16 at 0:18

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.