Why do Fourier Series work?

Question

I would like to have an intuitive understanding of Fourier Series. I mean, I know the formulas: $$ f(t) =\frac{a_0}{2}+\sum_{n=1}^\infty a_n\cos(n\pi tL)+\sum_{n=1}^\infty b_n \sin(n\pi tL) $$ And given a Wave, I know how to do the calculations, however, I have no idea why this formula works besides the fact it sums an infinity of sines and cosines.

How did Fourier arrive at this formula? And what's it meaning? I would be thankful if one could give me a brief explanation or point me reference material.

Thanks in advance

Answer 1

$\newcommand{\Vector}[1]{\mathbf{#1}}\newcommand{\vece}{\Vector{e}}$The linked questions provide good answers, but may be at the technical end of "intuitive". Here's a fast-and-loose conceptual motivation:

If $\bigl(V, \langle\ ,\ \rangle\bigr)$ is an $N$-dimensional real inner product space, and if $\{\vece_{n}\}_{n=1}^{N}$ is an (ordered) orthonormal basis, then an arbitrary vector $v$ in $V$ may be written as a linear combination $$ v = \sum_{n=1}^{N} \langle v, \vece_{n}\rangle \vece_{n}. \tag{1} $$ Indeed, $\{\vece_{n}\}$ is a basis of $V$, so there exist real coefficients $a_{k}$ such that $$ v = \sum_{k=1}^{N} a_{k} \vece_{k}. \tag{2} $$ Taking the inner product of each side with $\vece_{n}$ gives $\langle v, \vece_{n}\rangle = a_{n}$ because the basis $\{\vece_{n}\}$ is orthonormal.

Loosely, one might expect a similar conclusion to hold if $V$ is infinite-dimensional. Getting the definitions and hypotheses right, and proving a version of (1) in this new setting, is why any "honest" answer is bound to be technical. Phrases in quotes below are not mathematically correct, and therefore require careful inspection and/or justification.

Intuitively, let $L > 0$ be real, let $V$ be "the space of real-valued functions" on $[-L, L]$, and define an "inner product" by $$ \langle f, g\rangle = \frac{1}{L} \int_{-L}^{L} f(t) g(t)\, dt. $$ The functions $$ C_{n}(t) = \begin{cases} 1/\sqrt{2}, & n = 0, \\ \cos(n\pi t/L), & n > 0; \end{cases}\qquad S_{n}(t) = \sin(n\pi t/L),\quad n > 0; $$ turn out (by elementary calculus and trigonometry) to form an "orthonormal basis" of $V$.

Loosely, we expect that if $f$ is a function, we can express $f$ as an infinite sum of these basis functions, and the coefficients are the inner products of $f$ with the basis elements, i.e. (for $n > 0$), \begin{align*} a_{0} &= \langle f, 1\rangle = \frac{1}{L} \int_{-L}^{L} f(t)\, dt, \\ a_{n} &= \langle f, C_{n}\rangle = \frac{1}{L} \int_{-L}^{L} f(t) \cos(n\pi t/L)\, dt, \\ b_{n} &= \langle f, S_{n}\rangle = \frac{1}{L} \int_{-L}^{L} f(t) \sin(n\pi t/L)\, dt, \\ f(t) &= \frac{a_{0}}{2} + \sum_{n=1}^{\infty} (a_{n} C_{n}(t) + b_{n} S_{n}(t), \\ &= \frac{a_{0}}{2} + \sum_{n=1}^{\infty} a_{n} \cos(n\pi t/L) + b_{n} \sin(n\pi t/L). \end{align*} (The "special" factor of $1/2$ on the constant term arises because $C_{0} = 1/\sqrt{2} \neq 1$.)

Answer 2

I don't know for sure exactly how Fourier realized that you could represent any periodic function as the countable sum of sines and cosines, but I strongly suspect the realization that this might be the case came from studying solutions to simple, separable PDEs (most first PDE courses will cover this in some depth).

A Partial Differential Equation (PDE) describes the rate of change of a quantity that depends on more than one independent variable, for example if we consider the heat equation in one dimension:

\begin{equation} u_t(t,x) = ku_{xx}u(t,x), \quad k \in \mathbb{R}, \end{equation}

we have that $u:\mathbb{R_+}\times(-L,L) \to \mathbb{R}$ is a scalar quantity corresponding to temperature that depends on time $t$ and one spatial variable $x$. This is a simple model for how the distribution of heat in an iron bar of length $2L$ changes with time. The equation says that the rate of change of temperature at one point $x$, with respect to time is proportional to the second spatial derivative of heat at that point (intuitively, the second spatial derivative can be interpreted as a measure of the nonuniformity of the heat distribution - if the temperature gradients are very large, the temperature will change more quickly. This hopefully aligns with your intuition about how heat conduction works).

One of the nice things about the heat equation is that it's a $\textit{separable}$ equation, that is if we suppose the solution $u(t,x)$ can be written as the product of two terms that depend only on one of each of the independent variables, so that $u(t,x) = T(t)X(x)$ for two unknown functions $T$ and $X$ of time and space respectively, then we actually can work it through, get an answer and justify that our assumption was correct. Plugging this into the heat equation, we get that

\begin{equation} T'(t)X(x) = kT(t)X''(x) \implies \frac{X''(x)}{X(x)} = \frac{T'(t)}{kT(t)}. \end{equation}

Now, $\frac{X''(x)}{X(x)}$ is a function of $x$ only and $\frac{T'(t)}{kT(t)}$ is a function of $t$ only. Since $t$ and $x$ are independent, the only way this equality can hold for all $t$ and $x$ is if both ratios are constant (this is kind of tricky until you see it, think carefully about it for a while). I'll call this constant $-\lambda^2$ for reasons that will become clear.

We then have that

\begin{equation} X''(x) = -\lambda^2X(x), \end{equation}

which has general solution $X(x) = A\cos(\lambda x) + B\sin(\lambda x)$. Starting to look promising? If we put boundary conditions on the bar so that the temperature at each end is fixed at zero, then the corresponding conditions on $X$ are $X(-L) = X(L) = 0$. What does this mean for our general solution?

\begin{equation} A\cos(\lambda(-L)) + B\sin(\lambda(-L)) = 0 \,\, \text{and}\,\, A\cos(\lambda(L)) + B\sin(\lambda(L)) = 0 \implies \cos(\lambda L) = 0 \end{equation}

By adding together using the evenness/oddness of cos and sin. This implies $\lambda = (\pi/2 + n\pi)/L$ for any $n \in \mathbb{N}$ as these are all zeroes of the cosine function. Now, by linearity, if any such $n$ gives a solution, so must the sum of all such $n$, and so with a little trig rearrangement and relabelling to remove the phase shift, this gives

\begin{equation} X(x) = \sum_{n=1}^{\infty}\left(A_n\sin\left(\frac{n\pi x}{L}\right) + B_n\cos\left(\frac{n\pi x}{L}\right)\right) \end{equation}

Where obviously I'm playing a bit fast and loose but hopefully it gives you the idea. Notice I haven't addressed anything to do with $t$, that's a story for a different time, but I do think that given describing an unknown function (a spatial solution of the heat equation) as a sum of sines and cosines drops very easily out of the model, it's not then such a great leap to extend this idea to any unknown functions (either periodic or bounded such that they admit periodic extension), and so represents a possible path that Fourier and his contemporaries might have taken.

Answer 3

Although others provided a deeper mathematical explanation, there is also the musical/sound explanation. I do not know exactly when this was first discovered, but I would believe that even back in the days of the Greeks, people knew of "harmonics" in the musical sense; that is, certain vibrations add together to produce sounds that are consonant if the different vibrations are of integer relation and dissonant if the vibrations are not.

As a musician, I found Fourier Series a very intuitive concept since I had already been taught these things in music class. If you have software that allows it (GNU Octave is what I use), try playing individual sine waves as audio, and try adding together different frequencies together. Square waves and sawtooth waves are common shapes, and have a very distinct timbre.

So in some sense, musicians have known Fourier's Theorem for thousands of years, but the mathematical connection was not apparent.

Answer 4

Whenever this topic comes up, I find a lot of conjecture that reverses the order of the History of this subject. Some real History can be found in Dieudonne's "A History of Functional Analysis". I'll summarize part of what is found there.

Fourier was looking at Heat Conduction. He was fascinated by Heat. And he was able to come up with a macroscopic theory of heat by deducing how this mysterious substance "heat" would flow. He ended up with the classical Fourier heat equation for temperature: $$ \frac{\partial u}{\partial t}= k\nabla^{2}u. $$ He then proceeded to solve such equations by inventing what is now called the Method of Separation of Variables. This was no doubt motivated in part by considering earlier known separated expressions for the displacements of a vibrating string. But the novel idea of proposing separated solutions $u(x,t)=T(t)X(x)$, dividing by $TX$ to separate the variables, and concluding the existence of a separation parameter was Fourier's. This was Fourier's method, and it led to new results. The method was proposed in almost the same form as it is taught today.

Separated solutions of the vibration problem for the string were known to Fourier and others, and had been proposed decades before Fourier's work. And the integral orthogonality conditions of the trigonometric functions were even known to hold. The integral orthogonal conditions were one way to isolate the constant coefficients in the solution, assuming the initial displacement was known. However, people believed that these conditions imposed restrictions on what initial data could be used. It was not proposed that a general function could be expanded in a trigonometric series. Fourier made an amazing leap for the time by postulating that the trigonometric expansions would have to hold for every initial (mechanical) function. And that was a radical departure from the accepted Mathematical thought on the subject at the time. It was so radical that it kept his original treatise on Heat Conduction from being published for almost two decades. Lagrange, Poisson, and others fought him on this idea. After all, how could a series of analytic functions be anything but analytic; so they could not imagine piecewise functions being expanded in this way, even if they were joined continuously.

The idea that every function could be so expanded was motivated in part by Fourier's unwarranted, but unrelenting faith in his model of heat, and the idea that separated solutions must be fully general. This was new territory, and he was challenging accepted thought on the subject. In page after page of amazingly detailed identities, he showed that expansions of common functions really did work out. He even showed how to expand $cos$ in a series of $sin$ functions, resulting in power series of odd powers for $cos$, at least over part of the interval; this addressed another objection. He showed that discontinuous cases could be handled, and that convergence would be to the mean of the left- and right-hand limits. Fourier had to fight to get others to believe that he might be correct in his belief that such expansion could be general.

Many still unfairly accuse Fourier of not having been precise at all. To Fourier's credit, the Dirichlet kernel integral expression for the truncated trigonometric Fourier series was in Fourier's original work. But because that worked was banned from publication for so long, Dirichlet was credited with this first general proof of convergence under conditions of smoothness. Actually, according to experts, it appears that Fourier's discussion of convergence using the Dirichlet integral was sound and predated Dirichlet's proof by decades.

Addressing the expansion issues was a major driving force behind rigorous Mathematics. At the time of Fourier, the following did not exist

• The notion of a general function beyond explicit formulae
• The Riemann integral, measures, and Lebesgue integration
• The definition of real numbers
• General notions of convergence
• Methods for studying PDEs
• Inner product space or Cauchy-Schwarz inequality
• Norms and topology
• Stokes and Divergence Theorems

Historians credit the desire to resolve Fourier's many conjectures as a major driving force behind the development of modern rigorous Mathematics. I think that anyone who studies these methods and sees that they work has to be puzzled and ask, "Why?" Maybe the answer is Spectral Theory, or Group Theory, or Operator Algebras, or Real Analysis, or Complex Analysis. All of these answers have something to do with it, and all of these subjects were directly and profoundly influenced by the general study of Fourier expansion.

Answer 5

Fourier was looking at guitar strings. He reasoned that the waveform (of the string) changes with time and the amplitude at the ends are allways zero. He intuitively reasoned that any waveform of the strings can be represented as a weighted sum of sine/cosine waves that can fit into the guitar by having zero amplitude at the ends (i.e. the harmonics). He then proposed that by removing the restriction that the amplitude must be zero at the ends, any continuous function can be represented as a sum of sine/cosine waves (waves with different wavelengths/frequencies).