# Fourier Transform: Understanding change of basis property with ideas from linear algebra

The notion of Fourier transform was always a little bit mysterious to me and recently I was introduced to functional analysis. I am a beginner in this field but still I am almost seeing that the Fourier transform can be viewed as a change of basis in a space of functions. I read the following article here which tries to build an intuition:

Now, I can see that the Fourier and Inverse Fourier tranforms are projecting and projecting back a function $f(x)$ onto and from the basis of complex exponentials, $e^{i2\pi sx}$, respectively: $$F(s) = \int_{-\infty}^{\infty}f(x)e^{-i2\pi sx}dx$$

$$f(x) = \int_{-\infty}^{\infty}F(s)e^{i2\pi sx}ds$$

1)If I understand correctly, this operation is akin to regular linear algebra change of basis operations $a=Mb$ and $b=M^{-1}a$. Roughly, in this case $M$ is a matrix of uncountable many rows and columns where each row is $e^{-i2\pi sx}$, a function of $x$, and similarly $M^{-1}$ has rows as $e^{i2\pi sx}$, functions of $s$. Is this interpretation correct?

2)I don't have the exact rigour for this but intuitively think, if 1) is a correct interpretation then we should obtain from the "infinite dimensional" matrix multiplication $MM^{-1}$ something which resembles an infinite dimensional identity matrix. To test that, I built the inner product where $s$ is held fixed and equal in both terms from two objects $M$ and $M^{-1}$, which should correspond to a "diagonal" element of $MM^{-1}$: $\int_{-\infty}^{\infty}e^{i2\pi sx}e^{-i2\pi sx}dx = \int_{-\infty}^{\infty}e^{0}dx=\infty$. So this is not $1$ as expected from an identity matrix. What is the reason of that?

If you have an orthonormal basis $\{ e_{k} \}_{k=1}^{N}$ on a finite-dimensional space, such as what you would obtain with Gram-Schmidt, then every vector $x$ is expressed as $$x = \sum_{k} (x,e_{k})e_{k}.$$ This extends to $L^{2}[0,2\pi]$ using $e_{k} =\frac{1}{\sqrt{2\pi}}e^{ikx}$: $$f = \sum_{k}(f,e_{k})e_{k}$$ The importance of this basis is that it consists of eigenvectors of $Lf=\frac{1}{i}\frac{d}{dx}$ because $Le_{k}=ke_{k}$. So this basis diagonalizes the differentiation operator. Finally, the same thing holds in a continuous sense on $L^{2}(\mathbb{R})$ with \begin{align} f & = \int_{k} (f,e_{k})e_{k} \\ & = \frac{1}{2\pi}\int_{-\infty}^{\infty}\left(\int_{-\infty}^{\infty}f(t)e^{-ikt}dt\right)e^{ikx}dk \end{align} This is a generalization rather than a precise extension because $e_{k}=\frac{1}{\sqrt{2\pi}}e^{ikx}$ is not--strictly speaking--an eigenfunction of $L=\frac{1}{i}\frac{d}{dx}$ because $e_{k} \notin L^{2}(\mathbb{R})$ due to the fact that the function is not square integrable. However, for every $\delta > 0$, the following is square integrable $$e_{k,\delta}=\frac{1}{\sqrt{2\pi}}\int_{k-\delta}^{k+\delta}e_{k}(x)dk,$$ and, in the norm of $L^{2}$, it becomes closer and closer to an eigenvector with eigenvalue $k$ as $\delta\downarrow 0$: $$\|Le_{k,\delta}-ke_{k,\delta}\| < \delta\|e_{k,\delta}\|.$$ So the Fourier transform is the coefficient function and the expansion of $f$ looks very much like a "continuous" (i.e., integral) expansion of $f$ in approximate eigenfunctions of the differentiation operator. (The $e_{k,\delta}$ are even mutually orthogonal if the intervals $(k-\delta,k+\delta)$ do not overlap.)

As a final note, to make this generalization more precise, $$\|x\|^{2} = \sum_{k}|(x,e_k)|^{2}$$ also holds for the continuous orthogonal expansion: $$\|f\|^{2} = \int_{k} |(x,e_{k})|^{2}dk.$$ This is how Parseval saw it, who is the person after whom Parseval's identity is named: $$\int_{-\infty}^{\infty}|f(x)|^{2}dx = \int_{-\infty}^{\infty}|\hat{f}(k)|^{2}dk.$$

• Thanks for the answer. Sorry if this is too obvious but to make it clearer: The general form of the Fourier transform: $\int_{-\infty}^{\infty}f(t)e^{-ikt}dt$ is actually not defined, since $e^{ikt}$ does not have a finite integral and not in $L^2(\mathbb{R})$ space. What is actually going on is that we are defining $e_{k,\delta}$ as basis vectors like $\sum_{k} f(t) e_{k,\delta}$ and then letting $\delta \to 0$. Is this right? May 29 '15 at 14:18
• @UfukCanBiçici : The function $e^{ikt}$ is not square integrable on $\mathbb{R}$, even though wave packets of such things are: $\int_{k-\delta}^{k+\delta}e^{ikx}dk \in L^{2}$, and these are approximate eigenfunctions with eigenvalue $k$. For $f \in L^{2}$, you may view the Fourier transform as a limiting coefficient function for expansions as you mention. Essentially, $\hat{f}$ becomes the coefficient of $e^{ikx}$ in the continuous expansion $f=\int_{-\infty}^{\infty}\hat{f}(k)\frac{ikx}{\sqrt{2\pi}}dk$ May 29 '15 at 14:26
• Then roughly speaking, can we say that this is why my rather intuitive matrix multiplication $MM^{-1}$ does not result in something meaningful, like an identity matrix; since I am enforcing $L^2$ dot product on functions which are not square integrable to begin with? May 29 '15 at 14:37
• @UfukCanBiçici : As a final note, it is worth mentioning that how you state it is consistent with how the notion of "continuous spectrum" arose. Discrete spectrum is where you have a discrete set of eigenvalues, and the continuous case is thought of as a limit of the discrete, transitioning to the continuous. May 29 '15 at 14:37
• @UfukCanBiçici : Concerning $M$, that's right. You have a limiting case. The generalization comes through a Unitary operator $U$, i.e., one which is invertible and norm preserving, $\|Uf\|=\|f\|$. In this context, the Fourier transform turns application of $\frac{1}{i}\frac{d}{dt}$ into multiplication, which is really an application of a "continuous" diagonal matrix. You have $\frac{1}{i}\frac{d}{dt} f = U^{-1}MUf$ where $U$ is the Fourier transform and $(Mf)(\lambda)=\lambda f(\lambda)$ is multiplication. That's the Mathematical generalization through a unitary change of basis. May 29 '15 at 14:43

You can imagine derivative as a matrix such as this, as the corresponding values of each element approach zero (limit definition):

Or integration as like this(riemann sum):

Moreover, fourier transform already has a matrix representation for discrete case

https://en.wikipedia.org/wiki/DFT_matrix

You need extend this matrix to infinity and shrink the corresponding interval to zero (riemann sum) to get the continuous transform.

$$\int_{-\infty}^{\infty}e^{0}dx=\infty$$

Think of this as summing up all the ones in the diagonal. The diagonal components have the value 1, they correspond to an infinitesimal interval (dx) and when you sum infinitely many of them you get infinity.

Alternatively think about what would happen if you multiplied the derivative matrix and the integral matrix. You would get the identity matrix. By integrating you would be summing the ones on the diagonal.

• Where do we use the matrix $A$ in our case exactly? And is obtaining an infinite dimensional identity matrix with diagonal elements at infinity expected then, in this case? May 29 '15 at 13:44
• You don't use infinite matrices. Matrices are used by computers because it is the only thing they can do well. May 29 '15 at 13:50
• I mean, from an intuitive point of view, not for using in a practical application. My goal is to get an understanding of the Fourier Transform with analogies to finite dimensional matrix-vector operations commonly used in linear algebra. May 29 '15 at 13:52
• We don't use the matrix A. We would use the matrix W in the wiki article. Derivative and integral matrices were given as an example of continuous linear transformations being turned into matrices (combined with the related limit definition). May 29 '15 at 13:54