6
$\begingroup$

I always wondered about the idea behind convolution. I get what the definition of the convolution does (and I saw all the animations), but what I don't understand is how it relates to so many topics in physics. It seems to me that it is not really an intuitive concept. I guess my question is - what were the ideas, thoughts of the guy that discovered convolution?

About two years ago I read a blog post about convolution that explains it pretty intuitively, but I forgot the name of the website.

$\endgroup$
  • $\begingroup$ Convolution is something like a weighted average operation: for each point you calculate a weighted (the weight being given by the other function) average of values of some function at (possibly infinite) surroundings of that point. $\endgroup$ – dtldarek Sep 20 '16 at 21:50
  • 1
    $\begingroup$ thoughts of the guy that discovered convolution? Do you know it was a guy? Okay, fine, it was Euler. $\endgroup$ – 6005 Sep 20 '16 at 21:51
  • $\begingroup$ With the keywords convolution and intuition, you can get many answers on the web, for example the answer of the Fields medallist Terry Tao in (mathoverflow.net/q/5892) $\endgroup$ – Jean Marie Sep 20 '16 at 21:59
  • 1
    $\begingroup$ @6005 Many people have "discovered" convolution by "turning around the subject" (I would be interested by a precise reference of such a paper by Euler) but recognizing it as a genuine operation on certain sets of functions could hardly be traced back before 1906 (I just checked the date) with Hilbert's "faltung" (german name for "convolution"). $\endgroup$ – Jean Marie Sep 20 '16 at 22:09
  • $\begingroup$ Convolutions also arise in signal processing applications, For some "intuition" as to "Why convolution?", see this answer of mine on the Signal Processing site dsp.SE. $\endgroup$ – Dilip Sarwate Sep 20 '16 at 22:48
1
$\begingroup$

Well, here I want to give you just an intuition not a whole proof and abstract notions (convolution products are defined in abstract algebra), so it is a good way to start with Fourier transform. Let's begin with this theorem

$\hat{f}.\hat{g}=\mathcal{F}[\frac{1}{2\pi}f*g]$

where $\hat{f}$ and $\hat{g}$ are the Fourier transform of the functions $f$ and $g$ respectively and $\mathcal{F}$ is the Fourier transform symbol.

$\hat{f}=\frac{1}{2\pi}\int_{-\infty}^{\infty}f(x)e^{i\omega x}dx$

$\hat{g}=\frac{1}{2\pi}\int_{-\infty}^{\infty}g(x)e^{i\omega x}dx$

We just want to know "Where did convolution formula come from?" So we just need to accept the identity above and extend it to see what happens?

$\hat{f}.\hat{g}=\frac{1}{2\pi}\int_{-\infty}^{\infty}f(x)e^{i\omega x}dx .\frac{1}{2\pi}\int_{-\infty}^{\infty}g(x)e^{i\omega x}dx$

$=\frac{1}{4\pi^2}\int_{-\infty}^{\infty}f(x)e^{i\omega x}dx\int_{-\infty}^{\infty}g(x)e^{i\omega x}dx$

Now if we extend the right hand side of the first equation mentioned, we obtain:

$\frac{1}{2\pi}(\frac{1}{2\pi}\int_{-\infty}^{\infty}(f*g)e^{i\omega x}dx) = \frac{1}{4\pi^2}\int_{-\infty}^{\infty}(f*g)e^{i\omega x}dx$

In the equality the term $\frac{1}{4\pi^2}$ is omitted from both side and we obtain:

$\int_{-\infty}^{\infty}f(x)e^{i\omega x}dx\int_{-\infty}^{\infty}g(x)e^{i\omega x}dx=\int_{-\infty}^{\infty}(f*g)e^{i\omega x}dx$

Now we can ask "How can we have the product of two different integrals in only one integral?" (Note that by accepting the convolution and the theorems of Fourier transform we got here!).

Let's make it easier by writing $\int_{\Omega}\phi d\mu(x) \int_{\Omega}\psi d\mu(x)$ and how can we make them into one integral. We know from real analysis that we can write the integral as:

$\int \phi d\mu = \sum_{1}^{n}a_j\mu(E_j)$ and $\int \psi d\mu = \sum_{1}^{n}b_j\mu(E_j)$

where $E_j$ are elementary sets of the domain. For easier manipulation (this is just an intuition not a proof!) we suppose that for all $E_j$ we have $\phi(E_j)=a_j$ and $\psi(E_j)=b_j$ and with assumption that $\mu(E_j)=\mu(E_i)=1$ for all $i$ and $j$ ($\mu$ here is the measure of the sets $E_j$) then we can write

$\int \phi d\mu\int \psi d\mu = \sum_{1}^{n}a_j\mu(E_j)\sum_{1}^{n}b_j\mu(E_j)=$

$(a_1+a_2+\dots+a_n)(b_1+b_2+\dots+b_n)=$ $a_1(b_1+b_2+\dots+b_n)+a_2(b_1+b_2+\dots+b_n)+\dots+a_n(b_1+b_2+\dots+b_n)$

For the first term $a_1(b_1+b_2+\dots+b_n)$, we see that $a_1$ is multiplied in all values of the function $g$ and this process is repeated for other terms. This is like translating the function $f$ (compute its values in $E_j$s) and then multiply it by function of $g$ then sum them (integrate the function $f(x-t)g(x)$ over the translation domain).

Now we can understand the meaning of convolution: it is a way to take the product of two integrals into only one integral, and for doing that, you need to translate one function (like $f$) then multiply it by the other ($g$) and then sum the multiplications (integrate them).

Any useful comment to complete the answer is appreciated.

| cite | improve this answer | |
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.