# Mathematical explanation behind a picture posted (lifted from facebook)

In this image given below, there is an actor's (famous south Indian actor Rajinikanth) image which can be seen only if you shake your head ! I had lifted this from Facebook.

I am just curious to know if there is any mathematical explanation for it. Is there any way to know how this image was created in the first place.

PS : If this question (although interesting) is inappropriate here, it is still okay and i hope it could be migrated to some stackexchange site.

ADDED

(...experiment to prove that it is a physical phenomenon)

After some comments expressing doubt as to whether this is a physical phenomenon, I have done a small experiment using a simple camera. I have have shot photos of the picture displayed on a LCD monitor in two different cases. In case-1, the camera was still and in case-2 the camera was shaking in a circular arc (in to and fro) about and axis in the vertical plane passing through the centre of the camera. (just as like we shake our head). I did it with hand just as we shake our head. I have given the photos below.

Case-1 (still camera)

Case-2 (shaking camera)

It can be observed that the face of Rajini is more clearly visible in Case-2 (shaking camera) than in the Case-1 (still camera) where the face is not clearly visible.

PS : Now there is really no need to shake our head.

Added 2

after a recommendation by Willie, here, I have added Case-3 where the camera is shaking vertically (parallel to the stripes in the picture).

It can be observed that not much of an effect there when camera is shaking parallel to the stripes.

Case-3 (shaking camera vertically)

-
What sort of mathematical explanation are you looking for? –  user9413 Oct 11 '11 at 15:28
Just for your information, you can also see the photo if you unfocus your vision, no need to shake your head and cause unnecessary headaches. –  Asaf Karagila Oct 11 '11 at 15:29
It's not mathematics, it's physiology. They drew a grid of crisp black bars across a faint photograph. In mechanistic language: as long as you see the image clearly, the eyes' built-in edge-detection signal overwhelms everything else, but if you move your head quickly from side to side there isn't time for the edges to be detected, and the vision falls back on average brightness. –  Henning Makholm Oct 11 '11 at 15:32
Actually, if I tilt my LCD screen, I can see the image. No need for head-shaking! –  Guess who it is. Oct 11 '11 at 15:37
@Rajesh: I fail to see the connection between your question and the one you link to. Music is well-known to have links with mathematics. Optical illusions...less so. –  user1729 Oct 11 '11 at 15:55

## 2 Answers

What you are seeing is a physical manifestation of the mathematical operation known as the convolution.

First let me show you some pictures; we'll get into the mathematics afterwards. We start with the original

I take the image, desaturated the colours, and duplicated another layer, and pixel-wise added the layers after some translation. With a horizontal translation that is half the "wavelength" of the black bars, we get

With a translation of the same number of pixels, but vertically, we get

and finally, a diagonal translation at -45 degrees.

So what is going on? Why did I say that this is a manifestation of convolution?

Recall that the convolution of two functions defined on (say) the real line $\mathbb{R}$ is defined to be

$$f * g (x) = \int_{\mathbb{R}} f(y) g(x-y) dy$$

In a course in Fourier analysis, one is taught to emphasize that this is the dual operation of multiplication. That is, convolution in physical space corresponds to (point-wise) multiplication in Fourier space. This immediately gives the following interpretation of a convolution in signal processing:

Convolving a signal $f$ by a function $\psi$ is the same as applying a frequency dependent filter $\hat{\psi}$ to the signal $f$.

Another way of looking at the convolution, however, after staring at the above definition for a bit, is that

A convolution is a way of taking weighted average of a signal with its translates. The weight depends on the amount of translation.

It is in this second sense that we will first look at the phenomenon you asked above. In the second image of this post, I averaged the signal with its translation horizontally by half the wavelength of the black bars. Hence this is a convolution. Similarly, in the third/fourth image of this post, I averaged the original with a vertical/diagonal translation. They are also convolutions. And you see that this reproduces the observation you made that the direction in which you shake your head/camera produces an effect on the image seen/captured.

So how is the process of shaking your head of shaking a camera a process of convolution? The idea is that the image you see with your eyes and you capture with a camera do not come from photons all emitted at the same instant in time (special relativity notwithstanding). In your vision, there is the well-known phenomenon of persistence of vision which posits that the perceived image is actually made up of photons arriving in a 40 millisecond interval. Similarly, the shutter-speed of a camera determines how long a camera registers light, and so a camera set on 1/25 for the shutter-speed will "open its eye" for 40 milliseconds, and the image registered on the CCD or on film will be photons arriving in that window.

Now, if you shake your head or camera so that the retina or the CCD or the film moves significantly during that 40 milliseconds, each of your retina cell, each of the photoelements on the CCD, or each of the dye pigments on the film will be exposed to photons originating from different spatial positions. (I am grossly simplifying here, but that's the moral of the story.)

To summarise: your eyes and cameras already take convolution of the incoming signal in time when they compose the image. By shaking the apparati you convert the temporal convolution to a spatial convolution. Which means that you are taking a weighted average of the image and its spatial translations, which is why what you see and capture on camera can be analogously described by digitally manipulating the image via an averaging/convolution procedure.

Note that this corresponds somewhat with Henning's comment to your question. The "eye's edge detection" he mentions is, roughly speaking, a description of how the eye is sensitive to different spatial frequencies of a signal (not to be confused with the actual electromagnetic frequencies with determines the colour). By shaking your head you apply a convolution operator, which in frequency space introduces a cut-off for high spatial frequency components. Buy reducing the high spatial frequency components, your eye is forced to get its information from the lower-frequency components in which the image of the Indian Actor hide. (There's some technical inaccuracies in this paragraph about how human physiology works and how it interacts with the shaking of the head, but I think this simpler picture illustrates the idea better.)

At this point I should mention that the idea of taking spatial convolutions of images and the exchange between temporal and spatial convolutions with the motion of the camera is not only useful for optical illusions. It actually has industry application in automatic image deblurring.

-
Nice explanation! +1. –  Jonas Teuwen Oct 19 '11 at 13:01
You have increased my love for math. –  The Chaz 2.0 Oct 19 '11 at 14:03
A most excellent essay! –  ItsNotObvious Oct 19 '11 at 19:55
I suspect that convolution plays a similar role in explaining the phenomenon of holography. –  Mike Jones Jan 2 '12 at 23:54
@MikeJones They absolutely do, that's essentially all it is in fact. en.wikipedia.org/wiki/Fourier_optics –  Steven-Owen Jun 16 '12 at 13:34

(this is in response to the answer by Willie..I have added this here as i couldn't insert an image in the comments...i hope it is ok in this special case.)

The notion of spatial frequency has been used by Willie in his answer. He mentions that the information pertaining to the face of the actor is present in the lower spatial frequency components, and the stripes correspond to high spatial frequency components. According to his answer, when we shake the head/apparati, we are convolving the picture with a low pass filter with some cut off frequency, and this operation allows only lower spatial frequency components in the final image there by forcing the eye to interpret the information in the lower spatial frequency components which is nothing but the face of the actor.

Here I argue that the notion of spatial frequency is not useful in all circumstances, for example consider an image shown below. The upper half of this image is taken from the original image and the lower half is taken from the second image of the answer by Willie. The second image of the Willie's answer is result of convolving the original image with a low pass filter. Hence it does not contain high spatial frequency components.

The new image formed here is a combination of two images. The upper half contain high spatial frequency components in the form of stripes. The lower half does not contain high spatial frequency components. But if we consider the entire image as one signal then it contains high spatial frequency components. But there are no stripes in the lower half of it and the lower half of the actor's face is clearly visible without any need for head shaking. In order to view the upper half of the actor we still need head shaking. In this case spatial frequency is of no use to characterize the stripes. If there are high spatial frequency components then we can say that there could be stripes, but we cannot say whether they are present only in the upper half or lower half or everywhere. Simply speaking the spatial frequency notion cannot give any information about where the stripes are present in the image. But the human eye does so well that it can see the face in the lower half of the image where the stripes are not present !

-
actually, the one thing you didn't take into account is that in your case you can do separation of variables. In essence the shaking of head in one preferred direction means that your are convolving against a function which is constant in the complementary direction. In other words, you are only taking a frequency filter in the $x$ direction, applying to one constant $y$ slice at a time.... –  Willie Wong Oct 19 '11 at 19:37
@Willie : I am not able to get what intended in your comment in the current context. I agree that here the stripes only in one direction are considered. But my point of argument is not about the direction of stripes or the separability of variables and i deliberately want to keep this issue aside and consider only the case of vertical stripes. I intend to say that the presence of stripes is indicated in the Fourier spectrum of the signal (my argument applies irrespective of the dimensionality of the signal) by the presence of high frequency components....... –  Rajesh D Oct 21 '11 at 4:09
...But the Fourier spectrum does not give any idea of where the stripes are present. (whether everywhere are only at certain locations). But from the figure i've posted, it can be seen that the presence of stripes affect the vision only ate the vicinity of location where the stripes are present. The bottom half of the picture is still clear even though the stripes are present in upper half. This localization property is not evident from the use of Fourier transform and hence the explanation using Fourier transform and convolution, in my opinion does not give the full picture. –  Rajesh D Oct 21 '11 at 4:12
what you are looking at is related to microlocalisation and separation of variables. The latter I already explained up top: if you take only the Fourier transform in the $x$ variable, you will see that you have functions $\tilde{f}(\xi,y)$ where the "high frequency" components from the black bars only occur for certain $y$. But more importantly, you see to be assuming that I meant that your eye is taking the two-dimensional Fourier transform of the signal and applying some sort of filter to it, when that is not at all what is meant. The point is that the "persistence of vision" allows –  Willie Wong Oct 21 '11 at 5:42
certain "time integrals" be expressed equivalently as certain "space integrals" which can then be seen to be a convolution, which has a Fourier space interpretation. But the convolution can be taken to be a purely physical space phenomenon, and I mentioned the Fourier space interpretation only to show you the two sides of the same coin: in this case it (weightly) averages pixels with nearby ones. I don't see how your picture does anything to contradict that. –  Willie Wong Oct 21 '11 at 5:45