Take the 2-minute tour ×
Mathematics Stack Exchange is a question and answer site for people studying math at any level and professionals in related fields. It's 100% free, no registration required.

One of the things I'm curious about is why do some functions describe something like this:

$$f(x,y) = e^{-\frac{x^2+y^2}{2\sigma^2}}$$

And people mostly take it for granted, throw it around for various kernels for processing images... But nobody takes the time to explain why there's a fraction in the exponent. What does it do? Square $e$ by the squares of inputs and then take a doubled square of sigma - root? What does that even mean?

Is there even any point of doing "the math" in your head for these wild equations? $f(1,1)$ with a $\sigma = 1$ is not that hard, it's $\frac{1}{e}$. Not to mention bigger inputs which are not so friendly.

share|improve this question
1  
$\sigma$ denotes the standard deviation of the corresponding random variable. See en.wikipedia.org/wiki/Normal_distribution (which describes mostly the one-variable case but there is a generalization to $n$ variables). –  Qiaochu Yuan Apr 2 '12 at 20:42
    
Err, $f(1,1)$ with $\sigma=1$ is $1/e$. –  anon Apr 2 '12 at 20:50
    
Apologies, I slipped up with the numerator. Indeed, it's $1/e$. –  ElcorAllFour Apr 2 '12 at 21:02
7  
Is this really worth down voting? I understand refraining from an upvote, but the OP is asking a question that he appears to have thought about and was unable to resolve, and it's not unclear what he is asking, which perhaps might just be rephrased of "what is the justification for using this kernal?" I just don't want to be scaring away well-intentioned newbies with answerable questions by down voting. –  guy Apr 2 '12 at 21:24
1  
The basic idea here is that $f(z) = e^{-z^2/2}/\sqrt{2 \pi}$ is the density for the standard normal random variable (mean $0$ and standard deviation $1$). For a normal random variable with mean $0$ but standard deviation $\sigma$ you scale the variable, so it's $e^{-(x/\sigma)^2/2}/\sqrt{2\pi \sigma^2}$. For the joint density of two independent normals you multiply the densities: $e^{-(x/\sigma)^2/2} e^{-(y/\sigma)^2/2} = e^{-(x^2+y^2)/(2 \sigma^2)}$ –  Robert Israel Apr 2 '12 at 21:28

2 Answers 2

up vote 14 down vote accepted

I understand why you are frustrated. A lot of people try to elude the extra mile of explaining the mathematics they employ or even understanding them.

Just like with software libraries, people use them either because they don't know or they are too lazy to write them personally. Truth be told, there are those who are on a deadline and don't have the time to reinvent the wheel.

And some are just rude and try to hide their ignorance behind harsh words. Although, sometimes it is tiresome when people don't do the research beforehand and demand an answer quickly. Fortunately, I'm in a good mood, so I'll try to assist you. And, to be honest, you did try to find an answer and are now asking a legitimate question formulated to the best of your abilities and knowledge of the codomain you're trying to get into from your predicament or domain (pun intended). So, let us try to get you there.

</rant>

I will assume that you're not versed (enough, at least) in the formal approach to mathematics, so I am going to try to demonstrate it in a more freeform, intuitive manner. As it should be done with those unfamiliar with the topic. A small example, if you will: "a regular grid is formed/consists of congruent parallelotopes" might mean nothing to you, but saying "A foundation is formed of matching bricks" yields a quick intuitive representation of what it is, all the while clearing the air of a more formal, precision-wise definition. It is crude, a bit imprecise, but effective, as Seven of Nine would put it.

It seems to me that you're not that interested in the actual normal distribution, but more in the "construction" of a bell curve and its inherent nature - the "wild exponent".

So, the famed $$ f(x) = ae^\frac{(x-b)^2}{2c^2}$$

What most people miss is that there is nothing intentionally magical about them. They were a natural consequence of human curiosity. It had been just a matter of time before someone tried to put a quadratic equation as an exponent to a reoccuring buddy of every mathematician:

$$\lim_{n \rightarrow \infty} (1 + \frac{1}{n})^n = L$$

$$ L = e = 2.7182818284590452353602874$$

$e$ is also not magical. It's just a repercussion of how we shaped mathematics over the centuries. It is accidental, but important. It's sort of like you, just ask your parents. Jokes aside, it was discovered by one Mr. Jacob Bernoulli, from the distinct family of Swiss scientists and mathematicians. And yes, his bro was Daniel. He was attempting to resolve some issues inherent to compound interest when he first ran into $e$. Many others have found it since and have given it different names to reference it in their work. One working with mathematics without any assistance from others, totally agnostic of the Euler's name $e$ could give it a totally different letter, like $p$. Before Euler assigned it the letter $e$, 2.7182818... had been known under the alias $b$.

You might be asking yourself why use e to model so much phenomena? Why not use an arbitrary number? That's a genuinely good question. $e$ has some very neat properties. When people delved into exploring exponentional functions (base $e$) they discovered that the derivative of $e^x$ is - get this - $e^x$.

The slope at any point along the $e^x$ curve is exactly equal to the yield of of the function $e^x$, or the y-coordinate of the ordered pair or simply - the height. Every other base $n^x$ yields a derivative of $n^xlog(n)$. So, $e$ was recognized as a special fellow very early on. The potential has been proved year after year while we discover many of its intricate uses.

So, let's get to the gist of it - colorfully named the "wild" exponents of $e$. Indeed, wild they are. As I said a moment ago, the gaussian function (the function that is currently bothering you in the question) was developed on top of the first attempts to apply a quadratic function as its exponent.

$$e^{ax^2 + bx + c}$$

And the result was interesting. But not that useful. How about we swap the leading coefficient $a$ with $-a$?

$$e^{-ax^2 + bx + c}$$

Well, that's a bit better, now we have a concave graph which is starting to resemble the "bell curve". You really have to appreciate computers today, we can simply analyze this function for many different coefficients and see how the function behaves in real time. For example, $a$ controls the width of the bell, $b$ controls the peak (modulated against the x-axis), $c$ defines a pure offset which is invariant of input.

Now, people wanted to express various things and one of the most prominent ones were located within statistics. The most violated example is IQ distribution over a sample of people. See, when you have a function such as that bell curve it really lends itself well to describing such phenomena. Remember coefficient $b$? It governs where the peak is located of our bell. Where the highest amount of something is located at exactly a specific $x$. People are the "highest amount of something" and that b governs the location of the bell over the values of IQ, a peak. An average IQ. 100. Get the picture?

Coefficients $a$ and $b$ can be linked into something useful. If $a$ governs the width between the y axis and the one side of the curve (hyperbola), then the total width will be expressed as $2a$. But remember that while the leading coefficient is truthful to the width, it is serving another purpose - keeping the graph concave. So - $w = -2a$. This gives us the possibility to calculate the mean by using those two coefficients. Mean = average.

$$\mu = \frac{b}{-2a}$$

Also, there is the concept of variance $\sigma^2$ which I won't delve into, but it can be logically replaced within the original concept of a function to make it more sensible to the problem of determining distribution. Suffice it to say, it describes how tightly the distribution of people are around the average IQ, or the mean. It is defined as $1/-2a$ which relates it to the width of the bell curve - logically.

Now, since the function describes 100% of the sample, you can't have a surface under the curve greater than - well - 1. You have to account for exactly 100% of the population or the sample. Otherwise it makes no sense. This isn't Putin's function. That means you've got to modulate the width with the variance to make sure it stays uniformal and true to the "100% rule". Therefore, you modulate the width with the variance to keep it within 1/2 both sides and "corrected" by the variance, adding up to a 1. Some comments noted that this also works to protect dimensionality of units (if used as opposed to the concept of analyzing a whole within statistics).

$$f(x) = \frac{1}{\sqrt{2\pi\sigma^2}}e^{-\frac{(x-\mu)^2}{2\sigma^2}}$$ $$f(x) = \frac{1}{\sigma\sqrt{2\pi}}e^{-\frac{(x-\mu)^2}{2\sigma^2}}$$

The first term $\frac{1}{\sigma\sqrt{2\pi}}$ is one of the "security" measures that enforce the Gaussian integral to resolve to a promised agreed upon 100% total. Study the Gaussian integral to find out more about integrating the Gaussian function which doesn't play ball with standard procedures.

That's the price of having an awesome constant like $e$. Anyways, this is really a simplification. Maybe even a bit too much, but if it helps you grasp the intricacies in the future, my time here was well spent.

In mathematics, everything makes sense. You just have to look for it. And be persistent. I've tried my best to reduce blasting you with symbol-riddled equations, formal talk and procedures since your interest is purely understanding. And remember - the Internet is your friend. There's a lot of data hidden. And you have to try real hard if you wish to understand more.

And that's the simple tale of the wild exponents of $e$.

share|improve this answer
    
I stopped hoping that anyone would do this question "justice". Thank you, kind sir! It maybe isn't the most formal of answers, but I really enjoyed it. Quite thorough! –  ElcorAllFour Apr 3 '12 at 1:47
    
You're most welcome! –  Domagoj Pandža Apr 3 '12 at 7:10

One way to interpret the necessity of a fraction is in the sense of units. If say $x$ and $y$ are measured in meters, then the argument of the exponential must be unitless, hence $\sigma$ must have the same units. Indeed, if you think in terms of probability, then $\sigma$ in the sense of standard deviation works for this.

share|improve this answer
    
Thank you for your input. You've made some nice points. –  ElcorAllFour Apr 3 '12 at 1:46

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.