Mathematics Stack Exchange is a question and answer site for people studying math at any level and professionals in related fields. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

Universal approximation theorem states that "the standard multilayer feed-forward network with a single hidden layer, which contains finite number of hidden neurons, is a universal approximator among continuous functions on compact subsets of $R^n$, under mild assumptions on the activation function."

I understand what this means, but the relevant papers are too far over my level of math understanding to grasp why it is true or how a hidden layer approximates non-linear functions.

So, in terms little more advanced than basic calculus and linear algebra, how does a feed-forward network with one hidden layer approximate non-linear functions? The answer need not necessarily be totally concrete.

I also posted this question at TCS, and CV. Previously, no one had given a solution. But now, here is a really excellent and comprehensive answer.

share|cite|improve this question
maybe or might be better suited for this question. – robjohn May 7 '13 at 22:02
@robjohn I was also wondering if cstheory might be a better fit. What would be the best way to approach that possibility? I assume the correct procedure would be to migrate the question, but I'm a bit wary of possibly migrating to the wrong place. – Matt Munson May 7 '13 at 22:14
Just repost this on cstheory. You can either delete the question here or leave it. If you leave it here, link it to the question on cstheory, and link to here from the question on cstheory. – robjohn May 7 '13 at 22:18
@robjohn cool, thanks. – Matt Munson May 7 '13 at 22:19

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Browse other questions tagged or ask your own question.