# Universal Approximation Theorem — Neural Networks

Universal approximation theorem states that "the standard multilayer feed-forward network with a single hidden layer, which contains finite number of hidden neurons, is a universal approximator among continuous functions on compact subsets of $R^n$, under mild assumptions on the activation function."

I understand what this means, but the relevant papers are too far over my level of math understanding to grasp why it is true or how a hidden layer approximates non-linear functions.

So, in terms little more advanced than basic calculus and linear algebra, how does a feed-forward network with one hidden layer approximate non-linear functions? The answer need not necessarily be totally concrete.

I also posted this question at TCS, and CV. Previously, no one had given a solution. But now, here is a really excellent and comprehensive answer.

• maybe cstheory.stackexchange.com or cs.stackexchange.com might be better suited for this question. – robjohn May 7 '13 at 22:02
• @robjohn I was also wondering if cstheory might be a better fit. What would be the best way to approach that possibility? I assume the correct procedure would be to migrate the question, but I'm a bit wary of possibly migrating to the wrong place. – Matt Munson May 7 '13 at 22:14
• Just repost this on cstheory. You can either delete the question here or leave it. If you leave it here, link it to the question on cstheory, and link to here from the question on cstheory. – robjohn May 7 '13 at 22:18
• @robjohn cool, thanks. – Matt Munson May 7 '13 at 22:19