3
$\begingroup$

I can imagine a lot of math's theorems and laws, but I can not imagine the "process" universal approximation theorem is talking about.

1) Can you explain in simple words the proof of universal approximation theorem?

2) Why neural networks with 1+ hidden layers are working?

3) Are neural networks analog of math's series?

$\endgroup$
  • $\begingroup$ In Wikipedia's terminology, you can approximately expand a continuous real-valued function on the unit hypercube into a finite linear combination of functions of the form $\varphi \circ a_i$ where $a_i$ are affine functions and $\varphi$ can be taken to be a single, rather "nice", function. If my limited understanding is correct, these $a_i$ form the "hidden layer" of the network. And yes, this expansion of a "generic" function into a sum of "nice" functions is similar to expansion into power series, Fourier series, or wavelet series. $\endgroup$ – Ian Mar 20 '17 at 12:31
  • $\begingroup$ @Ian I need the proof $\endgroup$ – Dmitry Nalyvaiko Mar 20 '17 at 12:35
  • $\begingroup$ Surely it can be looked up? Indeed the Wikipedia article gives 5 references... $\endgroup$ – Ian Mar 20 '17 at 12:36
  • $\begingroup$ @Ian my brain very poorly delves into the scientific terminology, that's why I ask here in the hope of getting an understandable explanation $\endgroup$ – Dmitry Nalyvaiko Mar 20 '17 at 12:39
3
$\begingroup$

This chapter gives nice geometric intuition. And you can find the proof easily around, follow Wikipedia references. The above mentioned book is generally good for understanding basics of neural networks.

$\endgroup$
  • $\begingroup$ ..What chapter? $\endgroup$ – MadHatter Oct 28 '17 at 15:51

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.