# An intuitive explanation for neural networks as function approximators ?

We use normal linear regression for modelling functions on datasets . But Can someone explain how neural networks help in approximating more complex ,especially non-linear functions ? intuitively , like what each layer adds to the whole process of approximation etc . Also , when does using artificial networks more prefferable ? What i'am looking for is an explanation of neural network as function approximators , and not a comparison with the biological neurons .

Thanks

-

There is some explanation for this in Duda and Hart's Pattern Recognition book. Look at section "6.2.2 Expressive power of multilayer networks". Directly quoting from there:

It is natural to ask if every decision can be implemented by such a three-layer network (Eq. 6). The answer, due ultimately to Kolmogorov but reﬁned by others, is “yes” — any continuous function from input to output can be implemented in a three-layer net, given suﬃcient number of hidden units nH , proper nonlinearities, and weights.

...

Speciﬁcally, Kolmogorov proved that any continuous function $g(x)$ deﬁned on the unit hypercube $I^n (I = [0, 1] \;\;\mathrm{and}\;\; n ≥ 2)$ can be represented in the form $g(x) = \sum_{j=1}^{2n+1} \Theta_j(\sum_{i=1}^d\psi_{ij}(x_i))$ for properly chosen functions $\Theta_j$ and $\psi_{ij}$. This equation can be expressed in neural network terminology as follows: each of $2n + 1$ hidden units takes as input a sum of $d$ nonlinear functions, one for each input feature $x_i$. Each hidden unit emits a nonlinear function $\Theta$ of its total input; the output unit merely emits the sum of the contributions of the hidden units.

-