This just hit me today. I am not too experienced with math or neural networks, but I am trying to find out about them in my own way so I can some day understand them well.
So I was thinking about how neural networks are connected to more familiar things that I know. This is just speculation, but I would like to know if I am on the right track at all.
Currently I think that neural networks are in fact "function adapters" (if that is the correct term in english) so that when they are learning, they are trying to adapt some invisible function so that the inputs match the wanted outputs.
If I wanted to do something like this by hand, I would of course adjust the terms of some function that I am adapting.
Like if I had a simple polynomial:
5x + 1
I would adjust the x until the function outputs what I want with the given input.
I think the x in this example might actually represent a weight in a neural network. It would make a lot of sense if this was the case!
And then there is the "back propagation", which I have not studied that much at all, but I think it has to do with correcting the other weights when adjusting one, because if the weights are the unknowns of a polynomial and I adjust some unknown in a polynomial - all the previous calculations would be off to account for this name input, because the old inputs use the same network / polynomial as the new input that the network / polynomial was just adjusted for. So this "back propagation" takes this into account and tries to minimize the error for the old inputs?
Am I on the right track here?
Are weights = unknowns in a polynomial
Is "back propagation" = Making sure that the polynomial gives same outputs for the old inputs, after the polynomial has been adjusted to work with a new input