Mathematics Stack Exchange is a question and answer site for people studying math at any level and professionals in related fields. It's 100% free, no registration required.

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

As far as I remember a neural network cannot forget anything. Does this mean that no matters how evolved the network is, it's always going to throw me back the right output if I feed it an input?

And when I say "right output" I mean "precise output" without getting a bit wrong

share|cite|improve this question
up vote 2 down vote accepted

neural networks only have a finite capacity to "remember" (called capacity). Assuming that you don't keep trying to update it with new information, then it would theoretically 'last forever'. But if you keep adjusting the weights to learn new and different things, eventually it will forget old patterns.

share|cite|improve this answer
Damn... thank you for your help – John Smith Jul 12 '12 at 18:19
some networks have more memory than others mind you. But yeah, since a neural network stores a finite amount of information it can only remember finitely many things. – Robert Mastragostino Jul 12 '12 at 18:23

No. Neural nets do not have infinite memory; they have a finite storage capacity.

The storage capacity depends on the configuration of the net(structure, neuronal connection types, number of neurons etc.).

Take for instance a Hopfield net: training it using too many exemplars will result in interference between stored patterns and hence "memory loss". (Its storage capacity is logarithmically dependent on the number of neurons in the structure.)

Now, there are some architectures that are far better at retaining new information as it's presented e.g. Adaptive Resonance Theory, but ultimately there is still a memory limit.

share|cite|improve this answer

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.