In machine learning a very common technique to use as a training algorithm (in NN) is the gradient descent rule. I understand that it is an iterative process of increasing each of the weights based on the partial derivative. Why could we not simply take partial derivative of all weights, set up set of lieanr equations, and solve them? Is it the computational cost?

-
my bad... spelling was not an issue, but funny comment indeed :) –  Bober02 May 6 '12 at 10:53