I am watching the Neural Network videos by Prof. Geoff Hinton. In there he talks about the problem with elliptical error surfaces and how they can be transformed to circular surfaces.

Slide: enter image description here

Link to the timestamped video:

1) Why is the first case with 101, 99 so elongated? Is it because the inputs are large (101, 99) as compared to output (0, 2)? I thought weights (not inputs) are the axes in the error surface. So, the elongation should depend upon discrepancies in weights.

2) Why are the output lines (red and green) almost parallel in the first case and almost perpendicular in the second case?



You must log in to answer this question.