Visualization of neural network error surfaces and learning trajectories helps to understand the influence of network structure, initialization, training parameters, input data and other factors on the neural learning process. It also may be useful be designing and modifying MLP training algorithms. The first two principal components of the weight matrix are used to determine orthogonal directions that capture almost all variance in the weight space. 3-dimensional plots show important characteristics of the error surfaces. Many issues are addressed using several datasets for illustration: general properties of the weight vectors, interesting directions in the weight space, local minima, plateaus and narrow funnels on error surfaces. Reduction of the effective number of training parameters in the weight space reconstructed from a small number of principal components is discussed. Keywords: neural networks, MLP, Error functions, visualization of error surfaces, neural learning trajectory, backpropagation.

Preprint for comments in PDF, 1114 KB. Finall version to appear in Control and Cybernetics (submitted 6/04, accepted 10/04).

BACK to the publications of W. Duch.
BACK to the on-line publications of the Department of Informatics, NCU.