Variable Step Search MLP Training Method

Miroslaw Kordos¹ and Wlodzislaw Duch^2,3
¹Department of Biomedical Informatics, Children's Hospital Research Foundation, Cincinnati, Ohio, USA
²Department of Informatics, Nicolaus Copernicus University, Grudziadzka 5, 87-100 Torun, Poland.
³School of Computer Engineering, Nanyang Technological University, Singapore.

Abstract.

MLP training process is analyzed and a variable step search-based algorithm (VSS) that does not require gradient information is introduced. This algorithm finds rough position of the minima in each single weight direction, and successively updates the weights. Only a small fragment of the network is analyzed for each update, making the method computationally efficient. The VSS algorithm is simpler to program than backpropagation, yet the quality of results and the speed of convergence are at the level of state-of-the-art Levenberg-Marquardt and scaled conjugate gradient algorithms.

Preprint for comments in PDF, 160 KB.
Reference: Kordos M, Duch W, Variable Step Search MLP Training Method. International Journal of Information Technology and Intelligent Computing 1 (2006) 45--56

BACK to the publications of W. Duch.
BACK to the on-line publications of the Department of Informatics, NCU.