The problem of extraction of crisp logical rules from neural networks trained with backpropagation algorithm is solved by transforming these networks into simpler networks performing logical functions. Two constraints are included in the cost function: regularization term inducing weight decay and additional term forcing the remaining weights to +/- 1. Networks with minimal number of connections are created, leading to a small number of crisp logical rules. A constructive algorithm is proposed, in which rules are generated consecutively by adding more nodes to the network. Rules that are most general, covering many training examples, are created first, followed by more specific rules, covering a few cases only. Generation of new rules is stopped when their application on the test dataset does not increase the number of correctly classified cases. Our constructive algorithm applied to the Iris classification problem generates two rules with three antecedents giving 98.7% accuracy. A single rule for the mushroom problem leads to 98.52% accuracy while three additional rules allow for perfect classification. The rules found for the three monk problems classify all the examples correctly.
International Joint Conference on Artificial Neural Networks (IJCNN'97), Houston, 9-12.6.1997, pp. 2384-2389

Paper in PDF format, 196KB

Projects on similar subject and BACK to the on-line publications of W. Duch.