Shannon entropy used in standard top-down decision trees does not guarantee the best generalization. Split criteria based on generalized entropies offer different compromise between purity of nodes and overall information gain. Modified C4.5 decision trees based on Tsallis and Renyi entropies have been tested on several high-dimensional microarray datasets with interesting results. This approach may be used in any decision tree and information selection algorithm.
Key words: Decision rules, entropy, information theory, information selection, decision trees.

Preprint for comments in PDF, 111 KB.

Reference: Maszczyk T, Duch W, Comparison of Shannon, Renyi and Tsallis Entropy used in Decision Trees.
Lecture Notes in Computer Science, Vol. xxx, pp. xxx-yyy, 2008

BACK to the publications of W. Duch.
BACK to the on-line publications of the Department of Informatics, NCU.