Title: New transfer functions for neural networks

• Objective: The choice of transfer functions may strongly influence complexity and performance of neural networks. Provide systematic analysis and taxonomy of transfer functions, their advantages over standard solutions.
• Participants: Norbert Jankowski and Wlodzislaw Duch
• Date: From 1993 - present
• Papers: main and related

Main papers on transfer functions:

1. Duch W (2005) Uncertainty of data, fuzzy membership functions, and multi-layer perceptrons.
   IEEE Transactions on Neural Networks 16(1): 10-23.
   Relations between fuzzy logic and neural transfer functions are analyzed and a few new transfer functions derived.
2. Duch W, Jankowski N, Transfer functions: hidden possibilities for better neural networks. 9th European Symposium on Artificial Neural Networks (ESANN), Brugge 2001. De-facto publications, pp. 81-94
3. Duch W, Adamczak R, Diercksen GHF, Constructive density estimation network based on several different separable transfer functions. 9th European Symposium on Artificial Neural Networks (ESANN), Brugge 2001. De-facto publications, pp. 107-112
4. Jankowski N, Duch W, Optimal transfer function neural networks. 9th European Symposium on Artificial Neural Networks (ESANN), Brugge 2001. De-facto publications, pp. 101-106
5. Duch W, Adamczak R, Diercksen GHF, Neural Networks from Similarity Based Perspective.
   New Frontiers in Computational Intelligence and its Applications. Ed. M. Mohammadian, IOS Press, Amsterdam 2000, pp. 93-108
6. Duch W, Jankowski N, Taxonomy of neural transfer functions,
   IEEE, International Joint Conference on Neural Networks 2000 (IJCNN), Vol. III, pp. 477-484
7. Duch W, Jankowski N (1999) Survey of neural transfer functions,
   Neural Computing Surveys 2: 163-213, and a copy (PDF) here.
8. Duch W, Adamczak R, Diercksen GHF (1999) Neural Networks in non-Euclidean spaces.
   Neural Processing Letters  10: 201-210
9. Duch W, Adamczak R, Diercksen GHF (1999) Distance-based multilayer perceptrons.
   Computational Intelligence for Modelling Control and Automation. Neural Networks and Advanced Control Strategies. Ed. M. Mohammadian, IOS Press, Amsterdam, pp. 75-80
10. Duch W and Jankowski N (1996) Bi-radial transfer functions.
    Proceedings of the Second Conference on Neural Networks and their applications, Orle Gniazdo, 30.IV-4.V.1996, pp. 131-137

Papers related to the transfer functions:

1. Duch W, Towards comprehensive foundations of computational intelligence. | PDF file.
   In: W. Duch and J. Mandziuk, Challenges for Computational Intelligence. Springer Studies in Computational Intelligence, Vol. 63, 261-316, 2007.
2. Kordos M, Duch W (2004), A Survey of Factors Influencing MLP Error Surface.
   Control and Cybernetics 33(4): 611-631.
   Neural error functions are influence by the choice of transfer functions, but the primiary goal here is to visualize the training process.
3. Duch W, Blachnik M, Fuzzy rule-based systems derived from similarity to prototypes.
   Lecture Notes in Computer Science, Vol. 3316 (2004) 912-917.
   Introduces transformations between distance functions and fuzzy membership functions that are used as transfer functions in the basis expansion networks, such as RBF or separable functions networks.
4. Duch W, Grabczewski K, Heterogeneous adaptive systems IEEE World Congress on Computational Intelligence, Honolulu, May 2002, pp. 524-529.
5. Duch W, Similarity based methods: a general framework for classification, approximation and association,
   Control and Cybernetics 29 (4) (2000) 937-968
6. Duch W, Adamczak R, Diercksen G.H.F, Classification, Association and Pattern Completion using Neural Similarity Based Methods.
   Applied Mathematics and Computer Science 10:4 (2000) 101-120
7. N. Jankowski. Ontogenic neural networks and their applications to classification of medical data . PhD thesis, Department of Computer Methods, Nicolaus Copernicus University, Torun, Poland, 1999.
8. N. Jankowski. Approximation and classification in medicine with IncNet neural networks. In Machine Learning and Applications. Workshop on Machine Learning in Medical Applications, pages 53-58, Chania, Greece, July 1999.
9. N. Jankowski. Flexible transfer functions with ontogenic neural. Technical report, Computational Intelligence Lab, DCM NCU, Torun, Poland, 1999.
10. N. Jankowski. Approximation with RBF-type neural networks using flexible local and semi-local transfer functions. In 4th Conference on Neural Networks and Their Applications, pages 77-82, Zakopane, Poland, May 1999.
11. Duch W, Grudzinski K and Diercksen G.H.F (1998) Minimal distance neural methods.
    World Congress of Computational Intelligence, May 1998, Anchorage, Alaska, IEEE IJCNN'98 Proceedings, pp. 1299-1304
12. Duch W and Jankowski N (1997) New neural transfer functions.
    Applied Mathematics and Computer Science 7 (1997) 639-658 (invited by the Editor)
13. Duch W and Diercksen GHF (1995) Feature Space Mapping as a universal adaptive system Computer Physics Communications 87: 341-371
14. Duch W (1994) Floating Gaussian Mapping: a new model of adaptive systems.
    Neural Network World 4:645-654
15. Duch W (1993) On the optimal processing functions for neural network elements, UMK-KMK-TR 6/93 report.
16. Links to many talks and to other papers on various subjects.

Links to other Duch-Lab projects, many talks and to other papers on various subjects.

Working log (local accsess only), maintained by Wlodzislaw Duch.