A Global Indicator for Measuring the Efficiency of Machine Learning Classifier Based on Multi-Criteria Approach

Abstract

The main challenge that faces any researcher in the field of machine learning is determining the quality of an indicator used for measuring the efficiency of classifier techniques. This issue based on Multiple-Criteria Decision Making (MCDM) has not been tackled by any researcher until now. The previous work concerned with a single classical criterion (Accuracy Level) ignoring other important criteria in real-life. This paper presents a novel indicator for measuring the efficiency of classifier techniques. This measure is a global indicator with multi-criteria approach based on the technique for preference by similarity to the ideal solution (TOPSIS). This indicator is characterized by its ability to taking in account all previous criteria. In addition, two novel criteria are created by authors: Learning Efficiency Ratio (LER), and the CPU time efficiency. The classifiers evaluation process includes the classical classifiers: Support Vector Machines (SVM), Multi-layer perceptron (MLP), Gene Expression Programming (GEP), Single Decision Tree (STR), and the techniques that achieved the best results in literature. Inaddition, the latest classifiers: Tropical Collective Machine Learning (TCML), and Dempster-Shafer Collective Machine Learning (DSCML) using the proposed indicator. The comparison is performed using twenty-five standard datasets (benchmarks). The results supported by statistical analysis (T-test) show the efficiency and effectiveness of the proposed global indicator for selecting the best classifier and its ability to measure the classifier efficiency based on multi-criteria. Results promise the optimistic use of the global indicator in the classifiers evaluation process for real-life problems.