Feature Selection: Traditional and Wrapping Techniques with Tabu Search

Abstract

Feature selection is an important step in improving the performance of machine learning algorithms. This paper describes a comparative study of traditional feature selection techniques and a wrapping technique with tabu search. To validate our wrapper with tabu search approach, we implemented three feature selection techniques: correlation, entropy, and principal component analysis. Nevertheless, to evaluate their performance, we used three classification algorithms: a J48 decision tree, a random forest, and an artificial neural network. We selected five datasets with a large number of features from public repositories. The experimental results showed that the subsets provided by tabu search have high performance with a J48 decision tree. Additionally, tabu search was better ranked than the other feature selection techniques. Finally, we consider that tabu search is a good alternative for feature selection paired with a simple classification algorithm.