Ensemble of Automatically-Designed Decision-Tree Induction Algorithms

• Grant number: 13/20058-3
• Support Opportunities: Scholarships in Brazil - Post-Doctoral
• Start date: January 01, 2014
• End date: February 28, 2014
• Field of knowledge: Physical Sciences and Mathematics - Computer Science - Computing Methodologies and Techniques
• Principal Investigator: André Carlos Ponce de Leon Ferreira de Carvalho
• Grantee: Rodrigo Coelho Barros
• Host Institution: Instituto de Ciências Matemáticas e de Computação (ICMC). Universidade de São Paulo (USP). São Carlos , SP, Brazil
• Associated research grant: 13/07375-0 - CeMEAI - Center for Mathematical Sciences Applied to Industry., AP.CEPID

Abstract

Decision-tree induction is one of the most employed methods to extract knowledge from data, since the representation of knowledge is very intuitive and easily understandable by humans. There are several distinct strategies for decision-tree induction, each one presenting advantages and disadvantages according to its corresponding inductive bias. These strategies have been continuously improved by researchers over the last 40 years, that is, there has been a continuous manual improvement of decision-tree design components throughout the years. Following recent breakthroughs in the automatic design of machine learning algorithms, a hyper-heuristic evolutionary algorithm called HEAD-DT has been proposed to automatically generate decision-tree induction algorithms. HEAD-DT works over several manually-designed decision-tree components which were developed in the last decades, and following a typical genetic search, it combines the most suitable components for the task at hand. In this project, we propose to significantly extend HEAD-DT in order to generate ensembles of automatically-designed decision-tree algorithms. We envision two possible frameworks for generating these ensembles: i) evolving a single decision-tree induction algorithm and generating the base classifiers that form the ensemble by modifying the training data set; and ii) evolving a set of distinct decision-tree induction algorithms, naturally generating distinct base classifiers to form the ensemble. Both frameworks will be compared to state-of-the-art ensemble methods and other classifiers such as SVMs and neural networks.