Ensembled Geometric Semantic Genetic Programming: An ensemble-based initialization technique for Geometric Semantic Genetic Programming

Abstract

Machine learning is a scientific field that seeks to empower computers with the ability to learn without having to be explicitly programmed. Given the ever growing number of sophisticated intelligent machine learning algorithms, investigators can find themselves “stuck” in a time and resources consuming process of model searching, tuning and testing. Bearing this in mind, Ensemble Methods and meta-learning algorithms have emerged as an attempt to automate the process of combining several different models in an intelligent, adaptive way. Within the field of machine learning, a specific set of algorithms, called Evolutionary Algorithms, mimic Darwin’s Theory of Evolution. As their name indicates, these algorithms incorporate concepts of evolution to the task learning process. They do so by evolving a set of individuals (i.e., possible solutions to a given problem) under pressures of natural selection and "survival of the fittest" mechanisms. Genetic Programming (GP) is an Evolutionary Algorithm that evolves computer programs (i.e., individuals) in order to perform a mapping between input and output features. Its extension, Geometric Semantic Genetic Programming (GSGP), allows us to perform the evolution and variation of the individuals on the semantic space (i.e., the space in which the output vectors lay) rather than on their syntax based structure. The generation of the initial population, commonly known as the initialization, has been proven to be a key factor for both GP and GSGP’s performance. This work proposes an initialization technique for GSGP that utilizes the combinatory power of Ensemble Methods in order to generate a “fit” initial population based on the predictions of a wide variety of machine learning algorithms, called Base Learners, making use of GSGP’s ability to evolve individuals solely based on their semantics (i.e., predictions). This initialization technique is called Ensembled Geometric Semantic Genetic Programming as it utilizes GSGP as a learning combiner that merges the knowledge obtained from different machine learning techniques by evolving their semantics via crossing between and mutating them over the course of several generations. The performance of the proposed initialization method (EGSGP) was tested on three different case studies. Results show that EGSGP significantly outperforms traditional GSGP (with the exception of one case study where EGSGP’s improved performance in comparison to GSGP did not translate into statistically significant results). Additionally, results show that EGSGP produces better results than the best base learner from the initial population being, thus, able to improve it. This improvement was present in all case studies only yielding, however, statistically significant results in one of them.