Semi-supervised relevance index for feature selection

Coelho, Frederico [Universidade Federal de Minas Gerais, Belo Horizonte, Brazil] Castro, Cristiano [Universidade Federal de Minas Gerais, Belo Horizonte, Brazil] Braga, Antônio P. [Universidade Federal de Minas Gerais, Belo Horizonte, Brazil] Verleysen, Michel [UCL]

This paper presents a new relevance index based on mutual information that is based on labeled and unlabeled data. The proposed index, which is based in Mutual Information, takes into account the similarity between features and their joint influence on the output variable. Based on this principle, a method to select features is developed to eliminate redundant and irrelevant features when the relevance index value is less then a threshold value. A strategy to set the threshold is also proposed in this work. Experiments show that the new method is capable of capturing important joint relations between input and output variables, which are incorporated into a new feature selection clustering approach.