Automatic identification of architecture smell discussions from stack overflow

Abstract

Architecture Smells (ASs), as one source of technical debt, indicate underlying problems at a high level of systems and negatively impact various system qualities, such as maintainability and evolvability. Detecting and refactoring ASs requires the relevant architectural knowledge and experience. Therefore, gathering the knowledge of ASs from various sources can facilitate ASs detecting and refactoring. However, manually identifying AS knowledge is time-consuming. Automatically and correctly identifying AS-related posts from Stack Overflow is a step toward utilizing the AS knowledge to help developers better maintain their systems. In this work, we propose an approach to automatically identify AS-related posts from Stack Overflow (SoF) by using machine learning algorithms. We evaluate the performance of 12 classifiers based on 3 feature extraction techniques and 4 classification algorithms with a created dataset of SoF posts (including 208 AS-related posts and 187 AS-unrelated posts). The results demonstrate that the SVM algorithm with Word2Vec achieved the best overall performance with an accuracy of 0.650, a precision of 0.613, a recall of 0.905, and an F1- score of 0.731. These results imply that the obtained model of the AS-related posts identification can be used to aid developers and researchers in collecting AS discussions from SoF.