Adding value to sensor data of civil engineering structures: automatic outlier detection

Abstract

This paper discusses the problem of outlier detection in datasets generated by sensors installed in large civil engineering structures. Since outlier detection can be implemented after the acquisition process, it is fully independent of particular acquisition processes as well as it scales to new or updated sensors. It shows a method of using machine learning techniques to implement an automatic outlier detection procedure, demonstrating and evaluating the results in a real environment, following the Design Science Research Methodology. The proposed approach makes use of Manual Acquisition System measurements and combine them with a clustering algorithm (DBSCAN) and baseline methods (Multiple Linear Regression and thresholds based on standard deviation) to create a method that is able to identify and remove most of the outliers in the datasets used for demonstration and evaluation. This automatic procedure improves data quality having a direct impact on the decision processes with regard to structural safety.