3D Point Cloud; Voxel; Feature extraction; Instance segmentation; Classification; 3D semantics; Ontology; Deep Learning
Abstract :
[en] Automation in point cloud data processing is central for efficient knowledge discovery. In this paper, we propose an instance segmentation framework for indoor buildings datasets. The process is built on an unsupervised segmentation followed by an ontology-based classification reinforced by self-learning. We use both shape-based features that only leverages the raw X, Y, Z attributes as well as relationship and topology between voxel entities to obtain a 3D structural connectivity feature describing the point cloud. These are then used through a planar-based unsupervised segmentation to create relevant clusters constituting the input of the ontology of classification. Guided by semantic descriptions, the object characteristics are modelled in an ontology through OWL2 and SPARQL to permit structural elements classification in an interoperable fashion. The process benefits from a self-learning procedure that improves the object description iteratively in a fully autonomous fashion. Finally, we benchmark the approach against several deep-learning methods on the S3DIS dataset. We highlight full automation, good performances, easy-integration and a precision of 99.99% for planar-dominant classes outperforming state-of-the-art deep learning. [fr] L'automatisation du traitement des données dans le nuage de points est essentielle pour une découverte efficace des connaissances. Dans cet article, nous proposons un cadre de segmentation des instances pour les ensembles de données des bâtiments intérieurs. Le processus est construit sur une segmentation non supervisée suivie d'une classification basée sur une ontologie renforcée par l'auto-apprentissage. Nous utilisons à la fois des caractéristiques basées sur la forme qui n'exploitent que les attributs X, Y, Z bruts ainsi que la relation et la topologie entre les entités voxel pour obtenir une caractéristique de connectivité structurelle 3D décrivant le nuage de points. Ces caractéristiques sont ensuite utilisées par le biais d'une segmentation non supervisée basée sur un plan pour créer des groupes pertinents constituant l'entrée de l'ontologie de classification. Guidées par des descriptions sémantiques, les caractéristiques de l'objet sont modélisées dans une ontologie par OWL2 et SPARQL pour permettre la classification des éléments structurels de manière interopérable. Le processus bénéficie d'une procédure d'auto-apprentissage qui améliore la description de l'objet de manière itérative et totalement autonome. Enfin, nous comparons l'approche à plusieurs méthodes d'apprentissage approfondi sur l'ensemble de données S3DIS. Nous soulignons l'automatisation complète, les bonnes performances, la facilité d'intégration et une précision de 99,99% pour les classes à dominance planaire, ce qui surpasse l'apprentissage profond de pointe.
Antoniou, G., Harmelen, F. Van, 2004. A semantic web primer. Armeni, I., Sener, O., Zamir, A. R., Jiang, H., Brilakis, I., Fischer, M., Savarese, S., 2016. 3D Semantic Parsing of Large-Scale Indoor Spaces, in: Conference on Computer Vision and Pattern Recognition (CVPR). Las Vegas, United States, pp. 1534-1543. https://doi. org/10. 1109/CVPR. 2016. 170
Ben-Shabat, Y., Avraham, T., Lindenbaum, M., Fischer, A., 2018. Graph based over-segmentation methods for 3D point clouds. Computer Vision and Image Understanding 174, 12-23. https://doi. org/10. 1016/j. cviu. 2018. 06. 004
Ben-Shabat, Y., Lindenbaum, M., Fischer, A., 2017. 3D Point Cloud Classification and Segmentation using 3D Modified Fisher Vector Representation for Convolutional Neural Networks. arXiv. org.
Ben Hmida, H., Boochs, F., Cruz, C., Nicolle, C., 2012. Knowledge Base Approach for 3D Objects Detection in Point Clouds Using 3D Processing and Specialists Knowledge. International Journal on Advances in Intelligent Systems 5, 1-14.
Clementini, E., Di Felice, P., 1997. Approximate topological relations. International Journal of Approximate Reasoning 16, 173-204. https://doi. org/10. 1016/S0888-613X(96)00127-2
De Lathauwer, L., De Moor, B., Vandewalle, J., 2003. A Multilinear Singular Value Decomposition. SIAM Journal on Matrix Analysis and Applications 21, 1253-1278. https://doi. org/10. 1137/s0895479896305696
Dietenbeck, T., Torkhani, F., Othmani, A., Attene, M., Favreau, J.-M. M., 2017. Multi-layer ontologies for integrated 3D shape segmentation and annotation, in: Studies in Computational Intelligence. Springer, Cham, pp. 181-206. https://doi. org/10. 1007/978-3-319-45763-5_10
Engelmann, F., Kontogianni, T., Hermans, A., Leibe, B., 2018a. Exploring Spatial Context for 3D Semantic Segmentation of Point Clouds, in: International Conference on Computer Vision (ICCV). IEEE, Istanbul, Turkey, pp. 716-724. https://doi. org/10. 1109/ICCVW. 2017. 90
Engelmann, F., Kontogianni, T., Schult, J., Leibe, B., 2018b. Know What Your Neighbors Do: 3D Semantic Segmentation of Point Clouds, in: European Conference on Computer Vision (ECCV). Munich, Germany. Feng, C. C., Guo, Z., 2018. Automating parameter learning for classifying terrestrial LiDAR point cloud using 2D land cover maps. Remote Sensing 10, 1192. https://doi. org/10. 3390/rs10081192
Horrocks, I., Patel-Schneider, P., Boley, H., Tabet, S., Grosof, B., Dean, M., 2004. SWRL: A Semantic Web Rule Language Combining OWL and RuleML [WWW Document]. W3C members. URL http://www. academia. edu/download/30680504/SWRL__A_Semantic_Web_Rule_Language_Combining_OWL_and_RuleM. pdf (accessed 5. 2. 20).
Kalinowski, P., Fidler, F., 2010. Interpreting Significance: The Differences Between Statistical Significance, Effect Size, and Practical Importance. Newborn and Infant Nursing Reviews 10, 50-54. https://doi. org/10. 1053/j. nainr. 2009. 12. 007
Koffka, K., 2013. Principles of Gestalt psychology. Routledge. Landrieu, L., Simonovsky, M., 2018. Large-Scale Point Cloud Semantic Segmentation with Superpoint Graphs, in: Conference on Computer Vision and Pattern Recognition (CVPR). Salt Lake City, United States, pp. 4558-4567. https://doi. org/10. 1109/CVPR. 2018. 00479
Liu, Y.-S., Ramani, K., 2009. Robust principal axes determination for point-based shapes using least median of squares. Computer aided design 41, 293-305. https://doi. org/10. 1016/j. cad. 2008. 10. 012
Nguyen, C., Starek, M. J., Tissot, P., Gibeaut, J., 2018. Unsupervised clustering method for complexity reduction of terrestrial lidar data in marshes. Remote Sensing 10, 133. https://doi. org/10. 3390/rs10010133
Ni, H., Lin, X., Zhang, J., Ni, H., Lin, X., Zhang, J., 2017. Classification of ALS Point Cloud with Improved Point Cloud Segmentation and Random Forests. Remote Sensing 9, 288. https://doi. org/10. 3390/rs9030288
Ponciano, J.-J., Trémeau, A., Boochs, F., 2019. Automatic Detection of Objects in 3D Point Clouds Based on Exclusively Semantic Guided Processes. ISPRS International Journal of Geo-Information 8, 442. https://doi. org/10. 3390/ijgi8100442
Ponciano, J. J., Karmacharya, A., Wefers, S., Atorf, P., Boochs, F., 2019. Connected Semantic Concepts as a Base for Optimal Recording and Computer-Based Modelling of Cultural Heritage Objects, in: RILEM Bookseries. Springer Netherlands, pp. 297-304. https://doi. org/10. 1007/978-3-319-99441-3_31
Poux, F., Billen, R., 2019a. Voxel-Based 3D Point Cloud Semantic Segmentation: Unsupervised Geometric and Relationship Featuring vs Deep Learning Methods. ISPRS International Journal of Geo-Information 8, 213. https://doi. org/10. 3390/ijgi8050213
Poux, F., Billen, R., 2019b. A Smart Point Cloud Infrastructure for intelligent environments, in: Lindenbergh, R., Belen, R. (Eds.), Laser Scanning: An Emerging Technology in Structural Engineering, ISPRS Book Series. Taylor & Francis Group/CRC Press, United States. https://doi. org/in generation
Poux, F., Hallot, P., Neuville, R., Billen, R., 2016a. SMART POINT CLOUD: DEFINITION AND REMAINING CHALLENGES. ISPRS Annals of Photogrammetry, Remote Sensing and Spatial Information Sciences IV-2/W1, 119-127. https://doi. org/10. 5194/isprs-annals-IV-2-W1-119-2016
Poux, F., Neuville, R., Hallot, P., Billen, R., 2017. MODEL FOR SEMANTICALLY RICH POINT CLOUD DATA. ISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences IV-4/W5, 107-115. https://doi. org/10. 5194/isprs-annals-IV-4-W5-107-2017
Poux, F., Neuville, R., Hallot, P., Billen, R., 2016b. Point clouds as an efficient multiscale layered spatial representation, in: Vincent, T., Biljecki, F. (Eds.), Eurographics Workshop on Urban Data Modelling and Visualisation. The Eurographics Association, Liège, Belgium. https://doi. org/10. 2312/udmv. 20161417
Poux, F., Neuville, R., Nys, G.-A., Billen, R., 2018. 3D Point Cloud Semantic Modelling: Integrated Framework for Indoor Spaces and Furniture. Remote Sensing 10, 1412. https://doi. org/10. 3390/rs10091412
Poux, Florent, Neuville, R., Van Wersch, L., Nys, G.-A., Billen, R., 2017. 3D Point Clouds in Archaeology: Advances in Acquisition, Processing and Knowledge Integration Applied to Quasi-Planar Objects. Geosciences 7, 96. https://doi. org/10. 3390/geosciences7040096
Qi, C. R., Su, H., Mo, K., Guibas, L. J., 2017a. PointNet: Deep learning on point sets for 3D classification and segmentation, in: Conference on Computer Vision and Pattern Recognition (CVPR). Honolulu, Hawaii, United States, pp. 77-85. https://doi. org/10. 1109/CVPR. 2017. 16
Qi, C. R., Yi, L., Su, H., Guibas, L. J., 2017b. PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space, in: Conference on Neural Information Processing Systems (NIPS). Long Beach, United States.
Quan, S., Ma, J., Hu, F., Fang, B., Ma, T., 2018. Local voxelized structure for 3D binary feature representation and robust registration of point clouds from low-cost sensors. Information Sciences 444, 153-171. https://doi. org/10. 1016/J. INS. 2018. 02. 070
Rusu, R. B., Blodow, N., Beetz, M., 2009. Fast Point Feature Histograms (FPFH) for 3D registration, in: International Conference on Robotics and Automation (ICRA). IEEE, Kobe, Japan, pp. 3212-3217. https://doi. org/10. 1109/ROBOT. 2009. 5152473
Son, H., Kim, C., 2017. Semantic as-built 3D modeling of structural elements of buildings based on local concavity and convexity. Advanced Engineering Informatics 34, 114-124. https://doi. org/10. 1016/j. aei. 2017. 10. 001
Tchapmi, L. P., Choy, C. B., Armeni, I., Gwak, J., Savarese, S., 2017. SEGCloud: Semantic Segmentation of 3D Point Clouds, in: International Conference on 3D Vision (3DV). Qingdao, China.
Truong-Hong, L., Laefer, D. F., Hinks, T., Carr, H., 2012. Flying Voxel Method with Delaunay Triangulation Criterion for Façade/Feature Detection for Computation. Journal of Computing in Civil Engineering 26, 691-707. https://doi. org/10. 1061/(ASCE)CP. 1943-5487. 0000188
Wang, J., Lindenbergh, R., Menenti, M., 2017. SigVox-A 3D feature matching algorithm for automatic street object recognition in mobile laser scanning point clouds. ISPRS Journal of Photogrammetry and Remote Sensing 128, 111-129. https://doi. org/10. 1016/j. isprsjprs. 2017. 03. 012
Wang, Y., Cheng, L., Chen, Y., Wu, Y., Li, M., 2016. Building point detection from vehicle-borne LiDAR data based on voxel group and horizontal hollow analysis. Remote Sensing 8, 419. https://doi. org/10. 3390/rs8050419
Weber, C., Hahmann, S., Hagen, H., 2010. Sharp feature detection in point clouds, in: International Conference on Shape Modeling and Applications. IEEE, Washington, United States, pp. 175-186. https://doi. org/10. 1109/SMI. 2010. 32
Xu, Y., Hoegner, L., Tuttas, S., Stilla, U., 2017. Voxel-and Graph-Based Point Cloud Segmentation of 3D Scenes Using Perceptual Grouping Laws, in: ISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences. ISPRS, Hannover, Germany, pp. 43-50. https://doi. org/10. 5194/isprs-annals-IV-1-W1-43-2017
Xu, Y., Tuttas, S., Hoegner, L., Stilla, U., 2018. Voxel-based segmentation of 3D point clouds from construction sites using a probabilistic connectivity model. Pattern Recognition Letters 102. https://doi. org/10. 1016/j. patrec. 2017. 12. 016
Zhu, Q., Li, Y., Hu, H., Wu, B., 2017. Robust point cloud classification based on multi-level semantic relationships for urban scenes. ISPRS Journal of Photogrammetry and Remote Sensing 129, 86-102. https://doi. org/10. 1016/j. isprsjprs. 2017. 04. 022
