Transfer learning in application to semantic segmentation of 3D indoor scenes scanned with a depth camera
Ewelina Rosiak, Artur Klepaczko · 2024
Recent achievements in the domain of computational intelligence and robotics trigger development and automation in various industry branches. One such domain is the architecture and construction, where the challenges related to efficient measurement, progress monitoring, maintenance, or verification of the agreement between a design and a built object, remain unsolved and may benefit from modern image and signal processing technology. Recognition of objects such as, e.g., walls, doors, or furniture in 3D scenes, allows the creation of so-called digital twins of the real-world environment. Within this study, we developed a series of deep learning models dedicated to semantic segmentation of point clouds representing interiors of buildings. Such representations are usually collected with expensive high-precision LiDAR scanners. The available neural network models, published in the literature, were, accordingly, trained using either such laser data or, alternatively, by the help of the synthetic ones. Our motivation was thus to enable semantic segmentation of RGBD point clouds acquired with a much cheaper sensor, i.e. a depth camera. In the absence of open real datasets, we propose to perform transfer learning and show that it is feasible to achieve reasonable segmentation accuracy on depth-camera data using a model pretrained on LiDAR scans.