Sparse Convolutional Neural Network for Localization and Orientation Prediction and Application to Drone Control
Jan Rodziewicz-Bielewicz, Marcin Korzeń · Frontiers in artificial intelligence and applications · 2024
Depth cameras used in visual feedback loops typically produce point clouds or their voxelized images. In the paper, we introduce an original sparse convolutional neural network structure tailored to work with 3D point clouds taken from RGB-D camera. The proposed structure is particularly effective for localization and orientation tasks and operates on the entire scene. We demonstrate its application by using the proposed neural network in a computer vision feedback system to track and control the drone indoors. The visual feedback loop measures the drone’s state and uses the information about its position and orientation to control the drone and allow it to fly along a given trajectory. In the experimental part we show the effectiveness of this approach to control the drone position in indoor spaces. We achieve trajectory-following accuracy comparable to that of the drone’s inbuilt controller’s ability to maintain a fixed position. We compare the proposed solution with a YOLO-based detector and sparse convolutional network to angle prediction operating on cropped and aligned drone point clouds.