Enhancing indoor mapping and localization in specular rich environments using deep learning and sensor fusion

Abstract

Robotic indoor mapping and localization are significantly challenged in environ ments with highly reflective or specular surfaces, which are common in hospitals and industrial settings. Specular reflections introduce severe artifacts in depth data from RGB-D sensors and degrade the performance of visual Simultaneous Localization and Mapping (SLAM) systems by creating unreliable features. This thesis presents a com prehensive solution to enhance robotic navigation in such specular-rich environments through a combination of deep learning and multi-sensor fusion. We propose a real-time filtering algorithm, RT-SpecFilter, which uses a Support Vector Machine (SVM) to detect and mitigate specular artifacts in point clouds from an Intel RealSense D435 camera. Furthermore, we conduct a comparative analysis of feature detectors, identifying Super Point as the most robust for environments with specular highlights. Finally, we develop the Multicam SP-VO system that leverages four wide FoV cameras and fuses their motion estimates with wheel odometry data using a pose-graph optimization framework. Exper imental results demonstrate that the proposed system significantly reduces orientation drift improves localization accuracy compared to reliance on wheel odometry alone and mitigates the specular artifacts during mapping, thereby enabling more robust and reli able autonomous navigation in challenging indoor spaces.

9