Measuring 3D-reconstruction quality in probabilistic volumetric maps with the Wasserstein Distance

In this study, we address the challenge of measuring 3D-reconstruction quality in large unstructured environments, when the map is built with uncertainty in the robot localization. The challenge lies in measuring the quality of a reconstruction against the ground-truth when the data is extremely sparse and where traditional methods, such as surface distance metrics, fail. We propose a complete methodology to measure the quality of the reconstruction, at a local level, in both structured and unstructured environments. Building upon the fact that a common map representation in robotics is the probabilistic volumetric map, we propose, along this methodology, to use a novel metric to measure the map quality based directly on the voxels’ occupancy likelihood: the Wasserstein Distance. Finally, we evaluate this Wasserstein Distance metric in simulation, under different level of noise in the robot localization, and in a real world experiment, demonstrating the robustness of our method.