USME: 统一SLAM度量与评测技术研究

USME: Unified SLAM Measurement and Evaluation

关键词:  SLAM性能评估框架  指标体系  仿真数据集  多体协同性能

Key words:  SLAM performance evaluation performance framework  Metrics system  Simulation dataset  Multi-body collaborative performance

摘要:

同时定位与建图(SLAM)技术近年来得到迅速发展，但由于缺乏在统一框架下对算法的度量和比较，对SLAM的客观评估和应用造成障碍。提出了统一SLAM度量与评测(USME)框架，从指标体系、数据集及评测方法三个维度为各种SLAM方法的性能度量及比较研究提供基准。针对不同场景，建立了包括长时间运行漂移量,闭环检测能力，存在相机遮挡、光照变化和运动物体时SLAM方法的鲁棒性，以及多体协同性能等的综合性能指标体系。基于三维仿真平台，以指标体系为基准建立了合成数据序列及对应数据集，以对性能指标进行度量与评估。还建立了平均指标均值的数据处理与评测方法，以综合评价不同参数选择对方法性能的影响。通过典型SLAM方法验证了上述方法的可行性。

Abstract:

The technology of simultaneous localization and mapping (SLAM) has developed rapidly in recent years, but the lack of a unified framework under which the measurement and comparison of algorithms can be carried out objectively has become a major obstacle to the objective evaluation and application of SLAM. A unified SLAM measurement and evaluation (USME) framework is proposed, which provides benchmarks for the performance measurement and comparative research of various SLAM methods in terms of metrics system, dataset and evaluation method. To adjust to different scenarios, we establish a comprehensive performance index system encompassing long-term drift, closed-loop detection capabilities, the robustness of the SLAM method when camera occlusion, illumination changes and moving objects exist, and multi-body coordination performance. Based on a three-dimensional simulation platform, synthetic data sequences and corresponding datasets based on the metrics system are established to measure and evaluate performance. We also present a data processing and evaluation method for the average value of the average metrics to comprehensively evaluate the impact of different parameter choices on the performance of the method. The feasibility of the above scheme is verified by a typical SLAM method.