ReRTG-SLAM:基于三段式全局优化的实时大规模三维场景重建方法

ReRTG-SLAM:Real-time 3D Reconstruction at Scale Based on Three-stage Global Optimization

摘要

3D 高斯泼溅(3DGS)能实现高保真三维重建却难以实时应用,SLAM可实时定位建图但对精细渲染关注不足,二者互补催生了实时高保真三维重建研究,但现有方法在实时重建场景中常面临性能瓶颈或重建质量缺陷.本文提出ReRTG-SLAM方法:首先分析3DGS与SLAM结合方法存在的局部过拟合、场景表达精度不足,尤其前后帧对过拟合高斯基元重整不足的问题;其次提出三段式全局优化策略,在每个关键帧仅优化当前帧与少量强约束历史帧构成的最小相关集,减少冗余计算;最后提出帧质量函数,筛选更多代表性关键帧以更高频触发全局优化,缓解过拟合并保持全局一致性.在Replica、TUM-RGBD和Azure数据集上与多强基线方法对比,实验结果表明,ReRTG-SLAM在多数场景中重建质量与性能均优于现有方法,可有效平衡局部拟合精度、全局一致性与实时性能,为大规模三维重建提供新解决方案.

Abstract

While 3D Gaussian Splatting (3DGS) enables high-fidelity 3D reconstruction,it struggles with real-time applicability;in contrast,SLAM excels at real-time localization and mapping yet falls short in prioritizing high-quality rendering.The complementary strengths of these two techniques have spurred research into real-time high-fidelity 3D reconstruction,but existing methods often face performance bottlenecks or reconstruction quality defects in real-time scenarios.To address these challenges,this paper proposes the ReRTG-SLAM method.First,it analyzes the limitations of existing 3DGS-SLAM integration approaches,including local overfitting,insufficient scene representation accuracy,and particularly the inadequacy of inter-frame information in rectifying overfitted Gaussian primitives.Second,it introduces a three-stage global optimization strategy that,for each keyframe,only optimizes the minimal relevant set composed of the current frame and a small number of strongly constrained historical frames,thereby reducing redundant computations.Finally,it presents a frame quality function to select more representative keyframes,triggering global optimization more frequently to alleviate overfitting and maintain global consistency.Comparative experiments against multiple strong baseline methods on the Replica,TUM-RGBD,and Azure datasets demonstrate that ReRTG-SLAM outperforms existing approaches in terms of both reconstruction quality and performance across most scenarios.It effectively balances local fitting accuracy,global consistency,and real-time performance,thus providing a novel solution for large-scale 3D reconstruction.

关键词

SLAM / 3D高斯泼溅 / 三维重建 / 全局一致性 / 优化

Key words

SLAM / 3D Gaussian splatting / 3D reconstruction / global consistency / optimization

[1] Feng D,Chen Z,Yin Y,et al.CaRtGS:computational alignment for real-time gaussian splatting SLAM[J].IEEE Robotics and Automation Letters,2025,10(5):4340-4347.
[2] Matsuki H,Murai R,Kelly P H J,et al.Gaussian splatting SLAM[C]//IEEE/CVF Conference on Computer Vision and Pattern Recognition,2024:18039-18048.
[3] Peng Z,Shao T,Liu Y,et al.RTG-SLAM:real-time 3D reconstruction at scale using Gaussian splatting[C]//Special Interest Group on Computer Graphics and Interactive Techniques Conference Conference Papers,2024:1-11.
[4] Yan C,Qu D,Xu D,et al.GS-SLAM:dense visual slam with 3D Gaussian splatting[C]//IEEE/CVF Conference on Computer Vision and Pattern Recognition,2024:19595-19604.
[5] Kerbl B,Kopanas G,Leimkuehler T,et al.3D Gaussian splatting for real-time radiance field rendering[J].ACM Transactions on Graphics,2023,42(4):1-17.
[6] Chung J,Oh J,Lee K M.Depth-regularized optimization for 3D gaussian splatting in few-shot images[C]//IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops,2024:811-820.
[7] Yang Z,Gao X,Zhou W,et al.Deformable 3D Gaussians for high-fidelity monocular dynamic scene reconstruction[C]//IEEE/CVF Conference on Computer Vision and Pattern Recognition,2024:20331-20341.
[8] Du H,Henry P,Ren X,et al.Interactive 3D modeling of indoor environments with a consumer depth camera[C]//13th International Conference on Ubiquitous Computing,2011:75-84.
[9] Kong J,Zhou T,Wang H,et al.Real-time high-accuracy 3D reconstruction based on texture and geometric features[C]//Proceedings of the 41st Chinese Control Conference,2022:6773-6777.
[10] Keller M,Lefloch D,Lambers M,et al.Real-time 3D reconstruction in dynamic scenes using point-based fusion[C]//International Conference on 3D Vision,2013:1-8.
[11] Whelan T,Salas Moreno R F,Glocker B,et al.Elastic fusion[J].The International Journal of Robotics Research,2016,35(14):1697-1716.
[12] Xu Y,Nan L,Zhou L,et al.HRBF-fusion:accurate 3D reconstruction from RGB-D data using on-the-fly implicits[J].ACM Transactions on Graphics,2022,41(3):1-19,doi:101145/3516521.
[13] Chen J,Bautembach D,Izadi S.Scalable real-time volumetric surface reconstruction[J].ACM Transactions on Graphics,2013,32(4):1-12,doi:10.1145/2461912.2461940.
[14] Dai A,Nieвner M,Zollhöfer M,et al.BundleFusion:real-time globally consistent 3D reconstruction using on-the-fly surface reintegration[J].ACM Transactions on Graphics,2017,36(4):1-17,doi:10.1145/3072959.3054739.
[15] Huang S,Chen H,Huang J,et al.Real-time globally consistent 3D reconstruction with semantic priors[J].IEEE Transactions on Visualization and Computer Graphics,2023,29(4):1977-1991.
[16] Newcombe R A,Lzadi S,Hilliges O,et al.Kinect fusion:real-time dense surface mapping and tracking[C]//10th IEEE International Symposium on Mixed and Augmented Reality (ISMAR),2011:127-136.
[17] Nieвner M,Zollhöfer M,Izadi S,et al.Real-time 3D reconstruction at scale using voxel hashing[J].ACM Transactions on Graphics,2013,32(6):1-10,doi:10.1145/2508363.2508374.
[18] Zhang Y,Xu W,Tong Y,et al.Online structure analysis for real-time indoor scene reconstruction[J].ACM Transactions on Graphics,2015,34(5):1-12,doi:10.1145/2768821.
[19] Huang J,Huang S,Song H,et al.DI-fusion:online implicit 3D reconstruction with deep priors[C]//IEEE/CVF Conference on Computer Vision and Pattern Recognition,2021:8932-8941.
[20] Sucar E,Liu S,Ortiz J,et al.iMAP:implicit mapping and positioning in real-time[C]//IEEE/CVF International Conference on Computer Vision,2021:6209-6218.
[21] Zhu Z,Peng S,Larsson V,et al.NICE-SLAM:neural implicit scalable encoding for SLAM[C]//IEEE/CVF Conference on Computer Vision and Pattern Recognition,2022:12776-12786.
[22] Yang X,Li H,Zhai H,et al.Vox-fusion:dense tracking and mapping with voxel-based neural implicit representation[C]//IEEE International Symposium on Mixed and Augmented Reality,2022:499-507.
[23] Johari M M,Carta C,Fleuret F.ESLAM:efficient dense SLAM system based on hybrid representation of signed distance fields[C]//IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR),2023:17408-17419.
[24] Wang H,Wang J,Agapito L.Co-SLAM:joint coordinate and sparse parametric encodings for neural real-time SLAM[C]//IEEE/CVF Conference on Computer Vision and Pattern Recognition,2023:13293-13302.
[25] Sandström E,Li Y,Van Gool L,et al.Point-SLAM:dense neural point cloud-based SLAM[C]//IEEE/CVF International Conference on Computer Vision,2023:18387-18398.
[26] Huang H,Li L,Cheng H,et al.Photo-SLAM:real-time simultaneous localization and photorealistic mapping for monocular,stereo,and RGB-D cameras[C]//IEEE/CVF Conference on Computer Vision and Pattern Recognition,2024:21584-21593.
[27] Keetha N,Karhade J,Jatavallabhula K M,et al.SplaTAM:splat,track & map 3D Gaussians for dense RGB-D SLAM[C]//IEEE/CVF Conference on Computer Vision and Pattern Recognition,2024:21357-21366.
[28] Straub J,Whelan T,Ma L,et al.The replica dataset:a digital replica of indoor spaces[J].arXiv Preprint arXiv:1906.05797,2019.
[29] Sturm J,Engelhard N,Endres F,et al.A benchmark for the evaluation of RGB-D SLAM systems[C]//IEEE/RSJ International Conference on Intelligent Robots and Systems,2012:573-580.

基金资助

四川省科技计划(重点研发)项目(2023YFG0122-LH)资助.