This study aims to propose a method for three-dimensional (3D) reconstruction of thoracoscopic surgical scenes to enhance intraoperative visualization and assist surgeons in performing precise operations. The method first employs a vision transformer (ViT) to estimate the depth from thoracoscopic images; it then predicts camera poses using a photometric constraint approach combined with an image pyramid; finally, the scene surface is reconstructed using a truncated signed distance function (TSDF). Experimental results on the thoracoscopic image dataset demonstrate that the proposed method achieves the best performance in depth estimation, with an absolute error of 0.056 and a root mean square error of 3.843, and in pose estimation, its rotation and translation errors are 0.071 4 and 0.002 8, respectively, outperforming other methods. Compared with existing techniques, this approach shows significant advantages in 3D reconstruction accuracy and intraoperative visualization guidance, providing intuitive and reliable intraoperative reference information for thoracoscopic minimally invasive surgery, holds potential clinical value.
HuiWen-yu, WuJin-yang, HuangJian-hua, et al. Experimental study on the accuracy evaluation of robot-assisted osteotomy of genioplasty[J]. Journal of Shanghai Jiao Tong University (Medical Science), 2022, 42(9): 1347-1352.
[4]
ChiJ N, MiaoJ, ChenJ H, et al. DSTAN: a deformable spatial-temporal attention network with bidirectional sequence feature refinement for speckle noise removal in thyroid ultrasound video[J]. Journal of Imaging Informatics in Medicine, 2024, 37(6): 3264-3281.
[5]
ZhouT H, BrownM, SnavelyN, et al. Unsupervised learning of depth and ego-motion from video[C]//2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Honolulu, 2017: 6612-6619.
[6]
MountneyP, StoyanovD, YangG Z. Three-dimensional tissue deformation recovery and tracking[J]. IEEE Signal Processing Magazine, 2010, 27(4): 14-24.
[7]
JosephJ, GeorgeS N, RajaK. Parameter-free matrix decomposition for specular reflections removal in endoscopic images[J]. IEEE Journal of Translational Engineering in Health and Medicine, 2023, 11: 360-374.
[8]
GodardC, Mac AodhaO, FirmanM, et al. Digging into self-supervised monocular depth estimation[C]//2019 IEEE/CVF International Conference on Computer Vision (ICCV). Seoul, 2019: 3827-3837.
[9]
KumarS P, PanchamiB. Optical flow glottovibrogram for the examination of vocal fold pathology[C]//2023 45th Annual International Conference of the IEEE Engineering in Medicine & Biology Society (EMBC). Sydney, 2023: 1-4.
[10]
ZhuH W, BaiS Q, ShiJ L, et al. IOFusion: instance segmentation and optical-flow guided 3D reconstruction in dynamic scenes[J]. Visual Computer, 2025, 41(2): 799-813.
[11]
ShenT C, MunkbergJ, HasselgrenJ, et al. Flexible isosurface extraction for gradient-based mesh optimization[J]. ACM Transactions on Graphics, 2023, 42(4): 1-16.
[12]
ZhouY F, ZhangM C, ZhuJ L, et al. A randomized block-coordinate adam online learning optimization algorithm[J]. Neural Computing & Applications, 2020, 32(16): 12671-12684.
[13]
TuranM, AlmaliogluY, AraujoH, et al. Deep EndoVO: a recurrent convolutional neural network (RCNN) based visual odometry approach for endoscopic capsule robots[J]. Neurocomputing, 2018, 275: 1861-1870.
[14]
ParasharS, PizarroD, BartoliA. Isometric non-rigid shape-from-motion with Riemannian geometry solved in linear time[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2018, 40(10): 2442-2454.
[15]
OzyorukK B, GokcelerG I, BobrowT L, et al. EndoSLAM dataset and an unsupervised monocular visual odometry and depth estimation approach for endoscopic videos[J]. Medical Image Analysis, 2021, 71: 102058.
[16]
ShaoS W, PeiZ C, ChenW H, et al. Self-supervised monocular depth and ego-motion estimation in endoscopy: appearance flow to the rescue[J]. Medical Image Analysis, 2022, 77: 102338.
[17]
CaoL, ZhuangS B, TianS, et al. A global structure and adaptive weight aware ICP algorithm for image registration[J]. Remote Sensing, 2023, 15(12): 3185.
[18]
SplietkerM, BehnkeS. Rendering the directional TSDF for tracking and multi-sensor registration with point-to-plane scale ICP[J]. Robotics and Autonomous Systems, 2023, 162: 104337.
[19]
MarchisottiD, ZappaE. Virtual simulation benchmark for the evaluation of simultaneous localization and mapping and 3D reconstruction algorithm uncertainty[J]. Measurement Science and Technology, 2021, 32(9):095404.