基于神经网络的实时光线追踪降噪

Real-Time Ray Tracing Denoising Based on Neural Networks

摘要

针对现有计算机硬件只支持低采样数的实时光线追踪,Monte Carlo积分在低采样样本情况下噪声明显的问题,使用神经网络技术,并权衡其质量和性能,解决低采样数下的Monte Carlo光线追踪噪声问题.首先,通过多尺度空洞卷积模块,融合图像局部特征细节及全局信息,在有限参数量下提升降噪质量;其次,引入时域的交叉注意力模块,提升神经网络对时域信息的复用能力,充分利用已降噪的历史帧,进一步提升图像当前帧的降噪质量和效率;最后,通过组合损失函数进一步增强网络对图像细节特征的提取能力,以及时域信息的稳定性和复用能力.在公开数据集BMFR上进行训练和验证的结果表明,该方法与目前的先进方法相比,在结构相似性(SSIM)性能保持相当的情况下,峰值信噪比(PSNR)平均提升3.7%,均方根误差(RMSE)平均降低23.5%,且运行时间满足实时性能.

Abstract

Monte Carlo-based ray tracing has been widely adopted in photorealistic rendering due to its physical accuracy, flexibility, and universality. However, existing hardware limitations restrict real-time implementations to low sample-per-pixel rates, where Monte Carlo integration exhibits significant noise under sparse sampling conditions. This paper employs neural network techniques to address the trade-off between quality and performance, effectively resolving noise issues in Monte Carlo ray tracing with low sample counts. The proposed framework incorporates three core innovations: a multi-scale dilated convolution module that enhances denoising quality by fusing local feature details with global contextual information with minimal parameter overhead; a temporal cross-attention mechanism that improves temporal coherence by adaptively reusing denoised historical frames; and a composite loss function combining reconstruction and temporal consistency terms to strengthen detail preservation and stability. Evaluated on the BMFR benchmark dataset, our method achieved an average 3.7% improvement in peak signal-to-noise ratio (PSNR) and a 23.5% reduction in root mean squared error (RMSE) compared to state-of-the-art approaches while maintaining comparable structure similarity index measure (SSIM) performance. The inference time meets real-time requirements.

关键词

Monte Carlo光线追踪 / 神经网络 / 实时光线追踪降噪 / 核预测网络

Key words

Monte Carlo ray tracing / neural network / real-time ray tracing denoising / kernel prediction network

参考文献

[1] Kemmerling M, Lütticke D, Schmitt R H. Beyond Games: A Systematic Review of Neural Monte Carlo Tree Search Applications[J]. Applied Intelligence, 2024, 54(1): 1020-1046.

[2] Lin Y F. The Review of Modeling and Rendering 3D Environment: A Survey[J]. Highlights in Science, Engineering and Technology, 2024, 93: 304-315.

[3] Kajiya J T. The Rendering Equation[C]// Proceedings of the 13th Annul Conference on Computer Graphics and Interactive Techniques. New York: ACM, 1986: 143-150.

[4] Metropolis N, Ulam S. The Monte Carlo Method[J]. Journal of the American Statistical Association, 1949, 44(247): 335-341.

[5] Schied C, Kaplanyan A, Wyman C, et al. Spatiotemporal Variance-Guided Filtering: Real-Time Reconstruction for Path-Traced Global Illumination[C]// Proceedings of High Performance Graphics. New York: ACM, 2017: 1-12.

[6] Bitterli B, Rousselle F, Moon B, et al. Nonlinearly Weighted First-Order Regression for Denoising Monte Carlo Renderings[C]// Proceedings of the 35th Annual Conference on Computer Graphics Forum. Goslar, Germany: Eurographics Association, 2016: 107-117.

[7] Koskela M, Immonen K, Mäkitalo M, et al. Blockwise Multi-order Feature Regression for Real-Time Path-Tracing Reconstruction[J]. ACM Transactions on Graphics, 2019, 38(5): 1-14.

[8] Chaitanya C R A, Kaplanyan A S, Schied C, et al. Interactive Reconstruction of Monte Carlo Image Sequences Using a Recurrent Denoising Autoencoder[J]. ACM Transactions on Graphics, 2017, 36(4): 1-12.

[9] Meng X X, Zheng Q, Varshney A, et al. Real-Time Monte Carlo Denoising with the Neural Bilateral Grid[C]// Proceedings of the 2020 European Conference on Computer Graphics. Goslar, Germany: Eurographics Association, 2020: 13-24.

[10] Bako S, Vogels T, Mcwilliams B, et al. Kernel-Predicting Convolutional Networks for Denoising Monte Carlo Renderings[J]. ACM Transactions on Graphics, 2017, 36(4): 97-1-97-14.

[11] Fan H M, Wang R, Huo Y C, et al. Real-Time Monte Carlo Denoising with Weight Sharing Kernel Prediction Network[C]// Proceedings of the 40th Annual Conference on Computer Graphics Forum. Goslar, Germany: Eurographics Association, 2021: 15-27.

[12] Chen L C, Zhu Y, Papandreou G, et al. Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation[C]// Proceedings of the European Conference on Computer Vision. Berlin: Springer, 2018: 801-818.

[13] Wang P Q, Chen P F, Yuan Y, et al. Understanding Convolution for Semantic Segmentation[C]// 2018 IEEE Winter Conference on Applications of Computer Vision. Piscataway, NJ: IEEE, 2018: 1451-1460.

[14] Hu J, Shen L, Sun G. Squeeze-and-Excitation Networks[C]// Proceedings of the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway, NJ: IEEE, 2018: 7132-7141.

[15] Dosovitskiy A, Beyer L, Kolesnikov A, et al. An Image Is Worth 16×16 Words:Transformers for Image Recognition at Scale[EB/OL]. (2020-10-22)[2025-04-10]. https://arxiv.org/abs/2010.11929.

[16] Wang H, Wang W N, Liu J. Temporal Memory Attention for Video Semantic Segmentation[C]// 2021 IEEE International Conference on Image Processing. Piscataway, NJ: IEEE, 2021: 2254-2258.

基金资助

吉林省科技发展计划项目重点研发项目(20260301004YY)