面向虚拟场景的大语言模型文本驱动连续地形生成方法研究

A Text-Driven Continuous Terrain Generation Method for Virtual Scenes Based on Large Language Models

摘要

随着游戏开发、虚拟现实 (Virtual Reality, VR) 及数字孪生等三维技术的快速发展，对高质量、大范围虚拟连续地形场景的自动化生成需求日益迫切。传统的手工交互方式操作繁琐且专业壁垒高，程序化生成方法虽能提升效率，但仍依赖图形界面和专业参数的调整，因而难以灵活捕捉用户语义意图。本文提出一种基于大语言模型 (Large Language Model, LLM) 的自动化文本驱动虚拟连续地形生成方法，旨在通过自然语言直接驱动生成过程，以实现高效、连续的虚拟地形场景构建。该方法采用多阶段任务分解策略，将地形生成流程拆分成 7 个步骤，融合提示词工程与程序化生成技术，构建从文本到大范围三维连续地形的端到端映射。针对 LLM 在复杂任务中的多跳推理缺陷、输出随机性及记忆丢失等局限，设计了跨阶段关键信息传递、错误重试与历史日志学习等机制，以提升系统可靠性和一致性。基于 Unity 引擎开发的原型系统验证了该方法的有效性，在 4 个典型地形主题上的测试结果表明，完整系统（Full Step System，FSS）的平均错误率仅为 12.0%，相较于移除关键机制的对比系统，错误率显著降低了约 19~47 个百分点。在效率方面，生成单个预览地形区块（1 024×1 024 顶点网格）的平均耗时约为 42.14 s，资源消耗可控。对比实验进一步表明本文方法生成的虚拟地形在生成效率、视觉一致性与可扩展性方面均达到预期要求。该方法显著提升了生成效率与易用性，为非专业用户通过自然语言快速构建高质量、大范围三维地形提供了可行的解决方案。

Abstract

With the rapid advancement of three-dimensional technologies including game development, Virtual Reality (VR) and digital twins, there is an increasingly urgent demand for the automatic generation of high-quality, large-scale continuous virtual terrain scenes. Conventional manual interactive workflows involve cumbersome operations and high professional thresholds. Although procedural generation methods can improve efficiency, they still rely on graphical interfaces and the tuning of specialized parameters, making it difficult to flexibly capture users’ semantic intentions. This paper proposes an automatic text-driven generation method for continuous virtual terrain based on the Large Language Model (LLM), which directly drives the generation pipeline through natural language to realize efficient construction of continuous virtual terrain scenes. A multi-stage task decomposition strategy is adopted in the proposed method to split the terrain generation pipeline into seven steps. By integrating prompt engineering and procedural generation techniques, an end-to-end mapping from text to large-scale continuous 3D terrain is established. To address the limitations of LLMs in complex tasks such as defects in multi-hop reasoning, output randomness and memory loss, mechanisms including cross-stage key information transmission, error retry and historical log learning are designed to improve the reliability and consistency of the system. A prototype system developed on the unity engine verifies the effectiveness of the proposed method. Test results on four typical terrain themes demonstrate that the Full Step System (FSS) achieves an average error rate of only 12.0%, which is significantly reduced by approximately 19–47 percentage points compared with the comparative systems without core mechanisms. In terms of efficiency, the average time consumption for generating a single preview terrain tile (1 024 × 1 024 vertex mesh) is about 42.14 s with controllable resource consumption. Further comparative experiments prove that the virtual terrain generated by the proposed method meets the expected requirements in generation efficiency, visual consistency and scalability. The proposed method remarkably improves generation efficiency and usability, offering a feasible solution for non-professional users to rapidly construct high-quality, large-scale 3D terrain via natural language.

关键词

人机交互 / 文本驱动 / 大语言模型 / 虚拟地形生成 / 程序化内容生成

Key words

human-computer interaction / text-driven / large language model / virtual terrain generation / procedural content generation

参考文献

[1] YIN Z, WANG Y, PAPATHEODOROU T, et al. Text2VRScene: Exploring the framework of automated text-driven generation system for VR experience[C]//2024 IEEE Conference on Virtual Reality and 3D User Interfaces (VR). Osaka, Japan: IEEE, 2024:701-711.

[2] VALENCIA-ROSADO L O, STAROSTENKO O. Methods for procedural terrain generation: A review[C]//Mexican Conference on Pattern Recognition. Cham, Switzerland: Springer International Publishing, 2019:58-67.

[3] FOURNIER A, FUSSELL D, CARPENTER L. Computer rendering of stochastic models[J].Communications of the ACM, 1982, 25(6):371-384.

[4] MUSGRAVE F K, KOLB C E, MACE R S. The synthesis and rendering of eroded fractal terrains[J].ACM Siggraph Computer Graphics, 1989, 23(3):41-50.

[5] CORDONNIER G, BRAUN J, CANI M P, et al. Large scale terrain generation from tectonic uplift and fluvial erosion[J].Computer Graphics Forum, 2016, 35(2):165-175.

[6] GAIN J E, MARAIS P C, STRABER W. Terrain sketching[C]//Proceedings of the 2009 Symposium on Interactive 3D Graphics and Games (I3D ‘09). Boston, Massachusetts: ACM, 2009:31-38.

[7] SPICK R, WALKER J A. Realistic and textured terrain generation using GANs[C]//Proceedings of the 16th ACM SIGGRAPH European Conference on Visual Media Production. London, United Kingdom: ACM, 2019:1-10.

[8] 江西林, 王少荣, 李文玉, 等. 以特征线和高程范围为约束的 DEM-cGAN 框架 [J].计算机辅助设计与图形学学报, 2021, 33(8):1191-1201.

[9] BORNE P, CZERKAWSKI M, MARTIN R, et al. MESA: Text-driven terrain generation using latent diffusion and global Copernicus data[C]//2025 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW). Seattle, WA, USA: IEEE,2025:3058-3066.

[10] ZHANG G, CHEN L, ZHANG Y, et al. Translating words to worlds: Zero-shot synthesis of 3D terrain from textual descriptions using large language models[J].Applied Sciences, 2024, 14(8):3257.

[11] LIU J H, ZHANG S K, ZHANG C, et al. Controllable procedural generation of landscapes[C]//Proceedings of the 32nd ACM International Conference on Multimedia. Melbourne, Australia: ACM, 2024:6394-6403.

[12] TIAN H. Is ChatGPT the ultimate programming assistant-How far is it[EB/OL]. (2023-04-24). https://arXivpreprintarXiv:2304.11938.

[13] TAMKIN A, BRUNDAGE M, CLARK J, et al. Understanding the capabilities, limitations, and societal impact of large language models[EB/OL]. (2021-02-04). https://arXiv preprint arXiv: 2102.02503.

[14] BANG Y, CAHYAWIJAYA S, LEE N, et al. A multitask, multilingual, multimodal evaluation of ChatGPT on reasoning, hallucination, and interactivity[EB/OL]. (2023-02-08). https://arXiv preprint arXiv: 2302.04023.

[15] BORJI A. A categorical archive of ChatGPT failures[EB/OL]. (2023-02-06). https://arXivpreprintarXiv:2302.03494.

[16] LIU H, NING R, TENG Z, et al. Evaluating the logical reasoning ability of ChatGPT and GPT-4[EB/OL]. (2023-04-07). https://arXivpreprintarXiv:2304.03439.

[17] ZALA A, CHO J, LIN H, et al. Envgen: Generating and adapting environments via LLMs for training embodied agents[EB/OL]. (2024-03-18). https://arXiv preprint arXiv: 2403.12014.

基金资助

国家自然科学基金(62272164)