算子化中间层范式:基于插槽机制的大语言模型通用水印框架

Operator-based intermediate-layer paradigm:A universal watermarking framework for large language models based on slot mechanism

摘要

大语言模型(LLMs)的广泛应用使得文本水印成为确权与溯源的关键技术。然而,现有大模型水印算法面临严重的碎片化问题:不同算法分别在 Logits 空间、词表划分或采样轨迹等不同层面进行干预。这种非标准化的设计导致水印逻辑与模型架构和解码策略产生强耦合,既阻碍了算法的灵活迁移,也使得跨机制的公平评测成为难题。针对上述问题,本文构建了一种面向大语言模型推理链路的通用中间层架构——水印插槽层(Slot layer)。通过将多样化的水印注入行为统一抽象为 Logits 空间内的标准化算子,水印插槽层成功实现了水印逻辑与骨干模型的结构性解耦。基于该架构,本文建立了覆盖可检测性、生成质量及鲁棒性的标准化评测基准,消除了模型与环境差异带来的实验偏差。实验与分析表明,水印插槽层不仅提供了一种普适的结构基础,更通过支持模块化热插拔,为水印技术的高效部署与标准化演进奠定了关键架构支撑。

Abstract

As large language models (LLMs) are increasingly deployed for content generation,text watermarking has emerged as a critical technology for source attribution and ensuring the reliability of generated content.However,existing watermarking techniques exhibit significant structural heterogeneity:some modify generation distributions through Logits bias,others adjust vocabulary based on static frequency statistics,while some introduce controlled random perturbations during decoding.These methods operate independently at different levels of implementation,which leads to a lack of a unified design paradigm.This disjointed approach results in a strong coupling between watermarking logic and specific model architectures,vocabulary configurations,and decoding strategies.Consequently,the migration and reuse of watermarking algorithms become more costly,and fair evaluation across different methods becomes challenging under a standardized benchmark.To address these structural challenges,we propose slot layer,a universal intermediate-layer architecture for LLMs inference.Positioned between the backbone model's Softmax Logits output and the sampling decision process,the slot layer abstracts various watermark injection behaviors into standardized Logits-processing operators through a unified interface.This design achieves structural decoupling between watermarking algorithms and the backbone model.By leveraging this unified entry point,we establish a comprehensive evaluation framework that covers detectability,generation quality,and adversarial robustness,enabling fair comparison of different mechanisms under consistent model environments and decoding conditions.Overall,the slot layer provides a universal structural foundation for text watermarking.It not only addresses the issue of evaluation fragmentation but also establishes a standardized architectural pathway for modular deployment,hot-swappable switching,and the integration of multiple watermarking mechanisms.

关键词

大语言模型 / 文本水印 / 结构解耦 / 统一评测 / 加工算子

Key words

large language models (LLMs) / text watermarking / structural decoupling / unified evaluation / processing operators

[1] OpenAI.Gpt-4 technical report[R/OL].(2023-03-14)[2026-04-13].https://openai.com/research/gpt-4.
[2] 张坤,李博,陈熙,等.基于语义的鲁棒文本水印算法[J].大数据,2024,10(6):49-61.
[3] CRFM.On the opportunities and risks of foundation models[EB/OL].(2021-08-16)[2025-12-04].https://crfm.stanford.edu/report.html.
[4] 秦中元,王甜甜,刘伟强,等.大语言模型水印技术研究进展[J].信息网络安全,2025,25(2):177-193.
[5] Dathathri S,See A,Ghaisas S,et al.Scalable watermarking for identifying large language model outputs[J].Nature,2024,634:818-823.
[6] Guo Daya,Yang Dejian,Zhang Haowei,et al.Deepseek-r1:incentivizing reasoning in llms through reinforcement learning[J].Nature,2025,645:633-636.
[7] Vaswani A,Shazeer N,Parmar N,et al.Attention is all you need[C]//Proceedings of the 31st International Conference on Neural Information Processing Systems (NIPS).Long Beach,CA,USA,2017:5998-6008.
[8] Kirchenbauer J,Geiping J,Wen Yuxin,et al.A watermark for large language models[C]//Proceedings of the 40th International Conference on Machine Learning (ICML).Hawaii:PMLR,2023:17061-17084.
[9] Zhao Xuandong,Ananth P,Li Lei,et al.Provable robust watermarking for AI-generated text[C]//Proceedingsof the 12th International Conference on Learning Representations (ICLR).Vienna,Austria,2024:1-18.
[10] Liu Han,Wang Kun,Zhang Min,et al.An entropy-based text watermarking detection method[C]//Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (ACL).Bangkok,Thailand,2024:11724-11735.
[11] Pan Liangyin,Liu Aiwei,He Ziwen,et al.Markllm:an open-source toolkit for LLM watermarking[C]//Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing:System Demonstrations.Miami,Florida,USA,2024:61-71.
[12] Zhang Ruisi,Koushanfar F.Watermarking large language models and the generated content:opportunities and challenges[PP/OL].arXiv(2024-10-21)[2025-12-04].https://arxiv.org/abs/2410.19096.
[13] Zhang Ruisi,Hussain S S,Neekhara P,et al.REMARK-LLM:a robust and efficient watermarking framework for generative large language models[C/OL]//33rd USENIX Security Symposium (USENIX Security 24).USENIX Association,2024:1813-1830[2025-12-04].https://www.usenix.org/system/files/usenixsecurity24-zhang-ruisi.pdf.
[14] Kirchenbauer J,Geiping J,Wen Yuxin,et al.On the reliability of watermarks for large language models[C]//Proceedings of the 12th International Conference on Learning Representations (ICLR).Vienna,Austria,2024:1-22.
[15] Casella G,Berger R L.Statistical inference[M].2nd ed.Pacific Grove:Duxbury,2002.
[16] Bai Jinze,Bai Shuai,Chu Yunfei,et al.Qwen technical report[PP/OL].arXiv(2023-09-28)[2026-04-13].https://arxiv.org/abs/2309.16609.
[17] Fan Angela,Lewis Mike,Dauphin Yann,et al.Eli5:long form question answering[PP/OL].arXiv(2019-07-21)[2026-04-13].https://arxiv.org/abs/1907.09190.
[18] Zhang Susan,Roller Stephen,Goyal Naman,et al.Opt:open pre-trained transformer language models[PP/OL].arXiv(2022-05-03)[2026-04-13].https://arxiv.org/abs/2205.01068.
[19] Wang Wenhui,Wei Furu,Dong Li,et al.Minilm:deep self-attention distillation for task-agnosticcompression of pre-trained transformers[PP/OL].arXiv(2020-02-24)[2026-04-13].https://arxiv.org/abs/2002.10957.
[20] Fawcett T.An introduction to ROC analysis[J].Pattern Recognition Letters,2006,27(8):861-874.
[21] Van Rijsbergen C J.Information retrieval[M].London:Butterworths,1979.
[22] Jelinek F,Mercer R L,Bahl L R,et al.Perplexity:a measure of the difficulty of speech recognition tasks[J].Journal of the Acoustical Society of America,1977,62(S1):S63.
[23] Reimers N,Gurevych I.Sentence-bert:sentence embeddings using siamese bert-networks[PP/OL].arXiv(2019-08-27)[2026-04-13].https://arxiv.org/abs/1908.10084.
[24] Rastogi S,Pruthi D.Revisiting the robustness of watermarking to paraphrasing attacks[C]//Proceedings ofthe 2024 Conference on Empirical Methods in Natural Language Processing (EMNLP).Miami,Florida,USA,2024:18100-18110.

基金资助

浙江省领军型创新创业团队(2024R02004)