基于迁移学习与知识蒸馏的管网工程领域语音识别

Speech recognition in pipeline engineering domain based on transfer learning and knowledge distillation

摘要

市政管道工程是城市建设的重点领域，传统信息录入手段记录专家施工经验效率低。语音识别技术可提升效率，但通用模型在该领域的准确率低。为此，提出一种基于迁移学习与知识蒸馏的管网工程领域语音识别模型。该模型采用端到端方式，通过迁移学习将开放域模型参数迁移到领域数据，再用知识蒸馏压缩模型，提升识别精度和效率。试验表明，迁移学习使字错误率降低6.2%,知识蒸馏使模型参数减少83.2 MB,推理速度提高。

Abstract

Municipal pipeline engineering is a key area in urban construction. Traditional method of recording expert construction guidance are inefficient. Speech recognition technology can improve efficiency but often has low accuracy in specialized domains. A speech recognition model was proposed for the pipeline engineering domain based on transfer learning and knowledge distillation. The model uses an end-to-end approach, adapts parameters from an open-domain model to the target domain via transfer learning, and then compresses the model using knowledge distillation.[Results]show that transfer learning reduces the word error rate by 6.2%, and knowledge distillation reduces model parameters by 83.2 MB while improving inference speed.

关键词

管网工程 / 专家语音识别 / 迁移学习 / 知识蒸馏 / 轻量化

Key words

pipeline engineering / expert speech recognition / transfer learning / knowledge distillation / lightweight design

基金资助

中国长江三峡集团有限公司科研项目(202103551)