高空直连试验台进气压力模拟系统DDPG前馈补偿智能控制

DDPG feedforward compensation intelligent control for intake pressure simulation system of high- altitude direct-connected test bench

摘要

提出一种基于深度强化学习的高空直连试验台进气压力模拟系统前馈补偿控制方法。研究并给出深度确定性策略梯度（deep deterministic policy gradient，DDPG）前馈补偿控制器的状态参数选取、动作输出设计、奖励函数设置等关键步骤，有效提高了前馈控制器的扰动感知能力，解决了单纯PID控制器主导所带来的智能体局部最优问题。仿真结果表明：与单一PID控制器相比，所设计的控制器在高空舱进气压力扰动和发动机流量扰动下，均实现了进气压力的无超调控制，且调节时间更短，验证了DDPG智能前馈补偿控制设计的快速性、稳定性和鲁棒性。

Abstract

A feedforward compensation control method for the intake pressure simulation system of high-altitude direct-connected test bench based on deep reinforcement learning was proposed. The key steps of state parameter selection， action output design and reward function setting of the Deep Deterministic Policy Gradient （DDPG） feedforward compensation controller were given， which effectively improved the disturbance perception ability of the controller and solved the agent local optimal problem caused by the dominance of a single PID controller. The simulation results show that， compared with the single PID controller， the controller designed can achieve no overshoot control of the intake pressure under the disturbance of the intake pressure of the high-altitude cabin and the disturbance of the engine flow， and the adjustment time is shorter. The rapidity， stability and robustness of the DDPG intelligent feedforward compensation control design are verified.

关键词

高空直连试验台 / 进气压力模拟系统 / 前馈补偿控制 / 强化学习 / 深度确定性策略梯度

Key words

high-altitude direct-connected test bench / intake pressure simulation system / feedforward compensation control / reinforcement learning / DDPG

基金资助

国家自然科学基金(61873172)

国家科技重大专项(2017-V-0014-0066)

中央引导地方科技发展专项(2021JH6/10500162)

辽宁省教育厅项目(JYT2020154)

热能动力技术重点实验室开放基金(TPL2020C01)