具有加性和乘性噪声的线性离散时间随机系统的无模型最优跟踪控制

Model-free optimal tracking control for linear discrete-time stochastic systems subject to additive and multiplicative noises

中文关键词  随机线性二次跟踪  随机噪声  无模型控制  Q-学习

英文关键词  stochastic linear quadratic tracking  stochastic noise  model-free control  Q-learning

中文摘要

      本文研究一类同时受加性和乘性噪声影响的离散时间随机系统的最优跟踪控制问题. 通过构造由原始系 统和参考轨迹组成的增广系统, 将随机线性二次跟踪控制(SLQT)的成本函数转化为与增广状态相关的二次型函数, 由此推导出用于求解SLQT的贝尔曼方程和增广随机代数黎卡提方程(SARE), 而后进一步针对系统和参考轨迹动 力学信息完全未知的情形, 提出一种Q-学习算法来在线求解增广SARE, 证明了该算法的收敛性, 并采用批处理最 小二乘法(BLS)解决该在线无模型控制算法的实现问题. 通过对单相电压源UPS逆变器的仿真, 验证了所提出控制 方案的有效性.

英文摘要

      This paper focuses on the optimal tracking control problem for a class of discrete-time stochastic systems with additive and multiplicative noises. By constructing an augmented system composed of the original system and the reference generator, the cost function of the stochastic linear quadratic tracking control (SLQT) is transformed into a quadratic function in terms of the augmented state. A Bellman equation and an augmented stochastic algebraic Riccati equation (SARE) for solving the SLQT are then derived. For the case of both the system and reference generator dynamics are completely unknown, a Q-learning algorithm is proposed to solve the augmented SARE, and its convergence is proved rigorously. Moreover, the online model-free control algorithm implements with the batch least square method (BLS). The effectiveness of the proposed control scheme is verified by the single-phase voltage-source UPS inverter.