基于Q学习算法的随机离散时间系统的随机线性二次最优追踪控制
作者:
作者单位:

作者简介:

通讯作者:

中图分类号:

O232;TP13

基金项目:

国家自然科学基金(61873099,62073144);广东省自然科学基金(2020A1515010441);广州市科技计划(202002030158,202002030389)


Stochastic linear quadratic optimal tracking control for stochastic discrete time systems based on Q-learning
Author:
Affiliation:

Fund Project:

  • 摘要
  • |
  • 图/表
  • |
  • 访问统计
  • |
  • 参考文献
  • |
  • 相似文献
  • |
  • 引证文献
  • |
  • 资源附件
  • |
  • 文章评论
    摘要:

    针对随机线性离散时间系统,利用Q学习算法求解无限时域的随机线性二次最优追踪控制(SLQT)问题.首先,假设通过命令生成器生成追踪所需的参考信号,并建立一个由原随机系统和参考轨迹系统组成的增广系统,把最优追踪问题转化为最优调节问题的形式.其次,为了在线求解随机系统的最优追踪问题,将随机系统转为确定性系统,并根据增广系统定义随机线性二次最优追踪控制的Q函数,在无需知道系统模型参数的情况下在线求解增广随机代数方程(GSAE).再次,证明了Q学习算法和增广随机代数方程的等价性,给出了Q学习算法实现步骤.最后,给出一个仿真实例说明Q学习算法的有效性.

    Abstract:

    For stochastic linear discrete time systems, a Q-learning algorithm is proposed in this paper to solve the stochastic linear quadratic optimal tracking control problem in the infinite time domain.First, it is assumed that the reference signal required for tracking is generated by the command generator, and an augmented system consisting of the original stochastic system and the reference trajectory system is established, then the optimal tracking problem is transformed into an optimal regulation problem.Second, in order to solve the optimal tracking problem online, the stochastic system is transformed into a deterministic one, the Q function of stochastic linear quadratic optimal tracking control is defined according to the augmented system, and the augmented stochastic algebraic equation is solved online without knowing the parameters of the system model.Third, the equivalence between the Q-learning algorithm and the augmented stochastic algebraic equation is proved, and the implementation steps of the Q-learning algorithm are given.Finally, a simulation example is given to illustrate the effectiveness of the proposed Q-learning algorithm.

    参考文献
    相似文献
    引证文献
引用本文

张正义,赵学艳.基于Q学习算法的随机离散时间系统的随机线性二次最优追踪控制[J].南京信息工程大学学报(自然科学版),2021,13(5):548-555
ZHANG Zhengyi, ZHAO Xueyan. Stochastic linear quadratic optimal tracking control for stochastic discrete time systems based on Q-learning[J]. Journal of Nanjing University of Information Science & Technology, 2021,13(5):548-555

复制
分享
文章指标
  • 点击次数:
  • 下载次数:
  • HTML阅读次数:
  • 引用次数:
历史
  • 收稿日期:2021-09-12
  • 最后修改日期:
  • 录用日期:
  • 在线发布日期: 2021-12-02
  • 出版日期:

地址:江苏省南京市宁六路219号    邮编:210044

联系电话:025-58731025    E-mail:nxdxb@nuist.edu.cn

南京信息工程大学学报 ® 2024 版权所有  技术支持:北京勤云科技发展有限公司