安全强化学习及其在机器人系统中的应用综述

Safe reinforcement learning and its applications in robotics: A survey

中文关键词  机器人  安全强化学习  约束马尔可夫决策过程  鲁棒性

英文关键词  robotics  safe reinforcement learning  constrained Markov decision process  robustness

中文摘要

      强化学习是一类通过与环境交互实现序贯优化决策的机器学习方法, 已经在游戏、推荐系统及自然语言处 理等任务中得到了应用. 然而, 强化学习算法应用于真实世界中的机器人系统时, 如何保证安全性仍然面临挑战. 近 年来, 针对机器人系统的安全强化学习方法研究已经成为热点方向, 获得了机器人和强化学习领域的广泛关注. 本 文结合现有的工作, 综述了安全强化学习理论和方法的重要成果和发展趋势, 并重点关注了现有方法在机器人领域 的适用性. 本文首先给出了安全强化学习的一般问题描述. 其次, 从方法和性能的角度重点介绍了该领域的最新重 要进展, 包括约束策略优化、控制障碍函数、安全过滤器和对抗性博弈训练等方法, 以及安全强化学习方法在地面 移动机器人系统、无人飞行器和其他机器人系统中的应用情况. 最后, 对该领域的未来研究方向进行了展望和探讨.

英文摘要

      Reinforcement learning is a kind of machine learning method that realizes sequential optimization decisions by interacting with the environment. It has been applied in games, recommendation systems and natural language processing. However, it is still a challenge to ensure the safety of reinforcement learning algorithms when applied to robotics in the real world. In recent years, the safe reinforcement learning methods for robotics systems have become a hot research direction, gaining extensive attention in robotics and reinforcement learning communities. This paper surveys important achievements and development tendency of safe reinforcement learning based on the existing work and focuses on their applicability in robotics. This paper first introduces the general problem description of safe reinforcement learning. Then we focus on the latest significant progress in this field from the perspective of method and performance, including constraint policy optimization, control barrier function, safety filter and adversarial training methods, and their applications in autonomous driving vehicles, unmanned aerial vehicles and other robotic systems. Finally, the future research direction of this field is prospected and discussed.