基于强化学习的波动鳍推进水下作业机器人悬停控制

Hovering control of an underwater vehicle-manipulator system propelled by undulatory fins via reinforcement learning

中文关键词  水下作业机器人  悬停控制  波动鳍  神经网络  强化学习

英文关键词  underwater vehicle-manipulator system  hovering control  undulatory fin  neural network  reinforcement learning

中文摘要

      本文针对波动鳍推进水下作业机器人的悬停控制问题开展研究. 首先, 给出了波动鳍推进水下作业机器人 的运动学模型、动力学模型和波动鳍的参数–力映射模型, 建立了基于马尔可夫决策过程的悬停控制训练框架. 其 次, 基于模型结构和训练策略, 使用强化学习的方法进行网络训练, 得到最佳的悬停控制器. 最终, 在室内水池中完 成了波动鳍推进水下作业机器人的悬停控制实验, 实验结果验证了所提方法的有效性.

英文摘要

      This paper addresses the hovering control of an underwater vehicle-manipulator system (UVMS) propelled by undulatory fins. First, the kinematic and dynamical models of the UVMS and a mapping model between the control parameters of undulatory fins and the driving force of the UVMS are introduced, and a hovering control training framework based on Markov decision process (MDP) is designed. Then, based on the framework and training strategies, the hovering controller is fully trained via reinforcement learning method. Finally, the well-trained controller is applied in the real environment, and the experimental results demonstrate that the proposed method can accomplish the UVMS’s hovering control effectively.