吕友豪, 贾袁骏, 庄圆, 董琦. 基于多模态信息融合的四足机器人避障方法(“智能机器人自主协同”专题)[J]. 工程科学学报. DOI: 10.13374/j.issn2095-9389.2023.07.01.002
引用本文: 吕友豪, 贾袁骏, 庄圆, 董琦. 基于多模态信息融合的四足机器人避障方法(“智能机器人自主协同”专题)[J]. 工程科学学报. DOI: 10.13374/j.issn2095-9389.2023.07.01.002
Obstacle avoidance method for quadruped robots based on multimodal information fusion[J]. Chinese Journal of Engineering. DOI: 10.13374/j.issn2095-9389.2023.07.01.002
Citation: Obstacle avoidance method for quadruped robots based on multimodal information fusion[J]. Chinese Journal of Engineering. DOI: 10.13374/j.issn2095-9389.2023.07.01.002

基于多模态信息融合的四足机器人避障方法(“智能机器人自主协同”专题)

Obstacle avoidance method for quadruped robots based on multimodal information fusion

  • 摘要: 本文提出了一种多模态强化学习的神经网络模型以解决四足机器人自主避障问题。当前基于学习的自主避障在机器人应用中取得长足进步,但大多数方法仍然以无外部感知的智能体为研究对象,依赖现实迁移技术来训练最终泛化到适应挑战性的复杂地形。本文的主要思路是,编码器、惯性测量单元等本体感受信息仅提供即时反应的接触测量,不足以支持在复杂场景下自主避障等决策任务,而配备雷达、相机等外部传感器的智能体可以提前感知、预测环境变化,学会通过规划自主穿越有障碍物和不平坦地形的环境。因此本文充分结合了四足机器人本体感受信息和机载外部传感器信息,在对各单独模态数据进行编码以提取有效特征的基础上,利用Transformer层的注意力机制进行多模态信息的融合支撑机器人动作决策,采用Actor-Critic强化学习架构在仿真环境中进行试错训练,最终为四足机器人提供安全运动策略。本文在具有不同障碍物、不平坦地形等具有挑战性的模拟环境中评估所提出的方法,进行多组消融实验,我们观测到该方法可以有效地提高算法避障成功率。除此之外,提出的算法由于引入注意力机制,四足机器人在动态未知环境下的也具有一定的可靠性。

     

    Abstract: In this paper, a multi-modal reinforcement learning neural network model is proposed to solve the quadruped robot motion planning problem. Current learning-based motion planning has made great progress in robotics applications currently, but most approaches still rely on domain randomization to train and eventually generalize to an intelligent body without external sensing that can adapt to challenging terrain. The main idea of this paper is that proprioceptive information such as encoders and inertial measurement units only provide contact measurements for immediate response, while intelligent bodies equipped with external sensors such as radar and cameras can learn to traverse environments with obstacles and uneven terrain autonomously through planning by predicting environmental changes many steps in advance. Therefore, this paper fully combines the quadruped robot proprioceptive information and on-board external sensor information, encodes each individual modal data to extract effective features, and then uses the attention mechanism of Transformer layer to fuse multimodal information to support the robot action decision, and uses the Actor-Critic reinforcement learning architecture for trial-and-error training in a simulation environment to finally provide quadruped robot motion with a safe motion strategy. Specifically, in terms of model design, we use a fully connected neural network to encode the ontology information vector, ConvNet to encode the image information, and PointNet to encode the point cloud information. We hope to minimize the information loss while improving the training and inference speed of the model to better meet the real-time and decision effectiveness of the task. In this paper, the proposed method is evaluated in a challenging simulation environment with different obstacles and uneven terrain for multiple sets of ablation experiments, and we observe that the method can effectively improve the algorithm's obstacle avoidance success rate. In addition to that, the proposed algorithm is still reliable in case of one or more sensor modal failures or unknown environment.

     

/

返回文章
返回