梁鸿涛, 王耀南, 华和安, 钟杭, 郑成宏, 曾俊豪, 梁嘉诚, 李政辰. 无人集群系统深度强化学习控制研究进展[J]. 工程科学学报, 2024, 46(9): 1521-1534. DOI: 10.13374/j.issn2095-9389.2023.07.30.001
引用本文: 梁鸿涛, 王耀南, 华和安, 钟杭, 郑成宏, 曾俊豪, 梁嘉诚, 李政辰. 无人集群系统深度强化学习控制研究进展[J]. 工程科学学报, 2024, 46(9): 1521-1534. DOI: 10.13374/j.issn2095-9389.2023.07.30.001
LIANG Hongtao, WANG Yaonan, HUA Hean, ZHONG Hang, ZHENG Chenghong, ZENG Junhao, LIANG Jiacheng, LI Zhengchen. Deep reinforcement learning to control an unmanned swarm system[J]. Chinese Journal of Engineering, 2024, 46(9): 1521-1534. DOI: 10.13374/j.issn2095-9389.2023.07.30.001
Citation: LIANG Hongtao, WANG Yaonan, HUA Hean, ZHONG Hang, ZHENG Chenghong, ZENG Junhao, LIANG Jiacheng, LI Zhengchen. Deep reinforcement learning to control an unmanned swarm system[J]. Chinese Journal of Engineering, 2024, 46(9): 1521-1534. DOI: 10.13374/j.issn2095-9389.2023.07.30.001

无人集群系统深度强化学习控制研究进展

Deep reinforcement learning to control an unmanned swarm system

  • 摘要: 随着无人集群在物流运输、农业管理、军事行动等场景的试验和应用,其面临的作业环境和任务内容日趋复杂,亟需设计效率更高、泛化能力更强、适应性更好的控制算法. 将人工智能引入到无人集群系统控制的研究中,能够大幅提升现有无人集群的能力,完成复杂的作业任务. 深度强化学习具有深度学习和强化学习的优点,无人集群系统深度强化学习控制研究受到了国内外科研人员的广泛关注,涌现出许多标志性成果. 本文将从原理、特点等方面阐述深度强化学习概念,深入分析深度强化学习的多种典型算法,并讨论无人机集群的各类控制需求,进而介绍深度强化学习在无人机集群控制领域的典型研究成果,最后针对该领域研究成果的落地转化总结了应用前景和面临的挑战.

     

    Abstract: Recently, testing and using micro-unmanned vehicles, such as unmanned aerial vehicles (UAVs), in scenarios such as supply transportation, agricultural management, and military operations have become more common. It is no longer sufficient to control a single UAV to accomplish all missions. With the increasing complexities associated with operating and task requirements, an unmanned swarm requires a series of algorithms with higher efficiency, greater generalization ability, and better adaptability than the earlier algorithms. A combination of unmanned swarms with artificial intelligence is becoming a common solution to manage the above requirements. Deep reinforcement learning (DRL) is a machine learning method that combines deep learning (DL) and reinforcement learning (RL); therefore, this method has the advantages of DL and RL. Using an RL method, an agent can learn from the environment by trial and error and make decisions that autonomously obtain high scores. However, when the given environment is complex, the decision function of the agent may be too difficult to implement and then the agent cannot make the correct decision. The DL method has strong fitting ability. A suitable deep neural network can simulate any linear or nonlinear function. If the DL method is used to simulate the decision function in RL, the hybrid method can solve the problem that an agent cannot solve and make a correct decision in a complex environment. The combination of an unmanned swarm and a DRL method has been widely studied. This paper introduces the concept of DRL from the perspective of principles and characteristics. This paper analyzes several typical DRL algorithms, discusses the various control requirements of a UAV swarm, and then focuses on the achievements of combining DRL and a UAV swarm control. Finally, this paper presents viewpoints on the application prospects and challenges related to landing and transformation in the combination field. The concept of an unmanned swarm originated from the study of the behavior of biological groups. Several species of bees, ants, birds, fish, and other creatures exhibit complex group behaviors. These clusters comprise many independent individuals in accordance with certain aggregation rules to form a coordinated, orderly group movement mechanism. Similar to biological clusters, in the field of robotics or UAVs, unmanned swarm systems are crowded intelligent systems. These systems consist of multiple homogeneous or heterogeneous unmanned equipment to achieve mutual behavior coordination and jointly complete specific tasks through interactive feedback and incentive response of information. In practical applications, an unmanned swarm system needs to meet the requirements of an open environment, a changeable situation, limited resources, and real-time responses. This system needs to have multicore collaborative capabilities such as distributed collaborative perception, intelligent collaborative decision-making, and robust collaborative control. The distributed intelligent collaborative control method based on DRL can fully meet the control requirements of high intelligence and robustness of unmanned cluster systems.

     

/

返回文章
返回