王若男, 董琦. 基于学习机制的多智能体强化学习综述[J]. 工程科学学报, 2024, 46(7): 1251-1268. DOI: 10.13374/j.issn2095-9389.2023.08.08.003
引用本文: 王若男, 董琦. 基于学习机制的多智能体强化学习综述[J]. 工程科学学报, 2024, 46(7): 1251-1268. DOI: 10.13374/j.issn2095-9389.2023.08.08.003
WANG Ruonan, DONG Qi. Multiagent game decision-making method based on the learning mechanism[J]. Chinese Journal of Engineering, 2024, 46(7): 1251-1268. DOI: 10.13374/j.issn2095-9389.2023.08.08.003
Citation: WANG Ruonan, DONG Qi. Multiagent game decision-making method based on the learning mechanism[J]. Chinese Journal of Engineering, 2024, 46(7): 1251-1268. DOI: 10.13374/j.issn2095-9389.2023.08.08.003

基于学习机制的多智能体强化学习综述

Multiagent game decision-making method based on the learning mechanism

  • 摘要: 强化学习作为人工智能领域的重要分支,以其在多智能体系统决策中的卓越表现,成为当前主流方法. 然而,传统的多智能体强化学习算法在面对维度爆炸、训练样本稀缺和难以迁移等方面仍然存在困难. 为了克服这些挑战并提升算法性能,本文从学习机制的角度入手,深入研究学习机制与强化学习的深度融合,以推动多智能体强化学习算法的发展. 首先,介绍了多智能体强化学习算法的基本原理、发展历程以及算法所面临的难点. 随后,引入了基于学习机制的多智能体强化学习方法这一种新兴方向. 这些学习机制,如元学习和迁移学习,被证明可以有效提升多智能体的学习速度,并缓解维度爆炸等问题. 按照课程学习、演化博弈、元学习、分层学习、迁移学习等学习机制在多智能体强化学习中的应用进行了综述,通过罗列这些方法的研究成果,论述了各种方法的局限性,并提出了未来改进的方向. 总结了这类融合算法在实际应用中取得的提升成果和实际应用,具体列举了基于学习机制的多智能体强化学习算法在交通控制、游戏领域的实际应用案例. 同时,对这类融合算法未来在理论、算法和应用方面的发展方向进行了深入分析. 这涵盖了对新颖理论的探索、算法性能的进一步优化,以及在更广泛领域中的推广应用. 通过这样的综述和分析,为未来多智能体强化学习算法的研究方向和实际应用提供了有益的参考.

     

    Abstract: Reinforcement learning, a cornerstone in the expansive landscape of artificial intelligence, has asserted its dominance as the prevailing methodology in contemporary multiagent system decision-making because of its formidable efficacy. However, the path to the zenith of algorithmic excellence is fraught with challenges intrinsic to traditional multiagent reinforcement learning algorithms, such as dimensionality explosion, scarcity of training samples, and the labyrinthine nature of migration processes. In a concerted effort to surmount these formidable challenges and propel the evolution of algorithmic prowess, this paper unfurls its inquiry from the perspective of learning mechanisms and undertakes an exhaustive exploration of the symbiotic integration of learning mechanisms and reinforcement learning. At the inception of this scholarly expedition, we meticulously delineate the rudimentary principles underpinning multiagent algorithms, present a historical trajectory tracing their developmental evolution, and cast a discerning eye upon the salient challenges that have been formidable impediments in their trajectory. The ensuing narrative charts a course into the avant-garde realm of multiagent reinforcement learning methods anchored in learning mechanisms, a paradigmatic shift that emerges as an innovative frontier in the field. Among these learning mechanisms, meta-learning and transfer learning are empirically validated as useful instruments in hastening the learning trajectory of multiagent systems and simultaneously mitigating the intricate challenges posed by dimensionality explosion. This paper assumes the role of a sagacious guide through the labyrinthine landscape of multiagent reinforcement learning, focusing on the manifold applications of learning mechanisms across diverse domains. A comprehensive review delineates the impact of learning mechanisms in curriculum learning, evolutionary games, meta-learning, hierarchical learning, and transfer learning. The research outcomes within these thematic realms are methodically cataloged, with a discerning eye cast upon the limitations inherent in each methodology and erudite propositions for the trajectory of future improvements. The discourse pivots toward synthesizing advancements and accomplishments wrought by fusion algorithms in practical milieus. This paper meticulously examines the transformative impact of fusion algorithms in real-world applications, with a detailed exposition of their deployment in domains as diverse as traffic control and gaming. Simultaneously, an incisive analysis charting the future trajectory of fusion algorithms is conducted. This prediction encompasses exploring nascent theories, refining algorithmic efficacy, and expanding dissemination and application across a broader spectrum of domains. Through this scholarly odyssey, this paper provides an invaluable compass for navigating the uncharted waters of future research endeavors and the judicious deployment of multiagent reinforcement learning algorithms in pragmatic scenarios.

     

/

返回文章
返回