基于云边协同的室内火灾监测大模型智能体研究

Large-model intelligent agent for indoor fire monitoring based on cloud-edge collaboration

  • 摘要: 室内火灾严重威胁生命财产安全,现有检测方法存在误报率高、预警与处置脱节等问题. 为此,本研究提出一种基于云边协同与多模态大模型的火灾监测智能体系统,实现从火灾感知到应急决策的全流程自动化. 系统由轻量火灾检测模型与云端多模态智能体协同构成. 首先,基于无参数量的特征提取模块(Hodges–Lehmann-based feature extraction, HLFE),提出基于霍奇斯–莱曼估计的轻量化混合检测模型(Hodges–Lehmann lightweight hybrid you only look once, HLLH-YOLO),在参数量降低20%以上的同时保持同级别模型的检测精度,可部署于边缘设备实时识别火情. 其次,构建基于视觉–语言大模型的火灾应急智能体FireAgent,对预警图像进行二次验证,并自动完成灾情分析、告警推送与检索增强(Retrieval-augmented generation, RAG)的处置建议生成. 实验表明,所提检测模型在精度与轻量化方面均优于主流方法,而智能体能有效解决火灾检测中的误报问题,智能分析现场的火灾情况并快速生成应急决策建议. 本研究为室内火灾防控提供了实时、可靠、一体化的解决方案.

     

    Abstract: Indoor fires severely threaten human lives and property. Although deep learning-based fire detection has advanced significantly, existing vision-based methods present two critical limitations: a high false alarm rate caused by environmental interference and lighting variations; and a disconnect between early warning and emergency response, where most systems merely trigger alarms without providing actionable decision support. Deploying computationally intensive detection models on resource-constrained edge devices remains challenging, while transmitting all video data to the cloud incurs excessive bandwidth and latency costs. Hence, this study proposes a cloud-edge collaborative intelligent agent system for indoor fire monitoring, which affords full-process automation from fire perception to emergency decision-making. The system integrates a lightweight detection model at the edge and a multimodal intelligent agent in the cloud, thus forming a cascaded “edge screening–cloud deliberation” pipeline. At the edge layer, a novel parameter-free feature-extraction module named Hodges–Lehmann-based feature extraction (HLFE) is proposed and embedded into the You Only Look Once verion 8 (YOLOv8s) architecture to construct an HLLH-YOLO model. Inspired by robust statistics, HLFE extracts three complementary statistical features—maximum pooling, adaptive average pooling, and the L2 norm—from feature maps and fuses them via pairwise averaging followed by a median operation. This design effectively suppresses outliers and noise-induced deviations without introducing any additional trainable parameters. Combined with varifocal loss for jointly optimizing classification and localization, HLLH-YOLO maintains detection accuracy comparable to mainstream models while reducing parameters by over 20%, thus enabling real-time deployment on edge devices for continuous video stream analysis. Once HLLH-YOLO triggers an alarm, the suspicious frame is uploaded to the cloud-based FireAgent. Instead of directly invoking a generic vision-language model (VLM), this study first constructs a dedicated multimodal evaluation dataset containing both actual fire scenarios and challenging interference samples. Subsequently, several open-source VLMs are benchmarked to identify Qwen2.5-VL-7B as the optimal backbone for accurate fire-scene understanding and secondary verification. FireAgent uses this model to rigorously verify the alert, thus eliminating residual false positives from the edge stage. Upon confirmation of a genuine fire, the visual information is automatically converted into a structured textual disaster description. Next, a large language model processes this description with retrieval-augmented generation. Consequently, relevant historical fire cases and safety protocols are retrieved to enable context-aware risk analyses, alarm notifications, and actionable emergency response suggestions, thereby closing the loop from perception to decision. To validate the framework, a dual-level dataset is constructed: a large-scale image dataset comprising approximately 30000 images encompassing diverse indoor scenarios, lighting conditions, and smoke occlusion levels for training and evaluating detection models, and a customized multimodal reasoning dataset for assessing VLM inference capabilities. Comparative experiments demonstrate that HLLH-YOLO outperforms mainstream methods in balancing accuracy and lightweight design, while FireAgent effectively resolves false alarms and rapidly generates reliable emergency decisions. This study offers a holistic cloud-edge solution that synergizes a structurally improved parameter-free detection module with a multimodal agent. By addressing both the algorithmic constraints of edge detection and the limitations of emergency response, the proposed system provides a real-time, reliable, and integrated approach to indoor fire prevention, monitoring, and emergency management.

     

/

返回文章
返回