• 《工程索引》(EI)刊源期刊
  • 综合性科学技术类中文核心期刊
  • 中国科技论文统计源期刊
  • 中国科学引文数据库来源期刊

留言板

尊敬的读者、作者、审稿人, 关于本刊的投稿、审稿、编辑和出版的任何问题, 您可以本页添加留言。我们将尽快给您答复。谢谢您的支持!

姓名
邮箱
手机号码
标题
留言内容
验证码

基于强化学习的工控系统恶意软件行为检测方法

高洋 王礼伟 任望 谢丰 莫晓锋 罗熊 王卫苹 杨玺

高洋, 王礼伟, 任望, 谢丰, 莫晓锋, 罗熊, 王卫苹, 杨玺. 基于强化学习的工控系统恶意软件行为检测方法[J]. 工程科学学报, 2020, 42(4): 455-462. doi: 10.13374/j.issn2095-9389.2019.09.16.005
引用本文: 高洋, 王礼伟, 任望, 谢丰, 莫晓锋, 罗熊, 王卫苹, 杨玺. 基于强化学习的工控系统恶意软件行为检测方法[J]. 工程科学学报, 2020, 42(4): 455-462. doi: 10.13374/j.issn2095-9389.2019.09.16.005
GAO Yang, WANG Li-wei, REN Wang, XIE Feng, MO Xiao-feng, LUO Xiong, WANG Wei-ping, YANG Xi. Reinforcement learning-based detection method for malware behavior in industrial control systems[J]. Chinese Journal of Engineering, 2020, 42(4): 455-462. doi: 10.13374/j.issn2095-9389.2019.09.16.005
Citation: GAO Yang, WANG Li-wei, REN Wang, XIE Feng, MO Xiao-feng, LUO Xiong, WANG Wei-ping, YANG Xi. Reinforcement learning-based detection method for malware behavior in industrial control systems[J]. Chinese Journal of Engineering, 2020, 42(4): 455-462. doi: 10.13374/j.issn2095-9389.2019.09.16.005

基于强化学习的工控系统恶意软件行为检测方法

doi: 10.13374/j.issn2095-9389.2019.09.16.005
基金项目: 国家自然科学基金资助项目(U1736117,U1836106);北京市自然科学基金资助项目(19L2029,9204028);北京市智能物流系统协同创新中心开放课题资助项目(BILSCIC-2019KF-08);北京科技大学顺德研究生院科技创新专项资金资助项目(BK19BF006);材料领域知识工程北京市重点实验室基本业务费资助项目(FRF-BD-19-012A)
详细信息
    通讯作者:

    E-mail:xluo@ustb.edu.cn

  • 中图分类号: TP273

Reinforcement learning-based detection method for malware behavior in industrial control systems

More Information
  • 摘要: 网络环境下的恶意软件严重威胁着工控系统的安全,随着目前恶意软件变种的逐渐增多,给工控系统恶意软件的检测和安全防护带来了巨大的挑战。现有的检测方法存在着自适应检测识别的智能化程度不高等局限性。针对此问题,围绕威胁工控系统网络安全的恶意软件对象,本文通过结合利用强化学习这一高级的机器学习算法,设计了一个检测应用方法框架。在实现过程中,根据恶意软件行为检测的实际需求,充分结合强化学习的序列决策和动态反馈学习等智能特征,详细讨论并设计了其中的特征提取网络、策略网络和分类网络等关键应用模块。基于恶意软件实际测试数据集进行的应用实验验证了本文方法的有效性,可为一般恶意软件行为检测提供一种智能化的决策辅助手段。
  • 图  1  总体结构

    Figure  1.  Framework

    图  2  测试集上的准确率

    Figure  2.  Accuracy in the test dataset

    图  3  测试集上查准率和查全率随迭代次数的变化

    Figure  3.  Precision and recall in the test dataset

    表  1  分类结果的混淆矩阵

    Table  1.   Confusion matrix

    Confusion matrixPrediction : maliciousPrediction : benign
    Truth : malicious257 (TP)43 (FN)
    Truth : benign4 (FP)296 (TN)
    下载: 导出CSV

    表  2  删除比例最高和最低的各5个API函数

    Table  2.   Five API functions with the highest and lowest deletion rates

    API FunctionsNumber of deleting operationNumber of retaining operationRate of deleting operation
    VirtualAllocEx1742090.454308
    IsDBCSLeadByte891350.397321
    GetSystemDirectoryA1012060.328990
    CreateThread381060.263889
    GetDC822290.263666
    GetProcAddress028830
    CloseHandle028530
    LocalFree019390
    GetModuleFileNameW014850
    lstrlenW014600
    下载: 导出CSV
  • [1] 时忆杰. 移动互联环境下工业控制系统安全问题研究[学位论文]. 北京: 北京邮电大学, 2016

    Shi Y J. Research on the Key Security Issues of Mobile and Open Industrial Control System[Dissertation]. Beijing: Beijing University of Posts and Telecommunications, 2016
    [2] Demontis A, Melis M, Biggio B, et al. Yes, machine learning can be more secure! A case study on android malware detection. IEEE Trans Dependable Secure Comput, 2019, 16(4): 711 doi: 10.1109/TDSC.2017.2700270
    [3] Sharif M, Lanzi A, Giffin J, et al. Impeding malware analysis using conditional code obfuscation // Proceedings of the Network and Distributed System Security Symposium. San Diego, 2008: 1939
    [4] Xiao X, Wang Z, Li Q, et al. Back-propagation neural network on Markov chains from system call sequences: a new approach for detecting Android malware with system call sequences. IET Inf Secur, 2016, 11(1): 8
    [5] Su X, Zhang D F, Li W J, et al. A deep learning approach to android malware feature learning and detection // 2016 IEEE Trustcom/BigDataSE/ISPA. Tianjin, 2016: 244
    [6] Li G L, Gomez R, Nakamura K, et al. Human-centered reinforcement learning: a survey. IEEE Trans Human Mach Syst, 2019, 49(4): 337 doi: 10.1109/THMS.2019.2912447
    [7] Wu C S, Shi J Y, Yang Y X, et al. Enhancing machine learning based malware detection model by reinforcement learning // Proceedings of the 8th International Conference on Communication and Network Security. Qingdao, 2018: 74
    [8] Mnih V, Kavukcuoglu K, Silver D, et al. Human-level control through deep reinforcement learning. Nature, 2015, 518(7540): 529 doi: 10.1038/nature14236
    [9] Schultz M, Eskin E, Zadok F, et al. Data mining methods for detection of new malicious executables // Proceedings of the IEEE Symposium on Security and Privacy. Oakland, 2001: 38
    [10] Santos I, Brezo F, Ugarte-Pedrero X, et al. Opcode sequences as representation of executables for data-mining-based unknown malware detection. Inf Sci, 2013, 231: 64 doi: 10.1016/j.ins.2011.08.020
    [11] Zhang J X, Qin Z, Yin H, et al. IRMD: Malware variant detection using opcode image recognition // Proceedings of the IEEE 22nd International Conference on Parallel and Distributed Systems. Wuhan, 2016: 1175
    [12] Tandon G, Chan P. Learning rules from system call arguments and sequences for anomaly detection // Proceedings of the International Workshop on Data Mining for Computer Security. Melbourne, 2003: 20
    [13] Willems C, Holz T, Freiling F. Toward automated dynamic malware analysis using CWSandbox. IEEE Secur Privacy, 2007, 5(2): 32 doi: 10.1109/MSP.2007.45
    [14] Rieck K, Trinius P, Willems C, et al. Automatic analysis of malware behavior using machine learning. J Comput Secur, 2011, 19(4): 639 doi: 10.3233/JCS-2010-0410
    [15] Ki Y, Kim E, Kim H K. A novel approach to detect malware based on API call sequence analysis. Int J Distrib Sens Netw, 2015, 11(6): 659101 doi: 10.1155/2015/659101
    [16] Busoniu L, Babuška R, De Schutter B. A comprehensive survey of multiagent reinforcement learning. IEEE Trans Syst Man Cybern Part C Appl Rev, 2008, 38(2): 156 doi: 10.1109/TSMCC.2007.913919
    [17] Zhang T Y, Huang M L, Zhao L, et al. Learning structured representation for text classification via reinforcement learning // Proceedings of the Thirty-Second AAAI Conference on Artificial Intelligence. New Orleans, 2018: 6053
  • 加载中
图(3) / 表(2)
计量
  • 文章访问数:  897
  • HTML全文浏览量:  802
  • PDF下载量:  46
  • 被引次数: 0
出版历程
  • 收稿日期:  2019-09-15
  • 刊出日期:  2020-04-01

目录

    /

    返回文章
    返回