• 《工程索引》(EI)刊源期刊
  • 中文核心期刊(综合性理工农医类)
  • 中国科技论文统计源期刊
  • 中国科学引文数据库来源期刊

留言板

尊敬的读者、作者、审稿人, 关于本刊的投稿、审稿、编辑和出版的任何问题, 您可以本页添加留言。我们将尽快给您答复。谢谢您的支持!

姓名
邮箱
手机号码
标题
留言内容
验证码

基于ALBERT与双向GRU的中医脏腑定位模型

张德政 范欣欣 谢永红 蒋彦钊

张德政, 范欣欣, 谢永红, 蒋彦钊. 基于ALBERT与双向GRU的中医脏腑定位模型[J]. 工程科学学报. doi: 10.13374/j.issn2095-9389.2021.01.13.002
引用本文: 张德政, 范欣欣, 谢永红, 蒋彦钊. 基于ALBERT与双向GRU的中医脏腑定位模型[J]. 工程科学学报. doi: 10.13374/j.issn2095-9389.2021.01.13.002
ZHANG De-zheng, FAN Xin-xin, XIE Yong-hong, JIANG Yan-zhao. Localization model of traditional Chinese medicine Zang-fu based on ALBERT and Bi-GRU[J]. Chinese Journal of Engineering. doi: 10.13374/j.issn2095-9389.2021.01.13.002
Citation: ZHANG De-zheng, FAN Xin-xin, XIE Yong-hong, JIANG Yan-zhao. Localization model of traditional Chinese medicine Zang-fu based on ALBERT and Bi-GRU[J]. Chinese Journal of Engineering. doi: 10.13374/j.issn2095-9389.2021.01.13.002

基于ALBERT与双向GRU的中医脏腑定位模型

doi: 10.13374/j.issn2095-9389.2021.01.13.002
基金项目: 国家重点研发计划云计算和大数据专项资助项目(2017YFB1002304)
详细信息
    通讯作者:

    E-mail: xieyh@ustb.edu.cn

  • 中图分类号: TP391.1

Localization model of traditional Chinese medicine Zang-fu based on ALBERT and Bi-GRU

More Information
  • 摘要: 脏腑定位,即明确病变所在的脏腑,是中医脏腑辨证的重要阶段。本文旨在通过神经网络模型搭建中医脏腑定位模型,输入症状文本信息,输出对应的病变脏腑标签,为实现中医辅助诊疗的脏腑辨证提供支持。将中医的脏腑定位问题建模为自然语言处理中的多标签文本分类问题,基于中医的医案数据,提出一种基于预训练模型ALBERT和双向门控循环单元(Bi-GRU)的脏腑定位模型。对比实验和消融实验的结果表明,本文提出的方法在中医脏腑定位的问题上相比于多层感知机模型、决策树模型具有更高的准确性,与Word2Vec文本表示方法相比,本文使用的ALBERT预训练模型的文本表示方法有效提升了模型的准确率。在模型参数上,ALBERT预训练模型相比BERT模型降低了模型参数量,有效减小了模型大小。最终,本文提出的脏腑定位模型在测试集上F1值达到了0.8013。

     

  • 图  1  脏腑定位模型结构

    Figure  1.  Zang-fu localization model structure

    图  2  ALBERT模型结构

    Figure  2.  ALBERT model structure

    图  3  GRU单元

    Figure  3.  GRU unit

    图  4  双向GRU模型示意图

    Figure  4.  Bi-GRU model diagram

    表  1  脏腑定位数据格式

    Table  1.   Zang-fu location data format

    No.SymptomsTag
    1Legs ache, and wake up unable to sleep, along with hemoptysis and a sore throatspleen, kidney, heart
    2The patient had high blood pressure, weakness in the right limb, and pain in the left upper armliver, kidney
    下载: 导出CSV

    表  2  训练过程中的参数

    Table  2.   Parameters in the training process

    Parameter nameParameter value
    Max_seq_lenth128
    GRU_units128
    Dropout0.4
    Learning_rate1×10−4
    Epochs10
    Batch_size128
    下载: 导出CSV

    表  3  多标签分类对比实验结果

    Table  3.   Comparative experimental results of multiple label classification

    No.MethodPrecisionRecallF1-value
    1Word2Vec+Bi-GRU0.80150.76530.7830
    2MLP Classifier0.70910.70670.7079
    3Decision Tree Classifier0.67440.66330.6688
    4ALBERT+Bi-GRU0.83010.77450.8013
    下载: 导出CSV

    表  4  BERT与ALBERT对比实验结果

    Table  4.   Comparative experimental results of BERT and ALBERT

    IdMethodPrecisionRecallF1-valueTime/sModel_
    parameters/
    MB
    1BERT+Bi-GRU0.82530.77830.801199.8219363.3
    2ALBERT+Bi-GRU0.83010.77450.801384.704537.3
    下载: 导出CSV

    表  5  多标签分类消融实验结果

    Table  5.   Ablation experiment multiple label classification results

    MethodPrecisionRecallF1-value
    ALBERT0.77110.73150.7508
    ALBERT+Bi-GRU0.83010.77450.8013
    下载: 导出CSV
  • [1] Xu Q. Mining the Syndrome Factor Distribution of AECOPD by the Attribution Model Built by Directed Graph [Dissertation]. Chengdu: Chengdu University of TCM, 2017

    许强. 基于有向图的证素归因模型挖掘AECOPD的证素分布规律[学位论文]. 成都: 成都中医药大学, 2017
    [2] Yin D, Zhou L, Zhou Y M, et al. Study on design of graph search pattern of knowledge graph of TCM classic prescriptions. Chin J Inf Tradit Chin Med, 2019, 26(8): 94 doi: 10.3969/j.issn.1005-5304.2019.08.019

    尹丹, 周璐, 周雨玫, 等. 中医经方知识图谱“图搜索模式”设计研究. 中国中医药信息杂志, 2019, 26(8):94 doi: 10.3969/j.issn.1005-5304.2019.08.019
    [3] Liu C, Gao J L, Dong Y, et al. Study on TCM syndrome differentiation and diagnosis model based on BP neural network for syndrome elements and their common combinations in patients with borderline coronary lesion. Chin J Inf Tradit Chin Med, 2021, 28(3): 104

    刘超, 高嘉良, 董艳, 等. 基于BP神经网络的冠状动脉临界病变患者证候要素及其常见组合中医辨证诊断模型研究. 中国中医药信息杂志, 2021, 28(3):104
    [4] Chu N. Research on Hybrid Intelligent Based Syndrome Differentiation System for Traditional Chinese Medicine [Dissertation]. Shanghai: Shanghai Jiaotong University, 2012

    褚娜. 基于混合智能的中医辨证系统研究[学位论文]. 上海: 上海交通大学, 2012
    [5] Yang K M. Research on Clinical Data Mining Technology of Diabetes TCM [Dissertation]. Kunming: Kunming University of Science and Technology, 2013

    杨开明. 糖尿病中医临床数据挖掘技术研究[学位论文]. 昆明: 昆明理工大学, 2013
    [6] Zhou L, Li G G, Sun Y, et al. Construction of intelligent syndrome differentiation and formula selection of compound structure model. World Chin Med, 2018, 13(2): 479 doi: 10.3969/j.issn.1673-7202.2018.02.057

    周璐, 李光庚, 孙燕, 等. 复合结构智能化辨证选方模型的构建. 世界中医药, 2018, 13(2):479 doi: 10.3969/j.issn.1673-7202.2018.02.057
    [7] Shu X, Cao Y, Huang X, et al. Construction of prediction model of qi deficiency syndrome in acute ischemic stroke based on neural network analysis technique. Glob Tradit Chin Med, 2019, 12(11): 1650 doi: 10.3969/j.issn.1674-1749.2019.11.007

    舒鑫, 曹云, 黄幸, 等. 基于神经网络分析技术的急性缺血性卒中气虚证预测模型构建的研究. 环球中医药, 2019, 12(11):1650 doi: 10.3969/j.issn.1674-1749.2019.11.007
    [8] Shen C B, Wang Z H, Sun Y G. A multi-label classification algorithm based on label clustering. Comput Eng Softw, 2014, 35(8): 16 doi: 10.3969/j.issn.1003-6970.2014.08.004

    申超波, 王志海, 孙艳歌. 基于标签聚类的多标签分类算法. 软件, 2014, 35(8):16 doi: 10.3969/j.issn.1003-6970.2014.08.004
    [9] Huang Z Q. Multi-Label Classification and Label Completion Algorithm Based on K-Means [Dissertation]. Anqing: Anqing Normal University, 2020

    黄志强. 基于K-means的多标签分类及标签补全算法[学位论文]. 安庆: 安庆师范大学, 2020
    [10] Li D Y, Luo F, Wang S G. A multi-label emotion classification method for Chinese text based on CNN and tag features. J Shanxi Univ Nat Sci Ed, 2020, 43(1): 65

    李德玉, 罗锋, 王素格. 融合CNN和标签特征的中文文本情绪多标签分类. 山西大学学报(自然科学版), 2020, 43(1):65
    [11] Joulin A, Grave E, Bojanowski P, et al. Bag of Tricks for Efficient Text Classification // Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics. Valencia, 2017: 427
    [12] Yi S X, Yin H P, Zheng H Y. Public security event trigger identification based on Bidirectional LSTM. Chin J Eng, 2019, 41(9): 1201

    易士翔, 尹宏鹏, 郑恒毅. 基于BiLSTM的公共安全事件触发词识别. 工程科学学报, 2019, 41(9):1201
    [13] Chen G B, Ye D H, Xing Z C, et al. Ensemble application of convolutional and recurrent neural networks for multi-label text categorization // 2017 International Joint Conference on Neural Networks (IJCNN). Anchorage, 2017: 2377
    [14] Yogatama D, Dyer C, Ling W, et al. Generative and discriminative text classification with recurrent neural networks[J/OL]. ArXiv Preprin (2017-03-06) [2020-12-29]. https://arxiv.org/abs/1703.01898v1
    [15] Wang B X. Disconnected Recurrent Neural Networks for Text Categorization // Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics. Melbourne, 2018: 2311
    [16] Kim Y. Convolutional Neural Networks for Sentence Classification // Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP). Doha, 2014: 1746
    [17] Mikolov T, Sutskever I, Chen K, et al. Distributed representations of words and phrases and their compositionality[J/OL]. arXiv preprint (2013-10-16) [2021-5-22]. https://arxiv.org/abs/1310.4546
    [18] Pennington J, Socher R, Manning C. Glove: Global Vectors for Word Representation // Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP). Doha, 2014: 1532
    [19] Vaswani A, Shazeer N, Parmar N, et al. Attention is all you need [J/OL]. arXiv preprint (2017-6-12) [2021-5-22]. https://arxiv.org/abs/1706.03762
    [20] Devlin J, Chang M W, Lee K, et al. BERT: Pre-Training of Deep Bidirectional Transformers for Language Understanding. //Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics. Minneapolis, Minnesota, 2018: 4171
    [21] Yang Z L, Dai Z H, Yang Y M, et al. Xlnet: Generalized autoregressive pretraining for language understanding[J/OL]. arXiv preprint (2019-6-19) [2021-5-23]. https://arxiv.org/abs/1906.08237
    [22] Liu Y, Ott M, Goyal N, et al. Roberta: A robustly optimized bert pretraining approach[J/OL]. arXiv preprint (2019-07-26) [2020-12-29]. http://arxiv.org/abs/1907.11692
    [23] Sanh V, Debut L, Chaumond J, et al. DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter[J/OL]. arXiv preprint (2019-10-02) [2020-12-29]. http://arxiv.org/abs/1910.01108
    [24] Lei J S, Qian Y. Chinese-text classification method based on ERNIE-BiGRU. J Shanghai Univ Electr Power, 2020, 36(4): 329 doi: 10.3969/j.issn.2096-8299.2020.04.003

    雷景生, 钱叶. 基于ERNIE-BiGRU模型的中文文本分类方法. 上海电力大学学报, 2020, 36(4):329 doi: 10.3969/j.issn.2096-8299.2020.04.003
    [25] Lan Z Z, Chen M, Goodman S, et al. ALBERT: A lite BERT for self-supervised learning of language representations. //ICLR 2020 : Eighth International Conference on Learning Representations. Addis Ababa, 2020
    [26] Chung J, Gulcehre C, Cho K, et al. Empirical evaluation of gated recurrent neural networks on sequence modeling [J/OL]. ArXiv Preprin (2018-08-13) [2020-12-29]. http://arxiv.org/abs/1412.3555
  • 加载中
图(4) / 表(5)
计量
  • 文章访问数:  150
  • HTML全文浏览量:  76
  • PDF下载量:  15
  • 被引次数: 0
出版历程
  • 收稿日期:  2021-01-13
  • 网络出版日期:  2021-03-02

目录

    /

    返回文章
    返回