Localization model of traditional Chinese medicine Zang-fu based on ALBERT and Bi-GRU
-
摘要: 脏腑定位，即明确病变所在的脏腑，是中医脏腑辨证的重要阶段。本文旨在通过神经网络模型搭建中医脏腑定位模型，输入症状文本信息，输出对应的病变脏腑标签，为实现中医辅助诊疗的脏腑辨证提供支持。将中医的脏腑定位问题建模为自然语言处理中的多标签文本分类问题，基于中医的医案数据，提出一种基于预训练模型ALBERT和双向门控循环单元（Bi-GRU）的脏腑定位模型。对比实验和消融实验的结果表明，本文提出的方法在中医脏腑定位的问题上相比于多层感知机模型、决策树模型具有更高的准确性，与Word2Vec文本表示方法相比，本文使用的ALBERT预训练模型的文本表示方法有效提升了模型的准确率。在模型参数上，ALBERT预训练模型相比BERT模型降低了模型参数量，有效减小了模型大小。最终，本文提出的脏腑定位模型在测试集上F1值达到了0.8013。Abstract: The rapid development of artificial intelligence (AI) has injected new vitality into various industries and provided new ideas for the development of traditional Chinese medicine (TCM). The combination of AI and TCM provides more technical support for TCM auxiliary diagnosis and treatment. In the history of TCM, many methods of syndrome differentiation have been observed, among which the differentiation of Zang-fu organs is one of the important methods. The purpose of this paper is to provide support for the localization of Zang-fu in TCM through AI technology. Localization of Zang-fu organs is a method of determining the location of lesions in such organs and is an important stage in the differentiation of Zang-fu organs in TCM. In this paper, the localization model of TCM Zang-fu organs through the neural network model was established. Through the input of symptom text information, the corresponding Zang-fu label for a lesion could be output to provide support for the realization of Zang-fu syndrome differentiation in TCM-assisted diagnosis and treatment. In this paper, the localization of Zang-fu organs was abstracted as multi-label text classification in natural language processing. Using the medical record data of TCM, a Zang-fu localization model based on pretraining models a lite BERT (ALBERT) and bidirectional gated recurrent unit (Bi-GRU) was proposed. Comparison and ablation experiments finally show that the proposed method is more accurate than multilayer perceptron and the decision tree. Moreover, using an ALBERT pretraining model for text representation effectively improves the accuracy of the localization model. In terms of model parameters, the ALBERT pretraining model greatly reduces the number of model parameters compared with the BERT model and effectively reduces the model size. Finally, the F1-value of the Zang-fu localization model proposed in this paper reaches 0.8013 on the test set, which provided certain support for the TCM auxiliary diagnosis and treatment.
-
表 1 脏腑定位数据格式
Table 1. Zang-fu location data format
No. Symptoms Tag 1 Legs ache, and wake up unable to sleep, along with hemoptysis and a sore throat spleen, kidney, heart 2 The patient had high blood pressure, weakness in the right limb, and pain in the left upper arm liver, kidney 
下载: 导出CSV
表 2 训练过程中的参数
Table 2. Parameters in the training process
Parameter name Parameter value Max_seq_lenth 128 GRU_units 128 Dropout 0.4 Learning_rate 1×10−4 Epochs 10 Batch_size 128
下载: 导出CSV
表 3 多标签分类对比实验结果
Table 3. Comparative experimental results of multiple label classification
No. Method Precision Recall F1-value 1 Word2Vec+Bi-GRU 0.8015 0.7653 0.7830 2 MLP Classifier 0.7091 0.7067 0.7079 3 Decision Tree Classifier 0.6744 0.6633 0.6688 4 ALBERT+Bi-GRU 0.8301 0.7745 0.8013
下载: 导出CSV
表 4 BERT与ALBERT对比实验结果
Table 4. Comparative experimental results of BERT and ALBERT
Id Method Precision Recall F1-value Time/s Model_
parameters/
MB1 BERT+Bi-GRU 0.8253 0.7783 0.8011 99.8219 363.3 2 ALBERT+Bi-GRU 0.8301 0.7745 0.8013 84.7045 37.3
下载: 导出CSV
表 5 多标签分类消融实验结果
Table 5. Ablation experiment multiple label classification results
Method Precision Recall F1-value ALBERT 0.7711 0.7315 0.7508 ALBERT+Bi-GRU 0.8301 0.7745 0.8013
下载: 导出CSV
-
参考文献
[1] Xu Q. Mining the Syndrome Factor Distribution of AECOPD by the Attribution Model Built by Directed Graph [Dissertation]. Chengdu: Chengdu University of TCM, 2017许强. 基于有向图的证素归因模型挖掘AECOPD的证素分布规律[学位论文]. 成都: 成都中医药大学, 2017 [2] Yin D, Zhou L, Zhou Y M, et al. Study on design of graph search pattern of knowledge graph of TCM classic prescriptions. Chin J Inf Tradit Chin Med, 2019, 26(8): 94 doi: 10.3969/j.issn.1005-5304.2019.08.019尹丹, 周璐, 周雨玫, 等. 中医经方知识图谱“图搜索模式”设计研究. 中国中医药信息杂志, 2019, 26(8):94 doi: 10.3969/j.issn.1005-5304.2019.08.019 [3] Liu C, Gao J L, Dong Y, et al. Study on TCM syndrome differentiation and diagnosis model based on BP neural network for syndrome elements and their common combinations in patients with borderline coronary lesion. Chin J Inf Tradit Chin Med, 2021, 28(3): 104刘超, 高嘉良, 董艳, 等. 基于BP神经网络的冠状动脉临界病变患者证候要素及其常见组合中医辨证诊断模型研究. 中国中医药信息杂志, 2021, 28(3):104 [4] Chu N. Research on Hybrid Intelligent Based Syndrome Differentiation System for Traditional Chinese Medicine [Dissertation]. Shanghai: Shanghai Jiaotong University, 2012褚娜. 基于混合智能的中医辨证系统研究[学位论文]. 上海: 上海交通大学, 2012 [5] Yang K M. Research on Clinical Data Mining Technology of Diabetes TCM [Dissertation]. Kunming: Kunming University of Science and Technology, 2013杨开明. 糖尿病中医临床数据挖掘技术研究[学位论文]. 昆明: 昆明理工大学, 2013 [6] Zhou L, Li G G, Sun Y, et al. Construction of intelligent syndrome differentiation and formula selection of compound structure model. World Chin Med, 2018, 13(2): 479 doi: 10.3969/j.issn.1673-7202.2018.02.057周璐, 李光庚, 孙燕, 等. 复合结构智能化辨证选方模型的构建. 世界中医药, 2018, 13(2):479 doi: 10.3969/j.issn.1673-7202.2018.02.057 [7] Shu X, Cao Y, Huang X, et al. Construction of prediction model of qi deficiency syndrome in acute ischemic stroke based on neural network analysis technique. Glob Tradit Chin Med, 2019, 12(11): 1650 doi: 10.3969/j.issn.1674-1749.2019.11.007舒鑫, 曹云, 黄幸, 等. 基于神经网络分析技术的急性缺血性卒中气虚证预测模型构建的研究. 环球中医药, 2019, 12(11):1650 doi: 10.3969/j.issn.1674-1749.2019.11.007 [8] Shen C B, Wang Z H, Sun Y G. A multi-label classification algorithm based on label clustering. Comput Eng Softw, 2014, 35(8): 16 doi: 10.3969/j.issn.1003-6970.2014.08.004申超波, 王志海, 孙艳歌. 基于标签聚类的多标签分类算法. 软件, 2014, 35(8):16 doi: 10.3969/j.issn.1003-6970.2014.08.004 [9] Huang Z Q. Multi-Label Classification and Label Completion Algorithm Based on K-Means [Dissertation]. Anqing: Anqing Normal University, 2020黄志强. 基于K-means的多标签分类及标签补全算法[学位论文]. 安庆: 安庆师范大学, 2020 [10] Li D Y, Luo F, Wang S G. A multi-label emotion classification method for Chinese text based on CNN and tag features. J Shanxi Univ Nat Sci Ed, 2020, 43(1): 65李德玉, 罗锋, 王素格. 融合CNN和标签特征的中文文本情绪多标签分类. 山西大学学报(自然科学版), 2020, 43(1):65 [11] Joulin A, Grave E, Bojanowski P, et al. Bag of Tricks for Efficient Text Classification // Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics. Valencia, 2017: 427 [12] Yi S X, Yin H P, Zheng H Y. Public security event trigger identification based on Bidirectional LSTM. Chin J Eng, 2019, 41(9): 1201易士翔, 尹宏鹏, 郑恒毅. 基于BiLSTM的公共安全事件触发词识别. 工程科学学报, 2019, 41(9):1201 [13] Chen G B, Ye D H, Xing Z C, et al. Ensemble application of convolutional and recurrent neural networks for multi-label text categorization // 2017 International Joint Conference on Neural Networks (IJCNN). Anchorage, 2017: 2377 [14] Yogatama D, Dyer C, Ling W, et al. Generative and discriminative text classification with recurrent neural networks[J/OL]. ArXiv Preprin (2017-03-06) [2020-12-29]. https://arxiv.org/abs/1703.01898v1 [15] Wang B X. Disconnected Recurrent Neural Networks for Text Categorization // Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics. Melbourne, 2018: 2311 [16] Kim Y. Convolutional Neural Networks for Sentence Classification // Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP). Doha, 2014: 1746 [17] Mikolov T, Sutskever I, Chen K, et al. Distributed representations of words and phrases and their compositionality[J/OL]. arXiv preprint (2013-10-16) [2021-5-22]. https://arxiv.org/abs/1310.4546 [18] Pennington J, Socher R, Manning C. Glove: Global Vectors for Word Representation // Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP). Doha, 2014: 1532 [19] Vaswani A, Shazeer N, Parmar N, et al. Attention is all you need [J/OL]. arXiv preprint (2017-6-12) [2021-5-22]. https://arxiv.org/abs/1706.03762 [20] Devlin J, Chang M W, Lee K, et al. BERT: Pre-Training of Deep Bidirectional Transformers for Language Understanding. //Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics. Minneapolis, Minnesota, 2018: 4171 [21] Yang Z L, Dai Z H, Yang Y M, et al. Xlnet: Generalized autoregressive pretraining for language understanding[J/OL]. arXiv preprint (2019-6-19) [2021-5-23]. https://arxiv.org/abs/1906.08237 [22] Liu Y, Ott M, Goyal N, et al. Roberta: A robustly optimized bert pretraining approach[J/OL]. arXiv preprint (2019-07-26) [2020-12-29]. http://arxiv.org/abs/1907.11692 [23] Sanh V, Debut L, Chaumond J, et al. DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter[J/OL]. arXiv preprint (2019-10-02) [2020-12-29]. http://arxiv.org/abs/1910.01108 [24] Lei J S, Qian Y. Chinese-text classification method based on ERNIE-BiGRU. J Shanghai Univ Electr Power, 2020, 36(4): 329 doi: 10.3969/j.issn.2096-8299.2020.04.003雷景生, 钱叶. 基于ERNIE-BiGRU模型的中文文本分类方法. 上海电力大学学报, 2020, 36(4):329 doi: 10.3969/j.issn.2096-8299.2020.04.003 [25] Lan Z Z, Chen M, Goodman S, et al. ALBERT: A lite BERT for self-supervised learning of language representations. //ICLR 2020 : Eighth International Conference on Learning Representations. Addis Ababa, 2020 [26] Chung J, Gulcehre C, Cho K, et al. Empirical evaluation of gated recurrent neural networks on sequence modeling [J/OL]. ArXiv Preprin (2018-08-13) [2020-12-29]. http://arxiv.org/abs/1412.3555 -
点击查看大图
图(4) / 表(5)
计量
- 文章访问数: 159
- HTML全文浏览量: 90
- PDF下载量: 16
- 被引次数: 0
