• 《工程索引》(EI)刊源期刊
  • 中文核心期刊(综合性理工农医类)
  • 中国科技论文统计源期刊
  • 中国科学引文数据库来源期刊

留言板

尊敬的读者、作者、审稿人, 关于本刊的投稿、审稿、编辑和出版的任何问题, 您可以本页添加留言。我们将尽快给您答复。谢谢您的支持!

姓名
邮箱
手机号码
标题
留言内容
验证码

自注意力指导的多序列融合肝细胞癌分化判别模型

贾熹滨 孙政 杨大为 杨正汉

贾熹滨, 孙政, 杨大为, 杨正汉. 自注意力指导的多序列融合肝细胞癌分化判别模型[J]. 工程科学学报, 2021, 43(9): 1149-1156. doi: 10.13374/j.issn2095-9389.2021.01.13.003
引用本文: 贾熹滨, 孙政, 杨大为, 杨正汉. 自注意力指导的多序列融合肝细胞癌分化判别模型[J]. 工程科学学报, 2021, 43(9): 1149-1156. doi: 10.13374/j.issn2095-9389.2021.01.13.003
JIA Xi-bin, SUN Zheng, YANG Da-wei, YANG Zheng-han. Self-attention guided multi-sequence fusion model for differentiation of hepatocellular carcinoma[J]. Chinese Journal of Engineering, 2021, 43(9): 1149-1156. doi: 10.13374/j.issn2095-9389.2021.01.13.003
Citation: JIA Xi-bin, SUN Zheng, YANG Da-wei, YANG Zheng-han. Self-attention guided multi-sequence fusion model for differentiation of hepatocellular carcinoma[J]. Chinese Journal of Engineering, 2021, 43(9): 1149-1156. doi: 10.13374/j.issn2095-9389.2021.01.13.003

自注意力指导的多序列融合肝细胞癌分化判别模型

doi: 10.13374/j.issn2095-9389.2021.01.13.003
基金项目: 国家自然科学基金资助项目(61871276,U19B2039)
详细信息
    通讯作者:

    E-mail:yangzhenghan@vip.163.com

  • 中图分类号: TP183

Self-attention guided multi-sequence fusion model for differentiation of hepatocellular carcinoma

More Information
  • 摘要: 结合影像学和人工智能技术对病灶进行无创性定量分析是目前智慧医疗的一个重要研究方向。针对肝细胞癌(Hepatocellular carcinoma, HCC)分化程度的无创性定量估测方法研究,结合放射科医师的临床读片经验,提出了一种基于自注意力指导的多序列融合肝细胞癌组织学分化程度无创判别计算模型。以动态对比增强核磁共振成像(Dynamic contrast-enhanced magnetic resonance imaging, DCE-MRI)的多个序列为输入,学习各时序序列及各序列的多层扫描切片在分化程度判别任务的权重,加权序列中具有的良好判别性能的时间和空间特征,以提升分化程度判别性能。模型的训练和测试在三甲医院的临床数据集上进行,实验结果表明,本文所提出的肝细胞癌分化程度判别模型取得相比几种基准和主流模型最高的分类计算性能,在WHO组织学分级任务中,判别准确度、灵敏度、精确度分别达到80%,82%和82%。

     

  • 图  1  “自注意力”模型结构

    Figure  1.  Structure of the "self-attention" model

    图  2  5个增强序列的2D影像与3D建模、2D原始数据及数据增强结果展示

    Figure  2.  Five enhanced sequences of 3D reconstruction, 2D raw data, and the corresponding data augmentation results

    图  3  HCC三分类和四分类任务中“自注意力”模型的embedding space和混淆矩阵。(a)三分类任务训练前的特征空间;(b)三分类任务训练后的特征空间;(c)三分类任务的混淆矩阵;(d)四分类任务训练前的特征空间;(e)四分类任务训练后的特征空间;(f)四分类任务的混淆矩阵

    Figure  3.  Feature distributions at the embedding space before and after training and the corresponding confusion matrix of the WHO and Edmonson classification tasks: (a) feature space of the model in three classification tasks before training; (b) feature space of the model in three classification tasks after training; (c) confusion matrix in three classification tasks; (d) feature space of the model in four classification tasks before training; (e) feature space of the model in four classification tasks after training; and (f) confusion matrix in four classification tasks

    表  1  基于WHO分类标准的HCC类别数据分布

    Table  1.   Augmentation results for a dataset with HCC grading under the WHO grading system

    DatasetsWellModeratelyPoorly
    Training set5620854
    Test set3210426
    Total8831280
    下载: 导出CSV

    表  2  基于Edmonson分类标准的HCC类别数据分布

    Table  2.   Augmentation results for a dataset with HCC grading under the Edmonson grading system

    DatasetsIIIIIIIV
    Training set568812054
    Test set32406426
    Total8812818480
    下载: 导出CSV

    表  3  基于WHO分类标准的对比实验

    Table  3.   Detailed comparison of experimental results on the test set under the WHO grading standard

    ModelAccuracyRecallPrecisionF1-score
    Our method0.8021±0.04780.8231±0.04040.8215±0.05370.8221±0.0477
    MCF-3DCNN[15]0.7188±0.04050.6667±0.01950.7874±0.04160.7014±0.0337
    3D ResNet[22]0.7312±0.06270.7353±0.05570.7762±0.04430.7613±0.0537
    3D SE-ResNet[23]0.7453±0.06750.7342±0.07550.7627±0.07320.7665±0.0675
    3D SE-DenseNet[24]0.7854±0.04450.7923±0.06380.8117±0.04170.7913±0.0576
    下载: 导出CSV

    表  4  基于Edmonson分类标准的对比实验

    Table  4.   Detailed comparison of experimental results on the test set under the Edmonson grading standard

    ModelAccuracyRecallPrecisionF1-score
    Our method0.7734±0.03180.7889±0.04120.8089±0.04160.7896±0.0225
    MCF-3DCNN[15]0.6322±0.05220.5482±0.13380.6424±0.06570.6431±0.0824
    3D ResNet[22]0.7037±0.07310.7229±0.04420.7404±0.04210.7203±0.0336
    3D SE-ResNet[23]0.7108±0.06440.7492±0.05310.7637±0.07880.7566±0.0631
    3D SE-DenseNet[24]0.7227±0.03410.7762±0.04260.7738±0.04460.7876±0.0512
    下载: 导出CSV
  • [1] Yang J D, Hainaut P, Gores G J, et al. A global view of hepatocellular carcinoma: trends, risk, prevention and management. Nat Rev Gastroenterol Hepatol, 2019, 16(10): 589 doi: 10.1038/s41575-019-0186-y
    [2] Jiang Y, Sun A H, Zhao Y, et al. Proteomics identifies new therapeutic targets of early-stage hepatocellular carcinoma. Nature, 2019, 567(7747): 257 doi: 10.1038/s41586-019-0987-8
    [3] Lin H X, Wei C, Wang G X, et al. Automated classification of hepatocellular carcinoma differentiation using multiphoton microscopy and deep learning. J Biophoto, 2019, 12(7): e201800435
    [4] Jimenez H, Wang M H, Zimmerman J W, et al. Tumour-specific amplitude-modulated radiofrequency electromagnetic fields induce differentiation of hepatocellular carcinoma via targeting Cav3.2 T-type voltage-gated calcium channels and Ca2+ influx. EBioMedicine, 2019, 44: 209 doi: 10.1016/j.ebiom.2019.05.034
    [5] Shioga T, Kondo R, Ogasawara S, et al. Usefulness of tumor tissue biopsy for predicting the biological behavior of hepatocellular carcinoma. Anticancer Res, 2020, 40(7): 4105 doi: 10.21873/anticanres.14409
    [6] Parr R L, Mills J, Harbottle A, et al. Mitochondria, prostate cancer, and biopsy sampling error. Discov Med, 2013, 25;15(83): 213
    [7] Henken K, Van Gerwen D, Dankelman J, et al. Accuracy of needle position measurements using fiber Bragg gratings. Minim Invasive Ther Allied Technol, 2012, 21(6): 408 doi: 10.3109/13645706.2012.666251
    [8] Li J Z, Xue F, Xu X H, et al. Dynamic contrast enhanced MRI differentiates hepatocellular carcinoma from hepatic metastasis of rectal cancer by extracting pharmacokinetic parameters and radiomic features. Exp Ther Med, 2020, 20(4): 3643
    [9] Kaissis G A, Lohöfer F K, Hörl M, et al. Combined DCE-MRI- and FDG-PET enable histopathological grading prediction in a rat model of hepatocellular carcinoma. Eur J Radiol, 2020, 124: 108848 doi: 10.1016/j.ejrad.2020.108848
    [10] Khalifa F, Soliman A, El-Baz A, et al. Models and methods for analyzing DCE-MRI: A review. Med Phys, 2014, 41(12): 124301 doi: 10.1118/1.4898202
    [11] Yang D W, Jia X B, Xiao Y J, et al. Noninvasive evaluation of the pathologic grade of hepatocellular carcinoma using MCF-3DCNN: A pilot study. Biomed Res Int, 2019: 9783106
    [12] Chernyak V, Fowler K J, Kamaya A, et al. Liver imaging reporting and data system (LI-RADS) version 2018: Imaging of hepatocellular carcinoma in at-risk patients. Radiology, 2018, 289(3): 816 doi: 10.1148/radiol.2018181494
    [13] Suk H I, Lee S W, Shen D G. Hierarchical feature representation and multimodal fusion with deep learning for AD/MCI diagnosis. NeuroImage, 2014, 101: 569 doi: 10.1016/j.neuroimage.2014.06.077
    [14] Wang Q Y, Que D S. Staging of hepatocellular carcinoma using deep feature in contrast-enhanced MR images//2nd International Conference on Computer Engineering, Information Science & Application Technology (ICCIA 2017). Wuhan, 2016: 186
    [15] Jia X B, Xiao Y J, Yang D W, et al. Temporal-spatial feature learning of dynamic contrast enhanced-MR images via 3D convolutional neural networks//Chinese Conference on Image and Graphics Technologies. Singapore, 2018: 380
    [16] Jia X B, Xiao Y J, Yang D W, et al. Multi-parametric MRIs based assessment of Hepatocellular Carcinoma Differentiation with Multi-scale ResNet. TIIS, 2019, 13(10): 5179
    [17] Antropova N, Huynh B Q, Giger M L. A deep feature fusion methodology for breast cancer diagnosis demonstrated on three imaging modality datasets. Med Phys, 2017, 44(10): 5162 doi: 10.1002/mp.12453
    [18] Hu Q Y, Whitney H M, Giger M L. A deep learning methodology for improved breast cancer diagnosis using multiparametric MRI. Sci Rep, 2020, 10: 10536 doi: 10.1038/s41598-020-67441-4
    [19] Zhang T H, Fan S L, Guo X X, et al. Intelligent medical assistant diagnosis method based on data fusion. Chin J Eng, doi: 10.13374/j.issn2095-9389.2021.01.12.003

    张桃红, 范素丽, 郭徐徐, 等. 基于数据融合的智能医疗辅助诊断方法. 工程科学学报, doi: 10.13374/j.issn2095-9389.2021.01.12.003
    [20] Ye H, Chen Q J, Wu H M, et al. Classification of liver cancer images based on deep learning//International conference on Data Science, Medicine and Bioinformatics. Singapore, 2020: 184
    [21] Zhou L, Rui J G, Zhou W X, et al. Edmondson-Steiner grade: A crucial predictor of recurrence and survival in hepatocellular carcinoma without microvascular invasio. Pathol Res Pract, 2017, 213(7): 824 doi: 10.1016/j.prp.2017.03.002
    [22] He K M, Zhang X Y, Ren S Q, et al. Deep residual learning for image recognition//2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Las Vegas, 2016: 770
    [23] Hu J, Shen L, Sun G. Squeeze-and-excitation networks//2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City, 2018: 7132
    [24] Zhou Q, Zhou Z Y, Chen C M, et al. Grading of hepatocellular carcinoma using 3D SE-DenseNet in dynamic enhanced MR images. Comput Biol Med, 2019, 107: 47 doi: 10.1016/j.compbiomed.2019.01.026
    [25] Yoshinobu Y, Iwamoto Y, Xianhua H A N, et al. Deep learning method for content-based retrieval of focal liver lesions using multiphase contrast-enhanced computer tomography images//2020 IEEE International Conference on Consumer Electronics (ICCE). Las Vegas, 2020: 1
  • 加载中
图(3) / 表(4)
计量
  • 文章访问数:  185
  • HTML全文浏览量:  82
  • PDF下载量:  16
  • 被引次数: 0
出版历程
  • 收稿日期:  2021-01-13
  • 网络出版日期:  2021-03-20
  • 刊出日期:  2021-09-18

目录

    /

    返回文章
    返回