• 《工程索引》(EI)刊源期刊
  • 中文核心期刊(综合性理工农医类)
  • 中国科技论文统计源期刊
  • 中国科学引文数据库来源期刊

留言板

尊敬的读者、作者、审稿人, 关于本刊的投稿、审稿、编辑和出版的任何问题, 您可以本页添加留言。我们将尽快给您答复。谢谢您的支持!

姓名
邮箱
手机号码
标题
留言内容
验证码

骨架图引导的级联视网膜血管分割网络

姜大光 李明鸣 陈羽中 丁文达 彭晓婷 李瑞瑞

姜大光, 李明鸣, 陈羽中, 丁文达, 彭晓婷, 李瑞瑞. 骨架图引导的级联视网膜血管分割网络[J]. 工程科学学报. doi: 10.13374/j.issn2095-9389.2021.01.13.005
引用本文: 姜大光, 李明鸣, 陈羽中, 丁文达, 彭晓婷, 李瑞瑞. 骨架图引导的级联视网膜血管分割网络[J]. 工程科学学报. doi: 10.13374/j.issn2095-9389.2021.01.13.005
JIANG Da-guang, LI Ming-ming, CHEN Yu-zhong, DING Wen-da, PENG Xiao-ting, LI Rui-rui. Cascaded retinal vessel segmentation network guided by a skeleton map[J]. Chinese Journal of Engineering. doi: 10.13374/j.issn2095-9389.2021.01.13.005
Citation: JIANG Da-guang, LI Ming-ming, CHEN Yu-zhong, DING Wen-da, PENG Xiao-ting, LI Rui-rui. Cascaded retinal vessel segmentation network guided by a skeleton map[J]. Chinese Journal of Engineering. doi: 10.13374/j.issn2095-9389.2021.01.13.005

骨架图引导的级联视网膜血管分割网络

doi: 10.13374/j.issn2095-9389.2021.01.13.005
基金项目: 北京化工大学‒中日友好医院生物医学转化工程研究中心联合资助项目(XK2020-7);科技部重点研发资助项目(2020YFF0305100)
详细信息
    通讯作者:

    E-mail:ilydouble@gmail.com

  • 中图分类号: TP391

Cascaded retinal vessel segmentation network guided by a skeleton map

More Information
  • 摘要: 针对目前视网膜血管分割中存在的细小血管提取不完整、分割不准确的问题,从血管形状拓扑关系利用的角度出发,探索多任务卷积神经网络设计,提出骨架图引导的级联视网膜血管分割网络框架。该框架包含血管骨架图提取网络模块、血管分割网络模块和若干自适应特征融合结构体。骨架提取辅助任务用于提取血管中心线,能够最大限度地保留血管拓扑结构特征;自适应特征融合结构体嵌入在两个模块的特征层间。该结构体通过学习像素级的融合权重,有效地将血管拓扑结构特征与血管局部特征相融合,加强血管特征的结构信息响应。为了获得更完整的骨架图,骨架图提取网络还引入了基于图的正则化损失函数用于训练。与最新的血管分割方法相比,该方法在3个公共视网膜图像数据集上均获得第一名,在DRIVE,STARE和CHASEDB1中其F1值分别为83.1%,85.8%和82.0%。消融实验表明骨架图引导的视网膜血管分割效果更好,并且,基于图的正则化损失也能进一步提高血管分割准确性。通过将骨架提取模块和血管分割模块替换成不同的卷积网络验证了框架的普适性。

     

  • 图  1  骨架图引导的视网膜血管分割网络框架

    Figure  1.  Skeleton map-guided retinal vessel segmentation framework

    图  2  血管骨架

    Figure  2.  Vessel skeleton

    图  3  快速并行细化算法流程图

    Figure  3.  Flowchart of the fast, parallel thinning algorithm

    图  4  自适应特征融合模块和注意力门控的对比。(a)深层特征$ {{\boldsymbol{f}}_{\rm{g}}} $过滤浅层特征$ {{\boldsymbol{f}}_{\rm{l}}} $;(b)包含结构信息的骨架特征$ {{\boldsymbol{f}}_{\rm{s}}} $和血管特征$ {{\boldsymbol{f}}_{{\rm{ves}}}} $融合

    Figure  4.  Comparison of the self-adaptive feature fusion block and attention gating: (a) deeper features $ {{\boldsymbol{f}}_{\rm{g}}} $ filter shallower features $ {{\boldsymbol{f}}_{\rm{l}}} $; (b) vessel features $ {{\boldsymbol{f}}_{{\rm{ves}}}} $ fuse skeleton features $ {{\boldsymbol{f}}_{\rm{s}}} $ containing structural information

    图  5  消融实验中每轮训练后在不同验证集上的F1值。(a)DRIVE;(b)CHASEDB1

    Figure  5.  F1 on the validation set after each training iteration in ablation experiments: (a) DRIVE; (b) CHASEDB1

    图  6  框架在DRIVE(a)和CHASEDB1(b)数据集上的训练损失

    Figure  6.  Training loss of the framework on the DRIVE (a) and CHASEDB1 (b) datasets

    表  1  本文提出的方法和近期的先进方法在F1值、敏感性Se、准确率Acc、AUC的比较结果

    Table  1.   Comparison results between our proposed method and the recent advanced methods of the F1 score, Sensitivity, Accuracy, and AUC

    MethodDRIVESTARECHASEDB1
    F1/%Se/%Acc/%AUC/%F1/%Se/%Acc/%AUC/%F1/%Se/%Acc/%AUC/%
    Segment[37]76.595.497.575.896.198.076.396.197.8
    DS[11]87.395.098.076.797.198.876.797.799.0
    DUNet[47]78.997.098.674.397.398.782.397.298.6
    Cascade[49]80.976.595.481.375.296.478.177.396.0
    DualUNet[48]82.779.495.797.780.480.796.698.1
    CE-Net[34]83.195.597.878.495.897.9
    STD[40]81.597.098.6
    IterNet[46]82.277.995.798.181.577.297.098.880.779.796.698.5
    Our method83.183.797.198.885.886.497.199.182.084.597.799.1
    下载: 导出CSV

    表  2  第一组消融实验结果

    Table  2.   Results of the first ablation experiments

    Control groupDRIVECHASEDB1
    F1/%Se/%Acc/%AUC/%F1/%Se/%Acc/%AUC/%
    Single task
    network
    84.284.696.598.881.382.997.198.8
    +Skeleton
    extraction
    85.085.597.799.281.883.697.699.0
    +Structure loss85.186.397.799.382.084.597.799.1
    下载: 导出CSV

    表  3  第二组消融实验结果

    Table  3.   Results of the second ablation experiments

    NetworkDRIVECHASEDB1
    F1/%Se/%Acc/%AUC/%F1/%Se/%Acc/%AUC/%
    ResNet3483.183.797.198.882.084.597.199.1
    ResNet1882.983.597.098.781.783.897.699.0
    VGG1682.883.297.098.781.683.797.699.0
    下载: 导出CSV
  • [1] Cong M, Wu T, Liu D, et al. Prostate MR/TRUS image segmentation and registration methods based on supervised learning. Chin J Eng, 2020, 42(10): 1362

    丛明, 吴童, 刘冬, 等. 基于监督学习的前列腺MR/TRUS图像分割和配准方法. 工程科学学报, 2020, 42(10):1362
    [2] Ma B Y, Jiang S F, Yin D, et al. Image segmentation metric and its application in the analysis of microscopic image. Chin J Eng, 2021, 43(1): 137

    马博渊, 姜淑芳, 尹豆, 等. 图像分割评估方法在显微图像分析中的应用. 工程科学学报, 2021, 43(1):137
    [3] Tso M O M, Jampol L M. Pathophysiology of hypertensive retinopathy. Ophthalmology, 1982, 89(10): 1132 doi: 10.1016/S0161-6420(82)34663-1
    [4] Yu S, Xiao D, Kanagasingam Y. Machine learning based automatic neovascularization detection on optic disc region. IEEE J Biomed Heal Inform, 2018, 22(3): 886 doi: 10.1109/JBHI.2017.2710201
    [5] Becker C, Rigamonti R, Lepetit V, et al. Supervised feature learning for curvilinear structure segmentation // International Conference on Medical Image Computing and Computer-Assisted Intervention. Berlin, 2013: 526
    [6] Tolias Y A, Panas S M. A fuzzy vessel tracking algorithm for retinal images based on fuzzy clustering. IEEE Trans Med Imaging, 1998, 17(2): 263 doi: 10.1109/42.700738
    [7] Soares J V B, Leandro J J G, Cesar R M, et al. Retinal vessel segmentation using the 2-D Gabor wavelet and supervised classification. IEEE Trans Med Imaging, 2006, 25(9): 1214 doi: 10.1109/TMI.2006.879967
    [8] Sebbe R, Gosselin B, Coche E, et al. Segmentation of opacified thorax vessels using model-driven active contour // 2005 IEEE Engineering in Medicine and Biology 27th Annual Conference. Shanghai, 2006: 2535
    [9] Pal S, Chatterjee S, Dey D, et al. Morphological operations with iterative rotation of structuring elements for segmentation of retinal vessel structures. Multidimens Syst Signal Process, 2019, 30(1): 373 doi: 10.1007/s11045-018-0561-9
    [10] Chang C C, Lin C C, Pai P Y, et al. A novel retinal blood vessel segmentation method based on line operator and edge detector // 2009 Fifth International Conference on Intelligent Information Hiding and Multimedia Signal Processing. Kyoto, 2009: 299
    [11] Zhang Y S, Chung A C S. Deep supervision with additional labels for retinal vessel segmentation task // International Conference on Medical Image Computing and Computer-Assisted Intervention. Granada, 2018: 83
    [12] Ronneberger O, Fischer P, Brox T. U-net: Convolutional networks for biomedical image segmentation // International Conference on Medical Image Computing and Computer-Assisted Intervention. Munich, 2015: 234
    [13] Guo C L, Szemenyei M, Hu H, et al. Channel attention residual U-Net for retinal vessel segmentation [J/OL]. arXiv preprint (2020-10-20) [2021-6-10]. https://arxiv.org/abs/2004.03702
    [14] He K M, Zhang X Y, Ren S Q, et al. Deep residual learning for image recognition // 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Las Vegas, 2016: 770
    [15] Vaswani A, Shazeer N, Parmar N, et al. Attention is all you need// Proceedings of the 31st Conference on neural information processing systems (NIPS 2017). Long Beach, 2017: 5998
    [16] Hu J, Shen L, Sun G. Squeeze-and-excitation networks // 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City, 2018: 7132
    [17] Yu F, Koltun V. Multi-scale context aggregation by dilated convolutions //Proceedings of the 4th International Conference on Learning Representations. San Juan, 2016
    [18] Chen L C, Papandreou G, Kokkinos I, et al. DeepLab: semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected CRFs. IEEE Trans Pattern Anal Mach Intell, 2018, 40(4): 834 doi: 10.1109/TPAMI.2017.2699184
    [19] Chen L C, Szemenyei M, Schroff F, et al. Rethinking atrous convolution for semantic image segmentation[J/OL]. arXiv preprint (2017-12-5) [2021-6-10].https://arxiv.org/abs/1706.05587
    [20] Chen L C, Papandreou G, Papandreou G, et al. Encoder-decoder with atrous separable convolution for semantic image segmentation // Computer Vision – ECCV 2018. Munich, 2018: 833
    [21] Song H M, Wang W G, Zhao S Y, et al. Pyramid dilated deeper ConvLSTM for video salient object detection // Computer Vision – ECCV 2018. Munich, 2018: 744
    [22] Xie S N, Tu Z W. Holistically-nested edge detection // 2015 IEEE International Conference on Computer Vision (ICCV). Santiago, 2015: 1395
    [23] Orlando J I, Blaschko M. Learning fully-connected CRFs for blood vessel segmentation in retinal images // International Conference on Medical Image Computing and Computer-Assisted Intervention. Boston, 2014: 634
    [24] Ricci E, Perfetti R. Retinal blood vessel segmentation using line operators and support vector classification. IEEE Trans Med Imaging, 2007, 26(10): 1357 doi: 10.1109/TMI.2007.898551
    [25] Ganin Y, Lempitsky V. N4-fields: Neural network nearest neighbor fields for image transforms // Asian Conference on Computer Vision. Singapore, 2015: 536
    [26] Dollár P, Zitnick C L. Structured forests for fast edge detection // 2013 IEEE International Conference on Computer Vision. Sydney, 2013: 1841
    [27] Maninis K K, Pont-Tuset J, Arbeláez P, et al. Deep retinal image understanding // Medical Image Computing and Computer-Assisted Intervention – MICCAI 2016. Athens, 2016: 140
    [28] Simonyan K, Zisserman A. Very deep convolutional networks for large-scale image recognition //Proceedings of the 3th International Conference on Learning Representations. San Diego, 2015
    [29] Zhang S H, Fu H Z, Yan Y G, et al. Attention guided network for retinal image segmentation // Medical Image Computing and Computer Assisted Intervention – MICCAI 2019. Shenzhen, 2019: 797
    [30] Guo C L, Szemenyei M, Yi Y G, et al. SA-UNet: spatial attention U-net for retinal vessel segmentation [J/OL]. arXiv preprint (2020-10-20) [2021-6-10]. https://arxiv.org/abs/2004.03696
    [31] Mou L, Zhao Y T, Chen L, et al. CS-net: Channel and spatial attention network for curvilinear structure segmentation // Medical Image Computing and Computer Assisted Intervention – MICCAI 2019. Shenzhen, 2019: 721
    [32] Jiang Y, Tan N, Peng T T, et al. Retinal vessels segmentation based on dilated multi-scale convolutional neural network. IEEE Access, 2019, 7: 76342 doi: 10.1109/ACCESS.2019.2922365
    [33] Hatamizadeh A, Hosseini H, Liu Z Y, et al. Deep dilated convolutional nets for the automatic segmentation of retinal vessels[J/OL]. arXiv preprint (2019-7-21) [2021-6-10]. https://arxiv.org/abs/1905.12120
    [34] Gu Z, Cheng J, Fu H, et al. CE-net: Context encoder network for 2D medical image segmentation. IEEE Trans Med Imaging, 2019, 38(10): 2281 doi: 10.1109/TMI.2019.2903562
    [35] Mo J, Zhang L. Multi-level deep supervised networks for retinal vessel segmentation. Int J Comput Assist Radiol Surg, 2017, 12(12): 2181 doi: 10.1007/s11548-017-1619-0
    [36] Hu K, Zhang Z Z, Niu X R, et al. Retinal vessel segmentation of color fundus images using multiscale convolutional neural network with an improved cross-entropy loss function. Neurocomputing, 2018, 309: 179 doi: 10.1016/j.neucom.2018.05.011
    [37] Yan Z Q, Yang X, Cheng K T. Joint segment-level and pixel-wise losses for deep learning based retinal vessel segmentation. IEEE Trans Biomed Eng, 2018, 65(9): 1912 doi: 10.1109/TBME.2018.2828137
    [38] Zhang Z J, Fu H Z, Dai H, et al. ET-net: A generic edge-aTtention guidance network for medical image segmentation // Medical Image Computing and Computer Assisted Intervention – MICCAI 2019. Shenzhen, 2019: 442
    [39] Kang H, Gao Y Q, Guo S, et al. AVNet: A retinal artery/vein classification network with category-attention weighted fusion. Comput Methods Programs Biomed, 2020, 195: 105629 doi: 10.1016/j.cmpb.2020.105629
    [40] Zhang S H, Fu H Z, Xu Y W, et al. Retinal image segmentation with a structure-texture demixing network // Medical Image Computing and Computer Assisted Intervention – MICCAI 2020. Lima, 2020: 765
    [41] Zheng S M, Zhang T Y, Zhuang J W, et al. A two-stream meticulous processing network for retinal vessel segmentation[J/OL]. arXiv preprint (2020-1-15) [2021-6-10]. https://arxiv.org/abs/2001.05829
    [42] Zou B J, Dai Y L, He Q, et al. Multi-label classification scheme based on local regression for retinal vessel segmentation. IEEE/ACM Trans Comput Biol Bioinform, 2020, PP(99): 1
    [43] Zhang T Y, Suen C Y. A fast parallel algorithm for thinning digital patterns. Commun ACM, 1984, 27(3): 236 doi: 10.1145/357994.358023
    [44] Hakim L, Yudistira N, Kavitha M, et al. U-net with graph based smoothing regularizer for small vessel segmentation on fundus image // Proceedings of the 26th International Conference on Neural Information Processing. Sydney, 2019: 515
    [45] Oktay O, Schlemper J, Folgoc L L, et al. Attention U-net: Learning where to look for the pancreas. 2018[J/OL]. arXiv preprint (2018-5-20) [2021-6-10]. https://arxiv.org/abs/1804.03999
    [46] Li L Z, Verma M, Nakashima Y, et al. IterNet: retinal image segmentation utilizing structural redundancy in vessel networks // 2020 IEEE Winter Conference on Applications of Computer Vision (WACV). Snowmass, 2020: 3645
    [47] Jin Q G, Meng Z P, Pham T D, et al. DUNet: A deformable network for retinal vessel segmentation. Knowl Based Syst, 2019, 178: 149 doi: 10.1016/j.knosys.2019.04.025
    [48] Wang B, Qiu S, He H G. Dual encoding U-net for retinal vessel segmentation // Medical Image Computing and Computer Assisted Intervention – MICCAI 2019. Shenzhen, 2019: 84
    [49] Wang X H, Jiang X D, Ren J F. Blood vessel segmentation from fundus image by a cascade classification framework. Pattern Recognit, 2019, 88: 331 doi: 10.1016/j.patcog.2018.11.030
    [50] Niemeijer M, Staal J, van Ginneken B, et al. Comparative study of retinal vessel segmentation methods on a new publicly available database // Proceedings of SPIE‒The International Society for Optical Engineering, 2004, 5370 I: 648
    [51] Hoover A D, Kouznetsova V, Goldbaum M. Locating blood vessels in retinal images by piecewise threshold probing of a matched filter response. IEEE Trans Med Imaging, 2000, 19(3): 203 doi: 10.1109/42.845178
    [52] Owen C G, Rudnicka A R, Mullen R, et al. Measuring retinal vessel tortuosity in 10-year-old children: Validation of the computer-assisted image analysis of the retina (CAIAR) program. Invest Ophthalmol Vis Sci, 2009, 50(5): 2004 doi: 10.1167/iovs.08-3018
  • 加载中
图(6) / 表(3)
计量
  • 文章访问数:  6
  • HTML全文浏览量:  5
  • PDF下载量:  1
  • 被引次数: 0
出版历程
  • 收稿日期:  2020-12-30
  • 网络出版日期:  2021-09-07

目录

    /

    返回文章
    返回