Applying 3D U-Net Architecture to the Task of Multi-Organ Segmentation in Computed Tomography

  • 1 Khmelnytskyi National University, , Khmelnytskyi, Ukraine

Abstract

The achievement of high-precision segmentation in medical image analysis has been an active direction of research over the past decade. Significant success in medical imaging tasks has been feasible due to the employment of deep learning methods, including convolutional neural networks (CNNs). Convolutional architectures have been mostly applied to homogeneous medical datasets with separate organs. Nevertheless, the segmentation of volumetric medical images of several organs remains an open question. In this paper, we investigate fully convolutional neural networks (FCNs) and propose a modified 3D U-Net architecture devoted to the processing of computed tomography (CT) volumetric images in the automatic semantic segmentation tasks. To benchmark the architecture, we utilised the differentiable Sørensen-Dice similarity coefficient (SDSC) as a validation metric and optimised it on the training data by minimising the loss function. Our hand-crafted architecture was trained and tested on the manually compiled dataset of CT scans. The improved 3D UNet architecture achieved the average SDSC score of 84.8 % on testing subset among multiple abdominal organs. We also compared our architecture with recognised state-of-the-art results and demonstrated that 3D U-Net based architectures could achieve competitive performance and efficiency in the multi-organ segmentation task.

If the inline PDF is not rendering correctly, you can download the PDF file here.

  • [1] D. Shen, G. Wu, and H.-I. Suk, “Deep learning in medical image analysis,” Annual Review of Biomedical Engineering, vol. 19, no. 1, pp. 221–248, Jun. 2017. https://doi.org/10.1146/annurev-bioeng-071516-044442

  • [2] W. Liu, Z. Wang, X. Liu, N. Zeng, Y. Liu, and F. E. Alsaadi, “A survey of deep neural network architectures and their applications,” Neurocomputing, vol. 234, pp. 11–26, Apr. 2017. https://doi.org/10.1016/j.neucom.2016.12.038

  • [3] E. Shelhamer, J. Long, and T. Darrell, “Fully convolutional networks for semantic segmentation,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 39, no. 4, pp. 640–651, Apr. 2017. https://doi.org/10.1109/TPAMI.2016.2572683

  • [4] H. Suk, S. W. Lee, and D. Shen, “Hierarchical feature representation and multimodal fusion with deep learning for AD/MCI diagnosis,” NeuroImage, vol. 101, pp. 569–582, Nov. 2014. https://doi.org/10.1016/j.neuroimage.2014.06.077

  • [5] A. Hamidinekoo, E. Denton, A. Rampun, K. Honnor, and R. Zwiggelaar, “Deep learning in mammography and breast histology, an overview and future trends,” Medical Image Analysis, vol. 47, pp. 45–67, Jul. 2018. https://doi.org/10.1016/j.media.2018.03.006

  • [6] G. Litjens et al., “State-of-the-art deep learning in cardiovascular image analysis,” JACC Cardiovascular Imaging, vol. 12, no. 8 Part 1, pp. 1549–1565, Aug. 2019. https://doi.org/10.1016/j.jcmg.2019.06.009

  • [7] J.-Z. Cheng et al., “Computer-aided diagnosis with deep learning architecture: applications to breast lesions in US images and pulmonary nodules in CT scans,” Scientific Reports, vol. 6, no. 24454, Apr. 2016. https://doi.org/10.1038/srep24454

  • [8] T. Hirasawa et al., “Application of artificial intelligence using a convolutional neural network for detecting gastric cancer in endoscopic images,” Gastric Cancer, vol. 21, no. 4, pp. 653–660, Jan. 2018. https://doi.org/10.1007/s10120-018-0793-2

  • [9] Y. Hu et al., “Weakly-supervised convolutional neural networks for multimodal image registration,” Medical Image Analysis, vol. 49, pp. 1–13, Oct. 2018. https://doi.org/10.1016/J.MEDIA.2018.07.002

  • [10] H. Takiyama et al., “Automatic anatomical classification of esophagogastroduodenoscopy images using deep convolutional neural networks,” Scientific Reports, vol. 8, no. 7497, pp. 1–8, May. 2018. https://doi.org/10.1038/s41598-018-25842-6

  • [11] X. Xie, Y. Li, M. Zhang, and L. Shen, “Robust segmentation of nucleus in histopathology images via mask R-CNN,” Springer, pp. 428–436, Jan. 2019. https://doi.org/10.1007/978-3-030-11723-8_43

  • [12] Y. Ren, J. Ma, J. Xiong, Y. Chen, L. Lu, and J. Zhao, “Improved false positive reduction by novel morphological features for computer-aided polyp detection in CT colonography,” IEEE Journal of Biomedical and Health Informatics, vol. 23, no. 1, pp. 324–333, Jan. 2019. https://doi.org/10.1109/JBHI.2018.2808199

  • [13] Q. Dou et al., “3D deeply supervised network for automated segmentation of volumetric medical images,” Medical Image Analysis, vol. 41, pp. 40–54, Oct. 2017. https://doi.org/10.1016/j.media.2017.05.001

  • [14] O. Ronneberger, P. Fischer, and T. Brox, “U-Net: Convolutional networks for biomedical image segmentation,” in Medical Image Computing and Computer-Assisted Intervention, MICCAI 2015. Lecture Notes in Computer Science, Springer, Champ, vol 935, pp. 234–241, Nov. 2015. https://doi.org/10.1007/978-3-319-24574-4_28

  • [15] X. Zhou, T. Ito, and R. Takayama, “Three-dimensional CT image segmentation by combining 2D fully convolutional network with 3D majority voting,” in Deep Learning and Data Labeling for Medical Applications, DLMIA 2016. Lecture Notes in Computer Science, Springer, Cham, vol. 10008, pp. 111–120, Sep. 2016. https://doi.org/10.1007/978-3-319-46976-8_12

  • [16] M. Havaei et al., “Brain tumour segmentation with deep neural networks,” Medical Image Analysis, vol. 35, pp. 18–31, Jan. 2017. https://doi.org/10.1016/j.media.2016.05.004

  • [17] H. R. Roth, L. Lu, N. Lay, A. P. Harrison, A. Farag, A. Sohn, and R. M. Summers, “Spatial aggregation of holistically-nested convolutional neural networks for automated pancreas localisation and segmentation,” Medical Image Analysis, vol. 45, pp 94–107, Apr. 2018. https://doi.org/10.1016/j.media.2018.01.006

  • [18] E. Trivizakis et al., “Extending 2-D convolutional neural networks to 3-D for advancing deep learning cancer classification with application to MRI liver tumor differentiation,” IEEE J. Biomed. Heal. Informatics, vol. 23, no. 3, pp. 923–930, May 2019. https://doi.org/10.1109/JBHI.2018.2886276

  • [19] A. Sinha and J. Dolz, “Multi-scale guided attention for medical image segmentation,” arXiv:1906.02849 [cs.CV], Jun. 2019.

  • [20] Ö. Çiçek, A. Abdulkadir, S. S. Lienkamp, T. Brox, and O. Ronneberger, “3D U-net: Learning dense volumetric segmentation from sparse annotation,” Lect. Notes Comput. Sci. (including Subser. Lect. Notes Artif. Intell. Lect. Notes Bioinformatics), vol. 9901 LNCS, pp. 424–432, Oct. 2016. https://doi.org/10.1007/978-3-319-46723-8_49

  • [21] F. Milletari, N. Navab, and S. Ahmadi, “V-Net: Fully convolutional neural networks for volumetric medical image segmentation,” in 2016 Fourth International Conference on 3D Vision (3DV), pp. 565–571, Dec. 2016. https://doi.org/10.1109/3DV.2016.79

  • [22] W. Zhu et al., “AnatomyNet: deep learning for fast and fully automated whole-volume segmentation of head and neck anatomy,” The International Journal of Medical Physics and Practice, vol. 46, no. 2, pp. 576–589, Nov. 2018. http://dx.doi.org/10.1002/mp.13300

  • [23] H. Chen, Q. Dou, L. Yu, J. Qin, and P.-A. Heng, “VoxResNet: Deep voxelwise residual networks for brain segmentation from 3D MR images,” NeuroImage, vol. 170, pp. 446–455, Apr. 2018. https://doi.org/10.1016/j.neuroimage.2017.04.041

  • [24] H. R. Roth et al., “An application of cascaded 3D fully convolutional networks for medical image segmentation,” Computerized Medical Imaging Graphics, vol. 66, pp. 90–99, Jun. 2018. https://doi.org/10.1016/j.compmedimag.2018.03.001

  • [25] V. V. Romanuke, “An attempt of finding an appropriate number of convolutional layers in CNNs based on benchmarks of heterogeneous datasets,” Electrical, Control and Communication Engineering, vol. 14, no. 1, pp. 51–57, Jul. 2018. https://doi.org/10.2478/ecce-2018-0006

  • [26] V. V. Romanuke, “Appropriate number and allocation of ReLUs in convolutional neural networks,” Research Bulletin of the National Technical University of Ukraine “Kyiv Polytechnic Institute”, no. 1, pp. 69–78, 2017. https://doi.org/10.20535/1810-0546.2017.1.88156

  • [27] V. V. Romanuke, “Appropriate number of standard 2×2 Max Pooling layers and their allocation in convolutional neural networks for diverse and heterogeneous datasets,” Information Technology and Management Science, vol. 20, no. 1, pp. 12–19, Jan. 2018. https://doi.org/10.1515/itms-2017-0002

  • [28] P. M. Radiuk, “Impact of training set batch size on the performance of convolutional neural networks for diverse datasets,” Information Technology and Management Science, vol. 20, no. 1, pp. 20–24, Jan. 2017. https://doi.org/10.1515/itms-2017-0003

  • [29] The Cancer Imaging Archive, “TCIA Collections”. [Online]. Available: https://www.cancerimagingarchive.net/#collections-list. [Accessed: Feb. 11, 2019].

  • [30] K. H. Zou, S. K. Warfield, A. Bharatha, C. M. C. Tempany M. R. Kaus, et al., “Statistical validation of image segmentation quality based on a spatial overlap index,” Academic Radiology, vol. 11, no. 2, pp. 178–189, Feb. 2004. https://doi.org/10.1016/S1076-6332(03)00671-8

  • [31] Q. Huang, J. Sun, H. Ding, X. Wang, and G. Wang, “Robust liver vessel extraction using 3D U-Net with variant dice loss function,” Computers in Biology and Medicine, vol. 101, pp. 153–162, Oct. 2018. https://doi.org/10.1016/j.compbiomed.2018.08.018

  • [32] M. Abadi et al., “TensorFlow: A system for large-scale machine learning,” in 12th USENIX Symposium on Operating Systems Design and Implementation (OSDI ‘16), pp. 265–283, Nov. 2016. [Online]. Available: https://www.usenix.org/conference/osdi16/technical-sessions/presentation/abadi

  • [33] P. Radiuk, “Applying 3D U-Net architecture to the task of multi-organ segmentation in computed tomography,” GitHub, Inc., Feb. 2020. [Online]. Available: https://github.com/soolstafir/3D-U-Net-in-CT [Accessed: Mar. 01, 2020].

OPEN ACCESS

Journal + Issues

Search