Material decomposition of spectral CT images via attention-based global convolutional generative adversarial network

Xiao-Dong Guo; Peng He; Xiao-Jie Lv; Xue-Zhi Ren; Yong-Hui Li; Yuan-Feng Liu; Xiao-Hua Lei; Peng Feng; Hong-Ming Shan

doi:10.1007/s41365-023-01184-5

Your Location：

Home >

Browse articles >

Material decomposition of spectral CT images via attention-based global convolutional generative adversarial network

NUCLEAR CHEMISTRY, RADIOCHEMISTRY, AND NUCLEAR MEDICINE | Updated：2023-04-20

- Material decomposition of spectral CT images via attention-based global convolutional generative adversarial network
- Material decomposition of spectral CT images via attention-based global convolutional generative adversarial network
- 核技术（英文版） 2023年34卷第3期文章编号：45
- Affiliations：
  
  1.The Key Lab of Optoelectronic Technology and Systems of the Education Ministry of China, Chongqing University, Chongqing 400044, China
  2.Department of Vascular and Endovascular Surgery, First Affiliated Hospital of Zhengzhou University, Zhengzhou 450003, China
  3.Institute of Science and Technology for Brain-inspired Intelligence and MOE Frontiers Center for Brain Science, Fudan University, Shanghai 200433, China
  4.Shanghai Center for Brain Science and Brain-Inspired Technology, Shanghai 200031, China
- Author bio：
  
  † penghe@cqu.edu.cn
  ‡ hmshan@fudan.edu.cn
- Funds：
  
  National Natural Science Foundation of China(62101136);Shanghai Sailing Program(21YF1402800);Shanghai Municipal Science and Technology Major Project(2018SHZDZX01);ZJLab, Shanghai Municipal of Science and Technology Project(20JC1419500);Natural Science Foundation of Chongqing(CSTB2022NSCQ-MSX0360);Shanghai Center for Brain Science and Brain-inspired Technology
- DOI：10.1007/s41365-023-01184-5
  中图分类号：
Scan for full text
Material decomposition of spectral CT images via attention-based global convolutional generative adversarial network[J]. 核技术（英文版）, 2023,34(3):45

Xiao-Dong Guo, Peng He, Xiao-Jie Lv, et al. Material decomposition of spectral CT images via attention-based global convolutional generative adversarial network[J]. Nuclear Science and Techniques, 2023,34(3):45
Material decomposition of spectral CT images via attention-based global convolutional generative adversarial network[J]. 核技术（英文版）, 2023,34(3):45 DOI： 10.1007/s41365-023-01184-5.

Xiao-Dong Guo, Peng He, Xiao-Jie Lv, et al. Material decomposition of spectral CT images via attention-based global convolutional generative adversarial network[J]. Nuclear Science and Techniques, 2023,34(3):45 DOI： 10.1007/s41365-023-01184-5.

摘要

Abstract

Spectral computed tomography (CT) based on photon counting detectors can resolve the energy of every single photon interacting with the sensor layer and be used to analyze material attenuation information under different energy ranges, which can be helpful for material decomposition studies. However, there is a considerable amount of inherent quantum noise in narrow energy bins, resulting in a low signal-to-noise ratio, which can consequently affect the material decomposition performance in the image domain. Deep learning technology is currently widely used in medical image segmentation, denoising, and recognition. In order to improve the results of material decomposition, we propose an attention-based global convolutional generative adversarial network (AGC-GAN) to decompose different materials for spectral CT. Specifically, our network is a global convolutional neural network based on an attention mechanism that is combined with a generative adversarial network. The global convolutional network based on the attention mechanism is used as the generator, and a patchGAN discriminant network is used as the discriminator. Meanwhile, a clinical spectral CT image dataset is used to verify the feasibility of our proposed approach. Extensive experimental results demonstrate that AGC-GAN achieves a better material decomposition performance than vanilla U-Net, fully convolutional network, and fully convolutional denseNet. Remarkably, the mean intersection over union, structural similarity, mean precision, PAcc, and mean F1-score of our method reach up to 87.31%, 94.83%, 93.22%, 97.39%, and 93.05%, respectively.

关键词

Keywords

Photon-counting CTMaterial decompositionAttention mechanismGAN

references

M.J. Willemink, M. Persson, A. Pourmorteza et al., Photoncounting CT: technical principles and clinical prospects. Radiology 289, 293 (2018). doi: 10.1148/radiol.2018172656http://doi.org/10.1148/radiol.2018172656

P. He, B. Wei, W. Cong et al., Optimization of K-edge imaging with spectral CT. Med. Phys. 39, 6572 (2012). doi: 10.1118/1.4754587http://doi.org/10.1118/1.4754587

L. Ren, B. Zheng, H. Liu, Tutorial on X-ray photon counting detector characterization. J. X-Ray Sci. Technol. 26, 1-28 (2018). doi: 10.3233/XST-16210http://doi.org/10.3233/XST-16210

P.R. Mendonça, R. Bhotika, M. Maddah et al., Multi-material decomposition of spectral CT images, in Proc. SPIE 7622, Medical Imaging 2010: Physics of Medical Imaging, 7622w ( 2010). doi: 10.1117/12.844531http://doi.org/10.1117/12.844531

R.E. Alvarez, A. Macovski, Energy-selective reconstructions in X-ray computerised tomography. Phys. Med. Biol. 21, 733 (1976). doi: 10.1088/0031-9155/21/5/002http://doi.org/10.1088/0031-9155/21/5/002

W. Cong, Y. Xi, P. Fitzgerald et al., Virtual monoenergetic CT imaging via deep learning. Patterns 1, 100128 (2020). doi: 10.1016/j.patter.2020.100128http://doi.org/10.1016/j.patter.2020.100128

L. Lehmann, R. Alvarez, A. Macovski et al., Generalized image combinations in dual KVP digital radiography. Med. Phys. 8, 659 (1981). doi: 10.1118/1.595025http://doi.org/10.1118/1.595025

C. Mory, B. Sixou, S. Si-Mohamed et al., Comparison of five onestep reconstruction algorithms for spectral CT. Phys. Med. Biol. 63, 235001 (2018). doi: 10.1088/1361-6560/aaeaf2http://doi.org/10.1088/1361-6560/aaeaf2

IIS. Tilley., W. Zbijewski, J.W. Stayman, Model-based material decomposition with a penalized nonlinear least-squares CT reconstruction algorithm. Phys. Med. Biol. 64, 035005 (2019). doi: 10.1088/1361-6560/aaf973http://doi.org/10.1088/1361-6560/aaf973

W. Fang, D. Wu, K. Kim et al., Iterative material decomposition for spectral CT using self-supervised Noise2Noise prior. Phys. Med. Biol. 66, 155013 (2021). doi: 10.1088/1361-6560/ac0afdhttp://doi.org/10.1088/1361-6560/ac0afd

J. Noh, J.A. Fessler, P.E. Kinahan, Statistical sinogram restoration in dual-energy CT for PET attenuation correction. IEEE Trans. Med. Imaging 28, 1688 (2009). doi: 10.1109/TMI.2009.2018283http://doi.org/10.1109/TMI.2009.2018283

M. Petrongolo, X. Dong, L. Zhu, A general framework of noise suppression in material decomposition for dual-energy CT. Med. Phys. 42, 4848 (2015). doi: 10.1118/1.4926780http://doi.org/10.1118/1.4926780

W. Zhao, T. Niu, L. Xing et al., Using edge-preserving algorithm with non-local mean for significantly improved image-domain material decomposition in dual-energy CT. Phys. Med. Biol. 61, 1332 (2016). doi: 10.1088/0031-9155/61/3/1332http://doi.org/10.1088/0031-9155/61/3/1332

T. Niu, X. Dong, M. Petrongolo et al., Iterative image-domain decomposition for dual-energy CT. Med. Phys. 41, 041901 (2014). doi: 10.1118/1.4866386http://doi.org/10.1118/1.4866386

Z. Chen, L. Li, Material decomposition of energy spectral CT by AUTOMAP. IEEE Nucl. Sci. Conf. Proc. (2018). doi: 10.1109/NSSMIC.2018.8824439http://doi.org/10.1109/NSSMIC.2018.8824439

O. Ronneberger, P. Fischer, T. Brox, U-Net: convolutional networks for biomedical image segmentation, in Lect. Notes. Comput. SC. (2015), p. 234–241. doi: 10.1007/978-3-319-24574-4_28http://doi.org/10.1007/978-3-319-24574-4_28

D.P. Clark, M. Holbrook, C.T. Badea, Multi-energy CT decomposition using convolutional neural networks. Proc. SPIE 10573, 1057310 (2018). doi: 10.1117/12.2293728http://doi.org/10.1117/12.2293728

Z. Chen, L. Li, Robust multimaterial decomposition of spectral CT using convolutional neural networks. Opt. Eng. 58, 013104 (2019). doi: 10.1117/1.OE.58.1.013104http://doi.org/10.1117/1.OE.58.1.013104

K.C. Zimmerman, G. Sharma, A.K. Parchur et al., Experimental investigation of neural network estimator and transfer learning techniques for K-edge spectral CT imaging. Med. Phys. 47, 541 (2020). doi: 10.1002/mp.13946http://doi.org/10.1002/mp.13946

T.-H. Chan, K. Jia, S. Gao et al., PCANet: a simple deep learning baseline for image classification? IEEE Trans. Image Process. 24, 5017 (2015). doi: 10.1109/TIP.2015.2475625http://doi.org/10.1109/TIP.2015.2475625

S. Minaee, Y.Y. Boykov, F. Porikli et al., Image segmentation using deep learning: a survey. IEEE Trans. Pattern Anal. (2021). doi: 10.1109/TPAMI.2021.3059968http://doi.org/10.1109/TPAMI.2021.3059968

Z.-Q. Zhao, P. Zheng, S.-T. Xu et al., Object detection with deep learning: a review. IEEE Trans. Neural Netw. Learn. Syst. 30, 3212 (2019). doi: 10.1109/TNNLS.2018.2876865http://doi.org/10.1109/TNNLS.2018.2876865

K. Chen, L.-B. Zhang, J.-S. Liu et al., Robust restoration of low-dose cerebral perfusion CT images using NCS-Unet. Nucl. Sci. Tech. 33, 1-15 (2022). doi: 10.1007/s41365-022-01014-0http://doi.org/10.1007/s41365-022-01014-0

Z. Huang, J. Zhang, Y. Zhang et al., DU-GAN: generative adversarial networks with dual-domain U-net based discriminators for low-dose CT denoising. IEEE Trans. Instrum. Meas. 71, 4500512 (2022). doi: 10.1109/TIM.2021.3128703http://doi.org/10.1109/TIM.2021.3128703

X.-Y. Guo, L. Zhang, Y.-X. Xing et al., Study on analytical noise propagation in convolutional neural network methods used in computed tomography imaging. Nucl. Sci. Tech. 33, 1-14 (2022). doi: 10.1007/s41365-022-01057-3http://doi.org/10.1007/s41365-022-01057-3

J.-S. Liu, Y.-K. Zhang, H. Tang et al., Rollback reconstruction for TDC enhanced perfusion imaging. Nucl. Sci. Tech. 32, 1-11 (2021). doi: 10.1007/s41365-021-00918-7http://doi.org/10.1007/s41365-021-00918-7

C. Feng, K. Kang, Y. Xing, A multi-energy material decomposition method for spectral CT using neural network. Proc. SPIE 10573, 105734J (2018). doi: 10.1117/12.2294611http://doi.org/10.1117/12.2294611

Y. Xu, B. Yan, J. Zhang et al., Image decomposition algorithm for dual-energy computed tomography via fully convolutional network. Comput. Math. Method Model. (2018). doi: 10.1155/2018/2527516http://doi.org/10.1155/2018/2527516

X. Wu, P. He, Z. Long et al., Multi-material decomposition of spectral CT images via Fully Convolutional DenseNets. J. X-Ray Sci. Technol. 27, 461 (2019). doi: 10.3233/XST-190500http://doi.org/10.3233/XST-190500

W. Zhang, H. Zhang, L. Wang et al., Image domain dual material decomposition for dual-energy CT using butterfly network. Med. Phys. 46, 2037 (2019). doi: 10.1002/mp.13489http://doi.org/10.1002/mp.13489

H. Gong, S. Tao, K. Rajendran et al., Deep-learning-based direct inversion for material decomposition. Med. Phys. 47, 6294 (2020). doi: 10.1002/mp.14523http://doi.org/10.1002/mp.14523

M. Geng, Z. Tian, Z. Jiang et al., PMS-GAN: parallel multi-stream generative adversarial network for multi-material decomposition in spectral computed tomography. IEEE Trans. Med. Imaging 49, 571-584 (2020). doi: 10.1109/TMI.2020.3031617http://doi.org/10.1109/TMI.2020.3031617

D. Popescu, M. Deaconu, L. Ichim et al., Retinal blood vessel segmentation using Pix2Pix GAN, in MED. (2021), p. 1173–1178. doi: 10.1109/MED51440.2021.9480169http://doi.org/10.1109/MED51440.2021.9480169

Y.-J. Ma, Y. Ren, P. Feng et al., Sinogram denoising via attention residual dense convolutional neural network for low-dose computed tomography. Nucl. Sci. Tech. 32, 41 (2021). doi: 10.1007/s41365-021-00874-2http://doi.org/10.1007/s41365-021-00874-2

H. Lan, A.W. Toga, F. Sepehrband et al., SC-GAN: 3D self-attention conditional GAN with spectral normalization for multi-modal neuroimaging synthesis. bioRxiv (2020). doi: 10.1101/2020.06.09.143297http://doi.org/10.1101/2020.06.09.143297

I. Goodfellow, J. Pouget-Abadie, M. Mirza et al., Generative adversarial networks. Adv. Neural Inf. Process. Syst. (2014). doi: 10.1145/3422622http://doi.org/10.1145/3422622

C. Jin, T. Wang, X. Li et al., A transformer generative adversarial network for multi-track music generation. CAAI Trans. Intell. Technol. 7, 369-380 (2022). doi: 10.1049/cit2.12065http://doi.org/10.1049/cit2.12065

X. Li, H. Yu, Y. Xie et al., Attention-based novel neural network for mixed frequency data. CAAI Trans. Intell. Technol. 6, 301-311 (2021). doi: 10.1049/cit2.12013http://doi.org/10.1049/cit2.12013

J. Long, E. Shelhamer, T. Darrell, Fully convolutional networks for semantic segmentation, in Proc. CVPR. IEEE (2015), p. 3431–3440. doi: 10.1109/TPAMI.2016.2572683http://doi.org/10.1109/TPAMI.2016.2572683

Z. Long, P. He, X. Wu et al., Fully convolutional pyramidal residual network for material discrimination of spectral CT. IEEE Access 7, 167187 (2019). doi: 10.1109/ACCESS.2019.2953942http://doi.org/10.1109/ACCESS.2019.2953942

K. He, X. Zhang, S. Ren et al., Deep residual learning for image recognition, in Proc. CVPR. IEEE (2016), p. 770–778. doi: 10.48550/arXiv.1512.03385http://doi.org/10.48550/arXiv.1512.03385

C. Peng, X. Zhang, G. Yu et al., Large kernel matters—improve semantic segmentation by global convolutional network, in Proc. CVPR (IEEE, 2017), p. 4353-4361. doi: 10.48550/arXiv.1703.02719http://doi.org/10.48550/arXiv.1703.02719

S. Woo, J. Park, J. Y. Lee et al., CBAM: convolutional block attention module, in ECCV (2018), p. 3–19. doi: 10.1007/978-3-030-01234-2_1http://doi.org/10.1007/978-3-030-01234-2_1

P. Isola, J. Y. Zhu, T. Zhou et al., Image-to-image translation with conditional adversarial networks, in Proc. CVPR (IEEE, 2017), p. 1125-1134. doi: 10.1109/CVPR.2017.632http://doi.org/10.1109/CVPR.2017.632

浏览量

Downloads

CSCD

文章被引用时，请邮件提醒。

Submit

工具集

关联资源

Sinogram denoising via attention residual dense convolutional neural network for low-dose computed tomography

Damage produced on GaN surface by highly-charged Kr^q+ irradiation

Demonstration of Pm-147 GaN betavoltaic cells

Surface decomposition and annealing behavior of GaN implanted with Eu