Material decomposition of spectral CT images via attention-based global convolutional generative adversarial network

Material decomposition of spectral CT images via attention-based global convolutional generative adversarial network 1 first-author https://orcid.org/0000-0003-0673-5262 1 corresp † penghe@cqu.edu.cn † penghe@cqu.edu.cn penghe@cqu.edu.cn 1 1 1 2 1 1 https://orcid.org/0000-0002-7420-2161 3 4 corresp ‡ hmshan@fudan.edu.cn ‡ hmshan@fudan.edu.cn hmshan@fudan.edu.cn The Key Lab of Optoelectronic Technology and Systems of the Education Ministry of China, Chongqing University, Chongqing 400044, China The Key Lab of Optoelectronic Technology and Systems of the Education Ministry of China, Chongqing University, Chongqing 400044, China Department of Vascular and Endovascular Surgery, First Affiliated Hospital of Zhengzhou University, Zhengzhou 450003, China Department of Vascular and Endovascular Surgery, First Affiliated Hospital of Zhengzhou University, Zhengzhou 450003, China Institute of Science and Technology for Brain-inspired Intelligence and MOE Frontiers Center for Brain Science, Fudan University, Shanghai 200433, China Institute of Science and Technology for Brain-inspired Intelligence and MOE Frontiers Center for Brain Science, Fudan University, Shanghai 200433, China Shanghai Center for Brain Science and Brain-Inspired Technology, Shanghai 200031, China Shanghai Center for Brain Science and Brain-Inspired Technology, Shanghai 200031, China Spectral computed tomography (CT) based on photon counting detectors can resolve the energy of every single photon interacting with the sensor layer and be used to analyze material attenuation information under different energy ranges, which can be helpful for material decomposition studies. However, there is a considerable amount of inherent quantum noise in narrow energy bins, resulting in a low signal-to-noise ratio, which can consequently affect the material decomposition performance in the image domain. Deep learning technology is currently widely used in medical image segmentation, denoising, and recognition. In order to improve the results of material decomposition, we propose an attention-based global convolutional generative adversarial network (AGC-GAN) to decompose different materials for spectral CT. Specifically, our network is a global convolutional neural network based on an attention mechanism that is combined with a generative adversarial network. The global convolutional network based on the attention mechanism is used as the generator, and a patchGAN discriminant network is used as the discriminator. Meanwhile, a clinical spectral CT image dataset is used to verify the feasibility of our proposed approach. Extensive experimental results demonstrate that AGC-GAN achieves a better material decomposition performance than vanilla U-Net, fully convolutional network, and fully convolutional denseNet. Remarkably, the mean intersection over union, structural similarity, mean precision, PAcc, and mean F1-score of our method reach up to 87.31%, 94.83%, 93.22%, 97.39%, and 93.05%, respectively. Photon-counting CT Material decomposition Attention mechanism GAN Introduction 1 Traditional x-ray computed tomography (CT) employs an energy-integrating detector whose output is proportional to the energy integrated over the entire incident spectrum, while spectral responses of different materials are ignored [ 1 1 , 2 2 ]. To address this issue, photon-counting CT uses an energy-resolving detector to gain energy-dependent information, which can be applied to multi material detection and decomposition [ 3 3 , 4 4 ]. In classical material decomposition methods, the attenuation of objects is usually expressed as a linear combination, such as a linear combination of photoelectric absorption, incoherent (Compton) and coherent (Rayleigh) scattering [ 5 5 , 6 6 ] or several basis materials [ 7 7 ]. However, owing to the low signal-to-noise ratio of spectral CT images in a narrow energy bin, the material decomposition performance is affected by artifacts and noise, particularly for material components with similar x-ray absorption characteristics. Several methods have been proposed to solve these problems. These methods can be classified into three categories: one-step inversion, projection domain-based, and image domain-based methods. One-step inversion algorithms combine reconstruction and decomposition processes to realize material decomposition directly from projection data. Mory et al. investigated the convergence speed of five one-step algorithms. One-step inversion methods are typically slower than their two-step counterparts because of their high computational cost [ 8 8 ]. Tilley et al. presented a model-based material decomposition algorithm that enabled reconstruction and decomposition to be synchronously performed using a multienergy forward model [ 9 9 ]. Meanwhile, Fang et al. proposed an iterative one-step inversion material decomposition algorithm with a Noise2Noise prior. This algorithm estimated material images directly from projection data and used a Noise2Noise prior for denoising [ 10 10 ]. Projection domain-based methods first decompose projection data into sinograms of materials and then perform reconstruction to obtain material images. Noh et al. proposed two novel sinogram restoration methods based on statistical models and suppress noise [ 11 11 ]. Petrongolo et al. used convex optimization to reduce noise in decomposed images in the projection domain [ 12 12 ]. Image domain-based methods first reconstruct images and then perform decomposition in the image domain. Zhao et al. developed an image-domain material decomposition algorithm by incorporating an edge-preserving filter into a local highly constrained backprojection reconstruction framework [ 13 13 ]. Niu et al. formulated an algorithm that included the inverse of the estimated variance-covariance matrix of the decomposed images as a penalty weight in the least-square term [ 14 14 ]. Chen et al. attempted to use the end-to-end AUTOMAP for fitting solutions in the image domain [ 15 15 ]. By using simulated data, Clark et al. trained CNNs with a U-Net structure [ 16 16 ]. After training, they tested the CNNs on experimentally acquired dual energy-energy integrating detector- and photon counting detector-based micro-CT data in mice. Their U-Net performed a more robust spectral micro-CT decomposition compared to sensitivity-based decomposition [ 17 17 ]. Chen et al. proposed a convolutional material decomposition algorithm that exhibited robustness during multimaterial decomposition of spectral CT images even when obvious ring artifacts were present [ 18 18 ]. Zimmerman et al. separated the soft tissue, bone, and gadolinium nanoparticles of an ex vivo rat leg specimen into different basis images with machine learning and transfer learning [ 19 19 ]. Theoretically, the purpose of spectral CT material decomposition in the image domain is to decompose the different material components in spectral CT images, which is the same goal as that of image segmentation tasks performed in certain clinical applications. Lungs, bones, and tissues are treated as the base material. Recently, deep learning has been widely used for image segmentation [ 20 20 ], classification [ 21 21 ], object detection [ 22 22 ], image denoising [ 23 23 , 24 24 ], and artifact reduction [ 25 25 , 26 26 ]. Deep learning can be used to perform feature learning and hierarchical feature extraction to solve image segmentation problems; thus, it is feasible to use it improve the accuracy of material decomposition of spectral CT images. Currently, some relevant studies have used deep learning technology for material decomposition. Feng et al. designed an end-to-end network model to obtain a virtual monochromatic attenuation map from a multicolor reconstruction map for material decomposition [ 27 27 ]. Xu et al. proposed a deep neural network composed of a fully convolutional network (FCN) and a fully connected network to solve a material decomposition problem [ 28 28 ]. Wu et al. presented a fully convolutional DenseNet (FC-DenseNet) based on densely connected blocks of Densenet to classify and quantify different m aterials [ 29 29 ]. Zhang et al. studied a model-based butterfly network to perform image domain material decomposition for dual-energy CT [ 30 30 ]. Gong et al. developed a neural network called Incept-net, which had an encoder—decoder framework, for material decomposition of spectral CT images [ 31 31 ]. Inspired by the generative adversarial network, Geng et al. proposed a novel parallel multi-stream generative adversarial network (PMS-GAN) to perform projection-based multi-material decomposition in spectral CT, in which they designed multiple sub-generators to perform multimaterial decomposition simultaneously [ 32 32 ]. Popescu et al. applied Pix2Pix GAN in retinal images and achieved satisfactory results in the segmentation of blood vessels; their network included a U-Net type generator and PatchGAN type discriminator [ 33 33 ]. However, these two GAN-based studies did not apply an attention mechanism and ignored some key feature information. Ma et al. proposed a residual dense convolutional neural network for low-dose CT denoising via an attention mechanism, which achieved excellent performance [ 34 34 ]. Lan et al. introduced a self-attention mechanism to a conditional generative adversarial network, which enabled robust deep learning-based neuroimaging synthesis. The self-attention module calculated the similarity among pixels of the image [ 35 35 ], which is computationally expensive. Unlike the methods proposed in previous studies, our global attention mechanism can extract global information using considerably fewer parameters. To effectively improve the accuracy of material identification based on deep learning, we introduce the concept of adversarial learning to material decomposition research [ 36 36 , 37 37 ] and propose an attention-based global convolutional generative adversarial network (AGC-GAN) for material decomposition, which is capable of extracting feature information at multiple scales of spectral CT images. AGC-GAN employs large convolutional kernels to extract global feature information in feature maps at multi-scales and embeds an attention mechanism [ 38 38 ] in the fusion process of multi-layers of feature maps. Spectral CT images datasets were constructed by scanning a mouse specimen with a photon-counting CT system. Then, we performed experiments to evaluate the performance of AGC-GAN on this spectral CT image dataset. Compared to U-Net, FCN [ 39 39 ], and FCD [ 40 40 ] our proposed method achieved a better discrimination performance on three materials: lung, bone, and soft tissue. Methodology 1 OVERVIEW OF AGC-GAN 2 Although there exist similar structural features of spectral CT images among different energy bins, they contain different gray feature information. Therefore, it is possible to construct a sufficiently large dataset according to the gray differences and structural correlation of spectral CT images and to improve the accuracy of material decomposition in spectral CT images by deep learning technology. Deep learning can be used for feature information extraction and learning, allowing inherent laws and deep features of the available dataset to be learnt. In this study, we use deep learning technology to identify lung, bone, and soft tissue from the chest spectral CT images of a mouse. Generative adversarial networks (GANs) have recently become a powerful model in the deep learning field, opening a new avenue for deep learning frameworks. Based on the GAN algorithm, we designed AGC-GAN for material decomposition of spectral CT images. The structure of AGC-GAN is shown in Fig. 1 Fig. 1 . Our network consists of two sub-modules: a generator module and a discriminator module. First, spectral CT images are fed into the generator network to produce the corresponding material decomposition prediction maps. Then, the discriminator network uses the reference image and the prediction map to determine the authenticity. The generator produces material decomposition images with the help of feedback from the discriminator and deceives the discriminator, while the discriminator attempts to distinguish between the generative images and the reference images. Functionally, the adversarial learning process corrects inconsistencies between predicted decomposition maps and reference images. NETWORK STRUCTURE 2 GENERATOR NETWORK 3 As shown in Fig. 1 Fig. 1 , AGC-GAN has an attention-based global convolutional network (AGCN), which is a U-type network, as the generator. Resnet-152 [ 41 41 ] was employed as the backbone network for AGCN in this study, and the details of backbone are presented in Table 1 Table 1 . Initially, multi-scale feature maps of inputs were extracted from different stages in the backbone network. Subsequently, a global convolutional block (GCB) was employed to extract the global information of the feature map, and a boundary refinement block (BRB) was used to help the model learn the boundary information of the feature map [ 42 42 ]. The GCB uses a large convolution kernel to achieve global convolution, enabling the feature information at each level to be closely related to the pixel categories. However, the large kernel size convolution operation entails excessive computation and storage requirements during the learning process. Therefore, GCB adopts parallel asymmetric convolution to complete the large-kernel-size convolution, effectively reducing the number of parameters and computational burden of the network. 1 1   1 1 Conv2d (kerne_size, out_channels) 1 1 Resnet152-layer0 1 1 7×7, 64, stride 2 1 1 Resnet152-layer1 1 1 3×3 max pool, stride 2 1 1   1 1 <math id="M1"><mrow><mrow><mo>[</mo><mrow><mtable><mtr><mtd><mrow><mn>1</mn><mo>×</mo><mn>1</mn><mo>,</mo><mn>64</mn></mrow></mtd></mtr><mtr><mtd><mrow><mn>3</mn><mo>×</mo><mn>3</mn><mo>,</mo><mn>64</mn></mrow></mtd></mtr><mtr><mtd><mrow><mn>1</mn><mo>×</mo><mn>1</mn><mo>,</mo><mn>256</mn></mrow></mtd></mtr></mtable></mrow><mo>]</mo></mrow><mo>×</mo><mn>3</mn></mrow></math> 1 1 Resnet152-layer2 1 1 <math id="M2"><mrow><mrow><mo>[</mo><mrow><mtable><mtr><mtd><mrow><mn>1</mn><mo>×</mo><mn>1</mn><mo>,</mo><mn>128</mn></mrow></mtd></mtr><mtr><mtd><mrow><mn>3</mn><mo>×</mo><mn>3</mn><mo>,</mo><mn>128</mn></mrow></mtd></mtr><mtr><mtd><mrow><mn>1</mn><mo>×</mo><mn>1</mn><mo>,</mo><mn>512</mn></mrow></mtd></mtr></mtable></mrow><mo>]</mo></mrow><mo>×</mo><mn>8</mn></mrow></math> 1 1 Resnet152-layer3 1 1 <math id="M3"><mrow><mrow><mo>[</mo><mrow><mtable><mtr><mtd><mrow><mn>1</mn><mo>×</mo><mn>1</mn><mo>,</mo><mn>256</mn></mrow></mtd></mtr><mtr><mtd><mrow><mn>3</mn><mo>×</mo><mn>3</mn><mo>,</mo><mn>256</mn></mrow></mtd></mtr><mtr><mtd><mrow><mn>1</mn><mo>×</mo><mn>1</mn><mo>,</mo><mn>1024</mn></mrow></mtd></mtr></mtable></mrow><mo>]</mo></mrow><mo>×</mo><mn>36</mn></mrow></math> 1 1 Resnet152-layer4 1 1 <math id="M4"><mrow><mrow><mo>[</mo><mrow><mtable><mtr><mtd><mrow><mn>1</mn><mo>×</mo><mn>1</mn><mo>,</mo><mn>512</mn></mrow></mtd></mtr><mtr><mtd><mrow><mn>3</mn><mo>×</mo><mn>3</mn><mo>,</mo><mn>512</mn></mrow></mtd></mtr><mtr><mtd><mrow><mn>1</mn><mo>×</mo><mn>1</mn><mo>,</mo><mn>2048</mn></mrow></mtd></mtr></mtable></mrow><mo>]</mo></mrow><mo>×</mo><mn>3</mn></mrow></math> Low-resolution feature maps were then up-sampled by using an up-sampling block (USB) and merged with higher-resolution feature maps to generate new feature maps. The final prediction maps were generated after the last BRB, which was used to output the decomposition results of AGCN. The compositions of the GCB, BRB, and USB are shown in Table 2 Table 2 . 1 1 GCB 1 1 BRB 1 1 USB 1 1 1×15 Conv, 15×1 Conv 1 1 3×3 Conv 1 1 3×3 Transposed Conv 1 1 15×1 Conv, 1×15 Conv 1 1 BN + ReLu 3×3 Conv 1 1   To enhance the feature extraction performance of the generator network, we added an attention mechanism to the fusion of feature maps for attention mapping. The attention mechanism can help the network assign different weights to various parts of feature maps and extract critical information, enabling it to make more accurate judgments in the learning process. Recently, the attention mechanism has become a powerful tool for performing deep learning tasks. Woo et al. proposed the convolutional block attention module (CBAM), which improves the accuracy of networks by concatenating spatial and channel attention [ 43 43 ]. Inspired by the CBAM, we designed a channel attention block (CAB) and a spatial attention block (SAB) and incorporated them into the generator network. Figure 2 Figure 2 shows the structure of the CAB and SAB. The CAB contributes to assigning different weights for convolutional feature maps. These weights indicate the importance of channel-wise feature maps. Specifically, two different layers of feature maps provide the inputs of the CAB. First, feature maps of two different scales are concatenated in the channel dimension and then fed into 1×1 convolution (Conv), batch normalization (BN), and rectified linear unit (ReLU) layers sequentially after an average pooling (Avg-Pool) operation. Second, the channel attention map is obtained using the sigmoid activation function. Finally, the input feature map of the CAB is dot multiplied (DM) with the channel attention map and summed with the next-level feature map to obtain the feature information extracted by the CAB as the output. The SAB helps the network assign different weights to various spatial regions according to their feature content. Specifically, the SAB takes the output of the CAB as its input. First, the input feature maps are subjected to maximum pooling (Max-Pool) and Avg-Pool; subsequently, they are concatenated in the channel dimension. The feature map is then sequentially fed into 3×3 Conv, BN, and ReLU layers and activated using the sigmoid function to obtain the spatial attention map. Eventually, the input feature map is DM with this spatial attention map to obtain the final output feature map. Thus, more informative and discriminative features can be obtained using the CAB and SAB, thereby effectively enhancing the decomposition performance of the network. DISCRIMINATOR NETWORK 3 The discriminator network of AGC-GAN in this study is PatchGAN, which is also called a Markovian discriminator [ 44 44 ]. As shown in Fig. 1 Fig. 1 , the original spectral CT image and the corresponding decomposition result image are concatenated together as the input of discriminator. Specifically, the reference image is considered “real” for the discriminator, while the decomposition result image produced by the generator is considered “fake”. The discriminator mainly consists of 4 down blocks (DB) and 1 output convolutional layer that finally outputs a 30×30 size confidence map. The structural parameters of the discriminator network are listed in Table 3 Table 3 . 1 1 Down sampling block 1 1 1 4×4 Conv LeakyReLU 1 1 Down sampling block 2 1 1 <math id="M5"><mrow><mrow><mo>[</mo><mrow><mtable><mtr><mtd><mrow><mn>4</mn><mo>×</mo><mn>4</mn><mtext>Conv</mtext></mrow></mtd></mtr><mtr><mtd><mrow><mtext>BN</mtext></mrow></mtd></mtr><mtr><mtd><mrow><mtext>LeakReLU</mtext></mrow></mtd></mtr></mtable></mrow><mo>]</mo></mrow><mo>×</mo><mn>3</mn></mrow></math> LOSS FUNCTION 3 The loss of GAN based on binary cross-entropy loss calculated at the discriminator can be defined as <math display="block" id="M6"><mrow><msub><mi mathvariant="script">L</mi><mrow><mtext>GAN</mtext></mrow></msub><mo>=</mo><munder><mstyle mathsize="140%" displaystyle="true"><mrow><mi>min</mi></mrow></mstyle><mi>G</mi></munder><munder><mstyle mathsize="140%" displaystyle="true"><mrow><mi>max</mi></mrow></mstyle><mi>D</mi></munder><msub><mi>E</mi><mi>x</mi></msub><mo stretchy="false">[</mo><mi>log</mi><mo stretchy="false">(</mo><mi>D</mi><mo stretchy="false">(</mo><mi>x</mi><mo stretchy="false">)</mo><mo stretchy="false">)</mo><mo stretchy="false">]</mo><mo>+</mo><msub><mi>E</mi><mi>z</mi></msub><mo stretchy="false">[</mo><mi>log</mi><mo stretchy="false">(</mo><mn>1</mn><mo>−</mo><mi>D</mi><mo stretchy="false">(</mo><mi>G</mi><mo stretchy="false">(</mo><mi>z</mi><mo stretchy="false">)</mo><mo stretchy="false">)</mo><mo stretchy="false">)</mo><mo stretchy="false">]</mo></mrow></math> where G and D represent the generator and discriminator in GAN respectively, x is the corresponding label maps after decomposition, and z is the spectral CT images. The training procedure of GAN is a process of confrontation between G and D . For G , the output image produced by itself is expected to deceive the opponent D , so the optimization goal of D is to minimize <math id="M7"><mrow><msub><mi mathvariant="script">L</mi><mrow><mtext>GAN</mtext></mrow></msub></mrow></math> . In contrast, D attempts to distinguish the authenticity of the input image, so the optimization goal of D is to maximize <math id="M8"><mrow><msub><mi mathvariant="script">L</mi><mrow><mtext>GAN</mtext></mrow></msub></mrow></math> . As can be seen above, the process of training and learning between D and G resembles two people playing a Max-Min game, which is an adversarial process, and thus <math id="M9"><mrow><msub><mi mathvariant="script">L</mi><mrow><mtext>GAN</mtext></mrow></msub></mrow></math> is also referred to as generative adversarial loss. In this study, the objective loss function of the AGC-GAN designed herein consists of two components, the generative adversarial loss and the cross-entropy loss of the generator network, which is defined as shown in Eq. (2): <math display="block" id="M10"><mrow><msub><mi mathvariant="script">L</mi><mrow><mtext>AGC</mtext><mo>−</mo><mtext>GAN</mtext></mrow></msub><mo>=</mo><msub><mi mathvariant="script">L</mi><mrow><mtext>GAN</mtext></mrow></msub><mo stretchy="false">(</mo><mi>G</mi><mn>,</mn><mi>D</mi><mo stretchy="false">)</mo><mo>+</mo><msub><mi>λ</mi><mrow><mtext>CE</mtext></mrow></msub><msub><mi mathvariant="script">L</mi><mrow><mtext>CE</mtext></mrow></msub><mo stretchy="false">(</mo><mi>G</mi><mo stretchy="false">)</mo><mo>,</mo></mrow></math> where <math id="M11"><mrow><msub><mi mathvariant="script">L</mi><mrow><mtext>CE</mtext></mrow></msub><mo stretchy="false">(</mo><mi>G</mi><mo stretchy="false">)</mo></mrow></math> indicates the cross-entropy loss of generator, which acts a regularization item role. A hyperparameter λ CE refers to the regularization item weight of <math id="M12"><mrow><msub><mi mathvariant="script">L</mi><mrow><mtext>CE</mtext></mrow></msub><mo stretchy="false">(</mo><mi>G</mi><mo stretchy="false">)</mo></mrow></math> . The cross-entropy loss of the generator can be formulated as <math display="block" id="M13"><mrow><msub><mi mathvariant="script">L</mi><mrow><mtext>CE</mtext></mrow></msub><mo stretchy="false">(</mo><mi>G</mi><mo stretchy="false">)</mo><mo>=</mo><mo>−</mo><mstyle displaystyle="true"><munderover><mo>∑</mo><mrow><mi>m</mi><mo>=</mo><mn>1</mn></mrow><mi>M</mi></munderover><mrow><mstyle displaystyle="true"><munderover><mo>∑</mo><mrow><mi>n</mi><mo>=</mo><mn>1</mn></mrow><mi>N</mi></munderover><mrow><msub><mi>y</mi><mrow><mi>m</mi><mn>,</mn><mi>n</mi></mrow></msub><mi>log</mi><mo stretchy="false">(</mo><msub><mi>p</mi><mrow><mi>m</mi><mn>,</mn><mi>n</mi></mrow></msub><mo stretchy="false">)</mo></mrow></mstyle></mrow></mstyle></mrow></math> When calculating the cross-entropy loss of the decomposition network, the label image will be processed into the form of one-hot coding. M and N are the total number of pixels and the number of categories of materials in the decomposition image, respectively. y m,n is the category label value corresponding to the m -th pixel point in the decomposition image, and p m,n is the predicted value of the decomposed network output for the n -th material. SPECTRAL CT DATASET 2 To obtain our spectral CT image dataset, we used a spectral CT imaging system based on a photon counting detector to scan a mouse specimen in our study. The photon-counting detector (SANTIS 0804) produced by DECTRIS has four fully calibrated and independently adjustable energy gating thresholds, and its effective detection area is 1024×256 pixels, with a pixel size is 150 μM. The SANTIS 0804 is a hybrid photon counting (HPC) detector with two configurations: high-resolution mode and multi-energy mode. We set the detector mode of the spectral CT imaging system to multi-energy mode, where the maximum input photon counting rate is 4.0×10 8 photons/s/mm 2 . We selected one mouse of approximately 10 cm in length and 160-180 g in weight to be scanned. The scanned mouse specimen was treated with the required reagents. The scanning parameters for the mouse specimen in the experiment are listed in Table 4 Table 4 . The spectral CT system scanned the mouse three times, and the SANTIS 0804 detector was set with four different energy thresholds for each scan. Consequently, we obtained spectrum data of the mouse specimen in 12 different energy bins. We selected 10 distinctive spectral ranges from 12 energy channels, namely 25–90, 30–90, 35–90, 40–90, 45–90, 50–90, 55–90, 60–90, 65–90, and 70–90 keV. In each scanning process, 250 projection views were acquired for each energy bin. We selected 100 slices from the projection data to reconstruct CT images, and each spectral CT image mainly contained three materials: lung, bone, and soft tissue. The selected projection data was reconstructed via the Split-Bregman algorithm, and finally a dataset containing 1000 spectral CT images was constructed. 1 1 Tube voltage (kvp) 1 1 90 1 1 Tube current (μA) 1 1 200 1 1 Source to detector distance (mm) 1 1 350 1 1 Source to mouse specimen distance (mm) 1 1 210 1 1 Total number of angels 1 1 250 1 1 Length of detector 1 1 1024 1 1 Number of projections 1 1 256 In addition, we manually made labels for each spectral CT image by using LabelMe, a professional labeling software. Each label image had the same resolution as the original spectral CT image. Some of the original spectral CT images and labeled reference images in the final dataset are shown in Fig. 3 Fig. 3 . Because all the data were from one mouse, we performed 5-fold cross-validation experiments to avoid overfitting. We divided the 100 slices into five equal parts from mouse head to mouse tail. Every part contained 20 slices, with every slice having 10 images with 10 different energy bins. We used three parts for training, one part for validation, and the remaining part for testing. We repeated this process five times as shown in Fig. 4 Fig. 4 and reported the average results on these five testing parts to evaluate the performance of the model. EXPERIMENTAL RESULTS 1 To verify the feasibility of AGC-GAN for material decomposition of spectral CT images, U-Net, FCN, and FCD were selected for comparison. The programming language used in the experiments was Python 3.7, and the individual network was implemented based on the deep learning framework PyTorch 1.5.1. The hardware platform configuration for running the network is as follows: Intel i5-9600kf CPU, Nvidia TITAN V (12GB/Nvidia) GPU, and 16G DDR4 RAM. The initialization learning rate of each network was set to 0.001, the decay rate of the learning rate was 0.95/(2 epochs), and the parameters of each layer of network were optimized by the Adam (Adaptive Moment Estimation) algorithm. In every rotation, we used the same training dataset to train AGC-GAN and other comparison networks, and we then used the same test dataset to test the decomposition performance of each network. The differences among the results of each network are shown in Fig. 5 Fig. 5 . The results of the three slices are from rotation 1, 3, and 4. From Fig. 5 Fig. 5 , we can see that the decomposition results of AGC-GAN are closer to the ground truth compared with other networks. AGC-GAN achieves superior material decomposition results, particularly for lung and bones. In addition, to further evaluate the material decomposition performance of each network, we calculated intersection over union (IoU), structural similarity (SSIM) index, precision, pixel accuracy (PAcc), and F1-score (F1) metrics for each network. IoU is a standard measurement method to check the degree of similarity of the output with the ground truth. IoU can be expressed as <math display="block" id="M14"><mrow><mtext>IoU</mtext><mo>=</mo><mfrac><mrow><mtext>ground truth</mtext><mo> ∩ </mo><mtext>prediction</mtext></mrow><mrow><mtext>ground truth</mtext><mo> ∪ </mo><mtext>prediction</mtext></mrow></mfrac><mn>.</mn></mrow></math> SSIM index is used as a metric to measure the similarity between two given images. <math display="block" id="M15"><mrow><mtext>SSIM</mtext><mo>=</mo><mfrac><mrow><mrow><mo>(</mo><mrow><mn>2</mn><msub><mi>μ</mi><mi>x</mi></msub><msub><mi>μ</mi><mi>y</mi></msub><mo>+</mo><msub><mi>c</mi><mn>1</mn></msub></mrow><mo>)</mo></mrow><mrow><mo>(</mo><mrow><mn>2</mn><msub><mi>σ</mi><mrow><mi>x</mi><mi>y</mi></mrow></msub><mo>+</mo><msub><mi>c</mi><mn>2</mn></msub></mrow><mo>)</mo></mrow></mrow><mrow><mrow><mo>(</mo><mrow><msubsup><mi>μ</mi><mi>x</mi><mn>2</mn></msubsup><mo>+</mo><msubsup><mi>μ</mi><mi>y</mi><mn>2</mn></msubsup><mo>+</mo><msub><mi>c</mi><mn>1</mn></msub></mrow><mo>)</mo></mrow><mrow><mo>(</mo><mrow><msubsup><mi>σ</mi><mi>x</mi><mn>2</mn></msubsup><mo>+</mo><msubsup><mi>σ</mi><mi>y</mi><mn>2</mn></msubsup><mo>+</mo><msub><mi>c</mi><mn>2</mn></msub></mrow><mo>)</mo></mrow></mrow></mfrac></mrow></math> <math display="block" id="M16"><mrow><msub><mi>c</mi><mn>1</mn></msub><mo>=</mo><mo stretchy="false">(</mo><msub><mi>k</mi><mn>1</mn></msub><mi>L</mi><msup><mo stretchy="false">)</mo><mn>2</mn></msup><mn>,</mn><msub><mi>c</mi><mn>2</mn></msub><mo>=</mo><mo stretchy="false">(</mo><msub><mi>k</mi><mn>2</mn></msub><mi>L</mi><msup><mo stretchy="false">)</mo><mn>2</mn></msup></mrow></math> In Eqs. (5) and (6), μ x and μ y are the average of the pixel values. σ xy is the covariance of two images, <math id="M17"><mrow><msubsup><mi>σ</mi><mi>x</mi><mn>2</mn></msubsup></mrow></math> and <math id="M18"><mrow><msubsup><mi>σ</mi><mi>y</mi><mn>2</mn></msubsup></mrow></math> are the variances of the two images. k 1 and k 2 are normal constants. L is the dynamic range for pixel values. Precision indicates how many of the values predicted positive by the model are actually positive. It can be calculated as <math display="block" id="M19"><mrow><mtext>Precision</mtext><mo>=</mo><mfrac><mrow><mi>T</mi><mi>P</mi></mrow><mrow><mi>T</mi><mi>P</mi><mo>+</mo><mi>F</mi><mi>P</mi></mrow></mfrac></mrow></math> PAcc is a measurement method to indicate the ratio of correct prediction pixels and total pixels. It can be calculated by Eq. (8910): <math display="block" id="M20"><mrow><mtext>PAcc</mtext><mo>=</mo><mfrac><mrow><mi>T</mi><mi>P</mi><mo>+</mo><mi>T</mi><mi>N</mi></mrow><mrow><mi>T</mi><mi>P</mi><mo>+</mo><mi>T</mi><mi>N</mi><mo>+</mo><mi>F</mi><mi>P</mi><mo>+</mo><mi>F</mi><mi>N</mi></mrow></mfrac></mrow></math> <math display="block" id="M21"><mrow><mtext>Recall</mtext><mo>=</mo><mfrac><mrow><mi>T</mi><mi>P</mi></mrow><mrow><mi>T</mi><mi>P</mi><mo>+</mo><mi>F</mi><mi>N</mi></mrow></mfrac></mrow></math> <math display="block" id="M22"><mrow><mi>F</mi><mn>1</mn><mo>=</mo><mfrac><mrow><mn>2</mn><mo>×</mo><mtext>Precision</mtext><mo>×</mo><mtext>Recall</mtext></mrow><mrow><mtext>Precision</mtext><mo>+</mo><mtext>Recall</mtext></mrow></mfrac></mrow></math> Recall is the ratio of correctly predicted positive observations to the all observations in the actual class. F1-score is the weighted average of precision and recall. TP denotes true positive, TN is true negative, FP is false positive, and FN is false negative. <math display="block" id="M23"><mrow><mtext>MIoU</mtext><mo>=</mo><mfrac><mn>1</mn><mi>k</mi></mfrac><mtext>IoU</mtext><mn>,</mn></mrow></math> <math display="block" id="M24"><mrow><mtext>MPre</mtext><mo>=</mo><mfrac><mn>1</mn><mi>k</mi></mfrac><mtext>Precision</mtext><mn>,</mn></mrow></math> where k is the number of material categories. The above evaluation indexes are positively correlated with the decomposition performance of the network, and a higher index value indicates better material decomposition performance of the network. To evaluate the performance of our network, we used U-Net, FCN, and FCD on the same dataset. We also performed an ablation experiment to estimate the attention mechanism. The GCN-GAN network was AGC-GAN without an attention mechanism. The quantitative results of AGC-GAN and other networks in terms of IoU, SSIM, precision, PAcc, and F1-score are shown in Tables 5 Tables 5 , 6 6 , and 7 7 , respectively. Compared with U-Net, FCN, and FCD, AGC-GAN showed improvement in the IoU, SSIM, precision, PAcc, and F1-score metrics of three materials: lung organ, bone, and soft tissue. Moreover, the AGC-GAN incorporating attention and generative adversarial learning theory improved the IoU metrics to 87.00%, 80.37%, and 94.54% for lung organ, bone, and soft tissue materials, respectively. In terms of SSIM index, AGC-GAN achieved the best performance (94.83%). Moreover, AGC-GAN improved the precision metric on bone and tissue and achieved the same performance as GCN-GAN. For PAcc index, we calculated all the pixels predicted correctly. AGC-GAN achieved the best performance, up to 97.39%. As a balance index between precision and recall, F1-score can value the network incorporates. AGC-GAN achieved the highest F1-score at 93.05%. From the specific quantitative results, it can be seen that AGC-GAN achieved the highest values of MPre, MIoU, PAcc, and F1-score compared with U-Net, FCN, and FCD, indicating that AGC-GAN can outperform other networks in material decomposition. 1 1 Networks 4 1 IoU 1 1   1 1 Lung (%) 1 1 Bone (%) 1 1 Tissue (%) 1 1 MIoU (%) 1 1 U-Net 1 1 86.84±3.32 1 1 80.27±4.28 1 1 94.09±0.60 1 1 87.20±2.48 1 1 FCN 1 1 86.57±3.88 1 1 79.00±4.46 1 1 93.65±0.82 1 1 86.54±2.86 1 1 FCD 1 1 79.45±8.89 1 1 58.59±6.98 1 1 90.98±1.76 1 1 76.46±5.38 1 1 GCN-GAN 1 1 85.52±3.60 1 1 75.30±3.99 1 1 93.71±1.09 1 1 84.84±2.64 1 1 AGC-GAN 1 1 87.00±3.01 1 1 80.37±3.21 1 1 94.54±1.08 1 1 87.31±2.16 1 1 Networks 1 1 SSIM 4 1 Precision 1 1   1 1   1 1 Lung (%) 1 1 Bone (%) 1 1 Tissue (%) 1 1 MPre (%) 1 1 U-Net 1 1 94.21+0.53 1 1 93.37+2.29 1 1 88.04+5.35 1 1 97.04+0.91 1 1 92.95+1.77 1 1 FCN 1 1 93.96+0.62 1 1 93.80+1.72 1 1 87.83+4.91 1 1 96.94+1.00 1 1 92.89+2.00 1 1 FCD 1 1 92.37+1.11 1 1 89.01+3.64 1 1 66.24+8.93 1 1 95.71+0.63 1 1 83.65+4.20 1 1 GCN-GAN 1 1 93.32+0.74 1 1 93.39+3.09 1 1 82.39+3.61 1 1 96.74+1.21 1 1 90.84+1.69 1 1 AGC-GAN 1 1 94.83+0.81 1 1 93.85+1.19 1 1 88.68+1.87 1 1 97.13+0.99 1 1 93.22+0.85 1 1 Networks 1 1 PAcc 4 1 F1-score 1 1   1 1   1 1 Lung (%) 1 1 Bone (%) 1 1 Tissue (%) 1 1 Mean (%) 1 1 U-Net 1 1 97.21±0.41 1 1 92.48±0.19 1 1 88.05±2.75 1 1 96.48±0.32 1 1 92.47±1.53 1 1 FCN 1 1 97.18±0.42 1 1 92.31±2.25 1 1 87.34±2.83 1 1 96.25±0.43 1 1 92.10±1.73 1 1 FCD 1 1 96.32±1.76 1 1 88.21±1.31 1 1 73.54±5.59 1 1 95.26±0.97 1 1 86.42±5.11 1 1 GCN-GAN 1 1 97.35±0.87 1 1 92.12±2.13 1 1 85.72±2.67 1 1 96.75±0.58 1 1 91.53±1.65 1 1 AGC-GAN 1 1 97.39±0.47 1 1 93.00±1.75 1 1 88.97±2.00 1 1 97.19±0.58 1 1 93.05±1.29 DISCUSSION AND CONCLUSION 1 Photon-counting CT is a promising new technology for CT systems. It can provide spectral information for identifying different components than existing technologies, and it can improve the accuracy of material identification. However, the noise in images in narrow energy bins is very strong, making the grey value of images inaccurate. Traditional material decomposition methods do not provide satisfactory decomposition results. Although spectral CT images with different energy bins have different grey values, they have the same structural similarity. Finally, we proposed a novel methodology to improve the accuracy of material decomposition. We intend to replace traditional material decomposition with segmentation based on deep learning. This will help improve the accuracy and aid radiologists in diagnosis. Our global attention mechanism contains channel attention and spatial attention. For photon-counting CT data, every slice has ten images under ten different energy bins. Channel attention mechanism can assign different weights to every channel, while spatial attention mechanism assigns different weights on the same position of every channel. Thus, we can extract feature information of the different energy bins. We also utilized the global convolution block and boundary refinement block to realize global convolution. However, self-attention mechanism usually uses a three-feature space for calculating the similarity among image pixels, which is computationally expensive and requires more samples to avoid overfitting. We performed five-fold cross-validation experiments to test our algorithm. The proposed method achieved the best values for precision, IoU, PAcc, and F1-score metrics in comparison with three other networks, providing better performance for the material decomposition of spectral CT images. Note that the proposed AGC-GAN here is used for material decomposition of two-dimensional tomographic images. There is a high structural similarity and a strong correlation in the spatial structure for spectral CT images with different energy ranges. Therefore, we will use volume data to train a three-dimensional network model in our follow-up research. This model could learn more local feature information in the spatial dimension, improving the accuracy of material decomposition. In summary, we proposed a material decomposition method, named AGC-GAN, for spectral CT images. In our network, the generator generates the results of material decomposition, while the discriminator attempts to distinguish whether the inputs are the reference or generated image. The generator and discriminator facilitate the material decomposition accuracy of AGC-GAN by learning against each other in a confrontational manner. The experimental results demonstrated that compared with U-Net, FCN, and FCD, our network exhibited excellent results the material decomposition of spectral CT. We performed cross-validation experiments to avoid the overfitting problem. In the future, a large amount of photon-counting CT data will be collected to improve the generalization ability of our network. References M.J. Willemink , M. Persson , A. Pourmorteza et al. , Photoncounting CT: technical principles and clinical prospects . Radiology 289 , 293 ( 2018 ). doi: 10.1148/radiol.2018172656 http://doi.org/10.1148/radiol.2018172656 Photoncounting CT: technical principles and clinical prospects P. He , B. Wei , W. Cong et al. , Optimization of K-edge imaging with spectral CT . Med. Phys. 39 , 6572 ( 2012 ). doi: 10.1118/1.4754587 http://doi.org/10.1118/1.4754587 Optimization of K-edge imaging with spectral CT L. Ren , B. Zheng , H. Liu , Tutorial on X-ray photon counting detector characterization . J. X-Ray Sci. Technol. 26 , 1 - 28 ( 2018 ). doi: 10.3233/XST-16210 http://doi.org/10.3233/XST-16210 Tutorial on X-ray photon counting detector characterization P.R. Mendonça , R. Bhotika , M. Maddah et al. , Multi-material decomposition of spectral CT images , in Proc. SPIE 7622, Medical Imaging 2010: Physics of Medical Imaging , 7622w ( 2010 ). doi: 10.1117/12.844531 http://doi.org/10.1117/12.844531 Multi-material decomposition of spectral CT images R.E. Alvarez , A. Macovski , Energy-selective reconstructions in X-ray computerised tomography . Phys. Med. Biol. 21 , 733 ( 1976 ). doi: 10.1088/0031-9155/21/5/002 http://doi.org/10.1088/0031-9155/21/5/002 Energy-selective reconstructions in X-ray computerised tomography W. Cong , Y. Xi , P. Fitzgerald et al. , Virtual monoenergetic CT imaging via deep learning . Patterns 1 , 100128 ( 2020 ). doi: 10.1016/j.patter.2020.100128 http://doi.org/10.1016/j.patter.2020.100128 Virtual monoenergetic CT imaging via deep learning L. Lehmann , R. Alvarez , A. Macovski et al. , Generalized image combinations in dual KVP digital radiography . Med. Phys. 8 , 659 ( 1981 ). doi: 10.1118/1.595025 http://doi.org/10.1118/1.595025 Generalized image combinations in dual KVP digital radiography C. Mory , B. Sixou , S. Si-Mohamed et al. , Comparison of five onestep reconstruction algorithms for spectral CT . Phys. Med. Biol. 63 , 235001 ( 2018 ). doi: 10.1088/1361-6560/aaeaf2 http://doi.org/10.1088/1361-6560/aaeaf2 Comparison of five onestep reconstruction algorithms for spectral CT II S. Tilley ., W. Zbijewski , J.W. Stayman , Model-based material decomposition with a penalized nonlinear least-squares CT reconstruction algorithm . Phys. Med. Biol. 64 , 035005 ( 2019 ). doi: 10.1088/1361-6560/aaf973 http://doi.org/10.1088/1361-6560/aaf973 Model-based material decomposition with a penalized nonlinear least-squares CT reconstruction algorithm W. Fang , D. Wu , K. Kim et al. , Iterative material decomposition for spectral CT using self-supervised Noise2Noise prior . Phys. Med. Biol. 66 , 155013 ( 2021 ). doi: 10.1088/1361-6560/ac0afd http://doi.org/10.1088/1361-6560/ac0afd Iterative material decomposition for spectral CT using self-supervised Noise2Noise prior J. Noh , J.A. Fessler , P.E. Kinahan , Statistical sinogram restoration in dual-energy CT for PET attenuation correction . IEEE Trans. Med. Imaging 28 , 1688 ( 2009 ). doi: 10.1109/TMI.2009.2018283 http://doi.org/10.1109/TMI.2009.2018283 Statistical sinogram restoration in dual-energy CT for PET attenuation correction M. Petrongolo , X. Dong , L. Zhu , A general framework of noise suppression in material decomposition for dual-energy CT . Med. Phys. 42 , 4848 ( 2015 ). doi: 10.1118/1.4926780 http://doi.org/10.1118/1.4926780 A general framework of noise suppression in material decomposition for dual-energy CT W. Zhao , T. Niu , L. Xing et al. , Using edge-preserving algorithm with non-local mean for significantly improved image-domain material decomposition in dual-energy CT . Phys. Med. Biol. 61 , 1332 ( 2016 ). doi: 10.1088/0031-9155/61/3/1332 http://doi.org/10.1088/0031-9155/61/3/1332 Using edge-preserving algorithm with non-local mean for significantly improved image-domain material decomposition in dual-energy CT T. Niu , X. Dong , M. Petrongolo et al. , Iterative image-domain decomposition for dual-energy CT . Med. Phys. 41 , 041901 ( 2014 ). doi: 10.1118/1.4866386 http://doi.org/10.1118/1.4866386 Iterative image-domain decomposition for dual-energy CT Z. Chen , L. Li , Material decomposition of energy spectral CT by AUTOMAP . IEEE Nucl. Sci. Conf. Proc. ( 2018 ). doi: 10.1109/NSSMIC.2018.8824439 http://doi.org/10.1109/NSSMIC.2018.8824439 Material decomposition of energy spectral CT by AUTOMAP O. Ronneberger , P. Fischer , T. Brox , U-Net: convolutional networks for biomedical image segmentation , in Lect. Notes. Comput. SC. ( 2015 ), p. 234 – 241 . doi: 10.1007/978-3-319-24574-4_28 http://doi.org/10.1007/978-3-319-24574-4_28 U-Net: convolutional networks for biomedical image segmentation D.P. Clark , M. Holbrook , C.T. Badea , Multi-energy CT decomposition using convolutional neural networks . Proc. SPIE 10573 , 1057310 ( 2018 ). doi: 10.1117/12.2293728 http://doi.org/10.1117/12.2293728 Multi-energy CT decomposition using convolutional neural networks Z. Chen , L. Li , Robust multimaterial decomposition of spectral CT using convolutional neural networks . Opt. Eng. 58 , 013104 ( 2019 ). doi: 10.1117/1.OE.58.1.013104 http://doi.org/10.1117/1.OE.58.1.013104 Robust multimaterial decomposition of spectral CT using convolutional neural networks K.C. Zimmerman , G. Sharma , A.K. Parchur et al. , Experimental investigation of neural network estimator and transfer learning techniques for K-edge spectral CT imaging . Med. Phys. 47 , 541 ( 2020 ). doi: 10.1002/mp.13946 http://doi.org/10.1002/mp.13946 Experimental investigation of neural network estimator and transfer learning techniques for K-edge spectral CT imaging T.-H. Chan , K. Jia , S. Gao et al. , PCANet: a simple deep learning baseline for image classification? IEEE Trans . Image Process. 24 , 5017 ( 2015 ). doi: 10.1109/TIP.2015.2475625 http://doi.org/10.1109/TIP.2015.2475625 PCANet: a simple deep learning baseline for image classification? IEEE Trans S. Minaee , Y.Y. Boykov , F. Porikli et al. , Image segmentation using deep learning: a survey . IEEE Trans. Pattern Anal. ( 2021 ). doi: 10.1109/TPAMI.2021.3059968 http://doi.org/10.1109/TPAMI.2021.3059968 Image segmentation using deep learning: a survey Z.-Q. Zhao , P. Zheng , S.-T. Xu et al. , Object detection with deep learning: a review . IEEE Trans. Neural Netw. Learn. Syst. 30 , 3212 ( 2019 ). doi: 10.1109/TNNLS.2018.2876865 http://doi.org/10.1109/TNNLS.2018.2876865 Object detection with deep learning: a review K. Chen , L.-B. Zhang , J.-S. Liu et al. , Robust restoration of low-dose cerebral perfusion CT images using NCS-Unet . Nucl. Sci. Tech. 33 , 1 - 15 ( 2022 ). doi: 10.1007/s41365-022-01014-0 http://doi.org/10.1007/s41365-022-01014-0 Robust restoration of low-dose cerebral perfusion CT images using NCS-Unet Z. Huang , J. Zhang , Y. Zhang et al. , DU-GAN: generative adversarial networks with dual-domain U-net based discriminators for low-dose CT denoising . IEEE Trans. Instrum. Meas. 71 , 4500512 ( 2022 ). doi: 10.1109/TIM.2021.3128703 http://doi.org/10.1109/TIM.2021.3128703 DU-GAN: generative adversarial networks with dual-domain U-net based discriminators for low-dose CT denoising X.-Y. Guo , L. Zhang , Y.-X. Xing et al. , Study on analytical noise propagation in convolutional neural network methods used in computed tomography imaging . Nucl. Sci. Tech. 33 , 1 - 14 ( 2022 ). doi: 10.1007/s41365-022-01057-3 http://doi.org/10.1007/s41365-022-01057-3 Study on analytical noise propagation in convolutional neural network methods used in computed tomography imaging J.-S. Liu , Y.-K. Zhang , H. Tang et al. , Rollback reconstruction for TDC enhanced perfusion imaging . Nucl. Sci. Tech. 32 , 1 - 11 ( 2021 ). doi: 10.1007/s41365-021-00918-7 http://doi.org/10.1007/s41365-021-00918-7 Rollback reconstruction for TDC enhanced perfusion imaging C. Feng , K. Kang , Y. Xing , A multi-energy material decomposition method for spectral CT using neural network . Proc. SPIE 10573 , 105734J ( 2018 ). doi: 10.1117/12.2294611 http://doi.org/10.1117/12.2294611 A multi-energy material decomposition method for spectral CT using neural network Y. Xu , B. Yan , J. Zhang et al. , Image decomposition algorithm for dual-energy computed tomography via fully convolutional network . Comput. Math. Method Model. ( 2018 ). doi: 10.1155/2018/2527516 http://doi.org/10.1155/2018/2527516 Image decomposition algorithm for dual-energy computed tomography via fully convolutional network X. Wu , P. He , Z. Long et al. , Multi-material decomposition of spectral CT images via Fully Convolutional DenseNets . J. X-Ray Sci. Technol. 27 , 461 ( 2019 ). doi: 10.3233/XST-190500 http://doi.org/10.3233/XST-190500 Multi-material decomposition of spectral CT images via Fully Convolutional DenseNets W. Zhang , H. Zhang , L. Wang et al. , Image domain dual material decomposition for dual-energy CT using butterfly network . Med. Phys. 46 , 2037 ( 2019 ). doi: 10.1002/mp.13489 http://doi.org/10.1002/mp.13489 Image domain dual material decomposition for dual-energy CT using butterfly network H. Gong , S. Tao , K. Rajendran et al. , Deep-learning-based direct inversion for material decomposition . Med. Phys. 47 , 6294 ( 2020 ). doi: 10.1002/mp.14523 http://doi.org/10.1002/mp.14523 Deep-learning-based direct inversion for material decomposition M. Geng , Z. Tian , Z. Jiang et al. , PMS-GAN: parallel multi-stream generative adversarial network for multi-material decomposition in spectral computed tomography . IEEE Trans. Med. Imaging 49 , 571 - 584 ( 2020 ). doi: 10.1109/TMI.2020.3031617 http://doi.org/10.1109/TMI.2020.3031617 PMS-GAN: parallel multi-stream generative adversarial network for multi-material decomposition in spectral computed tomography D. Popescu , M. Deaconu , L. Ichim et al. , Retinal blood vessel segmentation using Pix2Pix GAN , in MED. ( 2021 ), p. 1173 – 1178 . doi: 10.1109/MED51440.2021.9480169 http://doi.org/10.1109/MED51440.2021.9480169 Retinal blood vessel segmentation using Pix2Pix GAN Y.-J. Ma , Y. Ren , P. Feng et al. , Sinogram denoising via attention residual dense convolutional neural network for low-dose computed tomography . Nucl. Sci. Tech. 32 , 41 ( 2021 ). doi: 10.1007/s41365-021-00874-2 http://doi.org/10.1007/s41365-021-00874-2 Sinogram denoising via attention residual dense convolutional neural network for low-dose computed tomography H. Lan , A.W. Toga , F. Sepehrband et al. , SC-GAN: 3D self-attention conditional GAN with spectral normalization for multi-modal neuroimaging synthesis . bioRxiv ( 2020 ). doi: 10.1101/2020.06.09.143297 http://doi.org/10.1101/2020.06.09.143297 SC-GAN: 3D self-attention conditional GAN with spectral normalization for multi-modal neuroimaging synthesis I. Goodfellow , J. Pouget-Abadie , M. Mirza et al. , Generative adversarial networks . Adv. Neural Inf. Process. Syst. ( 2014 ). doi: 10.1145/3422622 http://doi.org/10.1145/3422622 Generative adversarial networks C. Jin , T. Wang , X. Li et al. , A transformer generative adversarial network for multi-track music generation . CAAI Trans. Intell. Technol. 7 , 369 - 380 ( 2022 ). doi: 10.1049/cit2.12065 http://doi.org/10.1049/cit2.12065 A transformer generative adversarial network for multi-track music generation X. Li , H. Yu , Y. Xie et al. , Attention-based novel neural network for mixed frequency data . CAAI Trans. Intell. Technol. 6 , 301 - 311 ( 2021 ). doi: 10.1049/cit2.12013 http://doi.org/10.1049/cit2.12013 Attention-based novel neural network for mixed frequency data J. Long , E. Shelhamer , T. Darrell , Fully convolutional networks for semantic segmentation , in Proc. CVPR. IEEE ( 2015 ), p. 3431 – 3440 . doi: 10.1109/TPAMI.2016.2572683 http://doi.org/10.1109/TPAMI.2016.2572683 Fully convolutional networks for semantic segmentation Z. Long , P. He , X. Wu et al. , Fully convolutional pyramidal residual network for material discrimination of spectral CT . IEEE Access 7 , 167187 ( 2019 ). doi: 10.1109/ACCESS.2019.2953942 http://doi.org/10.1109/ACCESS.2019.2953942 Fully convolutional pyramidal residual network for material discrimination of spectral CT K. He , X. Zhang , S. Ren et al. , Deep residual learning for image recognition , in Proc. CVPR. IEEE ( 2016 ), p. 770 – 778 . doi: 10.48550/arXiv.1512.03385 http://doi.org/10.48550/arXiv.1512.03385 Deep residual learning for image recognition C. Peng , X. Zhang , G. Yu et al. , Large kernel matters—improve semantic segmentation by global convolutional network , in Proc. CVPR ( IEEE , 2017 ), p. 4353 - 4361 . doi: 10.48550/arXiv.1703.02719 http://doi.org/10.48550/arXiv.1703.02719 Large kernel matters—improve semantic segmentation by global convolutional network S. Woo , J. Park , J. Y. Lee et al. , CBAM: convolutional block attention module , in ECCV ( 2018 ), p. 3 – 19 . doi: 10.1007/978-3-030-01234-2_1 http://doi.org/10.1007/978-3-030-01234-2_1 CBAM: convolutional block attention module P. Isola , J. Y. Zhu , T. Zhou et al. , Image-to-image translation with conditional adversarial networks , in Proc. CVPR ( IEEE , 2017 ), p. 1125 - 1134 . doi: 10.1109/CVPR.2017.632 http://doi.org/10.1109/CVPR.2017.632 Image-to-image translation with conditional adversarial networks All authors contributed to the study conception and design. Material preparation, data collection and analysis were performed by Peng He, Xiao-Jie Lv, Xue-Zhi Ren, Yong-Hui Li, Yuan-Feng Liu, Xiao-Hua Lei, Peng Feng, and Hong-Ming Shan. The first draft of the manuscript was written by Xiao-Dong Guo and all authors commented on previous versions of the manuscript. All authors read and approved the final manuscript. 10.1007/s41365-023-01184-5 This work was supported by National Natural Science Foundation of China (No. 62101136), Shanghai Sailing Program (No. 21YF1402800), Shanghai Municipal Science and Technology Major Project (No. 2018SHZDZX01) and ZJLab, Shanghai Municipal of Science and Technology Project (No. 20JC1419500), Natural Science Foundation of Chongqing (No. CSTB2022NSCQ-MSX0360), and Shanghai Center for Brain Science and Brain-inspired Technology. 2022-05-24 2023-01-06 2023-01-10 2023-03-27 2023-04-20 © China Science Publishing & Media Ltd. (Science Press), Shanghai Institute of Applied Physics, the Chinese Academy of Sciences, ChineseNuclear Society and Springer Nature Singapore Pte Ltd. 2023 This is a post-peer-review, pre-copyedit version of an article published in Nuclear Science and Techniques. The final authenticated version is available online at: http://dx.doi.org/10.1007/s41365-023-01184-5 http://dx.doi.org/10.1007/s41365-023-01184-5 . Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law. 2023 The data that support the findings of this study are openly available in Science Data Bank at https://doi.org/10.57760/sciencedb.j00186.00030 and https://cstr.cn/31253.11.sciencedb.j00186.00030. Data availability Nuclear Science and Techniques 3 34