Model analysis of degraded NRIs

Owing to the physical limitations of CNR systems, low-flux NRIs typically suffer from multiple severe distortions, including noise, geometric unsharpness, and white spots. As the DL-based method requires a large number of clear and degraded images to learn the abstract relationship, we designed three types of NRI degradation models to simulate as many authentic distortion types as possible; these are expressed in the following equations. $g=p\left(D*H*f\right)+n,$ (1) where g denotes the real NRIs obtained from the imaging system, f represents the latent clear images, H is the Gaussian blur kernel, D denotes the defocus blur kernel, * is the convolution operation, p(⋅) denotes the Poisson noise interference function, and n denotes the additive Gaussian noise with a mean of zero and variance of σ². The geometric instability of NRIs can be regarded as a Gaussian blur and defocus blur, as adopted in [24, 25].

On the basis of (1), we define the second model of NRIs with white spots as follows: $g=p\left(D*H*f\right)+n+w,$ (2) where w denotes additive white spots. In addition, the third model of NRIs with gamma distribution noise is given as $g=(G)p\left(D*H*f\right)+n+w,$ (3) where G denotes the function of the multiplicative gamma noise interference. The three proposed degradation models consider a comprehensive range of potential distortion types existing in real NRIs.

NRI dataset construction

Although there are many public image databases for DL-based image processing, most are designed for natural images. Owing to the scarcity of available neutron sources and the high cost of NR, obtaining NRIs is considerably more difficult than obtaining natural images. To our best knowledge, open-access datasets for NRIs have not been reported. Consequently, natural image datasets have been used as substitutes for NRI processes [26, 27]. However, owing to different imaging principles and distortion types [28], existing natural image datasets cannot provide an ideal effect. Therefore, we built NRI datasets for the first time to enhance the performance of the proposed method for NR.

Well-captured NRIs obtained from the reactor and large-scale accelerators were chosen as ideal clear images (i.e., original images). Then, degradation models (1), (2), and (3) were randomly applied to the original images to obtain distorted images with different levels and types. Both the original and simulated distorted images were used to construct the NRI training datasets. The detailed simulation operations are as follows:

Distortion 1: Gaussian noise

Gaussian noise is the most widespread noise in all image types. We generated a set of 20 random numbers that conformed to a uniform distribution in the range (0–0.03) as the variances of the Gaussian noise. By using the function “imnoise(I, ’gaussian’, m, var_gauss)" in MATLAB, the distortion images with different-level Gaussian noise can be obtained. Here, I denotes the original image, ’gaussian’ denotes the noise type, m denotes the mean value, and var_gauss denotes the variance, respectively. Not all degraded NRIs can be improved to a good visual quality; thus, the parameter selection is empirical according to the image screening condition.

Distortion 2: Poisson noise

As the neutron fluence rate fluctuates significantly in time and space, NRIs inevitably suffer from distortion of the Poisson distribution noise, which is also called shot noise. It can be simulated using the MATLAB function “imnoise(I, ’poisson’)". When the intensity of the Poisson noise becomes sufficiently large, it degenerates into Gaussian noise.

Distortion 3: Gamma noise

Gamma noise is not the main distortion type in NRIs. However, to make the simulated images more consistent with the real NRIs, we further use the MATLAB function “gamrnd(⋅)" to enrich the datasets. Although gamma noise is not necessary for the majority of NRIs, a dataset with gamma noise has been shown to be highly effective in certain special NRIs.

Distortion 4: Unsharpness(i.e., blur)

Ideally, the neutron source and beam in NR systems should be point-like and parallel. However, owing to the size of the neutron sources, the collimator ratio (i.e., L/D, where L is the length of the beam collimator, and D is the diameter of the neutron source), and the distance between the object and the scintillation screen, NRIs are usually shown with geometric unsharpness. As the generation of defocus blur in an imaging system is similar to the geometric unsharpness of NRIs [29], we used defocus blur to approximate the geometric unsharpness. To simulate this process, we randomly select the defocus blur kernel with the size from 1 to 15 for 30 times by using the MATLAB functions “fspecial(⋅)" and “imfilter(⋅)."

Additionally, when a single neutron is absorbed by the scintillation screen, the spot dispersion generated at the scintillation screen causes inherent unsharpness, which can be characterized as a two-dimensional Gaussian function. Therefore, Gaussian blur was employed to approximate the inherent unsharpness. To simulate this process, we randomly select the Gaussian blur kernel with the size from 1 to 15 for 30 times by using the MATLAB functions “fspecial(⋅)" and “imfilter(⋅)". The standard deviation of the two-dimensional Gaussian kernel function can be adjusted with reference to the kernel size (e.g., 1/3 or 1/6).

Distortion 5: White spots

During the NR imaging process, the neutron–nucleus interactions generate high-energy particles (e.g., X-rays and γ-rays), which generate white spots upon collision with the imaging detector. Furthermore, the photosensitive element accumulates charge clouds imbued with radiation energy, gradually diffusing and affecting multiple pixels, yielding white spots. Among the various distortions that exist in NRIs, white spots are the most conspicuous and detrimental to image quality. In previous studies, white spots were simulated using only a few fixed shapes, which were not sufficiently realistic [11]. Therefore, we leveraged the capability of the GAN model to emulate real-world data so as to simulate white spots with enhanced authenticity and diversity [30]. Because the GAN model requires real samples to learn its essence, we collected white-spot samples from real NRIs with multiple distortions to ensure the feature consistency of the simulated white spots. After extensive training with 14,000 iterations, the generated white spots were highly similar to the real white spots and showed a remarkable level of fidelity and authenticity as compared with those in the previous studies. The comparison results are shown in Figure 2.

Fig. 2Full-screen View

Various white-spot images. (a), (b) real white spots with low resolution; (c), (d) simulated white spots with low resolution in previous works; (e), (f) real white spots with high resolution; and (g) and (h) simulated white spots with high resolution upon using the proposed method)

On the basis of the aforedescribed distortion simulations, we built three multi-distortion datasets of NRIs by using data augmentation techniques. For dataset A, real and clear NRIs were first selected as the original input images, which were then processed using defocus blur, Gaussian blur, Gaussian noise, and Poisson noise according to (1). Then, dataset B was constructed according to (2) with additional white spots compared with dataset A. Finally, we constructed dataset C with additional gamma noise compared with dataset B for some special NRIs. These datasets are illustrated in Figure 3.

Fig. 3Full-screen View

Illustration of the self-built datasets of NRIs

Multi-distortion suppression network

The overall architecture of the multi-distortion suppression network based on the GAN framework is shown in Figure 4; it mainly consists of two crucial components, namely, the generator and discriminator. The primary objective of the proposed network is to reconstruct latent clear NRIs from input-degraded NRIs with multi-distortions. The GAN model is employed to learn the true distribution of the input samples P_data(x) by using the generator G. Meanwhile, the discriminator is used D to determine the authenticity of the generated samples. Through iterative training, G can generate samples with a high degree of fidelity. $z\in F\to G\to x=G(z)\sim P_G\left(x\right)\approx P_{\text{data}}\left(x\right)$ (4) where z denotes the data conforming to an arbitrary distribution of F, and x denotes the output of generator G. After the iterations, the generator output P_G(x) progressively approximates the true distribution of P_data(x). The visual representation of the generator and discriminator is given as $z\to G\to x_{\text{real}}\cup x_{\text{fake}}\Rightarrow D\Rightarrow (Real/Fake)\mathrm{.}$ (5) The mathematical description of the GAN model employed in this study is given as $\begin{array}{l}\underset{G}{{\displaystyle \min }}\underset{D}{{\displaystyle \min }}V\left(D\mathrm{,}G\right)=\\ E_{x\sim p_{\text{original}\left(x\right)}}\text{log }D\left(x\right)+E_{z\sim p_{z\left(\text{degraded}\right)}}\left[\text{log}\left(1-D\left(G\left(z\right)\right)\right)\right],\end{array}$ (6) where x denotes the real and clear NRIs with high resolution, and z denotes the generated samples with degradation processing.

Fig. 4Full-screen View

(Color online) Proposed multi-distortion suppression network

Model training

The NRIs of the built datasets are randomly cropped into a series of 128×128 subimages as training samples. We define α and β as the degraded images and corresponding ground truth, respectively. The training pair in each iteration is defined as ${\left({\alpha }_i\mathrm{,}{\beta }_i\right)}_{i=1}^N$, where the batch size N is set to 16. The loss function significantly influences the effectiveness of model training. Currently, the most commonly used loss functions in neural networks are L1 and L2 loss functions. L2 loss exhibited a higher degree of confidence in handling Gaussian noise, whereas L1 loss demonstrated superior robustness against outliers [31]. The L1 and L2 losses can be mathematically expressed as $L1=\Vert F\left(a_i\right)-{\beta }_i\Vert ,$ (8) $L2=\parallel F\left({\alpha }_i\right)-{\beta }_i{\parallel }^2.$ (9) Considering the presence of multiple distortions in the NRIs, the Huber loss function is employed for generator training. The main difference between the Huber and L1 loss functions lies in the nonuniqueness of the derivative of L1 loss at the origin, which may hinder convergence. The Huber loss function entails the utilization of the modified L1 and L2 losses on when the discrepancy in the predicted value F(αi) and ground truth βi is large or small (i.e., absolute difference less than 1), respectively. Thus, the Huber loss function combines the advantages of both L1 and L2 loss functions. For a small discrepancy, the gradient remains relatively small, resulting in a smoother loss function compared to the standard L1 loss. Moreover, for a larger discrepancy, the gradient is sufficiently small to avoid gradient explosions existing in the regular L2 loss. The mathematical expression for the Huber loss is given as $\text{Huber}=\{\begin{array}{ll}0.5{\left(F\left({\alpha }_i\right)-{\beta }_i\right)}^2\mathrm{,}\hfill & \text{|F}\left({\alpha }_{\text{i}}\right)-{\beta }_{\text{i}}\text{|<1}\hfill \\ |F\left({\alpha }_i\right)-{\beta }_i|-\mathrm{0.5,}\hfill & \text{otherwise}\hfill \end{array}$ (10) For discriminator training, the binary cross-entropy (BCE) loss function is utilized; this can be mathematically expressed as $\text{BCE}=-\left(\zeta \text{log}\left(\widehat{\zeta }\right)+\left(1-\zeta \right)\text{log}\left(1-\widehat{\zeta }\right)\right),$ (11) where ζ is the input to the discriminator and $\widehat{\zeta }$ is the predicted value. During the training process, Adam [32] optimizer is employed with an initial learning rate of 0.01. In addition, 100 iterations (i.e., epochs) are performed to update the network parameters and optimize the network performance.

Suppression results for real NRIs with severe noise

Image denoising is the most fundamental and important task in image processing. Several noise suppression algorithms have been proposed. We selected ImageJ [12], the most widely used software in NR, and mainstream learning-based RIDNet [21] to verify the superiority of the proposed method. As evident from Figure 5, the proposed method exhibits the best visual quality for noise suppression.

Fig. 5Full-screen View

Suppression results for different methods on real NRIs with severe noises

Given the lack of ideal reference NRIs, classical full-reference image quality assessment (IQA) methods (e.g., peak signal-to-noise ratio (PSNR) [33]) and gradient magnitude similarity deviation (GMSD) [34]) cannot be used to evaluate the quality of the aforedescribed suppression results. Therefore, a no-reference image quality metric, RBNIQM, designed for NRIs [35] was employed to provide an objective quantitative evaluation. RBNIQM can predict the quality of NRIs with multi-distortions, including Gaussian noise, Poisson noise, and blur. The prediction scores of the RBNIQM method fall within the range of 0~1, and a lower prediction score indicates a higher image quality. An objective evaluation of the suppression results in Figure 5 via RBNIQM is presented in Table 1.

As evident from Table 1, the objective evaluation was consistent with the perceptual visual quality depicted in Figure 5.

Suppression results for real NRIs with severe blur

Thus far, deblurring of NRIs remains a major challenge. Figure 6(a) shows two real NRIs: a small motor and a floppy disk drive, from top to bottom [36]. The small motor and floppy disk drive images were obtained using a small L/D value of 115 and a near imaging distance equal to its own width, thereby leading to significant blur distortion rather than noise. The traditional steering-kernel-based Richardson–Lucy algorithm (SK-RL) [37], BM3D frames, and nonlinear variance stabilization (BM3D frames)[17], as well as learning-based RIDNet [21] and CBDNet [20] were considered for comparison with the proposed method on these two images, as shown in Figure 6(b)-(f). Notably, the suppression results in Figure 6(f) were obtained by the proposed method with dataset A. The visual results of Figure 6(a)-(f) also indicate that the proposed method shows the best distortion suppression performance in terms of blur compared with the other four methods.

Fig. 6Full-screen View

Suppression results of different methods on the real NRIs with severe blur

Next, we used the RBNIQM method to quantitatively evaluate the suppression results in Figure 6. Further, Table 2 indicates that the objective quality evaluation exhibited good consistency with visual perception. In addition, learning-based CBDNet and RIDNet show superiority in multi-distortion suppression compared to traditional image-processing methods such as SK-RL and BM3D frames. This is mainly because learning-based models exhibit good capability for abstract high-level feature (e.g., deep network) extraction rather than the low-level feature (e.g., time-domain or frequency-domain features) extraction of traditional methods. The preceding discussion substantiates the efficacy of the proposed methodology in addressing distortions, particularly in the case of blur.

Suppression results for real NRIs with severe white spots

Figure 7(a) shows two other real NRIs, a large motor and a bottle, from top to bottom. Unlike the NRIs shown in Figure 6 with severe blur, these two images were mainly degraded with white spots. From the perspective of human perception, white-spot distortion has a greater significance level than various types of noise. Therefore, white-spot suppression is crucial for NRIs.

Fig. 7Full-screen View

Suppression results of different methods on the real NRIs with severe white spots

ImageJ [18] is the most widely used software for white-spot removal by professionals in the NR field. In addition, the SK-RL [37], BM3D frames, nonlinear variance stabilization (BM3D frames) [17], improved robust principal component analysis (IRPCA) [11], RIDNet [21], CBDNet [20] and the proposed method were employed to demonstrate the multi-distortion suppression effects in Figures 7(b)–(h). The suppression results of Figure 7(h) from top to bottom were obtained by the proposed method with datasets C and B. A subjective comparison of the results in Figure 7 indicates that the BM3D frames outperformed the traditional SK-RL and classical ImageJ in terms of white-spot removal. ImageJ demonstrated effectiveness in white-spot removal. The main advantages lie in user-friendly operation and easy access to the Internet. As regards BM3D frames, the filter inevitably induces additional blur, which is not beneficial for subsequent defect identification and measurement tasks. Compared with IRPCA, RIDNet, and CBDNet, the proposed method always shows the best visual perception in both white-spot removal and noise and blur suppressions. The efficacy of the proposed method in handling the first image in Figure 7 is attributed to the incorporation of the degradation model (expressed in (3)). The first image in Figure 7 contains not only prominent white spots but also diverse noise (e.g., Gaussian noise, Poisson noise, and gamma noise). As regards the second image in Figure 7, the effectiveness of the proposed method can be attributed to the adoption of the GAN model with the coordinate attention mechanism, which can effectively boost the capability of the proposed network for feature extraction in the area of interest. This also helped the proposed method outperform other learning-based methods (i.e., RIDNet and CBDNet) when using the same datasets.

Because existing IQA methods do not consider white spots, evaluation of the suppression results regarding white spots is a significant challenge. Therefore, both subjective and objective metrics were employed to evaluate the performance of the proposed method and its counterparts. Four no-reference quality assessment methods, namely, BIQAA [38], BLIINDS [39], NIQE [40], and RBNIQM, were chosen as the objective metrics and compared with a subjective metric (i.e., mean opinion score (MOS)). The normalized scores predicted by the four no-reference quality assessment methods are shown in Figure 8. Except for RBNIQM, all the other methods showed better image quality with higher scores.

Fig. 8Full-screen View

Line graphs of predicted scores in Fig. 7 with different no-reference quality evaluation methods

The black line in Figure 8 denotes the MOS trend obtained by averaging the subjective scores from the following two groups of evaluators: professional researchers with experience in NR and other researchers without relevant experience. Specifically, the NRIs processed using different methods were first randomly shuffled, and the evaluators (25 in each of the professional and nonprofessional groups) were thereafter instructed to rank the images in ascending order of quality and assign scores incrementally from one. Finally, the average scores from different groups were computed according to the weights of 0.7 and 0.3, respectively, for the professional and nonprofessional groups as the subjective quality scores for each method (Table 3).

The trend lines in Figure 8 show that the score curves deviate severely from the subjective scores, indicating their inability to accurately assess the quality of NRIs with white spots. Although the RBNIQM method shows a relatively steady trend compared to the subjective quality assessment, the minimal variations indicate its drawback of low sensitivity to white spots. After an extensive comparison with state-of-the-art distortion suppression methods, we considered three representative methods—ImageJ, IRPCA, and the proposed method—to illustrate the local details of white-spot suppression (Figure 9).

Fig. 9Full-screen View

Illustrations of local details of NRI with different suppression methods

Evidently, the ImageJ method fails to eliminate white spots. It can only remove the brightest pixels of the white spots and exhibits inferior performance in preserving fine details. The IRPCA method achieves satisfactory results in removing white spots but has a limited capability in preserving image details. The proposed method effectively removes white spots and noise with good detail preservation.

To further validate the exceptional efficacy of the proposed method in suppressing white spots in NRIs, the same background regions (i.e., the red box in the machine image) of the NRI with different distortion suppression methods are shown from the perspective of a three-dimensional (3D) grayscale distribution in Figure 10. On the basis of the pixel distribution of the red-box region in Figure 10(b)-(h), we can conclude that both the IRPCA method and proposed method exhibit remarkable white-spot suppression effects. As the selected region is a background with no useful object information, a smoother 3D grayscale distribution is preferred.

Fig. 10Full-screen View

(Color online) 3D grayscale distribution of a local region in NRI with different suppression methods

Without loss of generality, the real NRI obtained by the CNR system is shown in Figure 11 to further demonstrate the effectiveness of the proposed method. This image was obtained in 1972 using an A-711 D-T neutron tube with a small L/D value of 7.5. The low neutron flux of the neutron tube required a long exposure time of 90 min. However, the image quality was inferior to that of NRIs obtained by the reactor. Next, ImageJ, SK-RL, and the proposed method were employed to demonstrate the multi-distortion suppression performance. As evident from the comparison, the ImageJ and SK-RL methods exhibit fewer effects than the original image. In contrast, the proposed method can improve image quality in terms of noise, blur, and even color restoration to a certain extent. Notably, the proposed method has good generalization ability in improving the visual quality of various NRIs including thermal neutron radiographic images and fast neutron radiographic images. Although the energy spectra of the neutrons used for NR differ, the resulting images have similar image characteristics (e.g., radiographic images) and degraded models (e.g., Gaussian noise, Poisson noise, blur, and white spots).

Fig. 11Full-screen View

Illustrations of multi-distortion suppression effects on a real NRI obtained by a D-T neutron-tube source

Guidance on the selection of datasets

As DL-based methods heavily rely on dataset design for specific targets, we herein provide recommendations on the selection of appropriate datasets for different NRIs. Dataset A was mainly designed for blur and noise and is suitable for handling noisy NRIs with severe blur. Building on dataset A, dataset B considers the white-spot distortion type. As a result, dataset B can be considered as the most widely applicable used dataset for suppressing multi-distortion in real NRIs. Additionally, there may also exist NRIs with severe distortions. We recommend the use of dataset C to obtain remarkable results. However, the training time and recovery performance for most NRIs were not as favorable as those in the case of dataset B. Notably, the proposed method emphasizes multi-distortion suppression rather than a single distortion type in NRIs. Multi-distortion suppression can be realized in NRIs in a single training step by selecting the appropriate dataset.