2.1 Overview of the proposed method

The proposed method is composed of DBN training and DBN prediction (Fig. 1). The DBN is trained according to the flowchart shown in Fig. 1(a). This technique consists of spectrum simulation, extraction of approximation coefficients of the spectrum, and training. First, the model of the related detector is designed, and simulation spectra of the radionuclides of interest are obtained using the Monte-Carlo N-Particle transport code (MCNP) (a software package for simulating nuclear processes). Second, approximation coefficients of the spectrum are extracted by wavelet decomposition and normalized to eliminate the intensity difference between different spectra. Then, the normalized approximation coefficients are used as the input to the DBN. Finally, the DBN is trained using the samples of the simulation spectra, which are encoded as the input of the DBN and their target output. For DBN prediction, the trained DBN is used to predict the isotopic composition of the spectrum measured in the real environment using the process shown in Fig. 1(b). This process consists of spectral measurement and background subtraction, extraction of the approximation coefficients of the spectrum, and identification. First, radionuclide and background spectra are detected by the detector and smoothed to reduce noise interference. Then, the subtraction spectrum is obtained by subtracting the portion of the smoothed background spectrum according to the ratio of the radionuclide scan time to the background scan time from the smoothed radionuclide spectrum. Second, the approximation coefficients of the subtraction spectrum are extracted by wavelet decomposition and normalized to remove the effect of the change in spectrum intensity. Then, the normalized approximation coefficients are used as the input for the DBN. Finally, the trained DBN is used to predict the isotopic composition of the spectrum measured in the real environment, and the identification performance of the proposed method is evaluated by comparing the output of the DBN with its target output.

Fig. 1.Full-screen View

Schematic diagram of the proposed algorithm

2.2 Approximation coefficients

The low-frequency content is the most important part of many signals, because this component gives the signal its identity. The high-frequency content, on the other hand, imparts flavor, or nuance. Approximations and details are often discussed in wavelet analysis. Approximations are high-scale, low-frequency components of the signal, whereas details are low-scale, high-frequency components. The original signal S would be decomposed into approximations and details after the signal underwent a single-level discrete wavelet transform. This process is accomplished by applying two complementary filters to the original signal, which emerges as two signals, as shown in Fig. 2(a).

Fig. 2.Full-screen View

Schematic of wavelet decomposition. (a) single-level and (b) multilevel discrete wavelet transforms

The decomposition process can be iterated, with successive approximations being decomposed in turn, and the number of iterations is called the decomposition level [Fig. 2(b)]. The cA₁, cA₂, and cA₃ are the approximations of the original signal S, while cD₁, cD₂, and cD₃ are the details at different decomposition levels. Given that the analytical process is iterative, the processes can theoretically be continued indefinitely. In practice, the suitable number of levels is based on the nature of the signal, and a number from 3 to 5 is recommended. Approximation coefficients cA_i (i=1, 2, 3, 4, 5) of the spectrum can be considered as the input to the DBN. The cA_i has the same spectral shape as the original spectrum, but it is more smoothed. The original spectrum (dimension: 1024) detected by an ` (Tl) detector is decomposed in this study by 3 levels of discrete wavelet transforms (mother wavelet: Daubechies wavelets) using the MATLAB Wavelet Toolbox [20]. The dimension of the extracted approximation coefficients is 128, which is much lower than the dimension of the original spectrum (dimension: 1024). After normalization, the extracted approximation coefficients are used as the input to the DBN.

2.3 DBN model

Deep learning is a machine learning paradigm that focuses on learning deep hierarchical data models. In this algorithm, a hierarchy of intermediate representations is hypothesized to be needed to learn a high-level representation of the data [21]. A DBN is one kind of deep learning. In machine learning, a DBN is a generative graphical model, or alternatively, a type of deep NN composed of multilayers of latent variables (hidden units), with connections between layers but not between units within each layer. A DBN can be viewed as a composition of restricted Boltzmann machines (RBMs), in which the hidden layer of each sub-network serves as the visible layer for the next [22, 23]. Its schematic network structure is shown in Fig. 3. The structure comprises one visible layer, three hidden layers (h¹, h², h³), and one output layer [24]. The number of RBMs is not constant, and is determined by the nature of the data. RBMs contain a layer of visible units that represent the data, and a layer of hidden units that learn to represent features that capture higher-order correlations in the data. This module is a primitive DBN.

Fig. 3.Full-screen View

Schematic overview of a deep belief network (DBN)

An RBM has only a visible layer and a hidden layer, and these layers are connected by a matrix of symmetrically weighted connections W. No connections are found within a layer. Given a vector of activities v for the visible units, the hidden units are all conditionally independent, facilitating the sampling of a vector h from the factorial posterior distribution over hidden vectors p(h|v,W). Sampling from p(v|h,W) is also easy. A learning signal can be obtained by starting with an observed data vector on the visible units, and alternating several times between sampling from p(h|v,W) and p(v|h,W). This signal is simply the difference between the pairwise correlations of the visible and hidden units at the beginning and end of the sampling. Several reports have provided more detailed descriptions of RBM [25, 26].

The variables of the final hidden layer represent the desired outputs, which can be encoded to the error backpropagation (BP) NN. The BP algorithm functions much better if the feature detectors (hidden units) in the hidden layers are initialized by learning a DBN that models the structure in the input data. The DBN algorithm was realized in this study using a MATLAB Toolbox named DeepLearnToolbox (https://github.com/rasmusbergpalm/DeepLearnToolbox), and the key parameters of the DBN used during the training process are as follows: the number of hidden layers is 1, and the dimension of the hidden layer is 1024. The dimension of the visible layer is 128 for extracting 128 approximation coefficients as the input to the DBN, the dimension of the output layer is 9 for selecting the spectra of 9 radionuclides (i.e., ⁵⁷Co, ⁷⁵Se, ⁶⁰Co, ¹³³Ba, ¹³⁷Cs, ¹⁹²Ir, ²⁴¹Am, ¹⁵²Eu, and ²³⁸Pu) as training samples, thus the established DBN model is 128×1024×9. Each neuron in the output layer indicated whether the related radionuclide existed or not, and the radionuclide existed in the environment only when the output value of the related output layer neuron was greater than or equal to the threshold (0.85).

3.1 Assessment method

A gamma ray spectrometry system, which consists of a 3 in × 3 in NaI (Tl) detector (ORTEC Inc.), a PC-based multichannel analyzer (ORTEC Inc.), and MAESTRO software installed on the PC were used to assess the proposed method, and the MAESTRO 7.01 (ORTEC Inc.) software was used to record the spectra. The energy of this detector ranged from 30 keV to 3,000 keV, and the resolution was approximately 7.7% (at 662 keV). The spectra were recorded by sending a request to MAESTRO 7.01 software using its software development kit. Any spectrum that satisfied the requirements could be detected. Table 1 lists the radionuclides utilized in the experiment, and these radionuclides are labeled as Radio-1, Radio-2, and Radio-3.

Given that the DBN can only identify the trained radionuclides, and its predictive ability depends on the number of samples, adding samples of more radionuclides during the training process of the DBN is recommended. However, the majority of radionuclides are not available in the laboratory. Simulation spectra of the radionuclides of interest (i.e., ⁵⁷Co, ⁷⁵Se, ⁶⁰Co, ¹³³Ba, ¹³⁷Cs, ¹⁹²Ir, ²⁴¹Am, ¹⁵²Eu, and ²³⁸Pu) can be obtained by the MCNP, which models the mentioned detector based on its characteristics. All radionuclides except ²³⁸Pu are industrial radionuclides. All simulation spectra are broadened with the response function of the mentioned detector.

Detection rate was used to assess the identification performance of the proposed algorithm. This rate is the ratio of the correctly identified data to the total amount of data, as shown in Eq. (1) [27], as follows:

where TP indicates true positive, TN indicates true negative, FP indicates false positive, and FN indicates false negative.

The accurate radionuclide identification distance (ARID) of each radionuclide was calculated from the detection rate results with respect to the distance from the source to the detector. The ARID is the maximum distance at which detection rates are reliably ascertained to be above the required level [28]. The detection rate of the acceptance level in this study is 98.3%, as recommended by the ANSI N42.35 standard [29].

3.2 DBN training

Training samples were selected. Simulation spectra of radionuclides of interest (i.e., ⁵⁷Co, ⁷⁵Se, ⁶⁰Co, ¹³³Ba, ¹³⁷Cs, ¹⁹²Ir, ²⁴¹Am, ¹⁵²Eu, and ²³⁸Pu) were simulated by the MCNP, and the normalized simulation spectra generated 511 (2⁹–1=511) spectra according to the theory of combinations, which were all used as training samples for the DBN. The conditions with 1 radionuclide corresponded to 9 combinations (i.e., ⁵⁷Co, ⁷⁵Se, 60Co, ¹³³Ba, ¹³⁷Cs, ¹⁹²Ir, ²⁴¹Am, ¹⁵²Eu, and ²³⁸Pu), the conditions with 2 radionuclides corresponded to 36 combinations (i.e., ⁵⁷Co+⁷⁵Se, ⁵⁷Co+⁶⁰Co,…, ¹⁵²Eu+²³⁸Pu), and the number of combinations for the conditions with 3, 4, 5, 6, 7, 8, and 9 radionuclides can be acquired according to the theory of combinations, which is how we obtained the 511 spectra. The 511 normalized spectra are encoded as the input to the DBN and their target output before DBN training, and the target output was composed of 9 figures ranging from 0 to 1. For example, the target output of the ⁵⁷Co spectrum would be [1 0 0 0 0 0 0 0 0] if we sequenced the radionuclides as ⁵⁷Co, ⁷⁵Se, ⁶⁰Co, ¹³³Ba, ¹³⁷Cs, ¹⁹²Ir, ²⁴¹Am, ¹⁵²Eu, and ²³⁸Pu. So, the target output of the ⁷⁵Se spectrum was [0 1 0 0 0 0 0 0 0], the target output of the ⁵⁷Co+ ⁷⁵Se spectrum was [1 1 0 0 0 0 0 0 0], and so on.

Whether the trained DBN has learned the training samples properly should be verified. If not, the related parameters (section: DBN model) of the DBN should be adjusted until the mean square error of the DBN on the training set reaches an acceptable value. Table 2 lists the results of the DBN training. The trained DBN learned the high-level features of the training set completely. Theoretically, this DBN could predict the isotopic compositions of the spectrum measured in the real environment for all radionuclides (i.e., ⁵⁷Co, ⁷⁵Se, ⁶⁰Co, ¹³³Ba, ¹³⁷Cs, ¹⁹²Ir, ²⁴¹Am, ¹⁵²Eu, and ²³⁸Pu) with a detection rate of 100%.

3.3 Effect of detection time on the proposed method

Experiments using different detection times were conducted, because detection time can lead to a change in spectral intensity. The detector was installed at a fixed location, and the radionuclides were placed in front of the detector at point A (Fig. 4). The background spectrum without any source was recorded for 300 s to generate the background template. The spectra of Radio-1, Radio-2, and Radio-3 placed at point A (Fig. 4) were recorded to evaluate the detection time effect on the identification performance of the proposed method. The measurement was repeated 10 times for each condition, and detection times were 1, 2, 3, 4, and 5 s. For example, the measurement for Radio-1 with monitoring point at A (Fig. 4) and a detection time of 1 s would be repeated 10 times. The same rules apply to other detection times and radionuclides. A total of 150 spectra were recorded, which were all used as verifying samples. 10 spectra could be recorded at each detection time for each radionuclide, resulting in 50 spectra for each radionuclide. Each spectrum has a sample number, the value of which was determined by the measurement sequence. For example, 10 Radio-1 spectra with a detection time of 1 s would have sample numbers 1 to 10, 10 Radio-1 spectra with a detection time of 2 s would have sample numbers 11 to 20, etc. The same rules apply to Radio-2 and Radio-3.

Fig. 4Full-screen View

(Color online)Sketch of the first experimental environment

We were focused only on the output neurons of the DBN for ²³⁸Pu, ⁶⁰Co, and ¹³⁷Cs to select three radionuclides (³⁸Pu, ⁶⁰Co, and ¹³⁷Cs) to evaluate the identification performance of the proposed method in the real environment, and all the results related to verifying samples in this study were computed by analyzing the output value of the above three output neurons. Fig. 5 shows the results for samples with different detection times when only ²³⁸Pu exists in the environment. The output value of the DBN for ²³⁸Pu is greater than the threshold (0.85) at different times when only ²³⁸Pu exists in the environment. The output value for ⁶⁰Co or ¹³⁷Cs is less than the threshold, indicating the presence of ²³⁸Pu in the environment, but the absence of ⁶⁰Co and ¹³⁷Cs. The results are consistent with the actual situation because of the normalization of approximation coefficients, which can eliminate the effect of the change in spectral intensity. Therefore, the identification performance of the proposed method is not affected by the detection time if the minimum net count of the spectrum of related radionuclides is satisfied. Figures 6 and 7 illustrate similar results. The net count of the spectrum of each radionuclide measured at point A (Fig. 4) with a detection time of 5 s, and the count of background spectrum with a detection time of 5 s are listed in Table 3. The dose rate of the background without a shielding apparatus was measured in the laboratory (approximately 0.1 μSv/h).

Fig. 5Full-screen View

(Color online) Output values of deep belief network (DBN) at different sample numbers when only ²³⁸Pu exists in the environment

Fig. 6Full-screen View

(Color online) Output values of DBN at different sample numbers when only ⁶⁰Co exists in the environment

Fig. 7Full-screen View

(Color online) Output values of DBN at different sample numbers when only ¹³⁷Cs exists in the environment

3.4 Effect of the number of radionuclides on the proposed method

Experiments with different numbers of radionuclides were performed, because this parameter can change the spectral shape. The detector was installed at a fixed location, and the radionuclides were placed in front of the detector at point A (Fig. 4). The spectra of the cases with different numbers of radionuclides (i.e., Radio-1, Radio-2, Radio-3, Radio-1+Radio-2, Radio-1+Radio-3, Radio-2+Radio-3, and Radio-1+Radio-2+Radio-3) placed at point A (Fig. 4) were recorded to evaluate the effect of the number of radionuclides on the identification performance of the proposed method. The measurement was repeated 10 times for each condition, and the detection time was 5 s. For example, the measurement for Radio-1 with monitoring point at A (Fig. 4) and a detection time of 5 s would be repeated 10 times. The same rules apply to other combinations. Hence, 70 spectra were recorded, which were all used as verifying samples. Three combinations (i.e., Radio-1, Radio-2, and Radio-3) correspond to the condition with 1 radionuclide. Three combinations (i.e., Radio-1+Radio-2, Radio-1+Radio-3, and Radio-2+Radio-3) correspond to the condition with two radionuclides. One combination (i.e., Radio-1+ Radio-2+ Radio-3) corresponds to the condition with three radionuclides.

Table 4 lists the results of samples with different numbers of radionuclides. All radionuclides (i.e., ²³⁸Pu, ⁶⁰Co, and ¹³⁷Cs) have a detection rate of 100%, which demonstrates that the proposed algorithm can predict the isotopic compositions of the spectra of multiple radionuclides. This phenomenon is due to the fact that the DBN was trained using samples with mixed spectra of multiple radionuclides. However, the training samples are composed of simulation spectra which differ from the spectra measured in the real environment, especially in the low-energy region, although their shapes are similar. This phenomenon may reflect the fact that the DBN has strong tolerance, indicating promising machine learning for spectrometry analysis.

3.5 Effect of detection distance on the proposed method

Experiments with different detection distances were conducted, because this parameter can also lead to a change in the spectral shape. The detector was installed at a fixed location, and the radionuclides were placed in front of the detector at points A, B,…, O (15 points) at 10 cm intervals (Fig. 8). The spectra of the radionuclides were recorded to evaluate the effect of detection distance on the identification performance of the proposed method. The measurement was repeated 10 times for each condition, and the detection time was 10 s. For example, the measurement for Radio-1 with monitoring point at A (Fig. 8) and a detection time of 10 s would be repeated 10 times. The same rules apply to other monitoring points or radionuclides. Thus, 450 spectra were recorded, which were all used as verifying samples.

Fig. 8Full-screen View

(Color online) Sketch of the second experimental environment

Fig. 9 shows the results of samples with different detection distances. The ARIDs of Radio-1 (²³⁸Pu with an activity of 8.89×10³ μCi and a detection time of 10 s), Radio-2 (⁶⁰Co with an activity of 1.59 μCi and a detection time of 10 s), and Radio-3 (¹³⁷Cs with an activity of 1.42 μCi and a detection time of 10 s) are greater than or equal to 60, 70, and 60 cm, respectively. The corresponding minimum detectable activities (MDAs) of Radio-1(²³⁸Pu with a detection distance of 60 cm and a detection time of 10 s), Radio-2 (⁶⁰Co with a detection distance of 70 cm and a detection time of 10 s), and Radio-3 (⁶⁰Cs with a detection distance of 60 cm and a detection time of 10 s) are less than or equal to 8.89 × 10³, 1.59, and 1.42 µCi, respectively [30, 31]. Thus, the identification performance of the proposed method is not affected by the detection distance under certain conditions, because the effect of detection distance on the proposed method can be eliminated by the normalization of approximation coefficients. The detection rate of radionuclides (i.e., ²³⁸Pu, ⁶⁰Co, and ¹³⁷Cs) is over 60% when the detection distance is greater than the ARID of the related radionuclide [Fig. 9(a)–(c)]. This result is due to the fact that the detection rate is a statistic, and the numerator TN of Eq. (1) remains almost constant, although the numerator TP in Eq. (1) decreases with an increase in distance. This phenomenon is due to the low percentage of error in identification, which indicates the proposed method is a robust method.

Fig. 9Full-screen View

(Color online) Single radionuclide detection rates (%) of samples with different detection distances: (a) 238Pu, (b) 60Co, and (c) 137Cs

The spectra of radionuclides (²³⁸Pu, ⁶⁰Co, and ¹³⁷Cs) placed in front of the detector according to their ARIDs are plotted to further visualize the identification performance of the proposed method (Fig. 10). The signal-to-noise ratio (SNR) is extremely low in this condition, and the corresponding characteristic peaks of radionuclides are not obvious (fewer counts in the photopeak region). However, the proposed method can recognize these radionuclides with a detection rate of 100%. Therefore, the proposed method is suitable for fast radionuclide identification, because it requires a lower spectrum count.

Fig. 10.Full-screen View

Spectra of radionuclides with the monitoring point at their ARIDs: (a) Radio-1, (b) Radio-2, and (c) Radio-3

3.6 Effect of radionuclides in motion on the proposed method

Radionuclides may be carried by people or transported by car in the real environment. Thus, these molecules have a certain velocity. The effect of radionuclides in motion on the identification performance of the proposed method should be evaluated. The detector was installed at a fixed location, and the radionuclides were placed in front of the detector at points A, B,…, O at 10 cm intervals (Fig. 8). An experiment was designed to assess the effect of radionuclides in motion on the identification performance of the proposed method. A person with Radio-1, Radio-2, and Radio-3 approached point A from the starting point O along the line segment OA at a speed of approximately 0.375 m/s. The whole process took approximately 4 s, and 33 spectra were recorded at intervals of approximately 0.1 s. All spectra were used as verifying samples.

Fig. 11 shows the results of the samples of the moving radionuclides. The output value of the DBN for ¹³⁷Cs is greater than the threshold (0.85) at time 2.5 s, indicating the identification of ¹³⁷Cs. The corresponding value for ²³⁸Pu is also greater than the threshold (0.85) at time 2.76 s, indicating the identification of ²³⁸Pu. The corresponding value for ⁶⁰Co is greater than the threshold (0.85) at time 3.0 s, indicating the identification of ⁶⁰Co. Thus, the output value of the DBN for related radionuclides is always beyond the threshold after the corresponding radionuclide is recognized. The proposed method exhibits enhanced identification performance for the spectra of moving radionuclides and requires only a short time to identify all radionuclides in the environment. This characteristic is recommended for fast radionuclide identification.

Fig. 11.Full-screen View

(Color online) Output values of DBN for samples of moving radionuclides at different times. Output channels of DBN for (a) ²³⁸Pu, (b) ⁶⁰Co, and (c) ¹³⁷Cs

Fig. 12 demonstrates the spectra in radionuclide identification. The spectrum has a low SNR at each time of identification of the related radionuclide, and its corresponding characteristic peak cannot be observed. The proposed method is not based on peak detection, but on information from the full spectrum. The proposed algorithm makes a judgment using information from the full spectrum, contrary to the method based on peak detection. Specialized knowledge, deconvolution, and other parameters previously used were not needed after the DBN was trained.

Fig. 12.Full-screen View

Spectra of radionuclides (i.e., Radio-1+ Radio-2+ Radio-3) at their time of identification. (a) t=2.5 s, (b) t=2.76 s, and (c) t=3.0 s.

The identification results of the spectra contained within Fig. 12(a), Fig. 12(b), and Fig. 12(c) using MAESTRO 7.01 software (a software for gamma-ray spectrometry analysis) were all "No peaks found," as shown in Fig. 13. This is primarily because there were lower counts of radionuclide spectrum in the photopeak region, so the MAESTRO 7.01 software couldn’t find a peak that satisfied its default requirement. Experimental results demonstrate once again that the proposed method is not based on peak detection, but on information from the full spectrum.

Fig. 13Full-screen View

(Color online). Identification results of MAESTRO 7.01 software for spectra of radionuclides (i.e., Radio-1+ Radio-2+ Radio-3) at the times of (a) t=2.5 s, (b) t=2.76 s, and (c) t=3.0 s.