Simulated annealing model

Ordinary simulated annealing algorithms produce only one new solution at a time, and based on the Metropolis criterion, the probability is determined to choose whether to accept the new state or not. Assuming the solution of the current moment search is x_t, the corresponding system energy (objective function) is E_t, and random disturbance is applied to the search point to generate a new solution x_t+1. Correspondingly, if the system energy is E_t+1, then the acceptance probability of the transition from x_t to x_t+1 of the search point is: $\text{P}=\{\begin{array}{cc}1,& E_{t+1}<E_t\\ {\text{e}}^{\frac{-\left(E_{t+1}-E_t\right)}{\text{k}T}},& E_{t+1}>E_t\end{array}$ (1)

Threshold oscillation SFLA model based on simulated annealing

To solve the problem of the SFLA easily falling into the local optima and its accuracy decreasing, this study improves the SFLA based on the simulated annealing model. The improved algorithm added a threshold shock strategy based on a simulated annealing strategy [27,28]. In the optimization process of the SFLA, all the populations participating in the population iteration accelerate to converge. The current frog population convergence was determined by the standard deviation of the frog solution in the community. When the mean deviation of each frog population decreases by a preset threshold, part of the population is divided into groups that oscillate near the optimal solution in their community, thus jumping out of the local optimum [29-31]. $\{\begin{array}{l}U\left(t\right)=\left[M^{m/2+k}|\left(k=-\frac{m\times \omega }{2},-\frac{m\times \omega }{2}+1,\mathrm{...},\frac{m\times \omega }{2}\right),\text{ k}\ne \text{0, }m\times \omega \%2=0\text{)}\right]\\ \\ U\left(t\right)=\left[M^{m/2+k}|\left(k=\frac{-m\times \omega +1}{2},\frac{-m\times \omega +1}{2}+1,\mathrm{...},\frac{m\times \omega -1}{2}\right)\text{, }m\times \omega \%2=1\text{)}\right]\end{array}$ (2) where, M represents frog memeplex, N frogs are allocated to m/2+k populations and each memeplex contains n frogs, satisfying N=m×n, ω is the proportion of the number of groups that are mapped out to the total number of groups. When the spatial standard deviation of the S-dimension solution of the frog memeplex meets the threshold value, several memeplex are selected successively from the middle segment memeplex to both sides of each memeplex, and the sampled memeplex is defined as the regional oscillating memeplex. U(t) indicates rearrangement of M in descending order for the evolution iteration of the next generation memeplex [32]. The specific algorithm flow is shown in Fig. 1.

Fig. 1Full-screen View

Improved SFLA global search process

Setting Model Parameters

First, the initial parameters were set. The input layer set the input size to 256 px. The 256 channels of the full-spectrum data contained U, Th, and K nuclide information. Setting the Epoch to 32 and iteration time to 150. The population size of the PSO algorithm used to optimize the neural net is n=40. The initial inertia weight w=0.9, which linearly reduces to 0.4 as the number of iterations gradually diminishes the influence of the prior particle velocity on the present velocity, and the acceleration constant c₁=c₂=2 [34-36]. Each population contained 100 frogs, the number of iterations within each population was 10, the number of global information exchanges was 50, and the attenuation coefficient D of the simulated annealing strategy was a random number between 0.3 and 0.5. The populations were divided into ten groups based on a thresholding oscillation mixed leapfrog algorithm using simulated annealing. The oscillating intervals were 0.01, 0.05, 0.1, and 0.2, of the value range of the independent variables [37].

Common CNN use convolutional and pooling layers instead of fully connected layers. Through deep learning, that is, an increase in the number of layers, the model learns more abstract features. This study adopted the CNN structure shown in Fig. 4 and set two convolution pooling layers: a fully connected layer and an output layer. Spectral data were entered from the left side of a 16 × 16 grid. After the feature extraction of the convolutional and pooling layers, the coefficients of the fully connected layers were updated using the improved SFLA-PSO. After network training, a new spectral line on the right side was obtained as the output.

Fig. 4Full-screen View

SFLA-PSO CNN model structure

Spectrum processing and analysis

The gamma spectrum data measured by the NaI (TI) detector were processed and analyzed using 256 experimental data points [38,39]. The improved SFLA-PSO neural network algorithm was used to process the measured spectrum [40], and the processing and results of the measured data were analyzed from two aspects: spectrum line comparison and signal-to-noise ratio before and after processing to verify the rationality of the model design in this study and test the noise reduction effect [41].

To objectively analyze the model effect, the special energy spectrum data, maximum peak area (Fig. 5 (a)), minimum peak area (Fig. 5 (b)), and spectrum of the average peak area (Fig. 5 (c)) were simultaneously analyzed.

Fig. 5Full-screen View

(Color online) Comparison of (a) maximum peak area spectra, (b) minimum peak area spectral line and (c) spectral lines of average peak area

By comparing the spectra of the airborne gamma ray spectrometer before and after calibration and recombination, it can be seen that after model training and recombination, the noise information in the spectrum was significantly reduced, and the information of the weak peaks was preserved. For the weak peak region in the maximum peak area spectrum (Fig. 5 (a)), the reconstructed spectrum not only removes noise information but also preserves and highlights the weak peak shape. In the minimum peak area spectrum (Fig. 5 (b)) before reconstruction, the weak peak information was hidden in the noise. After reconstruction, weak peak information was enhanced. In the average peak area spectrum (Fig. 5 (c)), the reconstructed spectrum also had significant effects on noise reduction and highlighted weak peaks.

To assess the ability of this model to extract spectral data, the signal-to-noise ratio was quantitatively analyzed after energy spectrum training. Equation (6) was used to calculate the signal-to-noise ratio (SNR), where y(n) represents the predicted value, ~ yy(n) represents the true value, and N_b represents the sample size.

The higher the expected value, the better the noise-reduction ability of this model. $\text{SNR}=10{\log }_{10}\frac{{\displaystyle {\sum }_{n=1}^{N_b}{y}^2\left(n\right)}}{{{\displaystyle {\sum }_{n=1}^{N_{\text{b}}}\left(y\left(n\right)-\stackrel{\sim }{y}\left(n\right)\right)}}^2}$ (6)

The RMSE is the square root of the ratio of the deviation between the predicted and real values and the observed number, which is used to measure the deviation between the denoised and original spectra. According to Eq. (7), the smaller the value, the higher the SNR and the more effective the training model in reducing noise. $\text{RMSE}=\sqrt{{\displaystyle \sum _{n=1}^{N_b}\frac{{\left(y\left(n\right)-\stackrel{\sim }{y}\left(n\right)\right)}^2}{N_b}}}$ (7)

In this study, 160 groups of reconstructed spectral data were selected to calculate the signal-to-noise ratio; the results are shown in Fig. 6.

Fig. 6Full-screen View

(a) Reconstituted spectrum SNR (b) reconstituted spectrum RMSE

Referring to the experimental results of Li Fei [42], the SNR value is generally approximately 30 and the RMSE value should be less than 10. After wavelet denoising was performed on the tested energy spectrum data, the mean values of the SNR and RMSE in Fig. 6 (a) and Fig. 6 (b) were 31.27 and 2.75, respectively, indicating that the denoising method used in this model was very effective.

The sample size also affects training accuracy. In this study, 500 standard spectra were selected as training samples, and preliminary model effect validation was performed. The PSO, SFLA, and stochastic gradient descent (SGD) are selected as controls. The convergence of the training process of each optimization model is shown in Fig. 7, where “Loss” represents the error of each iteration of the model, and “Epoch” represents the number of iterations of data in the training process.

Fig. 7Full-screen View

Convergence of each optimization algorithm

It can be seen that the convergence of the four optimization models well reflects the characteristics of each model. Owing to the rapid convergence effect of the PSO algorithm, the neural network falls into local extrema, but the particle swarm optimization algorithm has completed convergence after 30 iterations; therefore, the prediction accuracy is low. The SFLA converges slower than the PSO, which converges to the local extrema approximately 75 times, but its accuracy is slightly higher than that of the PSO. When the CNN uses SGD to optimize the parameters, the convergence time is the longest, and approximately 110 iterations of convergence are completed. However, it is closest to the optimal extreme point, showing the possibility of falling into the local extreme value. The SFLA-PSO CNN model has the highest training efficiency, more than 20 iterations of convergence, and the best extreme value determination effect, which proves that this method has better nonlinear fitting ability and higher prediction accuracy for full-spectrum airborne gamma processing.

Increasing the dataset size is an important method of overfitting. Improve the accuracy of the model as much as possible and ensure its computational efficiency. In this study, 2000, 5000, and 10000 spectral data were used as samples for model training to obtain the most optimal model structure.

The sample training effect for each dataset size is shown in Fig. 8. With an increase in the sample size, the accuracy of the model also improved, and the training time was greatly improved. However, the models trained with different sample sizes converged when the loss tended to zero, indicating that the sample size had no significant influence on the accuracy of the model prediction ability. This is probably because this model only studies three types of nuclides, and there are few features to extract; therefore, ideal learning results can be obtained with a small number of samples. To obtain effective measurement efficiency and accuracy, an SFLA-PSO CNN model developed using 5000 samples was selected for this study.

Fig. 8Full-screen View

Effects of training with different sample sizes