Data acquisition

In the data acquisition stage, the datasets were produced. For a single distorted negative exponential pulse, the pulse distortion time is assumed to be $t_{\text{jump}}$, and its mathematical model is given by Eq. (1), where A represents the amplitude of the negative exponential pulse, $\tau$ represents attenuation time constant, $T_{\text{clk}}$ represents sampling period: $V_{\text{e}}\left(t\right)=\{\begin{array}{ll}A\times \exp \left(-t\times \frac{T_{\text{clk}}}{\tau }\right),\hfill & t<t_{\text{jump}}\hfill \\ 0\hfill & t\ge t_{\text{jump}}\hfill \end{array}$ (1)

The mathematical model of a pulse sequence composed of N distortion negative exponential pulses is: $V_{\text{e}}\left(t\right)={\displaystyle \sum _{i=1}^N\left[u\left(t-T_i\right)A_i{e}^{\frac{-\left(t-T_i\right)}{\tau }}\right]} .$ (2)

Here, u(t) represents unit-step signal, $A_i$ is amplitude coefficient of the i^th nuclear pulse, $T_i$ is occurrence time of the i^th nuclear pulse, $\tau$ represents time constant. The negative exponential pulse sequence after discretization can be expressed as $V_{\text{e}}\left(k{T}_{\text{clk}}\right)={\displaystyle \sum _{i=1}^N\left[u\left(k{T}_{\text{clk}}-T_i\right)A_i{e}^{\frac{-\left(k{T}_{\text{clk}}-T_i\right)}{\tau }}\right]}.$ (3)

The distorted nuclear pulse sequence V_o(nT_clk) used for the parameter estimation is regarded as $N$ distorted negative exponential pulse sequences V_e(kT_clk) obtained after triangular shaping, and its mathematical model is as follows: $\begin{array}{c}{V}_{\text{o}}\left(n{T}_{\text{clk}}\right)=2V_{\text{o}}\left[\left(n-1\right)T_{\text{clk}}\right]-V_{\text{o}}\left[\left(n-2\right)T_{\text{clk}}\right]\\ +\left\{V_{\text{e}}\left[\left(n-1\right)T_{\text{clk}}\right]-V_{\text{e}}\left[\left(n-n_{\text{a}}-1\right)T_{\text{clk}}\right]\right\}\\ -e^{\frac{-T_{\text{clk}}}{\tau }}\left\{V_{\text{e}}\left[\left(n-2\right)T_{\text{clk}}\right]-V_{\text{e}}\left[\left(n-n_{\text{a}}-2\right)T_{\text{clk}}\right]\right\}.\end{array}$ (4)

Here, the dataset of the CNN-LSTM model established in this study was taken from the pulse amplitude sampling value of distortion pulses after triangular shaping, whereas the parameter set P was taken from the negative exponential pulses before triangular shaping. Taking the parameter set $P_i$ of the i^th negative exponential pulse as an example, this set includes the amplitude $A_i\left(i=1,2,\dots ,N\right)$, sampling period $T_{\text{clk}}$, time constant $\tau$ of the negative exponential pulse and the rising time $t_{\text{up}}$ of the triangular shaping. The matrix representation of the dataset is as follows: $\left[\begin{array}{ccc}{\left[V_{\text{o}}\left(T_{\text{clk}}\right)\right]}_1{\left[V_{\text{o}}\left(2\times T_{\text{clk}}\right)\right]}_1& \cdots & {\left[V_{\text{o}}\left(n\times T_{\text{clk}}\right)\right]}_1{P}_1\\ ⋮& \ddots & ⋮\\ {\left[V_{\text{o}}\left(T_{\text{clk}}\right)\right]}_N{\left[V_{\text{o}}\left(2\times T_{\text{clk}}\right)\right]}_N& \cdots & {\left[V_{\text{o}}\left(n\times T_{\text{clk}}\right)\right]}_N{P}_N\end{array}\right]$ (5)

The datasets in Eqs. (5) contains $N$ triangular-shaped pulse sequences, and the amplitude of each pulse corresponds to a row in the matrix. Each row contains $n+1$ columns. The first $n$ columns correspond to each amplitude value of the triangular shaping result of distortion pulses, and the last column represents the parameter set of the sequence, including $A_i$, $T_{\text{clk}}$, $\tau$ and $t_{\text{up}}$. The structures of the generated datasets are shown in Fig. 2.

Fig. 2Full-screen View

Generated datasets

As required, this study divided the dataset into training, test, and validation sets in a ratio of 7:2:1. In general, the training set accounts for a large proportion of the data and is used to train the generalization ability of the model, whereas the verification set is used to verify whether the model is overfitted. Once overfitting occurs, it must be eliminated by adding a dropout layer to randomly discard the connections of some neurons.

Hyperparameter optimization

The model adopted in this study addresses a series of nuclear pulse amplitude sequences. As a special recurrent neural network (RNN), LSTM is essentially different from an RNN in that the introduction of forgetting gates determines which information will be retained or forgotten by controlling the parameters. Therefore, LSTM can solve the problems of gradient disappearance and explosion of long time-series samples during the training process.

LSTM usually deals with long sequences and large sample data, but too large sample data will cause some difficulties during model training, such as computation complexity. Therefore, this study combined LSTM wSith a CNN. A convolutional neural network (CNN) is not as much of an algorithm as a feature extraction method, and usually includes a convolution layer (including an activation layer), a pooling layer, and a full link layer. The process of extracting features through convolutional neural network is essentially the process of solving the optimal parameter matrix. The relationship between input and output can be expressed using Eq. (6): $y=f\left(X,W\right)=WX+b,$ (6) where $W$ represents the weight parameter matrix, $X$ represents the input neuron matrix, as shown in Eq. (6), and $b$ represents the offset term. The structures and parameter settings of the CNN model are listed in Table 1. By setting multiple convolution and pooling layers, the CNN greatly reduced the number of samples in the dataset while completing feature extraction. The CNN model used in this study included two one-dimensional convolution layers and two pooling layers. The time step for each layer was set to one. The first convolution layer contained 64 convolution cores and output 64 eigenvectors, whereas the second contained 16 convolution kernels and output 16 eigenvectors. The size of the convolution core in both convolution layers is set to 3×3, the moving step is 1, the filling strategy is "same,” and the activation function is "relu.” The pooling layer adopts MaxPooling1D, which does not change the input signal size.

In the forward propagation process, the input neurons are kept unchanged, and the weight parameter matrix is initialized using a random strategy. The error between the output parameter set $P_i^{\text{'}}$ in the forward-propagation process and the actual pulse parameter set $P_i$ in the training set can be calculated using the loss function. The calculation method is expressed in Eq. (7), where the actual parameter set of the i^th sample is represented by $P_i$, and Pi the parameter set estimated by the forward propagation is represented by $P_i^{\text{'}}$. For the training set with $N$ samples, the mean square error (MSE) of the parameter set was considered as the function value of the loss function, which is represented by $L_{\text{MSE}}$. $L_{\text{MSE}}=\frac{1}{N}{\displaystyle \sum _{i=1}^N{\left(P_i-P_i^{\text{'}}\right)}^2}$ (7)

Subsequently, the back-propagation through time (BPTT) algorithm is applied to feed back the gradient of the loss function and $L_{\text{MSE}}$ to the CNN-LSTM network to update the weight matrix $W$, so as to reduce the error in subsequent iterations. To prevent a gradient explosion, this model sets the gradient clipping parameter clipnorm to 1 and clipvalue to 0.5.

Figure 3 shows the hyperparameter optimization process for the parameters and layers during the training process of the CNN-LSTM model. If the batch size is set too large, it can lead to memory overflow during the training process, and the model is prone to convergence to local optima, making it impossible to complete the training. If the batch size is too small, the rate of convergence of the model will be too slow, and the training time will be too long. Figure 3 shows the iterative loss values obtained for the training and validation sets when the number of layers of the LSTM model was five and the parameter batch size was set to 10 and 100, respectively. When the batch size was 100, the model converged at the 40th epoch with a high loss of 3 × 10⁵. When batch size is 10, the model converges normally, and the loss value after convergence approaches zero as much as possible.

Fig. 3Full-screen View

Process of the hyperparameter optimization

When setting the parameters for the LSTM model, theoretically speaking, the more layers there are, the more ideal the training results. The problem of vanishing gradient must also be considered. An increase in the number of layers results in a greater computational burden; therefore, when optimizing the hyperparameters, we usually set the number of layers to 3-6. Figure 3 shows the iterative loss values obtained for the training and validation sets for layers 3 and 5. It can be seen that when the batch size is 10, the attenuation speed of the LSTM model with five and three layers is close, but when the layer number is 3, the loss value after model convergence is still as high as 2.8 × 10⁵. When the number of layers was five, the loss values of the training and validation sets approached zero.

After hyperparameter optimization, five LSTM layers were set with an initial learning rate of 0.0001 and a batch size of 10, and Adam was selected as the optimizer. The generated network model structure is shown in Fig. 4, which includes the input layer, hidden layer, output layer, and backpropagation part. The hidden layer includes the CNN and LSTM models.

Fig. 4Full-screen View

Network model structure of CNN-LSTM. The detailed calculation procedure of the LSTM unit can be found in the article of Graves et al.[29]

Model training

To verify the effect of the CNN-LSTM model on the parameter estimation of the triangular shaping results under the condition of severe distortion of the negative exponential pulses, we took 10000 samples and divided them into training, verification, and test sets according to a ratio of 7:2:1, with a training period of 100 epochs. The change in the loss values obtained from the training and verification sets during the training process is shown in Fig. 5.

Fig. 5Full-screen View

Iterative graph of loss and accuracy on training and validation sets during model training

In general, with an increase in the number of training cycles, the loss values of the training and verification sets showed a downward trend. The verification set experiences some shocks in the later period, but soon tends to stabilize. Using the loss function, when the loss values of both the training and validation sets were low and relatively stable, the model during that period was saved as the best model. In the training process of the proposed model, the model at the 91st epoch was saved as the best model, with loss values of 2.7895 and 3.5507 for the training and validation sets, respectively, during this epoch.

Performance evaluation of parameter estimation

In the production of the test set, considering that the distortion degree of the pulses may affect the model test effect, the sample of the test set can be divided into two categories according to the pulse distortion time mentioned above: slightly distorted pulses and severely distorted pulses, whose triangular shaping results are shown in Fig.6. The rising time of the triangular shape, $t_{\text{up}}$, is bound. If the time of pulse distortion, $t_{\text{jump}}$, is before the $t_{\text{up}}$, the amplitude of the shaping result decreases significantly, as shown in Fig. 6, and then such pulses are marked as severely distorted pulses. On the other hand, if the time of pulse distortion is after $t_{\text{up}}$ (including the critical time $t_{\text{up}}$), the amplitude of the shaping result decreases slightly, as shown in Fig. 6; therefore, this type of pulse is marked as a slightly distorted pulse.

Fig. 6Full-screen View

Triangular shaping results of negative exponential pulse at different distortion moments

To control the impact of other variables, the amplitude parameter $A_i$ of Pulse1-Pulse8 is fixed at 2000 mV, the time interval $T_i$ of adjacent distorted pulses was 500 $T_{\text{clk}}$. The sampling period $T_{\text{clk}}$, is 50 ns, and the rising time of the triangular shaping was 100 $T_{\text{clk}}$. Consider two pulse sequences and their triangular shaping results for analysis, as shown in Fig. 7 and 8.

Fig. 7Full-screen View

Triangular shaping results of the seriously distorted pulses. a Seriously distorted pulses, b Triangular shaping results

Fig. 8Full-screen View

Triangular shaping results of the Slightly Distorted Pulses. a Slightly distorted pulses, b Triangular shaping results

In Fig. 7a, the amplitude value of Pulse1 suddenly changes to 0 at the 47th $T_{\text{clk}}$ and the amplitude value of Pulse2 suddenly changes to 0 at the 60th $T_{\text{clk}}$. The amplitude value of Pulse3 suddenly changes to zero at the 75th $T_{\text{clk}}$ and the amplitude value of Pulse4 suddenly changes to zero at the 90th $T_{\text{clk}}$. As mentioned above, the rising time of the triangular shaping remains at 100 $T_{\text{clk}}$, that is, Pulse1-Pulse 4 are distorted before the peak value of the triangular shaping. Therefore, the amplitude values of the shaping results exhibited a significant loss compared to the original pulses, as shown in Fig.7b.

In Fig. 8a, the amplitude value of Pulse5 suddenly changes to zero at the 100th $T_{\text{clk}}$ and the amplitude value of Pulse6 suddenly changes to zero at the 140th $T_{\text{clk}}$. The amplitude value of Pulse7 suddenly changes to zero at the 220th $T_{\text{clk}}$ and the amplitude value of Pulse8 suddenly changes to zero at the 280th $T_{\text{clk}}$. In Fig. 8a, the common feature of Pulse5-Pulse8 is that the distortion time is after the peak value of the triangular shaping. Therefore, the amplitude values of the shaping results have no large losses compared to the original pulses, as shown in Fig. 8b.

During the performance evaluation of the parameter estimation, the CNN-LSTM and LSTM models were used to estimate the parameters of the above two pulse sequences, and the output results are shown in Fig. 9. The real value of the eight pulse amplitudes was fixed at 2000 mV, but the amplitude loss of the triangular shaping result of the Pulse 1-Pulse 4 was very large.

Fig. 9Full-screen View

Amplitude Comparison Chart of the Distorted Pulses

Based on the parameter estimation results of the slightly and severely distorted pulses, the absolute and relative errors of the different methods used to estimate the pulse amplitudes are summarized in Table 2.

A_real represents the actual pulse amplitude. The measurement methods used in this study mainly included triangular shaping, CNN-LSTM, and LSTM models, the measurement results of which are represented by $A_{\text{Tri}}$, $A_{\text{CNN}-\text{LSTM}}$, and $A_{\text{LSTM}}$. This study used relative error indicators to evaluate the parameter estimation performance of each algorithm. Let $\delta$ represents the relative error, and Δ represents the absolute error, whose calculation process is given in Eqs. (8), (9), (10). ${\delta }_{\text{Tri}}=\frac{{\text{Δ}}_{\text{Tri}}}{A_{\text{real}}}\times 100\text{\%}=\frac{ABS\left(A_{\text{real}}-A_{\text{Tri}}\right)}{A_{\text{real}}}\times 100\%$ (8) $\begin{array}{c}{\delta }_{\text{CNN}-\text{LSTM}}=\frac{{\text{Δ}}_{\text{CNN}-\text{LSTM}}}{A_{\text{real}}}\times 100\%\\ =\frac{ABS\left(A_{\text{real}}-A_{\text{CNN}-LSTM}\right)}{A_{real}}\times 100\%\end{array}$ (9) ${\delta }_{\text{LSTM}}=\frac{{\text{Δ}}_{\text{LSTM}}}{A_{\text{real}}}\times 100\%=\frac{ABS\left(A_{\text{real}}-A_{LSTM}\right)}{A_{\text{real}}}\times 100\%$ (10)

For severely distorted pulses, the average relative error of triangular shaping was as high as 14.39%, whereas that of CNN-LSTM was only 0.14%, and that of LSTM was 1.05%. On the other hand, for slightly distorted pulses, the average relative error of triangular shaping was 0.39%, while that of CNN-LSTM was only 0.06%, and that of LSTM was 0.65%. It can also be observed from the estimation results that the two models can estimate the pulse amplitude very accurately, whether it is a severely or slightly distorted pulse. It is worth noting that although the performance of the CNN-LSTM model is slightly better than that of LSTM, both deep learning methods have very high accuracy in pulse parameter estimation. This study introduces a CNN because the input pulse sequence is relatively complex, and directly uses the LSTM model to process data that are too large. Therefore, using a CNN for sampling reduces the amount of data and saves computational resources.

Qualitative analysis

The amplitude of the distorted pulses estimated by the CNN-LSTM model was used to replace the original pulse amplitude, and a comparison diagram of the X-ray fluorescence spectrum is shown in Fig. 11. The red spectral lines represent the results after the peak correction. By magnifying the local characteristics of the shadow peak area in the logarithmic coordinate system, it can be observed that there is a weak peak to the left of the two characteristic peaks of strontium in Fig. 11. Because the chemical symbol of Sr is Sr, its two characteristic peaks are represented by Sr-1 and Sr-2. According to the principle of multichannel spectroscopy, the amplitude loss of triangular shaping results in a left shift in the counts (also known as the left shift of the peak position), and the left-shifted counts form a new shadow peak on the left side of the characteristic peak. In Fig. 11, the left shift of the two characteristic peaks of strontium forms the shadow peaks 1 and 2. After the parameter estimation of the distorted pulses using the CNN-LSTM model, the left shift of the peak position was effectively corrected, and the shadow peak was eliminated.

Fig. 11Full-screen View

Correction effect of shadow peak of strontium element characteristic peak

Qualitative analysis of the X-ray fluorescence spectrum before and after optimization showed that the CNN-LSTM model trained in this study could effectively correct the shadow peak caused by pulse distortion and optimize the X-ray fluorescence spectrum analysis results.

Quantitative analysis

As mentioned previously, the amplitude loss of shaping results in a left shift in the peak position. A shadow of the characteristic peak was formed on the left side of the characteristic peak. Here, the peak area $S$ represents the sum of the counts of a certain channel address interval. $S_{\text{shadow}}$ represents the area of the shadow peak formed by the left shift of the first characteristic peak of Sr, and $S_{\text{peakloss}}$ indicates the area loss of the first characteristic peak of strontium owing to the left shift of the peak position, whose computational processes are shown in Eqs. (11), (12), (13) and (14). $S_{\text{shadow}1}={\displaystyle \sum _{i=896}^{960}{\text{Count}}_{i-\text{origin}}}-{\displaystyle \sum _{i=896}^{960}{\text{Count}}_{i-\text{corrected}}}$ (11) $S_{\text{peakloss}1}={\displaystyle \sum _{i=960}^{1024}{\text{Count}}_{i-\text{corrected}}}-{\displaystyle \sum _{i=960}^{1024}{\text{Count}}_{i-\text{origin}}}$ (12)

The region of interest (ROI) of shadow peak 1 is 896-960, so $S_{\text{shadow}1}$ is numerically equal to the difference between the peak area of the channel address interval where the shadow peak is located before and after spectral peak correction, as shown in Eq. (11), and $S_{\text{shadow}1}$ is shown in the shaded area of Fig. 12a.

Fig. 12Full-screen View

Spectral comparison of different ROI before and after correction. a ROI of shadow1, b ROI of Sr-1, c ROI of shadow2, d ROI of Sr-2.

S_peakloss1 represents the corrected peak area loss in the ROI of the first characteristic Sr peak of strontium element after calling the model. The ROI of the selected characteristic peak was located in the channel address interval of 960-1024, as shown in the shaded area of Fig. 12b. $S_{\text{peakloss}1}$ is numerically equal to the difference in the peak area of the second characteristic peak ROI before and after the spectral peak correction, as shown in Eq. (12).

S_shadow2 represents the area of the shadow peak formed by the left shift of the second characteristic peak of strontium. The ROI of shadow peak 2 is 1024-1088, so S_shadow2 is numerically equal to the difference in the peak area of the shadow peak ROI before and after spectral peak correction, as shown in Eq. (13); and S_shadow2 is shown in the shaded area in Fig. 12c. $S_{\text{shadow}2}={\displaystyle \sum _{i=1024}^{1088}{\text{Count}}_{i-\text{origin}}}-{\displaystyle \sum _{i=1024}^{1088}{\text{Count}}_{i-\text{corrected}}}$ (13) $S_{\text{shadow}2}={\displaystyle \sum _{i=1088}^{1152}{\text{Count}}_{i-\text{corrected}}}-{\displaystyle \sum _{i=1088}^{1152}{\text{Count}}_{i-\text{origin}}}$ (14)

S_peakloss2 represents the corrected peak area loss of the second characteristic peak of Sr element after calling the model. The ROI of the selected characteristic peak was located in the channel address interval of 1088–1152, as shown by the shaded area in Fig. 12d. S_peakloss2 is numerically equal to the difference in the peak area of the second characteristic peak ROI before and after the spectral peak correction, as shown in Eq. (14).

To quantify the correction effect of the CNN-LSTM model on the X-ray spectra of the measured iron ore samples, two indicators, the correction ratio $R_{\text{c}}$ and the efficiency ratio $R_{\text{e}}$, were introduced. The correction ratio represents the proportion of the increment of the peak area of the two characteristic peak ROI after calling the model to the peak area of the corrected characteristic peak ROI, and the efficiency ratio represents the proportion of the increment of the peak area of the two characteristic peak ROI after calling the model to the peak area of the two shadow peak ROI. The calculation formulas are given by Eqs. (15) and (16), respectively. Ten measurements were performed on the iron ore samples, and the measurement results were analyzed, as shown in Table 3. $R_{\text{c}}=\frac{S_{\text{peakloss}1}+S_{\text{peakloss}2}}{{\displaystyle {\sum }_{i=960}^{1024}{\text{Count}}_{i-\text{corrected}}}+{\displaystyle {\sum }_{i=1088}^{1152}{\text{Count}}_{i-\text{corrected}}}}\times 100\%$ (15) $R_{\text{e}}=\frac{S_{\text{peakloss}1}+S_{\text{peakloss}2}}{S_{\text{shadow}2}+S_{\text{shadow}2}}\times 100\%$ (16)

From the comparison of the measurement results, it can be seen that the corrected X-ray spectrum obtained using the CNN-LSTM model to predict the pulse height has two typical features. First, the peak area of the characteristic peak ROI was improved compared with the original spectrum, and the standard deviation of the multiple measurement results was also significantly reduced. Second, the peak area of the shadow peak area was significantly reduced. According to the two characteristics above and the energy conservation theorem, it can be inferred that the peak area reduced in the shadow peak area should theoretically be corrected to the characteristic peak ROI, and the correction effect can be evaluated by $R_{\text{e}}$ defined above. Table 3 shows that approximately 86.27% of the peak area reduced in the shadow peak area can be corrected to the characteristic peak ROI, and the proportion of the corrected peak area to the peak area of the characteristic peak ROI is approximately 1.72%, which is of great significance for X-ray spectroscopy and elemental content analyses.