Estimation of Gaussian overlapping nuclear pulse parameters based on a deep learning LSTM model

Estimation of Gaussian overlapping nuclear pulse parameters based on a deep learning LSTM model 1 first-author 1 corresp Corresponding author, huanghongquan@cdut.cn Corresponding author, huanghongquan@cdut.cn huanghongquan@cdut.cn 1 1 1 1 1 1 College of Nuclear Technology and Automation Engineering, Chengdu University of Technology, Dongsanlu, Erxianqiao, Chengdu 610059, China College of Nuclear Technology and Automation Engineering, Chengdu University of Technology, Dongsanlu, Erxianqiao, Chengdu 610059, China A Long Short-Term Memory (LSTM) neural network has excellent learning ability applicable to time series’ of nuclear pulse signals. It can accurately estimate parameters associated with amplitude, time, and so on, in digitally shaped nuclear pulse signals—especially signals from overlapping pulses. By learning the mapping relationship between Gaussian overlapping pulses after digital shaping and exponential pulses before shaping, the shaping parameters of the overlapping exponential nuclear pulses can be estimated using the LSTM model. Firstly, the Gaussian overlapping nuclear pulse (ONP) parameters which need to be estimated received Gaussian digital shaping treatment, after superposition by multiple exponential nuclear pulses. Secondly, a dataset containing multiple samples was produced, each containing a sequence of sample values from Gaussian ONP, after digital shaping, and a set of shaping parameters from exponential pulses before digital shaping. Thirdly, the Training Set in the dataset was used to train the LSTM model. From these data sets, the values sampled from the Gaussian ONP were used as the input data for the LSTM model, and the pulse parameters estimated by the current LSTM model were calculated by forward propagation. Next, the loss function was used to calculate the loss value between the network-estimated pulse parameters and the actual pulse parameters. Then, a gradient-based optimization algorithm was applied, to feedback the loss value and the gradient of the loss function to the neural network, to update the weight of the LSTM model, thereby achieving the purpose of training the network. Finally, the sampled value of the Gaussian ONP for which the shaping parameters needed to be estimated was used as the input data for the LSTM model. After this, the LSTM model produced the required nuclear pulse parameter set. In summary, experimental results showed that the proposed method overcame the defect of local convergence encountered in traditional methods, and could accurately extract parameters from multiple, severely overlapping Gaussian pulses, to achieve optimal estimation of nuclear pulse parameters in the global sense. These results support the conclusion that this is a good method for estimating nuclear pulse parameters. Nuclear pulses S-K digital shaping Deep learning LSTM 1. Introduction 1 The parameter estimation problem associated with nuclear pulse signals is an important aspect of radioactivity measurement. Goulding [ 1 1 ] , Gerardi [ 2 2 ] , Noulis [ 3 3 ] et al. pointed out that Gaussian pulse signals have good performance in improving the signal–noise ratio and in energy resolution. Therefore, the Gaussian pulse signal is usually used as the signal after the nuclear pulse signal is shaped, however, the real Gaussian pulse form has an anti-causal part, making it difficult to implement analog systems [ 4 4 ] . With the development of digital signal processing methods and technologies, nuclear pulse digital formation technology has become widely used. Chen Shi-Guo [ 5 5 ] et al. proposed the recursive implementation of Gaussian pulse-shaping, based on wavelet analysis, while Kaiming Jiang [ 6 6 ] et al. proposed a pulse parameter extraction method, based on Sally–Key (S–K) digital shaping and a population technique. Hongquan Huang [ 7 7 ] et al. used a genetic algorithm to estimate S–K, Gaussian-shaped, overlapping pulse signal parameters. Pulse signals have complex and varied characteristics, however, which means very large amounts of data are required for pulse parameter extraction. In addition, due to the simplicity of its mathematical model, the extraction efficiency and accuracy of the traditional search algorithm become rapidly reduced once the number of overlapping pulses increases, or the level of overlap deepens. In recent years, deep learning technology has been continuously developing [ 8 8 ] . It has hidden layers that contain many nonlinear transformation structures, so its ability in fitting complex models by training more data is enhanced [ 9 9 - 12 12 9-12 ] . As part of the continuing development of this field, the recurrent neural network (RNN) method has been proven effective in processing time series problems [ 13 13 - 15 15 13-15 ] . Unfortunately, RNN may experience gradient disappearance, or gradient explosion, during network model training, however, this can be solved by a Long Short-Term Memory (LSTM) neural network replacing each hidden layer in the RNN with a memory cell composed of three control unit gates. According to the characteristics of deep learning, a multi-layer LSTM model, composed of multiple LSTM models, can map the abstract features of data to the network layer in higher dimensions, giving it more powerful learning and expressive abilities, for nonlinear sequences [ 16 16 - 18 18 16-18 ] . At present, research related to introducing deep learning technology into nuclear pulse parameter extraction is still at the preliminary stage, making it urgent to introduce the new, deep learning technology into the field. In this paper, continuous pulse signals have been discretized, and then classical S-K Gaussian shaping has been used to form discrete exponential pulses, in a dataset with the characteristics of a time series. Then, based on the characteristics of Gaussian overlapping pulses after shaping, an efficient and stable LSTM model for parameter extraction of Gaussian overlapping nuclear pulses (ONP) was explored. Our experimental results showed that the proposed method could effectively overcome the difficulty of extracting parameters from noise-containing overlapping pulses. The extracted parameters have shown high precision and demonstrate the good performance of the proposed method for estimating pulse parameters. 2. Principles and Algorithms 1 2.1 A shaping model for nuclear pulses 2 The transmission process for nuclear signals in the detection channel is characterized by pulses that can exhibit exponential, double exponential, triangular, step, trapezoidal or Gaussian form. The following is an example of extracting overlapping pulse parameters after S-K Gaussian shaping. The input S-K signals were multiple exponential pulses. 2.1.1 Superposition model of multiple exponential pulses 3 For overlapping pulses formed by the superposition of N exponentially decaying nuclear pulses, the mathematical model is as follows: <math display="block" id="M1"><mrow><msub><mi>V</mi><mtext>e</mtext></msub><mo stretchy="false">(</mo><mi>t</mi><mo stretchy="false">)</mo><mo>=</mo><mstyle displaystyle="true"><munderover><mo>∑</mo><mrow><mi>i</mi><mo>=</mo><mn>1</mn></mrow><mi>N</mi></munderover><mrow><mo stretchy="false">[</mo><mi>u</mi><mo stretchy="false">(</mo><mi>t</mi><mo>−</mo><msub><mi>T</mi><mi>i</mi></msub><mo stretchy="false">)</mo><msub><mi>A</mi><mi>i</mi></msub><msup><mi>e</mi><mrow><mfrac><mrow><mo>−</mo><mo stretchy="false">(</mo><mi>t</mi><mo>−</mo><msub><mi>T</mi><mi>i</mi></msub><mo stretchy="false">)</mo></mrow><mi>τ</mi></mfrac></mrow></msup><mo stretchy="false">]</mo></mrow></mstyle><mo>+</mo><mi>v</mi><mo stretchy="false">(</mo><mi>t</mi><mo stretchy="false">)</mo><mo>.</mo></mrow></math> In Eq. (1), u ( t ) represents the step signal, <math id="M2"><mrow><mtext> </mtext><msub><mi>A</mi><mi>i</mi></msub></mrow></math> is the amplitude coefficient of the i th nuclear pulse, <math id="M3"><mrow><msub><mi>T</mi><mi>i</mi></msub></mrow></math> represents the occurrence time of the i th nuclear pulse, <math id="M4"><mi>τ</mi></math> is the time constant, and <math id="M5"><mrow><msub><mi>v</mi><mi>t</mi></msub></mrow></math> represents noise. Discretization is performed with the sampling period <math id="M6"><mrow><msub><mi>T</mi><mtext>s</mtext></msub></mrow></math> , and the discretized exponential pulse is described as shown in Eq. (2): <math display="block" id="M7"><mrow><msub><mi>V</mi><mtext>e</mtext></msub><mrow><mo>(</mo><mrow><mi>m</mi><msub><mi>T</mi><mtext>s</mtext></msub></mrow><mo>)</mo></mrow><mo>=</mo><mstyle displaystyle="true"><munderover><mo>∑</mo><mrow><mi>i</mi><mo>=</mo><mn>1</mn></mrow><mi>N</mi></munderover><mrow><mo stretchy="false">[</mo><mi>u</mi><mrow><mo>(</mo><mrow><mi>m</mi><msub><mi>T</mi><mtext>s</mtext></msub><mo>−</mo><msub><mi>T</mi><mi>i</mi></msub></mrow><mo>)</mo></mrow><msub><mi>A</mi><mi>i</mi></msub><msup><mi>e</mi><mrow><mfrac><mrow><mo>−</mo><mo stretchy="false">(</mo><mi>m</mi><msub><mi>T</mi><mtext>s</mtext></msub><mo>−</mo><msub><mi>T</mi><mi>i</mi></msub><mo stretchy="false">)</mo></mrow><mi>τ</mi></mfrac></mrow></msup><mo stretchy="false">]</mo></mrow></mstyle><mo>+</mo><mi>v</mi><mrow><mo>(</mo><mrow><mi>m</mi><msub><mi>T</mi><mtext>s</mtext></msub></mrow><mo>)</mo></mrow><mo>.</mo></mrow></math> 2.1.2 S-K Gaussian shaping 3 S–K is a common Gaussian shaping circuit. Assuming that the resistance of the circuit is R, and the capacitance is C . The parameter is <math id="M8"><mrow><mi>K</mi><mo>=</mo><mi>R</mi><mi>C</mi><mo>/</mo><msub><mi>T</mi><mtext>s</mtext></msub></mrow></math> , and the digital Gaussian shaping method for exponentially overlapping nuclear pulses can be described as shown in Eq. (3): <math display="block" id="M9"><mrow><msub><mi>V</mi><mtext>o</mtext></msub><mrow><mo>(</mo><mrow><mi>m</mi><msub><mi>T</mi><mtext>s</mtext></msub></mrow><mo>)</mo></mrow><mo>=</mo><mfrac><mrow><mrow><mo>[</mo><mrow><mrow><mo>(</mo><mrow><mi>K</mi><mo>+</mo><mn>2</mn><msup><mi>K</mi><mn>2</mn></msup></mrow><mo>)</mo></mrow><msub><mi>V</mi><mtext>o</mtext></msub><mrow><mo>(</mo><mrow><mrow><mo>(</mo><mrow><mi>m</mi><mo>−</mo><mn>1</mn></mrow><mo>)</mo></mrow><msub><mi>T</mi><mtext>s</mtext></msub></mrow><mo>)</mo></mrow><mo>−</mo><msup><mi>K</mi><mn>2</mn></msup><msub><mi>V</mi><mtext>o</mtext></msub><mrow><mo>(</mo><mrow><mrow><mo>(</mo><mrow><mi>m</mi><mo>−</mo><mn>2</mn></mrow><mo>)</mo></mrow><msub><mi>T</mi><mtext>s</mtext></msub></mrow><mo>)</mo></mrow><mo>+</mo><mn>2</mn><msub><mi>V</mi><mtext>e</mtext></msub><mrow><mo>(</mo><mrow><mi>m</mi><msub><mi>T</mi><mtext>s</mtext></msub></mrow><mo>)</mo></mrow></mrow><mo>]</mo></mrow></mrow><mrow><mn>1</mn><mo>+</mo><mi>K</mi><mo>+</mo><msup><mi>K</mi><mn>2</mn></msup></mrow></mfrac><mo>.</mo></mrow></math> From the above, Eq. (4) can be obtained: <math display="block" id="M10"><mrow><msub><mi>V</mi><mtext>o</mtext></msub><mrow><mo>(</mo><mrow><mi>m</mi><msub><mi>T</mi><mtext>s</mtext></msub></mrow><mo>)</mo></mrow><mtext>=</mtext><mfrac><mrow><mrow><mo>{</mo><mrow><mrow><mo>(</mo><mrow><mi>K</mi><mo>+</mo><mn>2</mn><msup><mi>K</mi><mn>2</mn></msup></mrow><mo>)</mo></mrow></mrow></mrow><msub><mi>V</mi><mtext>o</mtext></msub><mrow><mo>(</mo><mrow><mrow><mo>(</mo><mrow><mi>m</mi><mo>−</mo><mn>1</mn></mrow><mo>)</mo></mrow><msub><mi>T</mi><mtext>s</mtext></msub></mrow><mo>)</mo></mrow><mo>−</mo><msup><mi>K</mi><mn>2</mn></msup><msub><mi>V</mi><mtext>o</mtext></msub><mrow><mo>(</mo><mrow><mrow><mo>(</mo><mrow><mi>m</mi><mo>−</mo><mn>2</mn></mrow><mo>)</mo></mrow><msub><mi>T</mi><mtext>s</mtext></msub></mrow><mo>)</mo></mrow><mo>+</mo><mn>2</mn><mstyle displaystyle="true"><munderover><mo>∑</mo><mrow><mi>i</mi><mo>=</mo><mn>1</mn></mrow><mi>N</mi></munderover><mrow><mrow><mo>[</mo><mrow><mi>u</mi><mrow><mo>(</mo><mrow><mi>m</mi><msub><mi>T</mi><mtext>s</mtext></msub><mo>−</mo><msub><mi>T</mi><mi>i</mi></msub></mrow><mo>)</mo></mrow><msub><mi>A</mi><mi>i</mi></msub><msup><mi>e</mi><mrow><mfrac><mrow><mo>−</mo><mrow><mo>(</mo><mrow><mi>m</mi><msub><mi>T</mi><mtext>s</mtext></msub><mo>−</mo><msub><mi>T</mi><mi>i</mi></msub></mrow><mo>)</mo></mrow></mrow><mi>τ</mi></mfrac></mrow></msup></mrow><mo>]</mo></mrow></mrow></mstyle><mo>+</mo><mrow><mrow><mn>2</mn><mi>v</mi><mrow><mo>(</mo><mrow><mi>m</mi><msub><mi>T</mi><mtext>s</mtext></msub></mrow><mo>)</mo></mrow></mrow><mo>}</mo></mrow></mrow><mrow><mn>1</mn><mo>+</mo><mi>K</mi><mo>+</mo><msup><mi>K</mi><mn>2</mn></msup></mrow></mfrac></mrow></math> In this formula, <math id="M11"><mrow><msub><mi>V</mi><mtext>o</mtext></msub><mrow><mo>(</mo><mrow><mi>m</mi><msub><mi>T</mi><mtext>s</mtext></msub></mrow><mo>)</mo></mrow></mrow></math> is the signal of the overlapping pulses, after S-K digital Gaussian shaping, and <math id="M12"><mrow><mi>v</mi><mrow><mo>(</mo><mrow><mi>m</mi><msub><mi>T</mi><mtext>s</mtext></msub></mrow><mo>)</mo></mrow></mrow></math> is noise. Parameter m is the number of sampling values, the total number of which is M , and m = 1, 2, 3,..., M. 2.2 Theories and techniques relevant to the LSTM model 2 The LSTM model includes forward propagation, back propagation through time (BPTT), and the Adam parameter optimization algorithm [ 19 19 , 20 20 ] . For a given sequence, a standard LSTM model is applied, and hidden layer and output sequences can be iterated through structures such as a forget gate, input gate, candidate information gate, and output gate. The mathematical models for these structures are shown in Eqs. (5)–(10): <math display="block" id="M13"><mrow><msub><mi>f</mi><mi>t</mi></msub><mo>=</mo><mi>σ</mi><mrow><mo>(</mo><mrow><msub><mi>W</mi><mtext>f</mtext></msub><mo>⋅</mo><mrow><mo>[</mo><mrow><msub><mi>h</mi><mrow><mi>t</mi><mo>−</mo><mn>1</mn></mrow></msub><mo>,</mo><msub><mi>x</mi><mi>t</mi></msub></mrow><mo>]</mo></mrow><mo>+</mo><msub><mi>b</mi><mtext>f</mtext></msub></mrow><mo>)</mo></mrow><mo>,</mo></mrow></math> <math display="block" id="M14"><mrow><msub><mi>g</mi><mi>t</mi></msub><mo>=</mo><mi>σ</mi><mo stretchy="false">(</mo><msub><mi>W</mi><mtext>i</mtext></msub><mo>⋅</mo><mrow><mo>[</mo><mrow><msub><mi>h</mi><mrow><mi>t</mi><mo>−</mo><mn>1</mn></mrow></msub><mo>,</mo><msub><mi>x</mi><mi>t</mi></msub></mrow><mo>]</mo></mrow><mo>+</mo><msub><mi>b</mi><mtext>i</mtext></msub><mo stretchy="false">)</mo><mo>,</mo></mrow></math> <math display="block" id="M15"><mrow><msub><mrow><mover><mi>C</mi><mo>~</mo></mover></mrow><mi>t</mi></msub><mo>=</mo><mi>tanh</mi><mrow><mo>(</mo><mrow><msub><mi>W</mi><mtext>C</mtext></msub><mo>⋅</mo><mo stretchy="false">[</mo><msub><mi>h</mi><mrow><mi>t</mi><mo>−</mo><mn>1</mn></mrow></msub><mo>,</mo><msub><mi>x</mi><mi>t</mi></msub><mo stretchy="false">]</mo><mo>+</mo><msub><mi>b</mi><mtext>C</mtext></msub></mrow><mo>)</mo></mrow><mo>,</mo></mrow></math> <math display="block" id="M16"><mrow><msub><mi>C</mi><mi>t</mi></msub><mo>=</mo><msub><mi>f</mi><mi>t</mi></msub><mo>⋅</mo><msub><mi>C</mi><mrow><mi>t</mi><mo>−</mo><mn>1</mn></mrow></msub><mo>+</mo><msub><mi>g</mi><mi>t</mi></msub><mo>⋅</mo><msub><mrow><mover><mi>C</mi><mo>~</mo></mover></mrow><mi>t</mi></msub><mo>,</mo></mrow></math> <math display="block" id="M17"><mrow><msub><mi>o</mi><mi>t</mi></msub><mo>=</mo><mi>σ</mi><mrow><mo>(</mo><mrow><msub><mi>W</mi><mtext>o</mtext></msub><mrow><mo>[</mo><mrow><msub><mi>h</mi><mrow><mi>t</mi><mo>−</mo><mn>1</mn></mrow></msub><mo>,</mo><msub><mi>x</mi><mi>t</mi></msub></mrow><mo>]</mo></mrow><mo>+</mo><msub><mi>b</mi><mtext>o</mtext></msub></mrow><mo>)</mo></mrow><mo>,</mo></mrow></math> <math display="block" id="M18"><mrow><msub><mi>h</mi><mi>t</mi></msub><mo>=</mo><msub><mi>o</mi><mi>t</mi></msub><mi>tanh</mi><mrow><mo>(</mo><mrow><msub><mi>C</mi><mi>t</mi></msub></mrow><mo>)</mo></mrow><mo>,</mo></mrow></math> where <math id="M19"><mrow><msub><mi>g</mi><mi>t</mi></msub></mrow></math> , <math id="M20"><mrow><msub><mi>f</mi><mi>t</mi></msub></mrow></math> , <math id="M21"><mrow><mtext> </mtext><msub><mi>C</mi><mi>t</mi></msub></mrow></math> , and <math id="M22"><mrow><msub><mi>o</mi><mi>t</mi></msub></mrow></math> represent the input gate, forget gate, cell state and output gate at the current time, respectively; <math id="M23"><mrow><msub><mi>h</mi><mi>t</mi></msub></mrow></math> is hidden state information, and <math id="M24"><mrow><msub><mover accent="true"><mi>C</mi><mo>˜</mo></mover><mi>t</mi></msub></mrow></math> represents the candidate information vector. <math id="M25"><mrow><msub><mi>C</mi><mrow><mi>t</mi><mo>−</mo><mn>1</mn></mrow></msub></mrow></math> represents the cell state value for the previous moment, and <math id="M26"><mrow><msub><mi>h</mi><mrow><mi>t</mi><mo>−</mo><mn>1</mn></mrow></msub></mrow></math> is the hidden state information of the previous moment. W and b represent the weight and bias of different gate functions, respectively. The LSTM training process uses the BPTT algorithm, which consists of four main steps. First, the output value of each LSTM memory cell is calculated by forward propagation; secondly, the error term of the memory cell is calculated backwards; then, the gradient of each weight is calculated according to the error term, before, finally, the gradient-based optimization algorithm is applied to update the weight. Common gradient optimization algorithms include the stochastic gradient descent (SGD) [ 21 21 ] , adaptive gradient (AdaGrad) [ 22 22 ] , root mean square propagation (RMSProp) [ 23 23 ] , and adaptive momentum estimation (Adam) [ 24 24 ] algorithms. 2.3 Parameter estimation for overlapping pulses 2 Overlapping pulse parameter estimation, after Gaussian shaping, mainly involves production of the data set, forward and back propagation training, based on the BPTT algorithm, and preservation of the model after training completion. 2.3.1 Data set production 3 Make a data set with n samples. The matrix representation of the data set is as follows: <math display="block" id="M27"><mrow><mrow><mo>[</mo><mrow><mtable><mtr><mtd><mrow><msub><mrow><mrow><mo>[</mo><mrow><msub><mi>V</mi><mtext>o</mtext></msub><mrow><mo>(</mo><mrow><msub><mi>T</mi><mtext>s</mtext></msub></mrow><mo>)</mo></mrow></mrow><mo>]</mo></mrow></mrow><mn>1</mn></msub></mrow></mtd><mtd><mrow><msub><mrow><mrow><mo>[</mo><mrow><msub><mi>V</mi><mtext>o</mtext></msub><mrow><mo>(</mo><mrow><mn>2</mn><msub><mi>T</mi><mtext>s</mtext></msub></mrow><mo>)</mo></mrow></mrow><mo>]</mo></mrow></mrow><mn>1</mn></msub></mrow></mtd><mtd><mo>⋯</mo></mtd><mtd><mrow><msub><mrow><mrow><mo>[</mo><mrow><msub><mi>V</mi><mtext>o</mtext></msub><mo stretchy="false">(</mo><mi>M</mi><msub><mi>T</mi><mtext>s</mtext></msub><mo stretchy="false">)</mo></mrow><mo>]</mo></mrow></mrow><mn>1</mn></msub></mrow></mtd><mtd><mrow><msub><mi>θ</mi><mn>1</mn></msub></mrow></mtd></mtr><mtr><mtd><mrow><msub><mrow><mrow><mo>[</mo><mrow><msub><mi>V</mi><mtext>o</mtext></msub><mrow><mo>(</mo><mrow><msub><mi>T</mi><mtext>s</mtext></msub></mrow><mo>)</mo></mrow></mrow><mo>]</mo></mrow></mrow><mn>2</mn></msub></mrow></mtd><mtd><mrow><msub><mrow><mrow><mo>[</mo><mrow><msub><mi>V</mi><mtext>o</mtext></msub><mrow><mo>(</mo><mrow><mn>2</mn><msub><mi>T</mi><mtext>s</mtext></msub></mrow><mo>)</mo></mrow></mrow><mo>]</mo></mrow></mrow><mn>2</mn></msub></mrow></mtd><mtd><mo>⋯</mo></mtd><mtd><mrow><msub><mrow><mrow><mo>[</mo><mrow><msub><mi>V</mi><mtext>o</mtext></msub><mo stretchy="false">(</mo><mi>M</mi><msub><mi>T</mi><mtext>s</mtext></msub><mo stretchy="false">)</mo></mrow><mo>]</mo></mrow></mrow><mn>2</mn></msub></mrow></mtd><mtd><mrow><msub><mi>θ</mi><mn>2</mn></msub></mrow></mtd></mtr><mtr><mtd><mo>⋮</mo></mtd><mtd><mo>⋮</mo></mtd><mtd><mo>⋱</mo></mtd><mtd><mo>⋮</mo></mtd><mtd><mo>⋮</mo></mtd></mtr><mtr><mtd><mrow><msub><mrow><mrow><mo>[</mo><mrow><msub><mi>V</mi><mtext>o</mtext></msub><mrow><mo>(</mo><mrow><msub><mi>T</mi><mtext>s</mtext></msub></mrow><mo>)</mo></mrow></mrow><mo>]</mo></mrow></mrow><mi>n</mi></msub></mrow></mtd><mtd><mrow><msub><mrow><mrow><mo>[</mo><mrow><msub><mi>V</mi><mtext>o</mtext></msub><mrow><mo>(</mo><mrow><mn>2</mn><msub><mi>T</mi><mtext>s</mtext></msub></mrow><mo>)</mo></mrow></mrow><mo>]</mo></mrow></mrow><mi>n</mi></msub></mrow></mtd><mtd><mo>⋯</mo></mtd><mtd><mrow><msub><mrow><mrow><mo>[</mo><mrow><msub><mi>V</mi><mtext>o</mtext></msub><mo stretchy="false">(</mo><mi>M</mi><msub><mi>T</mi><mtext>s</mtext></msub><mo stretchy="false">)</mo></mrow><mo>]</mo></mrow></mrow><mi>n</mi></msub></mrow></mtd><mtd><mrow><msub><mi>θ</mi><mi>n</mi></msub></mrow></mtd></mtr></mtable></mrow><mo>]</mo></mrow></mrow></math> Each row in Eq. (11) represented data from one sample, with the first M data from each sample being the sampled values of V o ( mT s ). V o ( mT s ) is the signal from the overlapping pulses after S–K digital Gaussian shaping, corresponding to this sample—and so V o ( mT s ) is also known as the Gaussian overlapping pulses signal. The parameters of the input signal V e ( mT s ) were set to <math id="M28"><mrow><msub><mi>A</mi><mi>i</mi></msub><mo stretchy="false">(</mo><mi>i</mi><mo>=</mo><mn>1</mn><mo>,</mo><mn>2</mn><mo>,</mo><mo>…</mo><mo>,</mo><mi>N</mi><mo stretchy="false">)</mo></mrow></math> , <math id="M29"><mrow><msub><mi>T</mi><mi>i</mi></msub><mo stretchy="false">(</mo><mi>i</mi><mo>=</mo><mn>1</mn><mo>,</mo><mn>2</mn><mo>,</mo><mo>…</mo><mo>,</mo><mi>N</mi><mo stretchy="false">)</mo></mrow></math> , K , and τ . These parameters were constituted as a set of parameters for the sample θ i , that is <math id="M30"><mrow><msub><mi>θ</mi><mi>i</mi></msub><mtext>=</mtext><mo stretchy="false">[</mo><msub><mi>A</mi><mn>1</mn></msub><mo>,</mo><msub><mi>A</mi><mn>2</mn></msub><mo>,</mo><mn>...</mn><mo>,</mo><msub><mi>A</mi><mi>N</mi></msub><mo>,</mo><msub><mi>T</mi><mn>1</mn></msub><mo>,</mo><msub><mi>T</mi><mn>2</mn></msub><mo>,</mo><mn>...</mn><mo>,</mo><msub><mi>T</mi><mi>N</mi></msub><mo>,</mo><mi>τ</mi><mo>,</mo><mi>K</mi><mo stretchy="false">]</mo></mrow></math> . For example, the set of parameters corresponding to the i th sample was described as θ i , and the sampling values of its Gaussian ONP were [ V o ( T s )] i , [ V o (2 T s )] i , [ V o (3 T s )] i , …, and [ V o ( MT s )] i . Next, the data set was divided into the Training Set , the Test Set , and the Validation Set , according to a certain ratio. The Training Set was used for training the LSTM model, while the Test Set was used to verify the generalization ability of the model, after completion of its model training. The Validation Set was then used to verify whether the model for training completion suffered from the over-fitting phenomenon. The reason for this was that in an over-fitting situation, if the model loss value for the training data was small, the prediction accuracy was higher, but when the loss function was larger, the prediction accuracy on the validation data became low, which would cause the trained model to lose the ability to generalize. Traditional machine learning models have often used the L1 and L2 regularization method to modify the loss function. However, for large deep neural networks, modification of the loss function alone cannot meet the actual needs, so in order to solve this difficulty, the Dropout algorithm was used in our study [ 25 25 - 27 27 25-27 ] . If the Dropout algorithm is used in the forward propagation process, a certain number of memory cells in a complex neural network will stop processing sequence information with a certain probability. Thus, the training process is carried out on the LSTM architecture of different combinations. This method can reduce the dependence of the neural network on certain local features, enhance the generalization ability of the LSTM model, and finally achieve the purpose of improving the performance of the neural network. The mathematical model for Dropout algorithm is as shown in Eqs 9120 and (13): <math display="block" id="M31"><mrow><msubsup><mi>r</mi><mi>j</mi><mrow><mrow><mo>(</mo><mi>l</mi><mo>)</mo></mrow></mrow></msubsup><mo>∼</mo><mi>B</mi><mi>e</mi><mi>r</mi><mi>n</mi><mi>o</mi><mi>u</mi><mi>l</mi><mi>l</mi><mi>i</mi><mrow><mo>(</mo><mi>p</mi><mo>)</mo></mrow></mrow></math> <math display="block" id="M32"><mrow><msup><mrow><mover><mi>y</mi><mo>~</mo></mover></mrow><mrow><mrow><mo>(</mo><mrow><mi>l</mi><mo>+</mo><mn>1</mn></mrow><mo>)</mo></mrow></mrow></msup><mo>=</mo><msup><mi>r</mi><mrow><mrow><mo>(</mo><mi>l</mi><mo>)</mo></mrow></mrow></msup><msup><mi>y</mi><mrow><mrow><mo>(</mo><mi>l</mi><mo>)</mo></mrow></mrow></msup></mrow></math> where the p is the probability that the LSTM memory cells stop propagation. The function <math id="M33"><mrow><msubsup><mi>r</mi><mi>j</mi><mrow><mrow><mo>(</mo><mi>l</mi><mo>)</mo></mrow></mrow></msubsup></mrow></math> is the retention probability of state information for the j th LSTM memory cell of the l th layer neural network, and obeys the Bernoulli distribution. Parameter <math id="M34"><mrow><msup><mi>y</mi><mrow><mrow><mo>(</mo><mi>l</mi><mo>)</mo></mrow></mrow></msup></mrow></math> is the output information for the l th layer neural network, and <math id="M35"><mrow><msup><mrow><mover><mi>y</mi><mo>~</mo></mover></mrow><mrow><mrow><mo>(</mo><mrow><mi>l</mi><mo>+</mo><mn>1</mn></mrow><mo>)</mo></mrow></mrow></msup></mrow></math> is the input information for the l +1 th layer neural network. 2.3.2 Forward propagation calculation of the pulses sampling value sequence 3 The forward propagation calculation process for the pulses sampling value sequence involves using the pulses sampling value sequence [ V o ( mT s )] i in the training set as the input data, and, after iteration through the multi-layer LSTM model, the process finally delivers the nuclear pulses parameter set <math id="M36"><mrow><msubsup><mi>θ</mi><mi>i</mi><mo>'</mo></msubsup></mrow></math> , which was estimated by the current neural network. In order to ensure that the sequence information from each time unit can be fully utilized by the network, the number of first-layer memory cells of the LSTM model was made equal to the number M of the nuclear pulse sampling values, [ V o ( mT s )] i . Next, the pulses signal [ V o ( mT s )] i in the training set was substituted into Eqs. (10)–(17) to obtain hidden state information, <math id="M37"><mrow><msub><mi>h</mi><mi>m</mi></msub></mrow></math> , and cell state information, <math id="M38"><mrow><msub><mi>C</mi><mi>m</mi></msub></mrow></math> . Therefore, the mathematical models for the forget gate, the input gate, the output gate and the memory cell included in the forward propagation process of the single-layer LSTM nuclear pulses parameter extraction were as shown in the following sub-sections. Calculation of the forget gate structure The forget gate structure could determine the retention probability of memory cell state information, and was calculated as shown in Eq. (14): <math display="block" id="M39"><mtable><mtr><mtd><msub><mi>f</mi><mi>m</mi></msub><mo>=</mo><mi>σ</mi><mrow><mo>(</mo><mrow><mstyle displaystyle="true"><munderover><mo>∑</mo><mrow><mi>m</mi><mo>=</mo><mn>1</mn></mrow><mrow><mi>K</mi><mo>+</mo><mn>2</mn><mo>+</mo><msub><mi>n</mi><mtext>c</mtext></msub></mrow></munderover><mrow><msubsup><mi>U</mi><mrow><mi>i</mi><mo>,</mo><mi>m</mi></mrow><mtext>f</mtext></msubsup><msub><mrow><mrow><mo>[</mo><mrow><msub><mi>V</mi><mtext>o</mtext></msub><mrow><mo>(</mo><mrow><mi>m</mi><msub><mi>T</mi><mtext>s</mtext></msub></mrow><mo>)</mo></mrow></mrow><mo>]</mo></mrow></mrow><mi>i</mi></msub></mrow></mstyle><mo>+</mo><mstyle displaystyle="true"><munderover><mo>∑</mo><mrow><mi>m</mi><mo>=</mo><mn>2</mn></mrow><mrow><mi>K</mi><mo>+</mo><mn>2</mn><mo>+</mo><msub><mi>n</mi><mtext>c</mtext></msub></mrow></munderover><mrow><msubsup><mi>W</mi><mrow><mi>i</mi><mo>,</mo><mi>m</mi></mrow><mtext>f</mtext></msubsup><msub><mi>h</mi><mrow><mi>m</mi><mo>−</mo><mn>1</mn></mrow></msub></mrow></mstyle><mo>+</mo><msubsup><mi>b</mi><mi>i</mi><mtext>f</mtext></msubsup></mrow><mo>)</mo></mrow></mtd></mtr><mtr><mtd><mo>=</mo><mfrac><mn>1</mn><mrow><mn>1</mn><mo>+</mo><msup><mi>e</mi><mrow><mo>−</mo><mrow><mo>(</mo><mrow><mstyle displaystyle="true"><munderover><mo>∑</mo><mrow><mi>m</mi><mo>=</mo><mn>1</mn></mrow><mrow><mi>K</mi><mo>+</mo><mn>2</mn><mo>+</mo><msub><mi>n</mi><mtext>c</mtext></msub></mrow></munderover><mrow><msubsup><mi>U</mi><mrow><mi>i</mi><mo>,</mo><mi>m</mi></mrow><mtext>f</mtext></msubsup><msub><mrow><mrow><mo>[</mo><mrow><msub><mi>V</mi><mtext>o</mtext></msub><mrow><mo>(</mo><mrow><mi>m</mi><msub><mi>T</mi><mtext>s</mtext></msub></mrow><mo>)</mo></mrow></mrow><mo>]</mo></mrow></mrow><mi>i</mi></msub></mrow></mstyle><mo>+</mo><mstyle displaystyle="true"><munderover><mo>∑</mo><mrow><mi>m</mi><mo>=</mo><mn>2</mn></mrow><mrow><mi>K</mi><mo>+</mo><mn>2</mn><mo>+</mo><msub><mi>n</mi><mtext>c</mtext></msub></mrow></munderover><mrow><msubsup><mi>W</mi><mrow><mi>i</mi><mo>,</mo><mi>m</mi></mrow><mtext>f</mtext></msubsup><msub><mi>h</mi><mrow><mi>m</mi><mo>−</mo><mn>1</mn></mrow></msub></mrow></mstyle><mo>+</mo><msubsup><mi>b</mi><mi>i</mi><mtext>f</mtext></msubsup></mrow><mo>)</mo></mrow></mrow></msup></mrow></mfrac></mtd></mtr></mtable></math> In (14), hm-1 is the hidden state information of the previous memory cell. The functions <math id="M40"><mrow><msubsup><mi>U</mi><mrow><mi>i</mi><mo>,</mo><mi>m</mi></mrow><mtext>f</mtext></msubsup></mrow></math> and <math id="M41"><mrow><msubsup><mi>W</mi><mrow><mi>i</mi><mo>,</mo><mi>m</mi></mrow><mtext>f</mtext></msubsup></mrow></math> are the input weights and cyclic weights of the m th sampling values [ V o ( mT s )] i in the i th sample in the forgotten gate structure, respectively. The <math id="M42"><mrow><msubsup><mi>b</mi><mi>i</mi><mtext>f</mtext></msubsup></mrow></math> function represents the bias of the i th sample in the forgotten gate structure. The function σ is the gate function, which is composed of a sigmoid function. This function can be used to output a value between 0 and 1, to determine the retention probability of state information. Calculation of the input gate structure The input gate structure is used to calculate new state information inside the memory cell, and its structure is similar to that of the forget gate. The weight and offset parameters are U g , W g , and b g , respectively, and its mathematical model is described by Eq. (15). <math display="block" id="M43"><mtable><mtr><mtd><msub><mi>g</mi><mi>m</mi></msub><mo>=</mo><mi>σ</mi><mrow><mo>(</mo><mrow><mstyle displaystyle="true"><munderover><mo>∑</mo><mrow><mi>m</mi><mo>=</mo><mn>1</mn></mrow><mrow><mi>K</mi><mo>+</mo><mn>2</mn><mo>+</mo><msub><mi>n</mi><mtext>c</mtext></msub></mrow></munderover><mrow><msubsup><mi>U</mi><mrow><mi>i</mi><mo>,</mo><mi>m</mi></mrow><mtext>g</mtext></msubsup><msub><mrow><mrow><mo>[</mo><mrow><msub><mi>V</mi><mtext>o</mtext></msub><mrow><mo>(</mo><mrow><mi>m</mi><msub><mi>T</mi><mtext>s</mtext></msub></mrow><mo>)</mo></mrow></mrow><mo>]</mo></mrow></mrow><mi>i</mi></msub></mrow></mstyle><mo>+</mo><mstyle displaystyle="true"><munderover><mo>∑</mo><mrow><mi>m</mi><mo>=</mo><mn>2</mn></mrow><mrow><mi>K</mi><mo>+</mo><mn>2</mn><mo>+</mo><msub><mi>n</mi><mtext>c</mtext></msub></mrow></munderover><mrow><msubsup><mi>W</mi><mrow><mi>i</mi><mo>,</mo><mi>m</mi><mo>−</mo><mn>1</mn></mrow><mtext>g</mtext></msubsup><msub><mi>h</mi><mrow><mi>m</mi><mo>−</mo><mn>1</mn></mrow></msub></mrow></mstyle><mo>+</mo><msubsup><mi>b</mi><mi>i</mi><mtext>g</mtext></msubsup></mrow><mo>)</mo></mrow></mtd></mtr><mtr><mtd><mo>=</mo><mfrac><mn>1</mn><mrow><mn>1</mn><mo>+</mo><msup><mi>e</mi><mrow><mo>−</mo><mrow><mo>(</mo><mrow><mstyle displaystyle="true"><munderover><mo>∑</mo><mrow><mi>m</mi><mo>=</mo><mn>1</mn></mrow><mrow><mi>K</mi><mo>+</mo><mn>2</mn><mo>+</mo><msub><mi>n</mi><mtext>c</mtext></msub></mrow></munderover><mrow><msubsup><mi>U</mi><mrow><mi>i</mi><mo>,</mo><mi>m</mi></mrow><mtext>g</mtext></msubsup><msub><mrow><mrow><mo>[</mo><mrow><msub><mi>V</mi><mtext>o</mtext></msub><mrow><mo>(</mo><mrow><mi>m</mi><msub><mi>T</mi><mtext>s</mtext></msub></mrow><mo>)</mo></mrow></mrow><mo>]</mo></mrow></mrow><mi>i</mi></msub></mrow></mstyle><mo>+</mo><mstyle displaystyle="true"><munderover><mo>∑</mo><mrow><mi>m</mi><mo>=</mo><mn>2</mn></mrow><mrow><mi>K</mi><mo>+</mo><mn>2</mn><mo>+</mo><msub><mi>n</mi><mtext>c</mtext></msub></mrow></munderover><mrow><msubsup><mi>W</mi><mrow><mi>i</mi><mo>,</mo><mi>m</mi><mo>−</mo><mn>1</mn></mrow><mtext>g</mtext></msubsup><msub><mi>h</mi><mrow><mi>m</mi><mo>−</mo><mn>1</mn></mrow></msub></mrow></mstyle><mo>+</mo><msubsup><mi>b</mi><mi>i</mi><mtext>g</mtext></msubsup></mrow><mo>)</mo></mrow></mrow></msup></mrow></mfrac></mtd></mtr></mtable></math> In (15), <math id="M44"><mrow><msubsup><mi>U</mi><mrow><mi>i</mi><mo>,</mo><mi>m</mi></mrow><mtext>g</mtext></msubsup></mrow></math> and <math id="M45"><mrow><msubsup><mi>W</mi><mrow><mi>i</mi><mo>,</mo><mi>m</mi></mrow><mtext>g</mtext></msubsup></mrow></math> are the input weights and cyclic weights, respectively, of the m th sampling values [ V o ( mT s )] i in the i th sample in the input gate structure. The function <math id="M46"><mrow><msubsup><mi>b</mi><mi>i</mi><mtext>f</mtext></msubsup></mrow></math> is the bias of the i th sample in the input gate structure. Status update of the memory cell First, the candidate information vector <math id="M47"><mrow><msub><mrow><mover><mi>C</mi><mo>~</mo></mover></mrow><mi>m</mi></msub></mrow></math> was created, using the tanh function. Then, the forget gate information, the previous memory cell state information, the input gate information, and the candidate information vector were used as update elements for the current memory cell state information. Therefore, the mathematical model of the status update information was as shown in Eq. (16). <math display="block" id="M48"><mtable><mtr><mtd><msub><mover><mi>C</mi><mo>~</mo></mover><mi>m</mi></msub><mtext>=</mtext><mi>tanh</mi><mrow><mo>(</mo><mrow><mstyle displaystyle="true"><munderover><mo>∑</mo><mrow><mi>m</mi><mo>=</mo><mn>1</mn></mrow><mrow><mi>K</mi><mo>+</mo><mn>2</mn><mo>+</mo><msub><mi>n</mi><mtext>c</mtext></msub></mrow></munderover><mrow><msubsup><mi>U</mi><mrow><mi>i</mi><mo>,</mo><mi>m</mi></mrow><mtext>C</mtext></msubsup><msub><mrow><mrow><mo>[</mo><mrow><msub><mi>V</mi><mtext>o</mtext></msub><mrow><mo>(</mo><mrow><mi>m</mi><msub><mi>T</mi><mtext>s</mtext></msub></mrow><mo>)</mo></mrow></mrow><mo>]</mo></mrow></mrow><mi>i</mi></msub></mrow></mstyle><mo>+</mo><mstyle displaystyle="true"><munderover><mo>∑</mo><mrow><mi>m</mi><mo>=</mo><mn>2</mn></mrow><mrow><mi>K</mi><mo>+</mo><mn>2</mn><mo>+</mo><msub><mi>n</mi><mtext>c</mtext></msub></mrow></munderover><mrow><msubsup><mi>W</mi><mrow><mi>i</mi><mo>,</mo><mi>m</mi><mo>−</mo><mn>1</mn></mrow><mtext>C</mtext></msubsup><msub><mi>h</mi><mrow><mi>m</mi><mo>−</mo><mn>1</mn></mrow></msub></mrow></mstyle><mo>+</mo><msubsup><mi>b</mi><mi>i</mi><mtext>C</mtext></msubsup></mrow><mo>)</mo></mrow></mtd></mtr><mtr><mtd><mo>=</mo><mfrac><mrow><mn>1</mn><mo>−</mo><msup><mi>e</mi><mrow><mo>−</mo><mn>2</mn><mrow><mo>(</mo><mrow><mstyle displaystyle="true"><munderover><mo>∑</mo><mrow><mi>m</mi><mo>=</mo><mn>1</mn></mrow><mrow><mi>K</mi><mo>+</mo><mn>2</mn><mo>+</mo><msub><mi>n</mi><mtext>c</mtext></msub></mrow></munderover><mrow><msubsup><mi>U</mi><mrow><mi>i</mi><mo>,</mo><mi>m</mi></mrow><mtext>C</mtext></msubsup><msub><mrow><mrow><mo>[</mo><mrow><msub><mi>V</mi><mtext>o</mtext></msub><mrow><mo>(</mo><mrow><mi>m</mi><msub><mi>T</mi><mtext>s</mtext></msub></mrow><mo>)</mo></mrow></mrow><mo>]</mo></mrow></mrow><mi>i</mi></msub></mrow></mstyle><mo>+</mo><mstyle displaystyle="true"><munderover><mo>∑</mo><mrow><mi>m</mi><mo>=</mo><mn>2</mn></mrow><mrow><mi>K</mi><mo>+</mo><mn>2</mn><mo>+</mo><msub><mi>n</mi><mtext>c</mtext></msub></mrow></munderover><mrow><msubsup><mi>W</mi><mrow><mi>i</mi><mo>,</mo><mi>m</mi><mo>−</mo><mn>1</mn></mrow><mtext>C</mtext></msubsup><msub><mi>h</mi><mrow><mi>m</mi><mo>−</mo><mn>1</mn></mrow></msub></mrow></mstyle><mo>+</mo><msubsup><mi>b</mi><mi>i</mi><mtext>C</mtext></msubsup></mrow><mo>)</mo></mrow></mrow></msup></mrow><mrow><mn>1</mn><mo>+</mo><msup><mi>e</mi><mrow><mo>−</mo><mn>2</mn><mrow><mo>(</mo><mrow><mstyle displaystyle="true"><munderover><mo>∑</mo><mrow><mi>m</mi><mo>=</mo><mn>1</mn></mrow><mrow><mi>K</mi><mo>+</mo><mn>2</mn><mo>+</mo><msub><mi>n</mi><mtext>c</mtext></msub></mrow></munderover><mrow><msubsup><mi>U</mi><mrow><mi>i</mi><mo>,</mo><mi>m</mi></mrow><mtext>C</mtext></msubsup><msub><mrow><mrow><mo>[</mo><mrow><msub><mi>V</mi><mtext>o</mtext></msub><mrow><mo>(</mo><mrow><mi>m</mi><msub><mi>T</mi><mtext>s</mtext></msub></mrow><mo>)</mo></mrow></mrow><mo>]</mo></mrow></mrow><mi>i</mi></msub></mrow></mstyle><mo>+</mo><mstyle displaystyle="true"><munderover><mo>∑</mo><mrow><mi>m</mi><mo>=</mo><mn>2</mn></mrow><mrow><mi>K</mi><mo>+</mo><mn>2</mn><mo>+</mo><msub><mi>n</mi><mtext>c</mtext></msub></mrow></munderover><mrow><msubsup><mi>W</mi><mrow><mi>i</mi><mo>,</mo><mi>m</mi><mo>−</mo><mn>1</mn></mrow><mtext>C</mtext></msubsup><msub><mi>h</mi><mrow><mi>m</mi><mo>−</mo><mn>1</mn></mrow></msub></mrow></mstyle><mo>+</mo><msubsup><mi>b</mi><mi>i</mi><mtext>C</mtext></msubsup></mrow><mo>)</mo></mrow></mrow></msup></mrow></mfrac></mtd></mtr></mtable></math> <math display="block" id="M49"><mrow><msub><mi>C</mi><mi>m</mi></msub><mo>=</mo><msub><mi>f</mi><mi>m</mi></msub><msub><mi>C</mi><mrow><mi>m</mi><mo>−</mo><mn>1</mn></mrow></msub><mo>+</mo><msub><mi>g</mi><mi>m</mi></msub><msub><mrow><mover><mi>C</mi><mo>~</mo></mover></mrow><mi>m</mi></msub></mrow></math> In these equations, C m represents the status value of the memory cell at the current moment, while f m represents the output value of the forget gate. C m- 1 represents the status value of the memory cell at the previous moment, g m represents the output value of the input gate, and <math id="M50"><mrow><msub><mrow><mover><mi>C</mi><mo>~</mo></mover></mrow><mi>m</mi></msub></mrow></math> represents the candidate vector. <math id="M51"><mrow><msubsup><mi>U</mi><mrow><mi>i</mi><mo>,</mo><mi>m</mi></mrow><mtext>C</mtext></msubsup></mrow></math> and <math id="M52"><mrow><msubsup><mi>W</mi><mrow><mi>i</mi><mo>,</mo><mi>m</mi></mrow><mtext>C</mtext></msubsup></mrow></math> are the input weights and cyclic weights of the m th sampling values [ V o ( mT s )] i in the i th sample in the input gate structure, respectively. Function <math id="M53"><mrow><msubsup><mi>b</mi><mi>i</mi><mtext>C</mtext></msubsup></mrow></math> is the bias of the i th sample in the input gate structure. Calculation of the output gate structure The output gate structure determines the hidden state information, h m . At first, the vector containing the hidden state information h m- 1 , from the previous memory cell, and the vector containing the current pulse sequence information [ V o ( mT s )] I , were calculated, using the sigmoid function. Then the cell state information C m of the memory cell was calculated, using the tanh function. Next, the output value of the tanh function was multiplied with the output value, o m , of the sigmoid function to determine the hidden state information, h m . Finally, it was necessary to transmit the hidden state information, h m , to the next layer of the network, and to transmit the information, h m , of the hidden state and the state information C m of the memory unit to the next memory cell of the same layer. Therefore, the mathematical model of the output gate was as shown in Eq. (18) and (19): <math display="block" id="M54"><mtable><mtr><mtd><msub><mi>o</mi><mi>m</mi></msub><mo>=</mo><mi>σ</mi><mrow><mo>(</mo><mrow><mstyle displaystyle="true"><munderover><mo>∑</mo><mrow><mi>m</mi><mo>=</mo><mn>1</mn></mrow><mrow><mi>K</mi><mo>+</mo><mn>2</mn><mo>+</mo><msub><mi>n</mi><mtext>c</mtext></msub></mrow></munderover><mrow><msubsup><mi>U</mi><mrow><mi>i</mi><mo>,</mo><mi>m</mi></mrow><mtext>o</mtext></msubsup><msub><mrow><mrow><mo>[</mo><mrow><msub><mi>V</mi><mtext>o</mtext></msub><mrow><mo>(</mo><mrow><mi>m</mi><msub><mi>T</mi><mtext>s</mtext></msub></mrow><mo>)</mo></mrow></mrow><mo>]</mo></mrow></mrow><mi>i</mi></msub></mrow></mstyle><mo>+</mo><mstyle displaystyle="true"><munderover><mo>∑</mo><mrow><mi>m</mi><mo>=</mo><mn>2</mn></mrow><mrow><mi>K</mi><mo>+</mo><mn>2</mn><mo>+</mo><msub><mi>n</mi><mtext>c</mtext></msub></mrow></munderover><mrow><msubsup><mi>W</mi><mrow><mi>i</mi><mo>,</mo><mi>m</mi><mo>−</mo><mn>1</mn></mrow><mtext>o</mtext></msubsup><msub><mi>h</mi><mrow><mi>m</mi><mo>−</mo><mn>1</mn></mrow></msub></mrow></mstyle><mo>+</mo><msubsup><mi>b</mi><mi>i</mi><mtext>o</mtext></msubsup></mrow><mo>)</mo></mrow></mtd></mtr><mtr><mtd><mo>=</mo><mfrac><mn>1</mn><mrow><mn>1</mn><mo>+</mo><msup><mi>e</mi><mrow><mo>−</mo><mrow><mo>(</mo><mrow><mstyle displaystyle="true"><munderover><mo>∑</mo><mrow><mi>m</mi><mo>=</mo><mn>1</mn></mrow><mrow><mi>K</mi><mo>+</mo><mn>2</mn><mo>+</mo><msub><mi>n</mi><mtext>c</mtext></msub></mrow></munderover><mrow><msubsup><mi>U</mi><mrow><mi>i</mi><mo>,</mo><mi>m</mi></mrow><mtext>o</mtext></msubsup><msub><mrow><mrow><mo>[</mo><mrow><msub><mi>V</mi><mtext>o</mtext></msub><mrow><mo>(</mo><mrow><mi>m</mi><msub><mi>T</mi><mtext>s</mtext></msub></mrow><mo>)</mo></mrow></mrow><mo>]</mo></mrow></mrow><mi>i</mi></msub></mrow></mstyle><mo>+</mo><mstyle displaystyle="true"><munderover><mo>∑</mo><mrow><mi>m</mi><mo>=</mo><mn>2</mn></mrow><mrow><mi>K</mi><mo>+</mo><mn>2</mn><mo>+</mo><msub><mi>n</mi><mtext>c</mtext></msub></mrow></munderover><mrow><msubsup><mi>W</mi><mrow><mi>i</mi><mo>,</mo><mi>m</mi><mo>−</mo><mn>1</mn></mrow><mtext>o</mtext></msubsup><msub><mi>h</mi><mrow><mi>m</mi><mo>−</mo><mn>1</mn></mrow></msub></mrow></mstyle><mo>+</mo><msubsup><mi>b</mi><mi>i</mi><mtext>o</mtext></msubsup></mrow><mo>)</mo></mrow></mrow></msup></mrow></mfrac></mtd></mtr></mtable></math> <math display="block" id="M55"><mtable><mtr><mtd><msub><mi>h</mi><mi>m</mi></msub><mo>=</mo><msub><mi>o</mi><mi>m</mi></msub><mo>·</mo><mi>tanh</mi><mrow><mo>(</mo><mrow><msub><mi>C</mi><mi>m</mi></msub></mrow><mo>)</mo></mrow></mtd></mtr><mtr><mtd><mo>=</mo><msub><mi>o</mi><mi>m</mi></msub><mo>·</mo><mfrac><mrow><mn>1</mn><mo>−</mo><msup><mi>e</mi><mrow><mo>−</mo><mn>2</mn><msub><mi>C</mi><mi>m</mi></msub></mrow></msup></mrow><mrow><mn>1</mn><mo>+</mo><msup><mi>e</mi><mrow><mo>−</mo><mn>2</mn><msub><mi>C</mi><mi>m</mi></msub></mrow></msup></mrow></mfrac></mtd></mtr></mtable></math> In this mathematical model, <math id="M56"><mrow><msubsup><mi>U</mi><mrow><mi>i</mi><mo>,</mo><mi>m</mi></mrow><mtext>o</mtext></msubsup></mrow></math> and <math id="M57"><mrow><msubsup><mi>W</mi><mrow><mi>i</mi><mo>,</mo><mi>m</mi></mrow><mtext>o</mtext></msubsup></mrow></math> are the input weights and cyclic weights, respectively, for the m th sampling values [ V o ( mT s )] i , in the i th sample in the output gate structure, respectively. Function <math id="M58"><mrow><msubsup><mi>b</mi><mi>i</mi><mtext>C</mtext></msubsup></mrow></math> is the bias of the i th sample in the output gate structure. The ONP parameter estimation model for the multi-layer LSTM neural network could be considered as a stack of multiple, single-layer LSTM models. The hidden state information, h m , was used as the transmission information between each neural network layer. Therefore, the extracted abstract information was passed between each network layer until the last layer of the LSTM network estimated the parameter set for the ONP, and forward propagation finished. 2.3.3 Back propagation training process for nuclear pulse sequences 3 Because the weights and bias of each LSTM memory cell are randomly assigned when defining a neural network, a loss function also needs to be designed. With this loss function, the error between the estimated pulse parameters <math id="M59"><mrow><msubsup><mi>θ</mi><mi>i</mi><mo>'</mo></msubsup></mrow></math> in the forward propagation and the true pulse parameters <math id="M60"><mrow><msub><mi>θ</mi><mi>i</mi></msub></mrow></math> in the data set can be calculated. At the same time, because this error is calculated from the loss function, it is also called the loss value. The Gaussian ONP parameters estimation model proposed in this paper has solved the problem of estimating specific values. This problem belonged in the category of regression models, so it is more appropriate to use the mean square error (MSE) function as the loss function. For the Training Set with q samples, the error value between the estimated pulse parameter set <math id="M61"><mrow><msubsup><mi>θ</mi><mi>i</mi><mo>'</mo></msubsup></mrow></math> and the true parameter set <math id="M62"><mrow><msub><mi>θ</mi><mi>i</mi></msub></mrow></math> was calculated using the MSE function, and the error value was taken as the loss value <math id="M63"><mrow><mi>L</mi><mi>o</mi><mi>s</mi><msub><mi>s</mi><mrow><mi>M</mi><mi>S</mi><mi>E</mi></mrow></msub></mrow></math> . The calculation formula for the loss function was therefore as shown in Eq. (20): <math display="block" id="M64"><mrow><mi>L</mi><mi>o</mi><mi>s</mi><msub><mi>s</mi><mrow><mtext>MSE</mtext></mrow></msub><mo>=</mo><mfrac><mn>1</mn><mi>q</mi></mfrac><mstyle displaystyle="true"><munderover><mo>∑</mo><mrow><mi>i</mi><mo>=</mo><mn>1</mn></mrow><mi>q</mi></munderover><mrow><msup><mrow><mrow><mo>(</mo><mrow><msub><mi>θ</mi><mi>i</mi></msub><mo>−</mo><msubsup><mi>θ</mi><mi>i</mi><mo>'</mo></msubsup></mrow><mo>)</mo></mrow></mrow><mn>2</mn></msup></mrow></mstyle><mo>.</mo></mrow></math> Then, using the Adam [ 24 24 ] algorithm based on gradient optimization, the loss value and the gradient of the loss function were fed back to the network to update the weight, and as the weights were updated in backpropagation, the set of nuclear pulse parameters estimated in the next forward propagation process would be more accurate. By analogy, with the parameter set of the nuclear pulses estimated by forward propagations, and by correcting the weight of the LSTM network model by backpropagation, the loss value between the ONP parameter set estimated by the LSTM model and the true ONP set gradually decreased, thereby achieving the purpose of training the network. Finally, in order to improve training efficiency and avoid LSTM model loss value oscillation in the later stages of training, a method was needed to determine the number of rounds after which the model stopped training. Because the Mean Absolute Error (MAE) function—as shown in Eq. (21)—has the property of avoiding mutual offsetting between deviations, it was used to determine the number of algorithm training rounds. <math display="block" id="M65"><mrow><mi>M</mi><mi>A</mi><mi>E</mi><mo>=</mo><mfrac><mn>1</mn><mi>q</mi></mfrac><mstyle displaystyle="true"><munderover><mo>∑</mo><mrow><mi>i</mi><mo>=</mo><mn>1</mn></mrow><mi>q</mi></munderover><mrow><mrow><mo>|</mo><mrow><msub><mi>θ</mi><mi>i</mi></msub><mo>−</mo><msubsup><mi>θ</mi><mi>i</mi><mo>'</mo></msubsup></mrow><mo>|</mo></mrow></mrow></mstyle></mrow></math> In use, the technique applied was to set a threshold based on the actual situation, and then when the MAE was less than that threshold, the training ended. At this time, the data in the Test Set was input into the model to test the generalization ability of the ONP parameter estimation model. 2.3.4 Saving the model after training 3 After the LSTM model with the ability to estimate the Gaussian ONP parameter set θ was trained, important information such as the model structure, weights, training configuration, and optimizer status of the training were saved as Hierarchical Data Format 5 (HDF5) files. When it becomes necessary to estimate ONP in the future, first, let the program load the trained model through the HDF5 file; next, the sampled value of the Gaussian ONP for which the shaping parameters need to be estimated is used as the input data of the LSTM model; finally, the LSTM model outputs the required set of nuclear pulses parameters. Therefore, the structure diagram of the Gaussian ONP parameter estimation algorithm, based on the deep learning LSTM model, is as shown in Fig. 1 Fig. 1 . 3. Experimental verification and discussion 1 In order to verify the feasibility of the proposed method in estimating ONP parameters, three examples have been prepared and reviewed in this paper. First, Example 1 is a contrast experiment, in which the operation of the proposed method is compared with the estimation of nuclear pulse parameters using the traditional optimization algorithm proposed by Hong-Quan Huang, Xiao-Feng Yang et al [ 7 7 ] . Second, as the time interval between adjacent exponential pulses becomes smaller, their overlap becomes more severe, in a scenario that will further increase the difficulty of estimating nuclear pulse parameters. Therefore, Example 2 was used to verify the effect of the proposed algorithm on the estimation of ONP parameters when the time interval between adjacent nuclear pulses was short. Finally, considering the difficulty of estimating nuclear pulse parameters caused by the overlap of multiple pulses, Example 3 involved designing a parameter extraction experiment using nine ONP. This example has been used to verify the parameter extraction effect of the algorithm under multiple nuclear pulse overlap conditions. In addition, in order to ensure the accuracy of the calculation under the premise of maximizing computational efficiency improvement, an adaptive algorithm was designed, to determine the number M of sampled values of the ONP V o ( mT s ), and to determine the number of LSTM model neural network layers. Thus, for the ONP V o ( mT s ) , if the k th nuclear pulses sampling value V o ( kT s ) satisfies the condition expressed in Eq. (22), the number M of sampled values of the ONP V o ( mT s ) could be obtained by applying Eq. (23). <math display="block" id="M66"><mrow><msub><mi>V</mi><mtext>o</mtext></msub><mrow><mo>(</mo><mrow><mi>k</mi><msub><mi>T</mi><mtext>s</mtext></msub></mrow><mo>)</mo></mrow><mo>≤</mo><mn>0</mn><mo><</mo><msub><mi>V</mi><mtext>o</mtext></msub><mrow><mo>[</mo><mrow><mrow><mo>(</mo><mrow><mi>k</mi><mo>−</mo><mn>1</mn></mrow><mo>)</mo></mrow><msub><mi>T</mi><mtext>s</mtext></msub></mrow><mo>]</mo></mrow></mrow></math> <math display="block" id="M67"><mrow><mi>M</mi><mo>=</mo><mrow><mo>⌈</mo><mrow><mn>1.2</mn><mi>k</mi></mrow><mo>⌉</mo></mrow></mrow></math> Combined with the above conditions, the number of LSTM model neural network layers could be determined through applying Eq. (24): <math display="block" id="M68"><mrow><mrow><mo>{</mo><mtable columnalign="left"><mtr><mtd><msup><mn>2</mn><mrow><mi>s</mi><mo>−</mo><mn>1</mn></mrow></msup><mo>≤</mo><mi>M</mi><mo><</mo><msup><mn>2</mn><mi>s</mi></msup><mo>=</mo><msub><mi>C</mi><mrow><mtext>num</mtext><mn>1</mn></mrow></msub></mtd></mtr><mtr><mtd><msub><mi>C</mi><mrow><mtext>num</mtext><mi>ξ</mi></mrow></msub><mtext>=</mtext><mi>ξ</mi><msub><mi>C</mi><mrow><mtext>num</mtext><mn>1</mn></mrow></msub></mtd></mtr></mtable></mrow></mrow></math> In Eq.(24), s is an arbitrary positive integer, C num1 is the number of memory cells in the first layer, <math id="M69"><mrow><msub><mi>C</mi><mrow><mtext>num</mtext><mi>ξ</mi></mrow></msub></mrow></math> is the number of memory cells in the last layer, and <math id="M70"><mi>ξ</mi></math> is the number of network layers. However, subject to hardware performance constraints, the number of network layers involved in this article did not exceed three, and the advantage of this structure type was that it could retain ONP sequence information to the maximum extent, and could minimize the scale of the network, allowing the adaptive algorithm mentioned above to effectively improve hardware computing efficiency. Example 1 Exponential pulses Input 1, Input 2, Input 3, and Input 4 were inputted into an S–K shaping circuit, with the characteristic time of <math id="M71"><mtext>τ</mtext></math> = 100 ns. Values for the amplitude parameter, <math id="M72"><mrow><msub><mi>A</mi><mi>i</mi></msub></mrow></math> , were 300, 150, 200, and 250 Counts, and time parameter, <math id="M73"><mrow><msub><mi>T</mi><mi>i</mi></msub></mrow></math> , values were 1, 100, 200, and 300, respectively. The white noise standard deviation was 5 Counts, and the sampling period, T s , was 5 ns. Taking RC = 250 ns, parameter K of S-K digital Gaussian pulses shaping algorithm was K = RC / T s = 50. By applying these conditions into Eqs. (1)–(4), it was found that k = 488. Then, by applying this k value into Eqs. (22)–(24), the number <math id="M74"><mi>ξ</mi></math> of layers of the network and the number C num1 of memory cells in the first layer were calculated, where the M equaled 586, s = 10, C num1 = 1024, C num2 = 2048, and C num3 = 3072. The Adam algorithm was used to update the weights, and the parameters of the Adam optimizer were set to <math id="M75"><mrow><msub><mtext>β</mtext><mn>1</mn></msub></mrow></math> = 0.9, and <math id="M76"><mrow><msub><mtext>β</mtext><mn>2</mn></msub></mrow></math> = 0.999. In addition, the number of pre-training rounds (named epochs) was set to 20 and stopped the next round of training when the MAE value was less than three. At the same time, each training round randomly generated 1000 sample sets, and these were divided into the Training Set, the Test Set, and the Validation Set, in the ratio of 8:1:1. In addition, groups of 10 samples were gathered into a batch of input networks (batch size = 10). The changes in the network loss and MAE values achieved through training, for the Training Set and the Validation Set, are shown in Fig. 2 Fig. 2 . As shown in Fig. 2 Fig. 2 , the loss value of the model in both the Training Set and the Validation Set decreased monotonically, showing that there was no over fitting phenomenon, and, in a saving on computational costs, this example did not require use of the Dropout algorithm. Finally, the corresponding parameters and errors of the Gaussian ONP estimated by the deep learning LSTM model and the traditional optimization algorithm that Huang [ 7 7 ] et al. used are shown in Table 1 Table 1 . Bold value indicates a smaller error value in the comparison experiment. Used to compare the accuracy of the estimation of two experimental parameters 1 1   1 1 A 1 1 1 A 2 1 1 A 3 1 1 A 4 1 1 T 1 1 1 T 2 1 1 T 3 1 1 T 4 1 1 True value 1 1 300 1 1 150 1 1 200 1 1 250 1 1 1 1 1 100 1 1 200 1 1 300 1 1 Calculated value (Huang) 1 1 305.62 1 1 150.85 1 1 189.33 1 1 256.69 1 1 1.17 1 1 99.52 1 1 198.63 1 1 299.9 1 1 Calculated LSTM value 1 1 294.44 1 1 150.06 1 1 199.95 1 1 248.92 1 1 0.9995 1 1 100.1 1 1 199.89 1 1 294.29 1 1 Error (Huang) 1 1 5.62(1.87%) 1 1 0.85(0.57%) 1 1 10.67(5.34%) 1 1 6.69(2.68%) 1 1 0.17 1 1 0.48 1 1 1.37 1 1 0.1 1 1 Error (LSTM) 1 1 5.56 (1.85%) 1 1 0.06 (0.04%) 1 1 0.05 (0.03%) 1 1 1.08 (0.43%) 1 1 0.0005 1 1 0.1 1 1 0.11 1 1 5.71 Figure 3 Figure 3 shows the results for nuclear pulse parameter estimation based on the LSTM model. Here, Figure 3a Figure 3a shows the exponential pulses before shaping, the Gaussian pulses after shaping, and the Gaussian ONP obtained using the LSTM parameter estimation method. Figure 3b Figure 3b shows the true exponential pulses and the exponential pulses obtained using the LSTM model, while Fig. 3c Fig. 3c shows both the true Gaussian pulses and the Gaussian pulses whose parameters were estimated using the LSTM model. From the experimental results, the relative errors for amplitude parameter, <math id="M78"><mrow><msub><mi>A</mi><mi>i</mi></msub></mrow></math> , were 1.85%, 0.04%, 0.03%, 0.43%, and the absolute errors for time parameter, <math id="M79"><mrow><msub><mi>T</mi><mi>i</mi></msub></mrow></math> , were 0.0005, 0.1, 0.11, and 5.71. This showed that the deep learning LSTM model provided in this paper has good parameter estimation ability when addressing the pulse parameter estimation problem, and that the accuracy of the estimation parameters had been improved to some extent, in comparison to the traditional optimization algorithm. Example 2 When the time interval of the adjacent exponential nuclear pulses is short, the overlap between the pulses will be more serious, making it very difficult to estimate ONP parameters. The purpose of this example was to verify the ability of the proposed method to estimate ONP parameters when the time interval between adjacent exponential nuclear pulses was small. Exponential pulses Input 1, Input 2, Input 3, and Input 4 were inputted into an S–K shaping circuit, with the characteristic time of <math id="M80"><mtext>τ</mtext></math> = 100 ns. The amplitude parameter, <math id="M81"><mrow><msub><mi>A</mi><mi>i</mi></msub></mrow></math> , values were 300, 150, 200, and 250 Counts, and time parameter, <math id="M82"><mrow><msub><mi>T</mi><mi>i</mi></msub></mrow></math> , values were 1, 100, 150, and 200 respectively. The white noise standard deviation was 5 Counts, and the sampling period, T s , was 5 ns. Taking RC = 250 ns, parameter K for the S-K digital Gaussian pulses shaping algorithm was K = RC / T s = 50. By bringing these conditions into equations (1)–(4), k = 382 was calculated, which was then inputted into equations (22)–(24), so that the number, <math id="M83"><mi>ξ</mi></math> , of network layers, and the number, C num1 , of memory cells in the first layer could be calculated, where M = 457, s = 9, C num1 = 512, C num2 = 1024, and C num3 = 1536. The Adam algorithm was used to update the weights, and the parameters of the Adam optimizer were set at <math id="M84"><mrow><msub><mtext>β</mtext><mn>1</mn></msub></mrow></math> =0.9, and <math id="M85"><mrow><msub><mtext>β</mtext><mn>2</mn></msub></mrow></math> = 0.999. In addition, the number of pre-training rounds, named epochs , was set to 10, and the next training round was stopped when the value of MAE was less than three. At the same time, in each training round, 1000 sample sets were randomly generated, and were divided into a Training Set, Test Set, and Validation Set, in the ratio of 8:1:1. Furthermore, each 2 samples were composed into a batch of input networks (batch size = 2). The final parameter estimation results and related images can be seen in Table 2 Table 2 and Fig. 4 Fig. 4 , respectively. 1 1 Items 1 1 A 1 1 1 A 2 1 1 A 3 1 1 A 4 1 1 T 1 1 1 T 2 1 1 T 3 1 1 T 4 1 1 True value 1 1 300 1 1 150 1 1 200 1 1 250 1 1 1 1 1 100 1 1 150 1 1 200 1 1 Calculated value 1 1 300.13 1 1 150.72 1 1 199.29 1 1 250.23 1 1 0.997 1 1 98.95 1 1 149.29 1 1 199.34 1 1 Error 1 1 0.13(0.043%) 1 1 0.72(0.48%) 1 1 0.71(0.36%) 1 1 0.23(0.09%) 1 1 0.003 1 1 1.05 1 1 0.71 1 1 0.66 According to the experimental results, the relative amplitude parameter ( <math id="M87"><mrow><msub><mi>A</mi><mi>i</mi></msub></mrow></math> ) errors were 0.043%, 0.48%, 0.36%, and 0.09%, and the absolute time parameter ( <math id="M88"><mrow><msub><mi>T</mi><mi>i</mi></msub></mrow></math> ) errors were 0.003, 1.05, 0.71, and 0.66, respectively. Therefore, even when the time interval between adjacent exponential pulses was short, the ONP shaping parameters estimated by the LSTM model still showed high precision. Example 3 To test the LSTM model's ability to estimate the parameters of multiple ONP, this example investigated LSTM model parameter estimation for nine overlapping pulses. Exponential pulses Input 1, Input 2, Input 3, Input 4, Input 5, Input 6, Input 7, Input 8, and Input 9 were inputted into an S–K shaping circuit, with the characteristic time of <math id="M89"><mtext>τ</mtext></math> = 100 ns. Values for the amplitude parameter, <math id="M90"><mrow><msub><mi>A</mi><mi>i</mi></msub></mrow></math> , were 300, 150, 200, 250, 650, 100, 550, 350, and 50 Counts, while values for the time parameter, <math id="M91"><mrow><msub><mi>T</mi><mi>i</mi></msub></mrow></math> , were 1, 100, 200, 300, 400, 500, 600, 700, and 800 respectively. The white noise standard deviation was 5 Counts, and the sampling period, T s , was 5 ns. Taking RC = 250 ns, the parameter K for the S–K digital Gaussian pulses shaping algorithm was K = RC / T s = 50. Bringing the above conditions into Eqs. (1)–(4), it was calculated that k = 382, which, when inputted into Eqs. (22)–(24), allowed the number of network layers, <math id="M92"><mi>ξ</mi></math> , and the number of memory cells in the first layer, C num1 , to be calculated. Thus it was calculated that M = 902, s = 10, C num1 = 1024, C num2 = 2048, and C num3 = 3072. The Adam algorithm was used to update the weights, and the Adam parameters were set to <math id="M93"><mrow><msub><mtext>β</mtext><mn>1</mn></msub></mrow></math> = 0.9, and <math id="M94"><mrow><msub><mtext>β</mtext><mn>2</mn></msub></mrow></math> = 0.999. In addition, the number of pre-training rounds, named epochs, was set to 30, and the next training round was stopped when MAE was less than five. At the same time, in each training round, 1000 sample sets were randomly generated, and were divided into the Training Set, the Test Set, and the Validation Set, according to the ratio of 8:1:1. In addition, due to the large amount of training data, the network was allowed to learn sample by sample (batch size = 1), in order to ensure accuracy. However, owing to the large number of pulses involved in this example, it was not effective visually to display the results. Therefore, for this example, just the final estimated parameter results from the LSTM model are shown, in Table 3 Table 3 . 1 1 Items 1 1 A 1 1 1 A 2 1 1 A 3 1 1 A 4 1 1 A 5 1 1 A 6 1 1 A 7 1 1 A 8 1 1 A 9 1 1 T 1 1 1 T 2 1 1 T 3 1 1 T 4 1 1 T 5 1 1 T 6 1 1 T 7 1 1 T 8 1 1 T 9 1 1 True value 1 1 300 1 1 150 1 1 200 1 1 250 1 1 650 1 1 100 1 1 550 1 1 350 1 1 50 1 1 1 1 1 100 1 1 200 1 1 300 1 1 400 1 1 500 1 1 600 1 1 700 1 1 800 1 1 Calculated value 1 1 300.44 1 1 150.86 1 1 200.54 1 1 250.62 1 1 649.9 1 1 98.07 1 1 549.72 1 1 350.14 1 1 49.11 1 1 1 1 1 100.61 1 1 199.57 1 1 300.04 1 1 400.26 1 1 500.55 1 1 600.05 1 1 700.3 1 1 799.78 1 1 Absolute error 1 1 0.44 1 1 0.86 1 1 0.54 1 1 0.62 1 1 0.1 1 1 1.93 1 1 0.28 1 1 0.14 1 1 0.89 1 1 0 1 1 0.61 1 1 0.43 1 1 0.04 1 1 0.26 1 1 0.55 1 1 0.05 1 1 0.3 1 1 0.22 1 1 Relative error % 1 1 0.15 1 1 0.57 1 1 0.27 1 1 0.248 1 1 0.015 1 1 1.93 1 1 0.051 1 1 0.04 1 1 1.78 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 Examining the experimental results, it can be seen that the relative amplitude parameter, <math id="M96"><mrow><msub><mi>A</mi><mi>i</mi></msub></mrow></math> , errors were 0.15%, 0.57%, 0.27%, 0.248%, 0.015%, 1.93%, 0.051%, 0.04%, and 1.78%, and the time parameter, <math id="M97"><mrow><msub><mi>T</mi><mi>i</mi></msub></mrow></math> , absolute errors were 0, 0.61, 0.43, 0.04, 0.26, 0.55, 0.05, 0.3, and 0.22. Therefore, even when the time interval between adjacent exponential pulses was short, the ONP shaping parameters estimated by the LSTM model still showed high precision. This showed that even under conditions of multiple ONP, estimating the nuclear pulse shaping parameters using the LSTM model was still an effective method. 4. Conclusion 1 The LSTM model in deep learning has been used to estimate parameters for ONP signals after the S–K Gaussian shaping, and the problems of the pre-shaping exponential signal and noise influence on the parameters were seen to have been satisfactorily overcome. Taking the measured ONP signal as a sample, the sampled value of the ONP signal was transmitted into the LSTM model as a separate part. The processing and transfer of sampled value information was based on the memory cell structure peculiar to the LSTM model. Finally, through continuous training, when sampled ONP values were input into the LSTM model, the LSTM model was able to estimate the shaping parameters of these nuclear pulses quickly and accurately. Because the network was learning the entire sample, all features in the sample were recorded by the network, which indicated that this method could overcome the traditional method defect of local convergence, and achieve optimal estimation of nuclear pulse parameters in the global sense. It has been demonstrated that the model proposed herein is a good method for estimating ONP parameters. Use of full-sample learning resulted, however, in a sharp increase in the amount of data that needed to be calculated during model training, in comparison with traditional methods, and so training time for the current LSTM model was longer than that required for the traditional method. In addition, the increased size of the model, caused by the increased amount of data, meant that the currently used hardware could not explore some additional aspects. Network structure optimization and computational efficiency improvement will become the primary features, therefore, of our forward research, in which, for example, using multiple types of deep neural networks, and re-optimizing the memory cells inside the LSTM could be explored. In addition, the examples presented here only considered parameter estimation issues for ONP based on S-K digital shaping. In future research, the problem of pulse parameter extraction in digital trapezoidal (triangular) forms could be addressed, and, the issue of baseline estimates could also be taken into account. References F.S. Goulding , Pulses-shaping in low-noise nuclear amplifiers: A physical approach to noise analysis . Nucl. Instrum. Meth. A 100 , 493 - 504 ( 1972 ). doi: 10.1016/0029-554X(72)90828-2 http://doi.org/10.1016/0029-554X(72)90828-2 Pulses-shaping in low-noise nuclear amplifiers: A physical approach to noise analysis G. Gerardi , L. Abbene , A. La Manna et al ., Digital filtering and analysis for a semiconductor X-ray detector data acquisition . Nucl. Instrum. Meth. A 571 , 378 - 380 ( 2007 ). doi: 10.1016/j.nima.2006.10.113 http://doi.org/10.1016/j.nima.2006.10.113 Digital filtering and analysis for a semiconductor X-ray detector data acquisition T. Noulis , C. Deradonis , S. Siskos et al ., Particle detector tunable monolithic Semi-Gaussian shaping filter based on transconductance amplifiers . Nucl. Instrum. Meth. A 589 , 330 - 337 ( 2008 ). doi: 10.1016/j.nima.2008.02.048 http://doi.org/10.1016/j.nima.2008.02.048 Particle detector tunable monolithic Semi-Gaussian shaping filter based on transconductance amplifiers S.G. Chen , S.Y. Ji , W.S. Liu , Gaussian pulses shaping of exponential decay signal based on wavelet analysis . ACTA PHYS SIN-CH ED . 57 , 2882 - 2887 ( 2018 ). doi: 10.3321/j.issn:1000-3290.2008.05.041 http://doi.org/10.3321/j.issn:1000-3290.2008.05.041 (in Chinese) Gaussian pulses shaping of exponential decay signal based on wavelet analysis S.G. Chen , S.Y. Ji , W.S. Liu et al ., Recursive implementation of Gaussian pulse shaping based on wavelet analysis . ACTA PHYS SIN-CH ED . 58 ( 5 ), 3041 - 3046 ( 2009 ). doi: 10.7498/aps.58.3041 http://doi.org/10.7498/aps.58.3041 (in Chinese) Recursive implementation of Gaussian pulse shaping based on wavelet analysis K.M. Jiang , H.Q. Huang , X.F. Yang et al ., Pulses Parameter Extraction Method Based on S-K Digital Shaping and Population Technique . Nuclear Electronics & Detection Technology . 37 , 2 ( 2017 ). (in Chinese) Pulses Parameter Extraction Method Based on S-K Digital Shaping and Population Technique H.Q. Huang , X.F. Yang , W.C. Ding et al ., Estimation method for parameters of overlapping nuclear pulses signal . Nucl. Sci. Tech . 28 , 12 ( 2017 ). doi: 10.1007/s41365-016-0161-z http://doi.org/10.1007/s41365-016-0161-z Estimation method for parameters of overlapping nuclear pulses signal G.E. Hinton , R.R. Salakhutdinov , Reducing the dimensionality of data with neural networks . Science . 313 , 5786 ( 2006 ). doi: 10.1126/science.1127647 http://doi.org/10.1126/science.1127647 Reducing the dimensionality of data with neural networks G. Dorffner , Neural networks for time series processing . NEURAL NETW WORLD . 6 , 1 ( 1996 ). Neural networks for time series processing Y. LeCun , Y. Bengio , G. Hinton , Deep learning . Nature . 521 , 7553 ( 2015 ). doi: 10.1038/nature14539 http://doi.org/10.1038/nature14539 Deep learning Y. Lecun , L. Bottou , Y. Bengio et al ., Gradient-based learning applied to document recognition . Proceedings of the IEEE 86 , 2278 - 2324 ( 1998 ). doi: 10.1109/5.726791 http://doi.org/10.1109/5.726791 Gradient-based learning applied to document recognition J. Du , B.L. Hu , Y.Z. Liu et al ., Study on quality identification of macadamia nut based on convolutional neural networks and spectral features . SPECTROSC SPECT ANAL . 38 , 1514 ( 2018 ). doi: 10.3964/j.issn.1000-0593(2018)05-1514-06 http://doi.org/10.3964/j.issn.1000-0593(2018)05-1514-06 Study on quality identification of macadamia nut based on convolutional neural networks and spectral features A. Graves , Generating Sequences With Recurrent Neural Networks(2013) . arXiv:1308.0850 Generating Sequences With Recurrent Neural Networks(2013) A. Graves , A. Mohamed , G. Hinton , Speech Recognition with Deep Recurrent Neural Networks(2013) . arXiv:1303.5778 Speech Recognition with Deep Recurrent Neural Networks(2013) R. Pascanu , C. Gulcehre , K. Cho et al ., How to Construct Deep Recurrent Neural Networks(2013) . arXiv:1312.6026 How to Construct Deep Recurrent Neural Networks(2013) S. Hochreiter , J. Schmidhuber , Long short-term memory . Neural. Comput . 9 , 1735 - 1780 ( 1997 ). doi: 10.1162/neco.1997.9.8.1735 http://doi.org/10.1162/neco.1997.9.8.1735 Long short-term memory A. Graves , M. Liwicki , S. Fernandez et al ., A novel connectionist system for unconstrained handwriting recognition . IEEE T Pattern Anal . 31 , 855 - 868 ( 2009 ). doi: 10.1109/tpami.2008.137 http://doi.org/10.1109/tpami.2008.137 A novel connectionist system for unconstrained handwriting recognition F.A. Gers , D. Eck , J. Schmidhuber , Artificial Neural Networks-Icann 2001 ( Vienna univ technol , vienna , 2001 ), pp. 669 - 676 Artificial Neural Networks-Icann 2001 P.J. Werbos , Backpropagation through time: what it does and how to do it . Proceedings of the IEEE . 78 , 1550 - 1560 ( 1990 ). doi: 10.1109/5.58337 http://doi.org/10.1109/5.58337 Backpropagation through time: what it does and how to do it A. Graves , J. Schmidhuber , Framewise phoneme classification with bidirectional LSTM and other neural network architectures . Neural Networks . 18 , 602 - 610 ( 2005 ). doi: 10.1016/j.neunet.2005.06.042 http://doi.org/10.1016/j.neunet.2005.06.042 Framewise phoneme classification with bidirectional LSTM and other neural network architectures S. Amari , Backpropagation and stochastic gradient descent method . Neurocomputing . 5 , 185 - 196 ( 1993 ). doi: 10.1016/0925-2312(93)90006-O http://doi.org/10.1016/0925-2312(93)90006-O Backpropagation and stochastic gradient descent method J. Duchi , E. Hazan , Y. Singer , Adaptive subgradient methods for online learning and stochastic optimization . J. Mach. Learn. Res . 12 , 2121 - 2159 ( 2011 ). Adaptive subgradient methods for online learning and stochastic optimization S. Yeung , O. Russakovsky , N. Jin et al ., Every moment counts: Dense detailed labeling of actions in complex videos . Int. J. Comput. Vision 126 , 375 - 389 ( 2018 ). doi: 10.1007/s11263-017-1013-y http://doi.org/10.1007/s11263-017-1013-y Every moment counts: Dense detailed labeling of actions in complex videos Kingma , P. Diederik , J. Ba , Adam: A Method for Stochastic Optimization(2014) . arXiv:1412.6980 Adam: A Method for Stochastic Optimization(2014) G.E. Hinton , N. Srivastava , A. Krizhevsky et al ., Improving neural networks by preventing co-adaptation of feature detectors(2012) . arXiv:1207.0580 Improving neural networks by preventing co-adaptation of feature detectors(2012) A. Krizhevsky , I. Sutskever , G.E. Hinton , ImageNet Classification with Deep Convolutional Neural Networks . COMMUN ACM . 60 , 84 - 90 ( 2017 ). doi: 10.1145/3065386 http://doi.org/10.1145/3065386 ImageNet Classification with Deep Convolutional Neural Networks X. Bouthillier , K. Konda , P. Vincent , et al ., Dropout as data augmentation(2015) . arXiv:1506.08700 Dropout as data augmentation(2015) 10.1007/s41365-019-0691-2 This work was supported by the National Natural Science Foundation of China (Nos. 41774140 and 11675028), the Scientific Research Fund of Sichuan Provincial Education Department (No. 18ZA0050), and the Scientific Research Innovation Team of Chengdu University of Technology (No. 10912-KYTD201701). 2019-05-22 2019-06-29 2019-07-03 2019-10-29 2019-11-01 2021-04-11 © China Science Publishing & Media Ltd. (Science Press), Shanghai Institute of Applied Physics, the Chinese Academy of Sciences, ChineseNuclear Society and Springer Nature Singapore Pte Ltd. 2019 This is a post-peer-review, pre-copyedit version of an article published in Nuclear Science and Techniques. The final authenticated version is available online at: http://dx.doi.org/10.1007/s41365-019-0691-2 http://dx.doi.org/10.1007/s41365-019-0691-2 . 2019 Nuclear Science and Techniques 11 30