Two-Body Random Ensemble (TBRE)

In the TBRE, the nuclear Hamiltonian includes only two-body interactions and is expressed as follows: $H={\displaystyle \sum _J{\displaystyle \sum _{j_1{j}_2}{\displaystyle \sum _{j_3{j}_4}{G}_{j_1{j}_2;j_3{j}_4}^J}}}{A}_J^{†}\left(j_1{j}_2\right)A_J\left(j_3{j}_4\right)\mathrm{.}$ (1) In Eq. (1), $G_{j_1{j}_2;j_3{j}_4}^J$ represents the matrix elements of the two-body interaction; $A_J^{†}\left(j_1{j}_2\right)$ denotes the creation operator of the nucleon pair with two nucleons on the j₁ and j₂ orbits coupled to total angular momentum J; and $A_J\left(j_3{j}_4\right)$ represents the annihilation operator of the nucleon pair.

In TBRE, the matrix elements $G_{j_1{j}_2;j_3{j}_4}^J$ in Eq. (1) are independent random numbers that follow a Gaussian distribution with a probability function $f\left(G_{j_1{j}_2{j}_3{j}_4}^J\right)=\frac{1}{\sqrt{2\pi }\sigma }\exp \left\{-\frac{{\left(G_{j_1{j}_2;j_3{j}_4}^J\right)}^2}{2{\sigma }^2}\right\}\text{.}$ (2)

Here, ${\sigma }^2=\frac{1}{2}\left(1+{\delta }_{j_1{j}_3}{\delta }_{j_2{j}_4}\right)\text{,}$ (3) to maintain the invariance of the statistical distribution of interaction matrix elements during an arbitrary single-particle transformation.

Classification neural network

The classification model presented in this paper utilizes an NN, which consists of an input layer, one or more hidden layers, and an output layer. The structure is illustrated in Fig. 1, (with one hidden layer shown as an example). The input layer receives the matrix elements of the two-body interactions in the shell model, specifically the $G_{j_1{j}_2;j_3{j}_4}^J$ values in Eq. (1), where the number of inputs is equal to the number of independent two-body interaction matrix elements in a specific shell-model space. Based on the corresponding input interactions, the output layer provides the probabilities of different spin states being the ground state. The number of outputs should equal the number of possible ground-state spins. The activation function used in this model was the rectified linear unit (ReLU) function [41], which is explained in Table 2. Assuming that the vector $\overrightarrow{x}=\left\{x_i\right\}$ represents the network input, that is, G two-body interaction matrix elements in Eq. (1), and $\overrightarrow{y}$ is the network output, whose elements correspond to the probability of each spin being the ground-state spin, the relationship (with one hidden layer) can be expressed analytically as follows: $y_k\left(\overrightarrow{x};\overrightarrow{\omega }\right)=a_k+{\displaystyle \sum _j{b}_{kj}}\text{ReLU}\left(c_j+{\displaystyle \sum }{d}_{ji}{x}_i\right)\mathrm{.}$ (4) Here, $\overrightarrow{\omega }=\left\{a_k\mathrm{,}{b}_{kj}\mathrm{,}{c}_j\mathrm{,}{d}_{ji}\right\}$ denotes the NN parameter vector.

Fig. 1Full-screen View

Schematic of the adopted NN classification model

The output layer introduces the softmax function[42], which transforms the unnormalized output values into non-negative probability values whose sum is one. $P_k={\text{Softmax}\left(\overrightarrow{y}\right)|}_k=\frac{e^{y_k}}{{\displaystyle {\sum }_k{e}^{y_k}}}\mathrm{.}$ (5) This operation preserves the differentiability property of the model and the relative order of unnormalized output values. It also allows the model output to be interpreted as a probability for each class, facilitating the direct interpretation and utilization of these probabilities for classification decisions. Therefore, they are frequently employed in NN models to solve classification problems. Here, Pk is the probability that the kth spin is the ground-state spin. Thus, the maximum of Pk determines the ground-state spin according to the $\overrightarrow{x}$ feature. All the elements Pk construct the predicted probability $\overrightarrow{P}$ from the NN model.

To train the NN model, we first prepared a training set consisting of N samples, $D=\{\left({\overrightarrow{x}}_1\mathrm{,}{S}_1\right)\mathrm{,}\left({\overrightarrow{x}}_2\mathrm{,}{S}_2\right)\mathrm{,}\dots \mathrm{,}\left({\overrightarrow{x}}_N\mathrm{,}{S}_N\right)\}$ out of ~100,000 shell-model calculations. Here, ${\overrightarrow{x}}_i$ includes two-body interaction matrix elements in a single shell-model calculation, while Si is the corresponding ground-state spin obtained from such a shell-model. Secondly, for each Si spin, we created the label ${\widehat{\overrightarrow{P}}}^i$ vectors, which is a hot-one vector, and only include one non-zero elements of value “1,” corresponding to a 100% probability of Si ground-state spin, and 0% probabilities of the rest other spins. Third, we defined the loss function to evaluate the similarity between the label ${\widehat{\overrightarrow{P}}}^i$ vector and the NN predicted ${\overrightarrow{P}}^i$ vector from Eq. (5). $L\left({\overrightarrow{P}}^i\mathrm{,}{\widehat{\overrightarrow{P}}}^i\right)=-{\displaystyle \sum }{P}_m^i\log {\widehat{P}}_m^i\mathrm{,}$ (6) which is a common loss function for training an NN model for classification problems. Using the training samples and corresponding loss function, we trained our network by adjusting the network parameter vector $\overrightarrow{\omega }$ with the adaptive moment estimation (Adam) optimization algorithm [43] to minimize the sum of the loss functions for all training samples. Consequently, an NN model with predictive capabilities was developed.

Shell Model Spaces

We performed approximately 100,000 TBRE calculations in six model spaces. These included four valence nucleons in the f_7/2 orbital of the virtual nucleus (simply expressed as (f_7/2)⁴), four valence nucleons in the $h_{11/2}$ orbital virtual nuclear (${\left(h_{11/2}\right)}^4$), and two, four, and six valence neutrons in the sd shell (corresponding to ¹⁸Ne, ²⁰Ne, and ²²Ne nucleus, respectively), and six valence neutrons in the pf shell (⁴⁶Ca nuclear). These six model spaces represent various levels of many-body complexity.

In the (f_7/2)⁴ space, the eigenvalues from the shell model are simple linear combinations of the two-body interaction matrix elements, as shown in Eq. (1.81) [44]. The ground state spin corresponds to the lowest eigenvalue associated with a specific spin. An NN without a hidden layer corresponds to linear combinations of the input two-body interaction matrix elements, followed by the application of the softmax operation to identify the smallest linear combinations. Both models employ similar calculation processes to determine the ground-state spin, where the weight parameters. Thus, dji parameters in Eq. (4) in the NN correspond to the cfp coefficients [44] in the shell model, and the softmax input to the NN is equivalent to the energy eigenvalues of the shell model. Because a hidden layer complicates the NN model and violates the correspondence between the NN and shell models, we excluded the hidden layer in our NN model for the (f_7/2)⁴ space.

For the (h_11/2)⁴ and ¹⁸Ne spaces, the complexity increases beyond the (f_7/2)⁴ space. Some eigenvalues remain linear combinations of two-body interaction matrix elements, whereas others must be obtained through diagonalization. Although some diagonalization processes with fewer than 5 are analytical, the weight parameters dji can no longer be related to cfp coefficients. Therefore, hidden layers can enhance the adaptability of NN models to nonlinear diagonalization [45]. In the ^{20, 22}Ne and ⁴⁶Ca spaces, the relationship between the eigenvalues and cfp coefficients is completely nonlinear. The transcendental nature of some of these relationships further necessitates hidden layers. Table 1 presents the TBRE sample sizes and input-output settings of our nn models for different spaces.

Optimization of the network architecture

For the (f_7/2)⁴ space, the shell model eigenvalues were linear combinations of two-body interaction matrix elements and the softmax input in the NN model. Therefore, both models employed similar calculation processes for determining the ground-state spin. As mentioned in Sect. 2.3, hidden layers were not required. In fact, an NN model without a hidden layer has already predicted ground-state spins in the (f_7/2)⁴ space with up to 98% accuracy.

For the (h_11/2)⁴ space, because certain eigenvalues displayed nonlinear correlations with the two-body interaction matrix elements, incorporating hidden layers into the model was essential for enhancing the prediction accuracy. We first added one hidden layer and empirically selected 64 hidden nodes for the test run. The results demonstrated an accuracy of 97%, which was a satisfactory outcome.

For the remaining four spaces, the arbitrary accuracies were not always optimal. Therefore, we attempted to improve the prediction accuracy of our NN classification model by including more hidden layers and increasing the number of neural nodes in the ¹⁸Ne, ²⁰Ne, ²²Ne, and ⁴⁶Ca model spaces.

First, the prediction accuracy improved when the number of neural nodes was doubled, as indicated by the difference between the prediction accuracies with N/2 and N neural nodes, as shown in Fig. 2. The absence of negative differences (Fig. 2) suggests that doubling the number of neural nodes consistently improves results, as expected. Furthermore, the differences reached a peak when N=32 for all four model spaces. For N>32, the prediction accuracy improved by only 0~2%. Considering that more nodes entail additional computational overhead, we believe that 32 nodes may be an optimal and balanced choice for this study.

Fig. 2Full-screen View

(Color online) Difference in the prediction accuracies of models employing N/2 and N neural nodes for ¹⁸Ne, ²⁰Ne, ²²Ne, and ⁴⁶Ca model spaces with a single hidden layer. Thirty-two nodes are recommended

Furthermore, we investigated the impact of hidden layers on the prediction accuracy. By employing 32 neural nodes in each layer, as shown in Fig 2, we present the difference in prediction accuracy between networks with $n-1$ and n hidden layers against the layer number n in Fig 3 for the ¹⁸Ne, ²⁰Ne, ²²Ne, and ⁴⁶Ca model spaces. The accuracy improved significantly with a single hidden layer, that is, n=1. However, these improvements diminished with the introduction of additional layers. Because additional layers also consume computational resources, a single hidden layer may be the optimal choice.

Fig. 3Full-screen View

(Color online) Difference between the prediction accuracies of networks employing n and $n-1$ hidden layers for the ¹⁸Ne, ²⁰Ne, ²²Ne, and ⁴⁶Ca model spaces. Thirty-two neural nodes in each hidden layer are recommended, as in Fig 2. A single hidden layer is recommended

Activation functions [46] play a crucial role in NNs by learning abstract features via nonlinear transformations. Common activation functions are the sigmoid (also called the logistic function), tanh (hyperbolic tangent), and ReLU functions. Table 2 presents the impact of different activation functions on the prediction accuracy of our NN model. of The tanh and ReLU functions exhibit similar model prediction accuracies for the five model spaces. However, because the tanh activation function includes an exponential operation, the computational overhead may be larger; therefore, we used the ReLU function throughout this study.

In summary, the optimal network configuration for the ¹⁸Ne, ²⁰Ne, ²²Ne, and ⁴⁶Ca model spaces considered in our analysis comprised one hidden layer with 32 ReLU neural nodes.

Model comparison

As shown in Fig. 1, we adopted a fully connected NN model. However, considering the recent application of Bayesian NNs (BNNs) in nuclear physics [33, 34, 37, 39] and the great success of convolutional NNs [47-50] (CNN) and recurrent NNs [51-53] (RNN), we compared these four networks in terms of accuracy, as shown in Fig 4.

Fig. 4Full-screen View

(Color online) Prediction accuracies with different NN models. We adopt the classic fully connected NN as demonstrated in Fig. 1. BNN stands for Bayesian NN; CNN for convolutional NN; RNN for recurrent NN

The BNN was implemented via Bayesian sampling of weights and biases facilitated by a variational inference algorithm to optimize model training. The model adopted 1000 iterations to update loss and accuracy. During the prediction process, we sampled 1000 times to obtain precise probability prediction results. The CNN included a convolutional, pooling, fully connected, and softmax layers. In particular, in the convolutional layer, the input channel was the number of input features, the output channel was defined as 16. The convolution kernel size was specified as three to facilitate feature extraction. Subsequently, in the pooling layer, the pooling kernel size and step size were set to two to reduce the dimensions of the feature map. The fully connected layer mapped the features extracted by the convolutional layer to the final classification result based on the feature dimensions and category counts of the task. The model parameters were adjusted and iteratively optimized throughout the construction process to ensure effective feature extraction and classification. A remarkable characteristic of the RNN is its ability to continuously transmit and share information through recurrent connections. In addition, the network calculation process defines the forward propagation function, which encompasses the output generated after the input calculation and subsequent prediction through the fully connected layer and softmax function. All four models, including the adopted classic softmax model, shared consistent parameters, including a training-to-test set ratio of 2:1, a learning rate of 0.01, over 1000 training epochs, and the use of the ReLU activation and Adam optimization functions.

According to Fig. 4, the CNN performs the worst, while the accuracies of both BNN and RNN are similar to that of the adopted network. However, the adopted networks had faster training speeds and required fewer computational resources. Therefore, we believe that the adopted network was the optimal choice for our study.

Feature selection

Feature selection is crucial in machine learning and data analysis because it enhances model performance, mitigates the risk of overfitting, boosts computational efficiency, streamlines model interpretation, and addresses problems related to noise and redundant information. This involves conducting a correlation analysis to assess the relationship between each feature and target variable. Subsequently, the features exhibiting a strong correlation with the target variable are selected, whereas the others are excluded from further training. Considering the nonlinear nature of both our feature and label data, we utilized the Spearman correlation coefficient ρ [54] for feature selection as $\rho =1-\frac{6{\displaystyle \sum }{d}_i^2}{n\left(n^2-1\right)}\mathrm{,}$ (7) where di and n represent the difference in the rank values of the i-th data pair and total number of observed samples, respectively.

We calculated the ρ coefficients for four high-dimensional model spaces, namely, ^18~22Ne and ⁴⁶Ca model spaces and used different threshold sizes to select input features with strong correlation. Only features with ρ values greater than the threshold were retained for further training. In Table 3, we list the number of elements of the two-body matrix, that is, the number of input features over a certain threshold, and the corresponding accuracy.

According to Table 3, the number of inputs after feature selection decreases with increasing threshold values, as anticipated. However, this reduction in the input number implies declining performance. Consequently, each input two-body matrix element within the four model spaces significantly affects the output and thus, the exclusion of any of these from network training is not recommended.

Accuracy

Figure 5 presents the evolution of the loss function during training. As expected, the loss functions of the six model spaces converge, indicating that the network parameters are optimal. The loss values of two single-j model spaces, namely, (f_7/2)⁴ and (f_11/2)⁴, decrease the most dramatically because the single-j spaces are simpler than the other 4 models. Furthermore, ⁴⁶Ca, ²⁰Ne, ²²Ne, and ¹⁸N all converge with large loss values, corresponding to the unsatisfactory accuracy described in Table 4. Thus, increasing the number of training epochs does not improve the accuracy.

Fig. 5Full-screen View

(Color online) Evolution of the loss functions during training

Table 4 shows the correlation between the prediction accuracy (%) of the NN for the ground-state spin and the dimensions of the six model spaces under investigation. The shell model eigenvalues of the (f_7/2)⁴ space are linear combinations of two-body interaction matrix elements; therefore, the NN model is equivalent to linear regression and achieves high prediction accuracy of up to 98%. The accuracy is 97% for the (h_11/2)⁴ space with one hidden layer; however, some eigenvalues in the (h_11/2)⁴ space exhibit nonlinear relationships with the two-body interaction matrix elements.

For the remaining four spaces, the accuracy decreased significantly with increasing dimensions. We obtained a Pearson correlation coefficient [55] of -0.753 between the prediction accuracy (%) and dimensions on a logarithmic scale, indicating a negative correlation between the two variables. As the dimensions of the space and the shell complexity increased, the ability of the NN model to predict the ground-state spin diminished. As shown in Fig. 2 and 3, introducing additional hidden layers or neural nodes does not significantly improve the performance of general classification NNs. Thus, to accurately predict the ground-state spin in the TBRE, the generalization capability of the NN is strongly challenged by the complexity of the quantum many-body system, and a more specialized NN architecture and activation function should be designed according to the cfp coefficient property and diagonalization process.

To obtain a more detailed picture of the prediction performance of the NN model for TBRE samples with a specific spin, Fig. 6 presents the confusion matrix for the NN models of the six model spaces. In the confusion matrices, the y- and x-axes represent the ground-state spin predicted by the NN (I_NN) and that obtained from the shell model calculations (I_SM), respectively. The gray scale indicates the probability of the shell model calculation yielding a ground-state spin of I_SM in the samples for which the NN predicts a ground-state spin of I_NN. The main diagonal of the confusion matrix appears predominantly dark, indicating a reasonably high degree of consistency between the NN and shell model for a specific ground-state spin. From a statistical perspective, the NN captures some correlations between the ground-state spin and two-body interaction matrix elements of the TBRE.

Fig. 6Full-screen View

Confusion matrices for predicting the ground-state spin using the NN model in the (f_7/2)⁴, (h_11/2)⁴, ¹⁸Ne, ²⁰Ne, ²²Ne, and ⁴⁶Ca TBRE calculations. The y- and x-axes represents the ground-state spin predicted by the NN (I_NN) and that obtained from the shell model calculations (I_SM), respectively. The gray scale represents the probability of the shell model calculation yielding a ground-state spin of I_SM in the samples for which the NN predicts a ground-state spin of I_NN

The data in Table 4 indicates that the prediction accuracy for the ground-state spin of the ²⁰Ne nucleus is lower than that for higher-dimensional ²²Ne. This finding is consistent with the observations shown in Fig. 6. Specifically, for ²⁰Ne, the difference in colors between the main diagonal and other regions is less pronounced than in other nuclei. This suggests that the prediction of the ground-state spin in ²⁰Ne space poses greater challenges to the NN, which may be related to some special properties of the cfp coefficients of ²⁰Ne. Further exploration of specific multibody complexity features in ²⁰Ne space is desirable.

To further evaluate the statistical performance of the NN model, Fig. 7 presents the distribution of ground-spin spins I (PI) obtained using both the shell model and the well-trained NN model with random interactions. The NN model is consistent with the shell model for all model spaces and partially succeeds in capturing the robust statistical properties of the TBRE.

Fig. 7Full-screen View

(Color online) Distribution of the ground-state spin I (PI) for (f_7/2)⁴, (h_11/2)⁴, ¹⁸Ne, ²⁰Ne, ²²Ne, and ⁴⁶Ca. The black square and red circle represent the PI obtained from shell model calculations with random interactions and that predicted by the NN model, respectively. The blue triangle and olive star represent the PI obtained with the empirical approach [28] applied to the shell model and NN model, respectively

G-I correlation

To predict PI in TBRE, Zhao et al. proposed a general empirical approach [19]. In their approach, one of the two-body interaction matrix elements was set to -1, whereas the rest were set to 0. This interaction was then input into the shell model, and the output ground-state spin I was recorded. If N independent two-body interaction matrix elements existed in the model space, the process was repeated N times, setting a different matrix element equal to -1 each time. Finally, the number of times the spin I was observed as the ground-state spin in N numerical experiments was represented as NI. The probability of spin I in the ground state was then estimated as follows: $P_I=N_I/N\mathrm{.}$ (8)

The empirical approach [19] attributes the "specific spin I as the ground-state spin" to a few two-body interaction matrix elements. If there are more two-body interaction matrix elements responsible for the spin I=0, the empirical rule provides a phenomenological explanation for the dominance of the ground state with zero spin.

Note that the empirical approach hints at the correlation between the two-body interaction matrix elements and ground-state spin, and its correlation determines the ground-state spin distribution, as shown in Fig. 7. Thus, the NN model with a good prediction of the TBRE ground-state spin distribution should also produce a similar correlation between the two-body interaction matrix elements ($G_{j_1{j}_2;j_3{j}_4}^{JT}$) and ground-state spin (I) as the shell model. Therefore, the element–spin (G-I) correlations in the shell model must be compared with those in the NN model.

In Tables 5, 6 and 7, the G-I correlations between the two-body interaction matrix elements ($G_{j_1{j}_2;j_3{j}_4}^J$ defined in Eq. (1)) and the ground-state spin (I), obtained from the empirical approach applied to both the shell and NN models, are listed for the (f_7/2)⁴ and (h_11/2)⁴ model spaces, Ne isotopes, and ⁴⁶Ca, respectively.

According to Table 5, in the (f_7/2)⁴ and (h_11/2)⁴ model spaces, the NN model produces G-I correlations that are perfectly consistent with those of the shell model. This explains the agreement between the shell and NN models shown in Figures 6(a,b) and 7(a,b). In Table 6, this perfect consistency can also be observed for ¹⁸Ne space in coordination with Figures 6(c) and 7(c). However, as the dimensions increase, the consistency of ^{20 22}Ne in Tables 6 and ⁴⁶Ca in Table 7 gradually decreases. In the ²⁰Ne space, there are 12 inconsistent G-I correlations out of 30 (40%) between the SM and NN; in ²²Ne, six out of 30 (20%) are inconsistent; and in ⁴⁶, 24 out of 94 (~26%) are inconsistent. These inconsistent rates are also correlated with the prediction accuracy for different model spaces, as shown in Table 4.

Furthermore, the empirical approach was also applicable to the trained NN model by setting one of the inputs of the NN to -1 and the rest to 0 and recording the ground-state spin (I) from the network. This approach also revealed the correlation between the interaction matrix elements and the predicted ground-state spin, as well as the PI distribution, of the well-trained NN model. Table 5 presents the correlations between the matrix elements and the spin obtained from the shell and NN models. The correlations from both models are identical, indicating that our NN model successfully learns the G-I correlation, as suggested by the empirical approach. Thus, it accurately reproduces the ground-state spin of the shell model for simple model.

With the G-I correlation from such an NN model, we counted the ground-spin Is emerging in the G-I correlation and then normalized them to the PI distribution, as guided by the empirical approach. The PI distributions based on the empirical approach using the NN model are shown in Fig. 7. The empirical approach from both the SM and NN models yields reasonably consistent PI distributions for all model spaces in this case, suggesting that the NN may effectively capture the correlation between the two-body interaction matrix elements and the ground-state spin, which further explains its remarkable performance in reproducing the statistical properties of the ground-state spin in the TBRE.