3.1　 Predictions of the binding energy based on the LDM

In this section, LightGBM is trained to learn the residual δ(Z,A) between the LDM and the experimental binding energies. For this purpose, the binding energies of the 2408 nuclei between ¹⁶O and ²⁷⁰Ds from AME2016 were split into training and test datasets. We note here that nuclei with proton (neutron) numbers smaller than 8 and with relatively large experimental uncertainties in AME2016 were not used. First, the influence of the training size on the predicted binding energy was examined, as shown in Fig. 1. We randomly selected 482 (~20% of the 2408 nuclei) nuclei to constitute the test set. The RMSD of LDM for the selected 482 nuclei was ~2.458 MeV, and after LightGBM refinement, the RMSD was reduced to 0.496, 0.272, and 0.233 MeV when 482, 1204, and 1926 nuclei were used to train the LightGBM, respectively. This means that LightGBM can capture the missing physics of the LDM and decode the correlation between the input features and the residual, to further improve the agreement with experimental data.

Fig. 1Full-screen View

(Color online) Upper panels: Locations of training data sets with (a) 20%, (b) 50%, and (c) 80% of the 2408 nuclei from AME2016 in the N–Z plane. Lower panels: The error between the experimental and LightGBM-refined LDM predicted binding energies for the test set (20% of the 2408 nuclei). Results obtained with the LightGBM-refined LDM trained with (d) 20%, (e) 50%, and (f) 80% of the 2408 nuclei, respectively. σ_pre is the RMSD of the original LDM, and σ_post is the RMSD of the LightGBM-refined LDM.

In addition, it can be seen that the deviation between the experimental and LightGBM-refined LDM predictions for nuclei with the small number of proton and neutron is usually larger, this may be because the microstructure effect in the light-mass nuclei is strong, and there are less data on the light-mass nuclei in the training set. The value of the RMSD fluctuates when the training and test data sets are randomly selected, because δ(Z,A) for some of the nuclei (i.e., nuclei around the magic number) are large and some are small. To evaluate this issue, we randomly split the 2408 nuclei into training and test data sets 500 times with each ratio (i.e., 4:1, 1:1, and 1:4), and the RMSD and its density distribution are plotted in Figs. 2 and 3. As observed in Fig. 2, fluctuation in the RMSD is the largest of all when the ratio of training size to test size is 1:4. The RMSD for 1926 nuclei predicted by LightGBM-refined LDM with a binding energy of 482 nuclei was ~0.508± 0.035 MeV, which is comparable to many physical mass models. With the training data set built from 1926 nuclei and the remaining 482 nuclei constitute the test data set, the RMSD was obtained to be 0.234± 0.022 MeV, which is better than that of many physical mass models.

Fig. 2Full-screen View

(Color online) RMSD for the test data from 100 runs. In each run, the 2408 nuclei were randomly split into training and test datasets at a ratio of 4:1 (blue), 1:1 (orange), and 1:4 (green).

Fig. 3Full-screen View

(Color online) Density distribution of RMSD between the experimental and predicted binding energy. The results from 500 runs for each set are displayed. Dashed lines denote a Gaussian fit to the distribution. The mean values and the standard deviation of the RMSD values are 0.508, 0.303, and 0.224 MeV and 0.035, 0.020, and 0.022 MeV for the three sets with different ratios of training to test size, respectively.

Figure 4 shows the residual δ(Z,A) obtained from the LDM and LightGBM-refined LDM. The results from nine runs with randomly selected 80% of 2408 nuclei as the training set and the remaining 20% as the test set are displayed. It can be seen that the residual δ(Z,A) obtained with the original LDM is large, especially for nuclei around the magic number, owing to the absence of shell effect in the LDM. After the refinement of LightGBM, δ(Z,A) is considerably reduced, especially for nuclei with mass number larger than 60. The performance of LightGBM for nuclei with mass number smaller than 60 is not as good as that for nuclei with large mass number, the same as we already observed in Fig. 1. This could be improved by feeding more relevant features to LightGBM.

Fig. 4Full-screen View

(Color online) Residual δ(Z,A) plotted as a function of mass number. Nine runs with a random splitting of the 2408 nuclei into training and test groups with a ratio of 4:1 are displayed. Blue and orange points denote δ(Z,A) for the test data obtained with the LDM and LightGBM-refined LDM, respectively. σ_pre is the RMSD of the original LDM, and σ_post is the RMSD of the LightGBM-refined LDM.

3.2　 Predictions of the binding energy based on different mass models

In the previous section, the capability of LightGBM to refine the LDM has been demonstrated. In this section, in addition to the LDM, three popular mass models, that is, DZ, WS4, and FRDM are tested as well. To do so, the δ(Z,A) between the experimental binding energy and the one obtained from each mass model is fed to LightGBM, and we randomly split the 2408 nuclei into training and test groups at a ratio of 4:1, and run 500 times for each mass model. The distribution of RMSD on the training and test datasets is shown in Fig. 5. In Table 3 , the performance of serval ML-refined mass models are compared. It can be seen that the typical value of RMSD on the training data set is only ~0.05–0.1 MeV, which is the smallest of all, to the best of our knowledge, the highest performance mass model. The typical value of RMSD on the test data set is ~0.2 MeV, which is also smaller than the others. In general, significant improvements of approximately 90%, 65%, 40%, and 60% after the LightGBM refinement on the LDM, DZ, WS4, and FRDM were obtained, indicating the strong capability of LightGBM to improve theoretical nuclear mass models. In addition, other approaches, such as the radial basis function (RBF) [11], two combinatorial radial basis functions (RBFs) [59], the kernel ridge regression (KRR) [40], and the RBF approach with odd-even corrections (RBFoe) [41], can also be used to improve the performance of nuclear mass models.

Fig. 5Full-screen View

(Color online) Density distribution of RMSD for training and test data sets. Results from 500 runs for each mass model (LDM, DZ, WS4, and FRDM) are displayed. Dashed lines denote a Gaussian fit to the distribution. The corresponding mean values and standard deviations are listed in Table 5. In each run, the 2408 nuclei were randomly split into training and test datasets at a ratio of 4:1.

In principle, if there exists a function that can precisely predict nuclear mass and LightGBM can find this function by learning the mapping between the mass and input features, the improvements of different mass models after the LightGBM refinement should be the same. However, the correlations between nuclear mass and input features are indeed very complicated. LightGBM can capture some of the missed ingredients of these mass models, thereby improving their performance in predicting nuclear mass. In general, the improvement in RMSD is significant for the nuclear mass model with large RMSD values and is relatively slight for the nuclear mass model with small RMSD values.

Very recently, the AME2020 was published; thus, it is interesting to see whether the LightGBM-refined mass models also work well for newly measured nuclei that appear in the AME2020 mass evaluation. A comparison of the binding energies obtained with the LDM, DZ, WS4, and FRDM and LightGBM-refined mass models on the 66 newly measured nuclei that appeared in the AME2020 are illustrated in Fig. 6. The RMSD values of the original mass models on these newly measured nuclei were 2.468, 0.821, 0.350, and 0.778 MeV for the LDM, DZ, WS4, and FRDM, respectively. After the refinement of LightGBM, the RMSD of the above four mass models was significantly reduced to 0.452, 0.320, 0.222, and 0.292 MeV.

Fig. 6Full-screen View

(Color online) Difference between the theoretical and the experimental binding energies (red horizontal line) obtained using the LDM, DZ, WS4, and FRDM (open diamonds) and LightGBM-refined mass models (solid squares). The results of 66 newly measured nuclei that appeared in the AME2020 mass evaluation are displayed. σ_pre and σ_post denote the RMSD of the original and LightGBM-refined mass models on the newly measured nuclei, respectively. The error of the predictions obtained using the LightGBM-refined mass model is the standard deviation of the predicted binding energy. It is obtained by running LightGBM 500 times with randomly splitting AME2016 data into training and test sets with a ratio of 4:1.

By increasing the capacity of the training set, the model can learn more information, and the RMSD is reduced. However, the uncertainty of RMSD increases with an increase in the percentage of the training set, because the less tested nuclei, the larger the fluctuations. In addition, the RMSD of the 66 newly measured nuclei that appeared in AME2020 are also displayed in Fig. 7. When the percentage of the training set was larger than 90%, the RMSD of the 66 newly measured nuclei slightly increased, indicating that overfitting started to occur. However, this overfitting problem is not very serious because LightGBM has a strong ability to prevent overfitting. In the present work, to avoid either a large RMSD value or large uncertainty of RMSD, the percentage of the training set was chosen as 80% in most cases.

Fig. 7Full-screen View

(Color online) RMSD of the LightGBM-refined LDM plotted as a function of the percentage of the training set.

3.3　 Extrapolation of neutron separation energy

Single and two-neutron separation energies are of particular interest because they provide information relevant to shell and subshell structures, nuclear deformation, paring effects, as well as the boundary of the nuclear landscape. They can be calculated using the following formulas:

Good performance of the LightGBM-refined mass models for the prediction of nuclear binding has been shown, and it is interesting to see whether the single and two-neutron separation energies can also be reproduced well on the same foot. Based on the calculation of the LightGBM-refined models, the RMSD of S_n of 2255 nuclei and S_2n of 2140 nuclei are displayed in Table 4 . Figure 8 compares the single neutron separation energy of Ca, Zr, Sn, and Pb isotopic chains given by different theoretical models and the experimental data from AME2016. All predictions are in good agreement with the experimental data when there is data, while discrepancies appear as the neutron number increases. The general trend of S_n as a function of neutron number obtained with LightGBM-refined LDM and WS4 are similar to those obtained with other nuclear mass models, for example, the odd-even staggering can also be observed.

Fig. 8Full-screen View

(Color online) Single-neutron separation energy of Ca, Zr, Sn, and Pb isotopic chains given by different models. The results obtained using the LightGBM-refined LDM and WS4 were compared with those from the FRDM and WS4, as well as recent theoretical calculations given by Xia et al. [16], Ma et al. [59], and Yu et al. [60].

The latest experimental measurements of the two-neutron separation energies of the four elements (Ca, Ti, Pm, Sm) were compared with various theoretical model calculations, as shown in Fig. 9. It can be seen that the newly measured S_2n can be well reproduced by the LightGBM-refined LDM and WS4 models; in particular, S_2n obtained with LightGBM-refined LDM is much closer to the experimental data than that obtained with the LDM. For example, the sharp decrease in S_2n around the magic number cannot be reproduced by the LDM, while this issue can be fixed after the refinement of LightGBM. The good performance of LightGBM-refined mass models on both S_n and S_2n, again indicating the strong capacity of LightGMB in refining the nuclear mass model.

Fig. 9Full-screen View

(Color online) Two-neutron separation energy of the neutron-rich nuclei on Ca, Ti, Pm, Sm isotopic chains given by different models. The red and green dots represent the experimental data from AME2016 and the latest measurements from Refs. [61, 62], respectively. The neutron numbers of the predicted drip line isotopes (Ca and Ti) in each nuclear mass model are also listed in the figure. Note that S_2n obtained with WS4 and LightGBM-refined WS4 almost completely overlap.

3.4　 Prediction of the residual δ(Z,A)=B_LDM-B_exp

It is known that the residual δ(Z,A)=B_LDM-B_exp, that is, the difference in binding energy between the experimental data and the one calculated with the LDM, in the vicinity of the magic numbers are usually large, and δ(Z,A) can reflect quantum-mechanical shell effects. It is always interesting to know if new magic numbers exist for exotic nuclei. For this purpose, the residual δ(Z,A) predicted with LightGBM based on learning the pattern of δ(Z,A) for nuclei appearing in the AME2016 database are displayed in Fig. 10. As can be seen, δ(Z,A) for Z=126 and N=184 nuclei are relatively large, which might reveal that they are magic numbers. However, it should be noted that they have been used as input features, that is, |Z-m| and |N-m|. By removing four magic number related features (i.e., Zm, Nm, |Z-m|, and |N-m|) in the training, the predicted results are displayed in the upper subfigure of Fig. 10. It is interesting to note that δ(Z,A) has local minima around (Z=20, N=50), (Z=28, N=82), and (Z=50, N=126), but there is no shell structure on δ(Z,A) for nuclei with Z ≥ 82 or N ≥ 126. This is understandable because ML algorithms have a strong ability to address interpolation tasks and become less efficient and reliable for extrapolation tasks, particularly for samples that are far away from the training samples. In this context, one does not expect new magic numbers to be explored by ML algorithms. Nevertheless, the high accuracy for predicting nuclear mass around nuclei with experimental data has been demonstrated, for example, the RMSD for the 66 newly measured nuclei that appeared in the AME2020 of the LightGBM-refined mass.

Fig. 10Full-screen View

(Color online) Residual δ(Z,A)=B_LDM-B_exp predicted with LightGBM. The 10 input features are listed in Table 2. The training set consisted of 80% of the nuclear masses appearing in the AME2016 database, which are displayed in the area encircled in red. The neutron drip lines obtained by using the LDM, WS4, LDM_lgb, and Ma [59] are shown with different symbols. The upper subfigure shows the residual δ (Z,A) predicted with LightGBM but with only six input features, that is, excluding four magic number-related features.

Fig. 11Full-screen View

(Color online) Importance ranking for the input features obtained with the SHAP package. Each row represents a feature, and the x axis is the SHAP value, which shows the importance of a feature for a particular prediction. Each point represents a nucleus, and the color represents the feature value (with red being high and blue being low).

3.5　 Interpretability of the model

As a decision-tree-based algorithm, one advantage of LightGBM is its excellent degree of explainability. This is important because, as a physicist, one expecting the ML algorithm not only has a good performance in refining nuclear mass models, but can also provide some underlying physics that is absent from the original nuclear mass models. Understanding what happens when ML algorithms make predictions that could help us further improve our knowledge about the relationship between the input feature quantity and the predicted value. One of the possible ways to understand how the LightGBM algorithm provides a particular prediction is to appreciate the most important features that drive the model. For this purpose, SHapley additive exPlanations (SHAP) [63], which is one of the most popular feature attribution methods, was applied to obtain the contributions of each feature value to the prediction. Figure 12 illustrates the ranking of importance of the input 10 features. The top is the most important feature, while the bottom is the most irrelevant feature for predicting the residual δ(Z,A) between the experimental and theoretical binding energies. It can be seen that the importance ranks of the input features are different for different mass models. Because shell effects are not included in the LDM, the residual δ(Z,A) around magic numbers is usually larger (also can be seen in Fig. 4). As a result, |N-m| and |Z-m| are more important for predicting δ(Z,A) between the LDM calculation and experimental data. To demonstrate the meaning of the SHAP value, the residual δ(Z,A) obtained from the LDM and SHAP values are shown in Fig. 12. The upper panel of Fig. 12, around the magic number, i.e., |N-m| is close to 0, larger difference between LDM calculated and experimental binding energy is existed, especially for nuclei with larger neutron number. A very similar behavior for the SHAP value can be seen in the lower panel. This implies that by adding a |N-m|-related term in the LDM, the accuracy of the LDM for calculating nuclear binding energy can be improved to some extent. For FRDM, the neutron number N represents the most relevant feature, and the SHAP value for smaller N is usually larger. Indeed, the residual δ(Z,A) for nuclei with a smaller neutron number N is larger has already been observed in the FRDM paper, that is, Fig. 6 of Ref. [64]. In addition, we see that N_pair, Zm, and Nm are three of the most irrelevant features for predicting the residual δ(Z,A). We have checked that the enhancement of RMSD is ~5% when they are not considered. It is worth noting that the distribution of B_th-B_exp for nuclei with different Zm (Nm) are slightly different, implying that shell corrections on nuclei mass are different in different regions, which deserves further study with the inclusion of more relevant features.

Fig. 12Full-screen View

(Color online) Upper panel: Residual δ(Z,A) obtained from the LDM plotted against |N-m| colored by neutron number. Each point represents a nucleus, and the color represents the number of neutrons in the nucleus. Lower panel: Same as the upper one, but with the SHAP value plotted instead of the residual δ(Z,A).