ACE-formatted libraries and MCNP simulation details

The ACE library production and MCNP simulation processes are shown in Fig. 1. The NJOY program is a popular nuclear data processing program that can generate nuclear data in multiple formats for shielding and criticality calculations based on the evaluated nuclear data library. The JOYPI code can generate NJOY inputs for the development of the ACE library. The ACE format library is a continuous point section library for Monte Carlo program calculations and can be processed by using the NJOY program. The main NJOY modules used to make the ACE library include RECONR, BROASDR, THERMR, PURR, and ACER. The RECONR module performs the point cross-section resonance reconstruction function based on the ENDF file data. The BROADR module performs the temperature-related Doppler broadening function. The THERMR module generates the cross sections for free scatters in the thermal energy range. The PURR module calculates the unresolved resonance probability tables, and the ACER module converts the previously generated data into a c-type ACE file for use in MCNP.

Fig. 1Full-screen View

Production and verification process for the ACE-formatted libraries. MODER, RECONR, BROADR, HEATR, GASPR, THERMR, PURR, ACER are the modules of the NJOY 2016 code. JOYPI is an automatic NJOY input card generating program.

In the benchmark suite of the ICSBEP manual, some benchmarks contain thermal neutron scattering materials, and the corresponding thermal neutron scattering sub-library (TSL library) needs to be processed for calculation. For the new versions of ENDF/B-VIII.0, JENDL-4.0, and JEFF-3.3, there are corresponding TSL libraries, but for CENDL-3.2 and CENDL-3.1, no TSL library is provided. Therefore, to maintain the consistency of verification for different evaluated nuclear data libraries, we used the verified and publicly available data of the ACE thermal scattering library of JEFF-3.3 to perform criticality calculations for the benchmarks with thermal neutron scattering materials. The purpose of this study is to verify the overall criticality calculation performance of neutron data libraries. The standard neutron ACE library used in this study was based on different data libraries and was produced using the NJOY2016 program. If there are moderators in the benchmark facilities, the TSL library must be considered in the calculations. The TSL library used for the benchmark with moderators was from the JEFF3.3 ACE library.

For all assemblies calculated by MCNP, 3000 source neutrons were run per kcode cycle. For all metal assembly benchmarks, 40 inactive cycles and 360 active cycles were run. For the solution assemblies, 40 inactive cycles and 760 active cycles were run. These numbers ensure that sufficient active cycles are run to obtain good statistics for $k_{\text{eff}}$ calculations [17]. All these benchmarks use the total nubar data in the MCNP input. Eigenvalue uncertainties were < 70 pcm, which is approximately an order of magnitude lower than most benchmark uncertainties.

Description for the criticality benchmarks suite

A set of 100 criticality safety benchmarks was selected and established for the MCNP code. Benchmarks were obtained from two reports: a suite of criticality benchmarks for validating nuclear data [17] and an expanded criticality validation suite for MCNP [18]. Among the 100 benchmark cases, 88 were from the first report and 12 were from the second one. Although all benchmark cases have standard ICSBEP names, the names are too long to be displayed in the chart when describing them; therefore, they are usually represented by abbreviations. The abbreviations of the benchmark cases used in this study are consistent with those of the above two reports.

The fission nuclides ²³³U, ²³⁵U, and ²³⁹Pu produce the majority of fission products in the reactor. The criticality benchmark suite in this study is made up of five major categories based on the major fission nuclides: critical assemblies utilizing ²³³U, intermediate-enriched ²³⁵U (IEU), highly enriched ²³⁵U (HEU), ²³⁹Pu, and mixed metal (MIX) assemblies. Within each category, there were bare, reflected, and solution assemblies. The classification of the assembly and the number of each classification are listed in Table 1. The ICSBEP benchmarks and their abbreviations and the benchmark $k_{\text{eff}}$ reference values used in this study are listed in Table 8 in Appendix A, and the calculated $k_{\text{eff}}$ values and statistical errors of each data library are listed in Table 9.

Statistical analysis methods

The benchmark results were analyzed using the statistical parameters $\delta k/\sigma$, ${\chi }^{\text{2}}$, and $\langle \left|\Delta \right|\rangle$.

${\chi }^{\text{2}}$ is a statistical parameter used to determine which evaluated nuclear data library is the most suitable for criticality calculations. $\langle \left|\Delta \right|\rangle$ is the measure of the average difference between the calculated and benchmark $k_{\text{eff}}$ eigenvalues. $\delta k/\sigma$ indicates the consistency of the evaluated library and benchmark value in each benchmark case.

We use the 3σ rule to evaluate the calculation results of the benchmark. In statistics, the 3σ rule is a shorthand used to remember the percentage of values that lie within an interval estimate in a normal distribution: 68%, 95%, and 99.7% of the values lie within one, two, and three standard deviations of the mean, respectively. In empirical sciences, the 3σ rule expresses a conventional heuristic that nearly all values are taken to lie within three standard deviations of the mean, and thus it is empirically useful to treat 99.7% probability as near certainty. The results can be considered identical if the relative difference between the $k_{\text{eff}}$ eigenvalues and the benchmark values was within the ±3σ interval. Note that we have bolded the values exceeding ±3σ in each benchmark to facilitate identification in the table.

$\langle \left|\Delta \right|\rangle$ and ${\chi }^2$ are defined by $\langle \left|\Delta \right|\rangle ={\displaystyle \sum _{i=1}^n\frac{|k_{{\text{eff }}_i}^{\text{calculation}}-k_{{\text{eff }}_i}^{\text{benchmark}}|}{n}},$ (1) ${\chi }^2={\displaystyle \sum _{i=1}^n\frac{{((k_{{\text{eff }}_i}^{\text{calculation}}-k_{{\text{eff }}_i}^{\text{benchmark}})/{\sigma }_{{}_i}^{\text{benchmark}})}^2}{n}},$ (2) where n is the benchmark number, ${\sigma }^{\text{benchmark}}$ is the benchmark experimental uncertainty, $k_{\text{eff}}^{\text{calculation}}$ and $k_{\text{eff}}^{\text{benchmark}}$ are the simulated $k_{\text{eff}}$ eigenvalue and benchmark $k_{\text{eff}}$ eigenvalue, respectively, and i and n are the specific benchmark and total number of benchmark cases, respectively. $\delta k/\sigma$ was used to provide a confidence level for the benchmarks. The relative difference $\delta k$ and the relative combined statistical uncertainty $\sigma$ are defined by $\delta k=\frac{k_{\text{eff}}^{\text{calculation}}-k_{\text{eff}}^{\text{benchmark}}}{k_{\text{eff}}^{\text{benchmark}}},$ (3) $\sigma \text{=}\sqrt{{({\sigma }^{\text{benchmark}}\cdot k_{\text{eff}}^{\text{benchmark}})}^2\text{+}{({\sigma }^{\text{calculation}}\cdot k_{\text{eff}}^{\text{calculation}})}^2}.$ (4) ${\chi }^{\text{2}}$ and $\langle \left|\Delta \right|\rangle$ are lumped parameters that can describe the overall performance of the data library in the corresponding types of benchmarks. $\delta k/\sigma$ can locate the performance of each data library in each benchmark.

Comparison of the $k_{\text{eff}}$ calculation results

The calculated values were compared with the reference value (Figs. 2–6). Most of the calculated values were close to the allowable error interval of the experimental value. The analysis and discussion of each benchmark type are as follows:

Fig. 2Full-screen View

(Color online) $k_{\text{eff}}$ comparison for ²³³U assemblies.

Fig. 3Full-screen View

(Color online) $k_{\text{eff}}$ comparison for IEU assemblies.

Fig. 4Full-screen View

$k_{\text{eff}}$ comparison for HEU assemblies

Fig. 5Full-screen View

$k_{\text{eff}}$ comparison for ²³⁹Pu assemblies.

Fig. 6Full-screen View

$k_{\text{eff}}$ comparison for MIX assemblies.

(1) For the ²³³U assemblies (Fig. 2), the calculated values in most cases were in good agreement with the experimental values. The calculated values of the benchmarks of 23umt4b deviated significantly from the experimental values.

(2) For the IEU assemblies (Fig. 3), the calculated values in most cases were in good agreement with the experimental values. For the ieumt4 benchmark, all the calculated results except JENDL-4.0 were overestimated compared to the experimental value, and, for the ieumt6 benchmark, all the calculated results were underestimated compared to the experimental value.

(3) For the HEU assemblies (Fig. 4), all calculated values were overestimated compared to the experimental values in the umet3k benchmark. For the umet9b, usol13c, umet8, and umet15 benchmarks, all calculated values were underestimated compared to the experimental values.

(4) For the ²³⁹Pu assemblies (Fig. 5), the calculated values in most cases were in good agreement with the experimental values. For the four benchmarks of the pusl cases (pusl11a, pusl11b, pusl11c, and pusl11d), the calculation results of CENDL-3.2 and CENDL-3.1 were overestimated compared to those of other data libraries. After comparing individual nuclides to each other, we found that ²³⁹Pu of CENDL-3.2 caused the overestimation of pusl11 series cases.

(5) For the MIX assemblies (Fig. 6), the calculated values in most cases were in good agreement with the experimental values. The calculated values of mixmet8-7 deviate greatly from the experimental values.

Discussion of the statistical results

The results of the three statistical parameters of each data library (Tables 2–6) were compared in this study. The analysis and discussion are as follows:

(1) For the ²³³U assemblies (Table 2), CENDL-3.1 exceeded 3σ in the four benchmarks of 23umt4a, 23umt4b, 23umt5a, and 23umt5b. JEFF-3.3 exceeded 3σ in the flat23 benchmark. For the 23umt4b benchmark, all databases exceeded 3σ. By analyzing the values of ${\chi }^{\text{2}}$ and $\langle \left|\Delta \right|\rangle$, it can be concluded that CENDL-3.2 offers significant improvement compared to CENDL-3.1.

(2) For the IEU assemblies (Table 3), CENDL-3.1 exceeded 3σ in the benchmarks of lst7-14 and ieumt1c. Six benchmarks exceeded 3σ in the JENDL-4.0 library. Analysis of the values of ${\chi }^{\text{2}}$ and $\langle \left|\Delta \right|\rangle$ showed that ENDFB-VIII.0 and JEFF-3.3 performed better than the other libraries.

(3) For the HEU assemblies (Table 4), CENDL-3.1 exceeded 3σ in the benchmarks of umet19 and ieumt1c. JEFF-3.3 exceeded 3σ in the umet4b benchmark. Five benchmarks exceeded 3σ in the JENDL-4.0 library. Analysis of the values of ${\chi }^{\text{2}}$ and $\langle \left|\Delta \right|\rangle$ showed that CENDL-3.2 is not significantly improved compared to CENDL-3.1.

(4) For the ²³⁹Pu assemblies (Table 5), CENDL-3.1, and CENDL-3.2 both exceeded 3σ in the benchmarks of pusl11c and pusl11d. For the pumet8b benchmark, except for ENDF/B-VII.1, all other data libraries exceeded 3σ. Analyzing the values of ${\chi }^{\text{2}}$ and $\langle \left|\Delta \right|\rangle$ showed that the ENDF/B-VII.1 library performs better than the other data libraries.

(5) For the MIX assemblies (Table 6), all libraries exceeded 3σ in the benchmark mixmet8-7. The ENDF/B-VII.1 library performs better than the other data libraries. Analysis of the values of ${\chi }^{\text{2}}$ and $\langle \left|\Delta \right|\rangle$ shows that CENDL-3.2 performs better than CENDL-3.1.

Based on the calculated values of all benchmarks, we used chi-square and average errors to analyze the overall performance of all data libraries in criticality calculations (Table 7). From the perspective of the calculation results of the new and old updates of the library, CENDL-3.2 has smaller chi-square and average deviations for all benchmarks than CENDL-3.1, which reflects that, for the type of benchmarks involved, CENDL-3.2 is superior to the previous version. The average deviation of ENDF/B-VIII.0 for all benchmarks is similar to that of ENDF/B-VII.1, both being near 230, and the chi-square is larger than that of ENDF/B-VII.1. By counting the number of benchmarks exceeding 3σ, it can be seen that the number of benchmarks exceeding 3σ for JENDL-4.0 and CENDL-3.1 is >10 and the number of benchmarks exceeding 3σ for ENDF/B-VIII.0 and JEFF-3.3 is ~5.

In general, based on the above 100 criticality benchmark calculation results, we can conclude that the criticality calculation performance of CENDL-3.2 is better than that of CENDL-3.1. Analysis of the chi-square and average deviations and the number of benchmarks exceeding 3σ demonstrates that the overall criticality calculation performance of ENDF/B-VIII.0 and JEFF3.3 is equivalent. The ENDF/B-VII.1 library performed the best. Compared with the JEFF-3.3 and ENDF/B-VIII.0 libraries, CENDL3.2 performs better in the calculation of ²³³U devices and performs poorly in the pusl11 series case calculation of Pu devices, and thus further improvement is needed.

Compared with CENDL-3.1, CENDL-3.2 [1] has more materials (272), of which the data of 134 nuclides are new or updated evaluations in the energy region of 10⁻⁵ eV to 20 MeV. Data for most of the key nuclides in nuclear applications (e.g., U, Pu, Th, and Fe) have been revised. Covariance data for the main reactions were added for 70 fission product nuclides in CENDL-3.2. Compared with ENDF/B-VII.1, ENDF/B-VIII.0 [2] employs major changes for neutron reactions on the important isotopes ¹H, ¹⁶O, ⁵⁶Fe, ²³⁵U, 238U, and ²³⁹Pu and other nuclides that impact simulations of nuclear criticality. The number of materials increased from 423 to 557. In addition, neutron reactions on light nuclei, structural materials, minor actinides, fission energy release, decay data, and charged particle reactions and thermal neutron scattering data have been notably updated.

Therefore, we expect that the criticality calculation performance of ENDF/B-VIII.0 is better than that of ENDF/B-VII.1. However, for different benchmark experiments, the calculation results of the new evaluated library ENDF/B-VIII.0 are not all better than those of the previous version, such as the three benchmarks of pusl11a, pusl11b, and pumet21b in the Pu assemblies. In the criticality calculation of these three benchmarks, compared with the experimental value, the $k_{\text{eff}}$ results of ENDF/B-VIII.0 are 200–500 pcm smaller than the calculation results of ENDF/B-VII.1. This is also one of the goals of our work, that is, to find benchmarks that are not sensitive to the newly evaluated library and provide a reference for subsequent evaluation work. The calculation results of Table 7 list only the statistical performance of the current benchmark cases. The results in Table 7 cannot explain that the calculation result of ENDF/B-VIII.0 must be better than the calculation result of ENDF/B-VII.1. This requires a specific analysis based on specific issues.

Discussion of the anomalous benchmarks

From the discussion in Sect. 3.2, the $\delta k/\sigma$ value of all data libraries exceeded 3σ in the three benchmarks of 23umt4b, pumet8b, and mixmet8-7, which indicates that there is a large deviation between the experimental and calculated values. This question needs to be addressed separately.

The 23umt4b (u233-fast-met-004) benchmark is a spherical device, divided into two layers: The inner layer contains ²³³U fuel and the outer layer is a reflective layer dominated by tungsten. The pumet8b (pu-met-fast-008b) benchmark is a spherical device, divided into two layers: The inner layer contains Pu fuel and the outer layer is a reflection layer of ²³²Th. The mixmet8-7 (mix-met-inter-008-case-7) benchmark is based on a k_∞ measurement. The configuration consists of a rectangular plate with three rectangular normal-uranium plates above it and the other three below it. The plates were enclosed on all sides of a rectangular steel sheath.

Physically, $k_{\text{eff}}$ is mainly dominated by fission nuclides, but the reflective layer also has an important influence on $k_{\text{eff}}$. The average deviation between the pumet8b and 23umt4b benchmarks and the experimental values was ~600 pcm. In fact, there are benchmarks that have a deviation of >600 pcm from the experimental value, such as the JEFF-3.3calculation result for pumt21b. For the mixmet8-7 benchmark, the $k_{\text{eff}}$ deviation between the calculated value of all libraries and the experimental value was within 1000–2000 pcm. In addition, the $k_{\text{eff}}$ results of the reference values from ENDF/B-VII.0 and ENDF/B-VI also deviate from the experimental values by >1000 pcm. The $k_{\text{eff}}$ result of MCNP6.2 based on ENDF/B-VII.1, found in the LA-UR-17-25040 report [19], is 1.0192, which has a deviation of 1620 pcm from the experimental value. At present, no report has been published to explain the reason for the large deviation between the calculated and experimental values for mixmet8-7, and further research is needed.

From the point of view of nuclear data, because the benchmark involves many nuclides, it is impossible to determine which nuclides that have a greater impact on $k_{\text{eff}}$ through qualitative analysis. The commonly used method is to determine the important reaction channels of key nuclides that have a greater impact on $k_{\text{eff}}$ through sensitivity and uncertainty analyses. This part of the work has not been done yet, and the sensitivity and uncertainty analysis will be further studied in our future work.