Devices under test

High-performance Xilinx XC7K325T FPGAs were chosen as the devices under test (DUTs), which have been widely used in cost-sensitive applications to achieve high-end connectivity, bandwidth, and signal processing capability [24]. A 28 nm bulk silicon process with a high-k metal gate was utilized in the DUT. All programmable routings or switches in the DUTs are controlled by CRAMs, which can achieve complex functions to satisfy the requirements of different space missions after rational configurations.

The CRAMs are formally arranged into “frames” (rows) and “bits” (columns) for Xilinx families, and the specific functions can be realized by configuring the customized bitstreams generated by the Vivado software into the CRAM part. The DUT has 22546 frames (including seven dummy frames) and 72846048 bits (i.e., 101 32-bit words for each frame). All frame data can be read back and compared with the golden bitstream, and the error data as well as the logical locations (frame# and bit#) are extracted after the comparisons.

Test system

A test system was developed to capture the irradiation vectors of the DUT. The test system consisted mainly of a motherboard and daughterboard. The DUT on the daughterboard was irradiated during the tests. The main control FPGA soldered on the motherboard was responsible for reading the contents of the DUT, transmitting commands, and identifying functional failures. Both the motherboard and daughterboard need a 12 V DC power supply. The high-speed interface was utilized to connect the two boards, and the configuration time was less than 0.5 ms. Additionally, we designed software on the host computer to control the test system. After the beam is turned off, the motherboard executes bitstream comparisons and uploads the results to the main control FPGA. The detailed information on the upsets and types of hard errors were captured and saved as log files for further analysis.

Irradiation parameters

Irradiation tests were performed at the Heavy Ion Research Facility in Lanzhou (HIRFL) at the Institute of Modern Physics, Chinese Academy of Sciences. High-energy ⁷⁸Kr ions and medium-energy ¹²⁹Xe ions were used to irradiate the DUT. The ¹²⁹Xe ions were accelerated using a sector separator cyclotron. The ⁷⁸Kr ions were from the cooling storage ring (CSR), and this was the first available high-energy single-event experiment of the CSR heavy-ion accelerator. To meet the ion range requirement, the substrate of the DUT was thinned to approximately 50 μm. It should be noted that the tests were performed under the nominal temperature (20 ℃). The standard core voltage of the DUT was 1.0 V. Moreover, several aluminum foils were used to achieve multiple linear energy transfer (LET) values. The DUT was fixed on a 5-dimensional platform to change its position and rotate the tilts (θ, ψ), as shown in Fig. 1.

Fig. 1Full-screen View

Ion beam with azimuth angle θ and zenith angle ψ incident on the DUT surface

Additionally, many testing trials were performed at a low fluence to reduce the occurrence of MCUs induced by several heavy ions, and the detailed explanations are presented in the next section. The experimental conditions and parameters are listed in Table 1. In both the ⁷⁸Kr and ¹²⁹Xe experiments, the number of upsets in each run was controlled at less than 400. Any trials with upsets exceeding 400 were discarded from the dataset. In our testing scheme, static modes were used to radiate the DUT. The static modes start with the configuration of our DUT with a bitstream and then with the beam on. During the radiation tests, the circuits inside the DUT do not work because the static modes do not rely on clock driving. Heavy ions only affect the configuration functionality of the circuits, that is, the CRAM cells. After reaching the preset heavy-ion fluence, the beam was turned off, and the radiated and golden bitstreams were compared to identify the upsets. To date, only a single testing trial has been completed. Before the next static trial, the configuration must be restarted to restore the functionality of the DUT. By repeating this operation, multiple trial results were obtained. Static modes have the benefit of preventing undesired occurrence of functional errors and mitigating the rates of burst errors during the reconfiguration process [25], which improves the efficiency of our MCU extractions. The heavy-ion testing platform of HIRFL can achieve accurate flux as well as control the preset fluence.

FMCUs

“False MCUs (FMCUs)”, typically defined as MCUs induced by several particles rather than single ones, significantly affect the determination of correct MCU results. Therefore, before irradiation tests, it is necessary to propose a test principle that prevents the occurrence of FMCUs as much as possible.

According to [26], the probability of FMCUs can be $P_k\left(n,p\right)\left(\text{collision}\right)\approx 1-\text{exp}\left(-\frac{p\left(p-1\right)\left(2k-1\right)}{2n}\right)\mathrm{.}$ (1)

In this equation, p is the number of upsets in each radiation test, k is the collision range, which was set to 32 (the length of a word in the Kintex-7 FPGA), and n is the number of bits in one specific memory. It can be observed that the probability of the FMCUs depends only on the number of upsets. Eq. (1) shows that when there are a few upsets, the probability of the FMCUs can be sufficiently low. This is the reason why we separated our tests into multiple rounds as illustrated in Sect. 2.3. A more specific approach to obtain as few upsets as possible is to control the fluence of the heavy ions in each round of the radiation test. In the analysis of the upsets, the minimum and maximum numbers of upsets were 86 and 272, respectively. Based on Eq. (1), the minimum and maximum probabilities of the FMCUs in our radiation designs were approximately 0.3% and 3.0%, respectively, which proved the statistical property of this calculation.

MCU extraction

MCUs can be extracted by determining the differences in the positional relationships between the actual upset data and theoretical predictions. Therefore, the first important task was to create a predictive model. In this model, only SBUs exist, referred to as only-SBU systems [22], where the relationships between single upsets show a specific frequency distribution. However, the actual radiation violates the above assumptions because heavy ions can affect more than one cell simultaneously to generate the MCUs. The charge-sharing effect is another essential factor in MCU generation. More specifically, because MCUs are generated by single particles, there must be a certain rule between the upsets because it is determined by the statistical spatial and temporal nature of the ion beams and the specific MCU sensitivities of devices. Hence, the relationships between the real upsets must exhibit different frequencies from the theoretical only-SBU predictions; that is, the relationships appearing more frequently than the predictive ones can be identified as being caused by the MCUs.

Before constructing the predictive model, we first assume that there is a rectangular space of length X and width Y, and this space contains N elements, where N is equal to X×Y. The elements in this space can be included by a set named R, where R = {A₁, A₂, A₃, …, A_N}, and A_i is the i_th arbitrary element in R. Each element A_i in this space has its unique coordinate representation, labelled as (x_i, y_i) formally like a position in the X-Y plane. Hence, the coordinates of all elements in this space can be defined from the bottom left (1, 1) to the top right corner (X, Y).

In fact, there are specific relationships between the elements in this space. To identify the relationships between them without repetition and loss, we should first organize them in increasing order, and the principle is given by

A_i = (x_i, y_i), A_j = (x_j, y_j) and

if

x_i < x_j

or

x_i = x_j and y_i < y_j

then A_i < A_j.

Subtraction can be used to determine the unique relationships among these elements. Hence, we can formally obtain several offsets, such as (Δx, Δy). We then define a set named the offset set (OS) that contains all these offsets. It is worth noting that the Δx of each offset in the OS must be equal to or greater than 0, and the contents of the OS can be determined by $\text{OS}=\left\{\left(A_j-A_i\right)|A_i<A_j, A_i,A_j\in R\right\}$ (2)

To illustrate the subtraction process more clearly, we used a 2×3 array named S as an example. The coordinates of all the six elements and their subtracted offsets are listed in Table 2.

In the example illustrated above, X and Y are selected as 2 and 3, respectively. We then return to the Kintex-7 FPGA. As illustrated in Sect. 2.1, the CRAM bits are arranged in rows and columns in the Xilinx FPGAs. Therefore, in the N-bit CRAM space mentioned above, X and Y were 22539 and 3232, respectively. Using Eq. (2), we can infer that the ranges of x and y of the offset (x, y) are from 0 to 22538 and -3231 to 3231, respectively, and they are both integers. The offsets (x, y) in the N-bit CRAM space are arbitrary combinations of all x and y within the above ranges; however, these offsets must meet the increasing order rules exactly at x = 0. Thereby, there is a total of N(N-1)/2 offsets with repetition. For an arbitrary offset (x, y), the probability that it appears in the OS is determined by $p\left(x,y\right)=\frac{2}{N\left(N-1\right)}\left(Y-\left|y\right|\right)\left(X-x\right).$ (3)

After each radiation test, N_u upsets (N_u << N) were observed. A set named U_n containing all the N_u upsets with increasing order is then created to calculate the new OS, and the next significant step is to determine the predictive frequency distributions. For these N_u upsets in an individual radiation trial, the number of offsets in the new OS is given by (4) to obtain N_OS. $N_{\text{OS}}=1/2\cdot N_u\cdot \left(N_u-1\right)$ (4)

In our predictive model, because we assume that these upsets are irrelevant to each other, the probability of an arbitrary offset (x, y) appearing m times in the total N_OS trials can be calculated using the classical binomial probability distribution $p_{x,y}^{N_{\text{OS}}}\left(m\right)=\left(\begin{array}{c}{N}_{\text{OS}}\\ m\end{array}\right)\cdot p{\left(x,y\right)}^m\cdot {\left(1-p\left(x,y\right)\right)}^{\left(N_{\text{OS}}-m\right)}$ (5)

Then, under this prediction, the number of offsets that appear m times in N_OS trials is simply the sum of all the conditions; that is, the probability of each offset (x, y) multiplied by its appearing times N_x,y in the new OS. $N\left(m\right)={\displaystyle \sum _{\left(x,y\right)}{N}_{x,y}{p}_{x,y}^{N_{\text{OS}}}}\left(m\right)$ (6)

To ensure that the extracted MCUs have more than 95% confidence, we set 0.05 as the maximum value of Eq. (6) to determine the cut-down value, labelled as m.

We illustrate the above extraction process using an example. In the case of the Kintex-7 FPGA, based on Eq. (2) and Eq. (3), we can first determine the probabilities of all offsets {(0, 1), (0, 2), …, (0, 3231), (1, -3231), (1, -3230), …, (1, 3231), (2, -3231),…, (22538, 3231)}. After a single radiation test, we observed 214 upsets, and the N_OS was 22791. According to Eq. (5), we can determine the probabilities of all the actual obtained offsets (those offsets are absolutely the subset of the above offsets) that are in the new OS appearing m times. We then sum the above probabilities of all offsets multiplied by their appearing times in an individual test round to determine the number of offsets appearing m times. By letting the left-hand term in Eq. (6) be equal to 0.05, we obtain a cut-down value. In this case, the cut-down value m was 2.9. Thus far, we have completed the single process of obtaining one cut-down value. Then, a question arises: How do we identify the offsets that are caused by MCUs? The answer is that offsets that emerge more frequently than 2.9 times are extracted as true offsets that are induced by MCUs, because they violate the irrelevance between upsets, that is, only-SBU theoretical prediction. By repeating the above process, we obtain many cut-down values, which rely on the number of rounds of the static tests performed. However, if we carry out too many rounds of tests, the calculation and comparison processes are bound to be huge. To address this problem, two rules are defined below to make MCU extraction more convenient and accurate.

A. Cut down rule

This rule helps extract the MCUs more conveniently. Because of the specific physical layout arrangement and the MCU sensitivities of devices, after the cutdown values of all the testing rounds are confirmed, these values and the occurrence numbers of the same offsets obtained from multiple rounds can be added separately to create a histogram containing all the possible offsets. Taking two rounds of radiation tests as an example, their cut down values are calculated to be 2.9 and 3.1, and offset (1, 0) occurs 30 and 39 times, respectively. We can then add them separately. Finally, the cut down value is 6.0, and the offset (1, 0) occurs 69 times. Based on this rule, the offset results of the ⁷⁸Kr and ¹²⁹Xe irradiations are shown in Fig. 2. Offsets that occurred above the cut down (dashed red and blue lines in Fig. 2) were identified to discriminate the MCUs, and the remaining offsets were discarded.

Fig. 2Full-screen View

(Color online) Results of offsets caused by ⁷⁸Kr and ¹²⁹Xe ions

B. Repetition rule

Here, we define another principle called the repetition rule. This rule is available for offsets that do not exceed the cut-down value. For example, during the analysis, some exclusive MCUs presented in Table 3 appeared at least 2 times in an individual round. Because the probability of the same-shape MCUs appearing more than two times can be calculated to be very low, the offsets marked in red were reconsidered and added to the true offset group. This rule helped recover anomalous offsets that were not in the OS, as confirmed by the cut down rule.

MCU extraction begins with the construction of 2-bit MCUs; then, the larger MCUs need to be further identified based on the 2-bit MCU results by determining if the offsets between them are included in the confirmed OS. This procedure is iteratively performed until the offsets of the maximum MCUs go against the OS.

Results analysis

Fig. 3 shows the detailed SBU and MCU results of the ⁷⁸Kr and ¹²⁹Xe irradiation tests. For the ⁷⁸Kr ions, the cross-sections of the SBUs and MCUs were almost the same. However, the MCUs dominated the ¹²⁹Xe ion-induced upsets, which demonstrated a high-energy deposition ability and a strong charge sharing effect of the ¹²⁹Xe ions. Moreover, for both tests, significant increases in the cross-sections of the angular irradiations were observed, indicating that the different ion trajectories can affect the MCU characteristics of the 28 nm SRAM-based FPGAs.

Fig. 3Full-screen View

(Color online) Detailed SBU and MCU results of (a) ⁷⁸Kr ion and (b) ¹²⁹Xe ion irradiation

Based on Fig. 2 and Table 3, the extracted logical sizes and shapes of the MCUs are presented in Table 4. We obtained diverse (up to 9-bit) MCU patterns based on the selection of the worst irradiation parameters; these MCU data can support an effective radiation hardening process under a harsh space environment. It is important to note that, because of the statistical distribution nature presented in this paper, this method can be applied to arbitrary memory-based devices. If other types of FPGAs are irradiated, the MCU results can be easily obtained by changing X and Y in Eq. (3). For the same generation of Xilinx FPGAs, owing to the same process technologies, the MCU information presented in Table 4 can be directly used to distinguish the MCU-induced cross-sections and failures.

The percentages of MCUs of different sizes are presented in Fig. 4(a) (for ⁷⁸ Kr ions) and Fig. 4(b) (for ¹²⁹Xe ions). It is clear that for the high LET ¹²⁹Xe ions, approximately 60% MCUs were the major problem for aerospace electronic systems. In Figs. 4(a) and 4(b), an apparent percentage dependence of the large (more than 3-bit) MCUs on the LET and tilts was observed, and the good consistency indicated that the MCU distinction method is suitable for high-density FPGAs. Under the same LET, all large offsets (i.e., large MCUs) were generated under tilted incidence, which also proved the effectiveness of the MCU extraction method.

Fig. 4Full-screen View

(Color online) Percentages and sizes of MCUs induced by (a) ⁷⁸Kr ions and (b) ¹²⁹Xe ions

Additionally, for the ¹²⁹Xe ion irradiation, 7-bit MCUs were observed in the (90°, 30°) directions; however, under the (0°, 30°) incidence, the offset (1, 4) helped identify the largest 8-bit MCUs. For θ or ψ varying from 0° to 45°, the percentages of the largest MCUs increased by 5-6 times, demonstrating that there are indeed different layout distances in the different directional irradiations. Moreover, large (more than 4-bit) MCUs appeared more frequently when ψ was equal to 30°, which identified worse tilted irradiation effects. The MCU results verified the impacts of the different sensitive region arrangements on the extracted MCU orientations, and the discrepancies of the cross-sections in different incident directions must be considered to obtain the worse and more accurate MCU responses.

Although the SBUs and 2-bit MCUs occupied almost the entire ⁷⁸Kr upset dataset, when the LET was 13.3 MeV/(mg/cm²) and the tilt was (0°, 30°), 9-bit MCUs were observed twice in an individual irradiation round. Compared with the maximum 8-bit MCUs found in the ¹²⁹Xe tests with the same (0°, 30°) incidence, the ⁷⁸Kr ion-induced larger MCUs revealed that although the high-energy heavy ions have a much smaller LET, the generated charges can still exceed the critical charge of the 28 nm FPGAs. Therefore, the ⁷⁸Kr ions have more severe effects than the ¹²⁹Xe ions, even with a considerably higher LET.

In fact, heavy-ion energy has a significant effect on MCUs. During the interaction of heavy ions with devices, a region formally like a cylinder, called the ionization track, is created along the path of heavy-ion incidence to deposit energy in the memory cells. There are two important parameters in this region. One is the radius, which identifies the range that the heavy ions can affect, and the other is the density of electron-hole pairs inside the ionization track, which determines the collected charge. Any cells within this track will be upset once their collected charge is equal to or greater than the critical charge. As reported in [27], heavy ions with higher energies have wider ionization tracks; this is because they can transfer more energy to the target atoms by the direct ionization process. These secondary atoms have higher kinetic energy that affects the subsequent atoms, finally generating a wider ionization track. In our testing scheme, because the ⁷⁸Kr ions at the LET of 13.3 MeV/(mg/cm²) had considerably higher energy than the ¹²⁹Xe ions, their wider ionization track can affect more cells. However, it should be noted that compared to the ¹²⁹Xe ions, the density of the electron-hole pairs inside the ionization track of the ⁷⁸Kr ions is lower. This is because the ⁷⁸Kr ions have a smaller LET (LET corresponds to the density of the electron-hole pairs). Therefore, the ⁷⁸Kr ions have a smaller probability of inducing cells to be upset exactly at the edge of the ionization track, which exceeds the critical charge of the 28 nm technology. However, in modern scaling devices, because the critical charge of cells is sufficiently low, these cells are more prone to be upset to generate large MCUs, which is the reason why we observed the largest 9-bit MCUs. Therefore, to avoid underestimating the MCU sensitivity, high-energy heavy ions are necessary for the evaluation of advanced devices.

We further compared the offset counts between the ⁷⁸Kr and ¹²⁹Xe ions in Fig. 2 and 5 to better understand the more serious influences of high-energy heavy ions. As shown in Fig. 2, the number of offsets (1, 1), (1, 0), (0, 1), (1, -1), (0, 2), and (1, 2) exceeded the separate cut-down values for both the ⁷⁸Kr and ¹²⁹Xe heavy ions. Although the number of offsets of the ⁷⁸Kr irradiation was less than that of the ¹²⁹Xe ions, except for (2, 0), (2, -1), (2, 1), (3, 0), and (3, 1); the worse effects of the ⁷⁸Kr ions were determined by finding a comparable number of offsets to the ¹²⁹Xe ions.

Fig. 5Full-screen View

(Color online) Count comparison of offsets between ⁷⁸Kr and ¹²⁹Xe ions, we zoomed in on the right part of Fig. 2

In Fig. 5, the offsets (2, 0), (2, -1), (2, 1), (3, 0), and (3, 1) of the ¹²⁹Xe ions appeared more frequently than those of the ⁷⁸Kr ions. To determine the causes of this phenomenon, Fig. 6(a) illustrates the percentages of the large lateral offsets of the ¹²⁹Xe radiation tests at various tilts. It can be observed that the offset (2, 0) percentages were almost the same (approximately 15%) for all the conditions, indicating that the ¹²⁹Xe ions can simultaneously affect cells with a logically lateral distance of 3. Additionally, as ψ increased, the remaining offsets appeared more frequently, proving that large lateral offsets were indeed induced by θ = 90°. Because we only irradiated our DUT with high-energy heavy ions in the θ = 0° direction, these large lateral offsets rarely occurred.

Fig. 6Full-screen View

(Color online) Percentages of large offsets at incidences of (a) θ = 90° and (b) ψ = 30° in ¹²⁹Xe radiation tests

When radiating the DUT in various θ directions, a similar conclusion can be made from Fig. 6(b); that is, the increased azimuth angles can induce larger offset percentages in the longitudinal orientation. When comparing the offset distances between (90°, 30°) and (0°, 30°) in Fig. 6, the largest offset (1, 4) revealed the worst MCU sensitivities when θ was equal to 0°, which is in good agreement with the largest 8-bit MCUs found in the ¹²⁹Xe radiation tests at (0°, 30°). The offsets in the longitude were larger than those in the lateral direction, demonstrating that the word direction was more sensitive than that between words.

As mentioned above, because irradiations under θ = 0° can cause the largest longitudinal offsets, the numbers of offsets (1, 2), (1, -2), (0, 3), (1, 3), (0, 4), and (1, 4) of the ⁷⁸Kr data were almost the same as that of the ¹²⁹Xe ions. It can be inferred that high-energy heavy ions in the rotated θ directions generate larger MCUs (more than 9-bit) by creating larger lateral offsets.

For modern scaling devices, the energy, LET, tilts, and irradiation directions are becoming serious issues in irradiation evaluations. Under tilted incidence, it is predicted that, as the ionization ability of high-energy heavy ions increases, the sizes and proportions of the MCUs will dramatically increase, which will seriously affect the function of FPGA-based on-orbit systems, which must be considered. Additionally, based on the worse 9-bit MCUs observed in the high-energy tests, high-energy heavy ions are recommended to fully characterize the MCUs during ground-based accelerator radiation tests. With the help of the general MCU distinction methods, more serious MCU impacts were accurately determined according to the worse irradiation evaluation. Moreover, as illustrated in Fig. 6, the selection of various directional evaluation schemes affects both the OS content and MCU features owing to the different ion energy transfer trajectories. Therefore, diverse MCUs in different directions should also be considered by hardening designers.