1 Introduction

A number of researchers have reported that the analysis of the evolution behavior of metals or alloys exposed to the complex and extreme environment in a reactor is a considerably exigent task [1–3]. The irradiation-induced damage to nuclear reactor components is one of the major problems that extremely detrimental to nuclear power generation because of the high cost of component replacement and the introduction of uncertainty in predicting the lifecycle of components. Neutron interactions with the material lead to atomic collisions that create point defects and defect clusters. A problem of particular interest is the accumulation of nanoscale helium bubbles in nickel-based superalloys used in nuclear reactors. Helium is mainly produced through the interaction of thermal neutrons emitted by the reactor core with nickel atoms when collisions occur. The nuclear reaction channel indicates that the nickel nucleus can crack into two nuclei of iron and helium after the absorption of one neutron. The presence of transmutation-generated helium has a critical role in the microstructural evolution of metals exposed to neutron irradiation [4–6]. When the concentration of helium atoms exceeds a certain threshold, helium coalesces into bubbles in the microstructure. Moreover, nano-sized helium bubbles tend to appear and develop in the matrix, dislocations, and grain boundaries of the material, leading to swelling, hardening, or embrittlement [7–9]. Helium bubbles in metals have a complex evolution mechanism during service life of the material that is crucial to reactor safety. To prevent the decline in the performance of metals or alloys caused by helium bubbles, gaining insight into the dynamic evolution of helium bubbles, especially their movement and aggregation, is critical. To accomplish the foregoing, the use of a suitable technique to identify and analyze the microstructure of helium bubbles is essential.

Several experimental tools including small-angle neutron scattering [10–12], positron annihilation spectrometry [13–15], grazing incidence small angle X-ray scattering, and transmission electron microscopy (TEM) are available to study the behavior of helium bubbles in metals or alloys [16–18]. Among these tools, TEM is typically employed to capture helium bubble images for investigating their morphological evolution at the nanometer scale spatial resolution [19, 20]. Technically, bubbles images appear as white and black dots at the same location under under-focused and over-focused imaging conditions, respectively; hence, they are clearly distinguishable. However, owing to the powerful magnification in microscopy, the helium bubble images acquired by TEM are extremely sensitive to the quality of sample preparation and exposure setting. A typical TEM image in which the artifact caused by the mixed foreground and background and the non-uniform distribution of illumination intensity are easily differentiated is shown in Fig. 1. However, in the current practice, because the location, size, and density of helium bubbles are marked and analyzed manually by researchers, the task of characterizing the helium bubbles remains labor-intensive and prone to error. Presently, sample preparation and helium bubble marking in the image require days or even weeks. The analysis results are susceptible to inaccuracies due to several factors, such as visual fatigue during marking, subjective human judgment, uncontrollable image quality, and bubbles with irregular shapes.

Fig. 1Full-screen View

Typical TEM image of helium bubbles

To facilitate the characterization of helium bubble morphology, an image processing algorithm or a machine vision system may be introduced as an alternative to automatically count and identify the bubbles in TEM images. By exploiting electronic engineering and machine intelligence, the tedious task performed by humans can be easily accomplished. The necessity of detecting and characterizing objects, such as bubbles and cells from images, has been observed in various industrial fields [21]; generally, shape-based and template-based approaches are employed. In the former, object shapes are normally presumed to follow certain shapes, such as circles or ellipses. This assumption allows the use of shape detectors, such as the Hough transform, to project the candidate points inside circles in the image into the parameter space of center coordinates and the radius of circles [22]. As the number of objects to be identified increases, these shape detectors generally have high computational consumption and require large memory storage. Moreover, small targets are typically undetected because the number of points is insufficient to cluster them in the parametric space. Both problems are associated with helium bubble images captured by TEM. In contrast, the template-based method relies on the sliding window technique rather than voting for each point on the circle. This means that a patch window containing the pattern of the object of interest is convoluted across all pixels in the image. The result with the highest similarity to the pattern is selected as the target object. This approach utilizes all the pixel information in the template to identify the location and shape of all objects of interest without the necessity of using high-frequency filters to extract the edges of objects. As a result, the robustness of detection is improved [23]. Similarly, the kurtosis wavelet energy [24], kurtosis curvelet energy [25], and hidden Markov random field-expectation maximization [26] are three efficient methods that may be implemented to analyze the statistical characteristics of helium bubbles. However, if the electronic lens magnification or the illumination conditions are altered during image acquisition, or if the objects are deformed or partially occluded, then the template-based approach may experience difficulty in correctly matching candidates.

As the shape and size of helium bubbles continue to evolve in the course of the investigation, sample preparation and image acquisition become difficult to control to achieve satisfactory reproducibility. Moreover, the image features of helium bubbles actually deviate from the hypothesis of traditional image processing approaches. This limits the applicability of these traditional techniques for automatic helium bubble detection. To resolve this, a more delicate machine learning approach has been proposed to augment the helium bubble identification process. The method is capable of achieving accuracy and reproducibility comparable to that accomplished by humans but requires a considerably less amount of time to generate the TEM data [27]. By combining traditional image processing and the latest convolutional neural network techniques, objects can be detected with improved exactness [28, 29]. To further enhance the detection precision, large datasets with qualified annotations are normally required for training and testing deep neural networks. However, although training using synthetic data has been employed in biological experiments, the complexity of the procedure involving TEM images may preclude the possibility of utilizing simulated images [30-34].

In contrast to the deep learning approach, where learning numerous labeled real or synthetic images is generally required to establish an accurate classifier, this paper proposes a new machine learning method. This proposed technique is implemented to automatically identify and analyze helium bubbles in the TEM images of nickel-based alloys implanted with helium ions at high temperatures. The images were processed and analyzed using the density-based spatial clustering of applications with noise (DBSCAN) algorithm and Gaussian Mixture Model (GMM). After determining the key parameters of the model, the optimized model could be used to identify and count the helium bubbles in the TEM images of the nickel-based alloy. In case only a few marked TEM images were available for training, the performance achieved by the proposed approach is comparable to manual identification and analysis of helium bubbles.

This paper is mainly composed of three parts. Section 1 describes the research background of the identification and analysis of helium bubbles. Section 2 presents the proposed analysis methods as well as the details of the datasets used for modeling and training. Section 3 elaborates on the experimental results obtained using the proposed model; further discussions are also presented. Finally, the paper concludes with a brief summary.

2 Method

2.2 Statistical analysis of helium bubbles

In this study, the helium bubble TEM image was processed and analyzed in three steps: First, the image was preprocessed, eliminating the background and noise pixels from the image. Second, the remaining pixels were clustered by DBSCAN. Finally, the GMM was applied to analyze the helium clusters. A schematic of the entire model and a detailed description of the implementation steps are shown in Fig. 2.

Fig. 2Full-screen View

Schematic of designed model

2.2.1 Background and noise pixel elimination

The pixel matrix of the helium bubble image has three channels, each with different intensity values. To afford utmost convenience in data processing and maintaining the three-dimensional signal information, merging the three channel datasets into a single channel dataset is more advantageous. The intensity of each pixel is converted into a single value using the following:

$gray =0.229\times R+0.587\times G+0.114\times B,$ (1)

where gray represents the combined intensity value of the components of the three channels; and R, G, and B are the red, green, and blue channel intensities, respectively.

Before attempting to remove the background intensity from the image matrix, the background threshold should be defined. The helium bubbles are assumed to be evenly distributed in the TEM image and have different intensity depths from the background. The matrix data of helium bubble intensities are considerably distinguishable from the background, and the most appropriate threshold value is derived by the training process.

After determining the background threshold value, the image matrix can be processed using the following:

$New image\left[i,j\right]=\{\begin{array}{c}0, if image\left[i,j\right]\le v_{\text{th}}\\ image\left[i,j\right]-v_{\text{th}}, if image\left[i,j\right]>v_{\text{th}}\end{array}$ (2)

where $v_{\text{th}}$ corresponds to the threshold of background.

The background pixel value of the helium bubble image has a non-uniform distribution owing to the non-uniform illumination intensity; hence, determining an appropriate background threshold is difficult. To resolve this, the histogram equalization technology is introduced to adjust the helium bubble brightness against the background. This approach enhances the imaging contrast and facilitates the identification of helium bubbles.

Histogram equalization is a simple and effective image enhancement technology for improving the contrast of an image with small dynamic range intensity. It alters the grayscale value of each pixel to distribute the intensity histogram across the entire intensity range; consequently, the new image tends to be clearer. For example, the grayscale level of an overexposed image is concentrated in the high brightness range, whereas that of an underexposed image is concentrated in the low brightness range. Through histogram equalization, the histogram of an original image can be transformed into a uniformly distributed (equalized) form. This transformation increases the dynamic range of the grayscale value difference among pixels to enhance the overall image contrast. In other words, histogram equalization enhancement is achieved by compressing the grayscale value with numerous pixels (i.e., the grayscale value that performs a major function in the image) and expanding the grayscale value with a small number of pixels (i.e., the grayscale value that does not have a primary function in the image). The original bright-field under-focused TEM image and the results of histogram equalization operation are shown in Fig. 3. The separation of the helium bubble from the background is evidently more difficult in some regions, especially in the upper-left area. Nevertheless, the image contrast is significantly improved after performing histogram equalization on the original TEM micrograph. Thus, to a certain extent, preprocessing through histogram equalization can aid in setting a proper threshold value for distinguishing helium bubbles from the background.

Fig. 3Full-screen View

(a) Original TEM image and (b) TEM image after histogram equalization processing

2.2.2 Clustering

Clustering analysis and detection, which are known to have led to the rapid development of data mining techniques, are widely applied to fields, such as pattern recognition, data analysis, market research, and image processing. As an unsupervised machine learning algorithm, its basic principle is to group similar objects into the same cluster. The more similar the objects in the cluster, the better the clustering effect. Highly similar helium bubbles evidently belong to a particular bubble type. Accordingly, a clustering detection algorithm is adopted to analyze the irradiated-alloy TEM micrograph for helium bubble detection and statistics. Specifically, the DBSCAN algorithm was applied to this study. This algorithm assigns cluster labels based on dense regions of points. In DBSCAN, the notion of density is defined as the number of points within a specified radius.

Numerous helium bubble clusters can be obtained after the background and noise are eliminated; thus, clustering the remaining pixels in the preprocessed TEM image data is necessary. This section presents the application of DBSCAN to cluster the calculated helium bubble pixels.

A special label is assigned to each sample point using the following criteria.

(1) A point is considered as a core point if a specified number of neighboring points falls within the specified radius.

(2) A border point is one that has fewer neighbors than the specified number within a specified threshold; however, it lies within the threshold radius of the core point.

(3) All points that are neither core nor border points are considered as noise points.

After labeling the points as core, border, or noise, the DBSCAN algorithm processes them following two simple steps.

Step 1: A separate cluster for each core point or connected group of core points is formed (core points are connected if they are not more distant than the threshold).

Step 2: Each border point is assigned to the cluster of its corresponding core point.

The DBSCAN algorithm can be used to identify the degree of pixel density and classify the pixels according to their distribution based on the pixel distribution density. At the same time, the relatively isolated pixels are marked as noise points, rather than helium bubbles.

Two parameters are extremely important in the DBSACN algorithm: eps and $nei{b}_{\text{th}}$. The first, eps, is the scanning radius whose value is set to 1 in this model; that is, only eight pixels around the current pixel were considered. The second, $nei{b}_{\text{th}}$, is a threshold value for identifying kernel points; its value is obtained by the training process. The main steps of DBSCAN are as follows:

1. Randomly choose an unvisited pixel as a starting point, and then find all of its neighbors within the scanning radius (eps) range.

2. If the number of neighboring pixels is greater than or equal to $nei{b}_{\text{th}}$, mark the current point and its neighboring pixels as kernel points, and put the neighbors in a queue. Otherwise, the current pixel is temporarily marked as a noise point.

3. Consider another unvisited point from the queue, find its neighboring pixels, mark the unvisited pixels as kernel points, and put them in the queue.

4. Repeat step 3 until the queue is empty; at this instance, the marked kernel points form a new cluster.

5. Repeat steps 1-4 until all pixels are marked as kernel points or noise points, and terminate the algorithm.

The shape of the cluster able to be detected from the above steps can be any of the forms; hence, DBSCAN remains considerably suitable for clustering helium bubbles with irregular shapes.

2.2.3 GMM

The GMM is a well-known basic statistical machine learning model widely applied to visual media fields. It is a type of unsupervised clustering model and density model consisting of a number of Gaussian function components. Within its framework, the feature density function is regarded as belonging to different component variables whose combination provides multimodal density. The GMM can afford greater flexibility and precision in modeling as well as deriving the underlying statistics of sample data; accordingly, it is popularly used in object recognition and classification.

Virtually all helium bubbles in metals or alloys irradiated by helium ions are spherical or ellipsoidal; thus, their two-dimensional images are represented by circles or ellipses. Based on this observation and GMM characteristics, the assumption that helium bubble images follow a normal distribution is reasonable. In addition, considering that some helium bubbles in TEM images may intersect or coincide, the multi-dimensional GMM, which is a probabilistic model for representing normally distributed subpopulations within the overall population, is applicable to helium cluster analysis. In the multi-dimensional GMM, knowing the subpopulation to which a data point belongs to is unnecessary; hence, the subpopulations are automatically learned.

The multi-dimensional GMM involves two types of parameters: the mixture component weights and component means as well as its covariance. For a GMM with K components, the $k^{\text{th}}$ component has a mean of $\overrightarrow{{\mu }_k}$ and a covariance matrix of ${\text{Σ}}_k$. The mixture component weights are defined as ${\varphi }_k$ for component $C_k$. To ensure that the total probability distribution normalizes to 1, ${\displaystyle \sum }_{i=1}^K{\varphi }_i=1$ is applied as constraint. The mathematical expressions of multi-dimensional GMM can be summarized as Eqs. (3)–(5):

$p(\overrightarrow{x})={\displaystyle \sum _{i=1}^K{\varphi }_{i}N\left(\overrightarrow{x}|{\overrightarrow{\mu }}_i,{\Sigma }_i\right)},$ (3) $N\left(\overrightarrow{x}|{\overrightarrow{\mu }}_i,{\Sigma }_i\right)=\frac{1}{\sqrt{{\left(2\pi \right)}^D\left|{\Sigma }_i\right|}}\exp \left(-\frac{1}{2}{\left(\overrightarrow{x}-{\overrightarrow{\mu }}_i\right)}^T{\Sigma }_i^{-1}\left(\overrightarrow{x}-{\overrightarrow{\mu }}_i\right)\right),$ (4) ${\displaystyle \sum _{i=1}^K{\varphi }_i}=1,$ (5)

where D is the dimension of the Gaussian distribution (in this work, D = 2 because the helium bubble cluster pixels have two dimensions).

If the component weights are not learned, then they can be considered as the a priori distribution over components such that p($x$)=${\varphi }_k$ (where $x$ is generated by component $C_k$). If they are learned, then they are the a posteriori estimates of the component probabilities given by the data.

Parameters, such as ${\varphi }_k$, $\overrightarrow{{\mu }_k}$, and ${\text{Σ}}_k$, may be evaluated using the expectation maximization (EM) algorithm. The EM algorithm is a numerical technique for maximum likelihood estimation and typically used when closed-form expressions for updating model parameters can be calculated. The EM is an iterative algorithm with the property that the maximum likelihood of data strictly increases with subsequent iterations. This guarantees that the local maximum or saddle point is approached.

The EM for GMM consists of two steps: the expectation step (E step) and the maximization step (M step). The main objective of E step is to compute the expectation of the log-likelihood estimation. It uses the current estimated distributions of latent variables based on the parameters inferred from the previous step. For the E step, the probability, ${\widehat{\omega }}_{ik}$, that $x_i$ is generated by component $C_k$ is evaluated using the following equation:

${\widehat{\omega }}_{ik}=\frac{{\widehat{\varphi }}_{k}N\left(x_i|{\overrightarrow{\mu }}_k,{\Sigma }_k\right)}{{\displaystyle \sum _{j=1}^K{\widehat{\varphi }}_{j}N\left(x_i|{\overrightarrow{\mu }}_j,{\Sigma }_j\right)}},$ (6)

where ${\widehat{\omega }}_{ik}=P(C_k|x_i,\stackrel{\rightharpoonup }{\varphi },\stackrel{\rightharpoonup }{\mu },\stackrel{\rightharpoonup }{\text{Σ}})$. In the M step, the main function is to calculate the parameters maximizing the expected log-likelihood from the E step. These parameters are then utilized to determine the distribution of latent variables in the next E step. Specifically, along with ${\widehat{\omega }}_{ik}$ calculated in the E step, ${\varphi }_k$, $\stackrel{\rightharpoonup }{{\mu }_k}$, and ${\text{Σ}}_k$ are renewed by Equations (7)-(9), respectively:

${\widehat{\varphi }}_k=\frac{1}{N}{\displaystyle \sum _{i=1}^N{\widehat{\omega }}_{ik}},$ (7) ${\overrightarrow{\mu }}_k=\frac{{\displaystyle \sum _{i=1}^N{\widehat{\omega }}_{ik}{x}_i}}{{\displaystyle \sum _{i=1}^N{\widehat{\omega }}_{ik}}},$ (8) ${\Sigma }_k=\frac{{\displaystyle \sum _{i=1}^N{\widehat{\omega }}_{ik}{\left(\overrightarrow{x}-{\overrightarrow{\mu }}_i\right)}^T\left(\overrightarrow{x}-{\overrightarrow{\mu }}_i\right)}}{{\displaystyle \sum _{i=1}^N{\widehat{\omega }}_{ik}}}.$ (9)

The E and M steps are alternately run until the parameters converge. The EM algorithm is employed to iteratively optimize the parameter estimate. Note that the EM requires the a priori selection of the model order. Usually, the user selects a suitable number approximately corresponding to the length of the training expression.

In this work, the pixel value, $pixel\left(i\right)$, is used as the pixel mesh weight required in the GMM. Therefore, Equations (7)–(9) can be transformed into Equations (10)–(12), respectively, in this model.

${\widehat{\varphi }}_k={\displaystyle \sum _{i=1}^N{\widehat{\omega }}_{ik}pixel(i)}/{\displaystyle \sum _{i=1}^{N}pixel(i)}$ (10) ${\overrightarrow{\mu }}_k=\frac{{\displaystyle \sum _{i=1}^N{\widehat{\omega }}_{ik}{x}_{i}pixel(i)}}{{\displaystyle \sum _{i=1}^N{\widehat{\omega }}_{ik}pixel(i)}}$ (11) ${\Sigma }_k=\frac{{\displaystyle \sum _{i=1}^N{\widehat{\omega }}_{ik}{\left[\left(\overrightarrow{x}-{\overrightarrow{\mu }}_i\right)pixel(i)\right]}^T\left(\overrightarrow{x}-{\overrightarrow{\mu }}_i\right)}}{{\displaystyle \sum _{i=1}^N{\widehat{\omega }}_{ik}pixel(i)}}$ (12)

Based on the foregoing statement and formulae, an automatic bubble detection model is developed, and the clustering classification algorithm is formulated. The results yielded by the GMM-based model provide the helium bubble information; that is, $\overrightarrow{{\mu }_k}$ describes the central points, and the covariance matrix contains the size and shape information of helium bubbles, respectively.

3 Results and discussion

As described in the previous section, first, the determination of four parameters is necessary: $b{g}_{\text{th}}$ (the threshold value of the pixel background), $nei{b}_{\text{th}}$ (the minimum number of neighboring points of a pixel), $cluste{r}_{\text{th}}$ (the minimum number of a cluster), and $n_{\text{dot}}$ (the number of pixels of a helium bubble). These parameters vary with a limited number of discrete integer values. The rational values can be conveniently determined by testing the combinations of all values. In this study, eight TEM images and corresponding annotated images were employed for model training to determine the best combination of the four parameters. Based on the generated results, eight candidate models were established and denoted as Models 1–8. For each TEM image, a combination of parameters is given at the highest F1 value, as well as the P and R results, as listed in Table 1.

Second, eight candidate models were formed using a combination of the four parameters after the training process. As listed in Table 2, four TEM images are selected to determine the optimal parameters for the proposed model with the F1 value as the evaluation criterion. The parameters of Model 5 are easily identified as having the best values; hence, they are chosen as candidates for the final model for bubble clustering selection.

Finally, four different TEM images were used to test the performance of the formulated model for bubble clustering. The corresponding performance evaluation results including the four datasets and their mean values are listed in Table 3.

The position of the helium bubble was evaluated under the Gaussian mixture analysis framework, and the coordinate deviation was compared with the manually marked images in this model. The mean bias for describing the deviation is computed using the following:

where $P_{\text{l}}$ and $P_{\text{M}}$ are the central points of helium bubbles manually marked and estimated using the model, respectively. Here, the Manhattan distance is adopted to compute the distance between $P_{\text{l}}$ and $P_{\text{M}}$.

The list in Table 3 indicates that the mean coordinate bias is 3 pixels and the results yielded by the model approximate those marked manually.

For example, Figs. 4–7 show the helium bubble detection results yielded by the formulated model and manual marking. In Fig. 4, the red points represent the helium bubbles detected by the described model, and the blue points represent those marked by the TEM analysis experts. The numbers of red and blue points are expressed as $n_{\text{red}}$ and $n_{\text{blue}}$, respectively.

Fig. 4Full-screen View

(Color online) (a) Original TEM, and (b) detected results with $n_{\text{red1}}$ = 63 and $n_{\text{blue1}}$ = 37

Fig. 5Full-screen View

(Color online) (a) Original TEM and (b) detected results with $n_{\text{red2}}$ = 33 and $n_{\text{blue2}}$ = 27

Fig. 6Full-screen View

(Color online) (a) Original TEM and (b) detected results with $n_{\text{red3}}$ = 38 and $n_{\text{blue3}}$ = 24

Fig. 7Full-screen View

(Color online) (a) Original TEM and (b) detected results with $n_{\text{red4}}$ = 46 and $n_{\text{blue4}}$ = 45

As shown in Figs. 4–7, the number of red points exceeds that of the blue points. Virtually all helium bubbles have been recognized by the formulated model. Moreover, some objects similar to the bubbles, but with no manual markings, have also been identified by the established model. The superior recall (i.e., 94%) achieved by the proposed model means that it is capable of automatically locating and marking most of the manually marked helium bubbles.

In contrast to the excellent recall performance, the precision level achieved by the model was relatively low. This may have been caused by two factors. First, some objects with a similar morphology as the bubbles are extremely dark or minuscule to be observed by human sight, but they can be identified by the machine learning method. Second, helium bubbles with irregular shapes may be manually marked as a single bubble; however, they are distinguished as multiple bubbles by the model. Using the proposed model, these two aspects may yield higher counting results compared with those manually marked. The foregoing may also explain the relatively low precision associated with the manual approach. Accordingly, the model established in this work is observed to be extremely sensitive. It can detect bubble-like objects and identify more candidates of helium bubble clusters than the manual marking approach. By considering Fig. 4 as an example, Fig. 8 indicates that the model developed in this work is considerably sensitive and capable of dealing with situations in which helium bubbles may have irregular shapes or small sizes.

Fig. 8Full-screen View

(Color online) Comparison of results obtained by model and manual marking

4 Summary

This study is the first to employ the DBSCAN clustering method and the GMM model for efficient identification and analysis of helium bubbles in the TEM images of alloys in a high-temperature irradiation environment. In the current practice, considering that the manual counting of bubbles is time-consuming, the ability of the formulated model to identify and count these bubbles automatically considerably saves time. The new method is found to have a higher recall but low precision performance because it can identify objects with irregular shapes and those unnoticed by the naked eye. The nature of automatic identification and the substantial time saved afforded by the designed method highlights new opportunities in the use of machine learning to expedite helium bubble identification and counting.

The results obtained in this study are based exclusively on TEM datasets under the under-focused condition of bright-field TEM micrographs. Nevertheless, the model is also expected to be capable of adapting to the circumstances of the over-focused condition. A more accurate helium bubble counting performance can be derived by considering the results of these two conditions together. Furthermore, the variation in the shape and size of bubbles must also be considered because helium bubbles evolve dynamically and diversely throughout the entire service. Hence, considerable work awaits completion and verification through follow-up research.