Introduction
Muon scattering tomography (MST) is a noninvasive imaging technique that uses cosmic-ray muons to reconstruct the internal structure of dense materials. It is a versatile and powerful technique, particularly beneficial for probing nuclear materials [1-4]. It aids in the identification of concealed radioactive substances and illicit nuclear materials [5-7] in various environments. For example, MST plays a crucial role in nuclear waste management and storage facility inspections [8, 9], offering a reliable method for assessing the integrity of storage facilities and monitoring the containment of hazardous materials. In addition, MST is indispensable in cargo scanning and security screening, where it helps detect hidden contraband in shipments [10, 11], thus ensuring security at the ports and borders. Finally, MST is also applied in radiation shielding studies, where it assists in evaluating and improving radiation shielding materials and techniques for both industrial and research purposes [12]. The precision and effectiveness of MST are crucial, as they directly impact the reliability of detection and analysis in these applications.
The Point of Closest Approach (PoCA) [13] algorithm is commonly used in the MST to reconstruct muon-scattering locations. Although PoCA is simple and computationally efficient, it has inherent limitations in terms of image clarity and resolution, primarily because of data sparsity and high noise levels in muon measurements.
To overcome these challenges, we propose integrating Reinforcement Learning (RL) [14]—a technique effective in optimizing complex, non-linear problems through iterative interactions with the environment—into MST. By adaptively tuning the algorithmic parameters, RL can mitigate the limitations of PoCA and significantly improve the image quality, enabling precise imaging that detects subtle changes in the monitored objects.
This study introduces a new paradigm for image reconstruction and has the potential to significantly advance the field of muon scattering tomography. By demonstrating the feasibility of RL in this context, we aim to pave the way for further exploration of adaptive learning techniques in tomography and related areas. In summary, the key contributions of this study are as follows:
Novel integration of RL with PoCA for MST, introducing an intelligent approach to enhance muon imaging.
Development of a new RL framework for MST, including a multidimensional state representation, a multi-domain action space, and a reference-based multi-objective reward system.
Comprehensive evaluation of the μRL-enhanced approach, demonstrating significant improvements in image quality metrics compared to traditional PoCA methods.
The remainder of this paper is organized as follows: Section 2 provides a background on muon scattering tomography and reinforcement learning, Section 3 describes the proposed methodology, Section 4 presents the experimental results, and Sect. 5 concludes the paper and outlines directions for future research.
Background
Muon scattering tomography techniques
Traditional MST algorithms aim to reconstruct muon scattering locations by analyzing deviations in muon trajectories as they pass through matter. PoCA is a foundational method used for this purpose because of its simplicity and computational efficiency. PoCA estimates the scattering point by determining the midpoint of the line connecting the closest points between the incoming and outgoing muon tracks. Mathematically, given an incoming muon track defined by the direction vector u and point Pin, and an outgoing track with direction vector v and point Pout, the PoCA point PPoCA is calculated by solving the following:
Despite its speed, PoCA’s single-scatter assumption smears edges and introduces artifacts once a muon undergoes multiple Coulomb interactions inside complex high-Z materials. Schultz et al. (2003) therefore introduced two statistical methods: (i) Maximum Likelihood Scattering (MLS), which uses only the scattering angle, and (ii) Maximum Likelihood Scattering and Displacement (MLSD), which utilizes both the scattering angle and lateral shift for accurate reconstruction, although the scale of voxel-wise updates increases with resolution [13, 15]. Liu et al. (2007) subsequently developed EM implementations for the scattering only model (MLS-EM) and the joint scattering and displacement model [16]; by introducing an ordered subsets scheme, they further sped these algorithms up to produce MLS-OSEM and MLSD-OSEM [17]. To curb noise without losing detail, Wang et al. (2007) reformulated the MLSD within a maximum a posteriori (MAP) framework, adding Gaussian (smooth) and Laplacian (sparse) priors and applying voxel-wise shrinkage after each step; MAP suppresses high-intensity noisy voxels, preserves edges, and remains robust even with low flux data [18]. Further refinement using the Gaussian Scale Mixture (GSM) likelihood matches the heavy-tailed scattering statistics with a weighted Gaussian mix, yielding cleaner, high-Z-focused images [19]. Together, PoCA, MLS, MLSD, MAP, and GSM span a spectrum of reconstruction methods, from PoCA speed to full Bayesian accuracy. Additionally, neural networks [20, 21] and support vector machines [22] have been applied to improve the muon reconstruction and material discrimination accuracy.
Compared to the aforementioned methods, which often require extensive iterations and detailed parameter tuning, RL can rapidly converge in complex, high-dimensional spaces, which is a more flexible and efficient high-dimensional optimization and decision-making approach. Its core advantage lies in the utilizing of real-time feedback to dynamically adjust parameters or optimization strategies to achieve optimal imaging quality. This adaptability and scalability position RL as a key factor for further enhancing the performance of MST and paving the way for more intelligent and efficient imaging systems.
Basics of Reinforcement Learning
Reinforcement Learning is a subfield of artificial intelligence, where an agent learns to make decisions by interacting with an environment to maximize a cumulative reward over time through trial-and-error exploration [14]. Unlike supervised learning, in which the model is trained on labeled data, RL involves learning from the consequences of actions performed in an environment. In RL, an agent explores its environment, performs actions, and receives feedback in the form of rewards or penalties, which it uses to improve its future decision making.
The core components of RL are as follows:
Agent: The decision maker that interacts with the environment. The agent takes action based on its current state and learns from the feedback it receives.
Environment: The external system in which the agent operates. The environment defines the rules of interaction and determines the consequences of the agent’s actions.
State (st): A representation of the environment at a specific point in time. It contains all the environmental information that the agent requires to make decisions regarding its actions. The state changes over time as the agent interacts with the environment.
Action (at): A decision or move made by the agent that impacts the environment. Actions can be discrete or continuous, depending on the problem at hand.
Reward (rt): The feedback the agent receives after taking an action in a particular state. Rewards are typically scalar values, with positive rewards indicating favorable outcomes and negative rewards indicating undesirable outcomes.
Policy (π): A strategy that maps states to actions. The policy defines the agent’s behavior and decides which actions to take based on the current state.
Transition dynamics (
In the case of MST, we aimed to optimize the PoCA reconstruction process by adjusting various parameters (actions) based on the observed image quality (state) and guiding these adjustments using a reward function that measures the improvement in reconstruction accuracy.
RL has various algorithms [23], and here we mention two of them. One common approach is Q-learning, in which the agent updates its Q-values (state-action value function) based on feedback from the environment and seeks the action that maximizes long-term rewards. Another approach is the Deep Q-Networks (DQN), which uses deep neural networks to approximate the Q-function, enabling the agent to handle complex, high-dimensional environments. In this study, we used Q-learning because of its simplicity.
Reinforcement learning for MST enhancement
Motivation
Advancements in machine learning, particularly reinforcement learning, have opened new possibilities for the optimization and enhancement of physical imaging systems. These techniques are particularly effective for adapting to dynamic conditions and refining strategies based on feedback from limited data. This adaptability is particularly valuable for systems that deal with complex, high-dimensional datasets. RL has proven to be a powerful tool across a wide range of imaging applications, including medical imaging [24, 25], computer vision [26-28], and industrial inspection. In medical imaging, RL has been applied to tasks such as landmark localization [29, 30], object detection [31], image segmentation [32, 33] and registration [34, 35]. It offers solutions to complex optimization problems, such as parameter tuning, strategy selection, and automation of diagnostic processes. For example, RL agents have been trained to adjust imaging parameters in real-time to minimize radiation exposure while maintaining image quality [36]. In the realm of computer vision, RL has been utilized for object recognition [37], image enhancement and denoising [38], as well as image generation and restoration [39]. In industrial inspection, RL has enabled the development of intelligent systems that dynamically control inspection equipment, detect defects, and optimize quality control processes [40].
In the case of MST, traditional PoCA algorithms often face challenges in terms of image clarity and resolution, primarily because of data sparsity and high noise levels inherent in the muon scattering measurements. Meanwhile, PoCA and improved PoCA algorithms typically rely on fixed parameter settings [41, 42], which are often based on prior knowledge or expert judgment, and lack the flexibility to adapt to changing conditions. Moreover, fixed parameters may struggle to handle variations in the muon flux or density distribution of target objects, particularly when dealing with more complex or heterogeneous scanned objects. By integrating RL into the MST reconstruction process, we aim to bring similar advancements to the field of muon scattering tomography, such that key PoCA parameters can be dynamically adjusted, enhancing the accuracy and efficiency of the imaging process.
Challenges in RL for MST
While RL has demonstrated significant success in the aforementioned imaging fields, its use for MST has not been explored to the best of our knowledge and faces unique challenges. One of the main difficulties arises from the sparse and noisy nature of muon imaging data, which often leads to incomplete or weak signal. Unlike other imaging domains, MST is further complicated by the stochastic interactions of cosmic-ray muons, making it difficult to reliably track their scattering and accurately reconstruct the internal structures of the objects. Additionally, RL algorithms typically require substantial computational resources owing to the iterative nature of learning through repeated interactions with the environment, which poses a limitation for MST, particularly in real-time scenarios. Moreover, developing an RL framework capable of managing the intricacies of the MST reconstruction is particularly challenging. For example, the reward function must effectively quantify the image quality in the MST, as existing metrics may not fully capture the unique characteristics of muon scattering. Therefore, these frameworks must not only integrate domain-specific knowledge of muon interactions but also be sufficiently efficient to handle noisy, sparse, and sometimes ambiguous data, ensuring that the agent can learn meaningful strategies for improving reconstruction accuracy.
In the next section, we describe the MST as an RL optimization problem and the implementation of our framework in detail.
Methodology
MST as an RL optimization problem
MST reconstruction requires the adjustment of multiple parameters, and optimizing these parameters is challenging, particularly with varying material compositions and measurement conditions. To address this, we modeled the MST reconstruction as an RL optimization problem, enabling the adaptive refinement of parameters based on continuous feedback from the image quality.
In this framework, the environment is defined by the scattering points calculation and density-mapping process according to the PoCA algorithm, which generates updated images from muon scattering data. The RL agent interacts with the environment by adjusting the parameters to improve the image quality over time. The agent learns the optimal configurations through feedback on each reconstruction, progressively enhancing the reconstruction process.
At each iteration, the agent observes the current state, which is represented by a set of image characteristics, along with feedback defined on a set of image quality metrics. The agent then selects an action according to an ε-greedy exploration and exploitation policy as follows: After executing the action, the agent receives a reward and updates its Q-values. As these Q-values change, the policy is refined to maximize long-term improvements. Through trial and error, the agent explores different parameter configurations, learns from the feedback, and adjusts its strategy. Over time, the agent converges to a policy that consistently improves the PoCA reconstruction.
For simple state-action spaces, classic methods such as Q-learning can be used, whereas more complex scenarios may require methods such as Deep Q-Networks (DQN) or actor-critic algorithms.
In summary, framing MST as an RL optimization problem allows for dynamic and adaptive parameter adjustments, thereby improving the image quality and structural accuracy.
System architecture
In this section, we develop the RL framework by leveraging a Q-learning agent to dynamically adjust the key parameters of the PoCA algorithm and post-processing steps, thereby optimizing the reconstruction process and enhancing the image quality in MST. The primary components of this framework are the action space, state representation, reward function, and agent. Each component is elaborated below to illustrate how they collaboratively facilitate effective optimization of the model.
Reinforcement learning framework
The system flow diagram of the RL framework is shown in Fig. 1. The process is split into two parts, as depicted by the two boxes: the main process (left side) and the Q-learning process (right side).
Main process:
The main loop starts by loading the necessary input files for an experiment, then calculates the initial density map. It sets up the PoCA environment and initializes the Q-learning agent for the subsequent steps.
The Q-learning process which is described in detail below.
The optimized parameters, and resulting density map and 2D projections are saved.
Q-learning process:
The agent begins by initializing the Q-table and setting the hyperparameters such as α, γ, and ε.
At each time step, the agent extracts state features from the current density map, including μ, σ2, γ1t, and γ2t. These features are then discretized to form the current state, st.
The agent uses an ε-greedy policy to choose an action: either a random action (with probability ε) or the action with the highest expected reward (with probability 1-ε).
The chosen action is executed, which involves adjusting parameters.
The environment updates the density map and the agent receives a reward based on factors which reflect the quality of the image.
The Q-table is updated using the Q-learning update rule, allowing the agent to learn and refine its policy.
As the process continues, the agent decays ε, and progressively improves its strategy for optimizing the reconstruction.
This Q-learning process loop continues until the maximum number of episodes is reached, at which point the agent policy has been optimized and the final parameters are output.
State, Action, and Reward design
The state representation is crucial for the agent to make informed decision-making. Within the RL framework, the state was designed to capture the current state of the density map. Instead of relying solely on raw pixel values, we extracted higher-level statistical features, such as the mean, variance, skewness, and kurtosis. These features offer a comprehensive view of image quality: the mean represents the overall intensity, variance reflects the spread of the mean intensity, and skewness and kurtosis capture asymmetry and the presence of extreme values (tailing) in the image intensity distribution, respectively. The state st at any given time t of the current density map Dt is represented as a feature vector:
The action space consists of a discrete set of actions that the agent can take to adjust the PoCA algorithm and subsequent post-processing steps. The actions are grouped into four categories, each targeting different aspects of the reconstruction process:
a. Adjust maximum scattering distance dmax:
The agent may increase or decrease dmax to determine the most suitable distance range for the PoCA calculation by selecting valid scatter points with
b. Adjust minimum scattering angle θmin:
The agent adjusts the minimum scattering angle threshold to filter the scattering events and improve the clarity of the reconstructed image. Increasing θmin excludes small-angle scattering (often noise), whereas decreasing it preserves subtle scattering details crucial for objects with heterogeneous internal structures.
c. Switch PoCA strategy:
Different strategies exist for calculating scattering points, such as midpoint calculation versus angle-weighted calculation. This action alternates between the two strategies to optimize the reconstruction of various materials and imaging objects.
d. Toggle Gaussian smoothing:
This action alternates between enabling and disabling Gaussian smoothing on the density map. Smoothing can help reduce noise but may also blur the significant features. This decision depends on the need to balance noise reduction with the preservation of significant features.
Thus, the full action set is
The reward function is central to guiding the agent’s learning process by evaluating improvements in image quality at each step. It considers multiple image quality metrics: clarity (edge sharpness), contrast (intensity difference), entropy (amount of uncertainty or randomness), and variance (spread of pixel intensities). By structuring the reward function in this multi-objective manner, the RL agent can simultaneously enhance various features of the reconstruction. If a particular action results in sharper features, greater contrast, and reduced noise, the agent receives a higher reward, reinforcing the effectiveness of the action and improving future decision making. We now break down each element of the reward function in detail.
a. Clarity reflects the sharpness and well-defined nature of the features in the image. A clearer image has well-defined boundaries and structures, making it easier to interpret. This can be quantified by measuring the edge strength in the density map. A suitable measure is the edge detection Sobel algorithm, formulated as
b. Contrast refers to the difference in intensity between objects and the background in the image. Higher contrast can make features more distinct. The contrast T(D) is as follows:
d. Variance measures the dispersion of pixel values. The definition we use is:
Each component contributes differently to the final image quality, therefore, they must be weighted accordingly in the reward function. By combining these factors, the agent can be guided to produce an image that is not only clear and sharp but also has a good contrast and appropriately distributed pixel intensities. To assess the improvements, the current density map was compared with the initial reference map from PoCA. The enhancement was quantified by evaluating the extent to which the new density map improved the key quality metrics. The reward can be expressed as
Agent Design
The RL agent in our framework employs the Q-learning algorithm [14, 43], which is a model-free RL method that aims to learn the optimal action-value function Q(s, a). This function estimates the expected cumulative reward of taking action at in state st and then following the optimal policy. The Q-learning update rule is defined as:
The agent employs an ε-greedy policy to balance exploration and exploitation during the learning process. With probability ε, the agent selects a random action from the action space
The agent continuously updates its policy by comparing the observed rewards associated with different actions. The reward structure is designed to reinforce actions that lead to improved image quality while penalizing actions that degrade performance. This feedback-driven approach enables the agent to learn an optimal policy
Experiments and results
Experimental setup and MST reconstruction
The dataset utilized in this study comprises muon scattering data acquired using a muon tracking system, the schematic diagram of which is shown in Fig. 2. Eight plastic scintillator arrays are central to the tracking system and are organized into four position detectors, each featuring two orthogonal layers—x and y. This dual-layer configuration ensures precise spatial positioning, thereby enabling the accurate tracking and reconstruction of muon events. In this configuration, the tracking system encompasses a substantial detection volume of 500 mm × 500 mm × 280 mm, with two groups of detectors positioned above and below. This arrangement facilitates comprehensive coverage and enhances the system capability to capture muon scattering events.
Figure 3 presents the geometry layouts of the experiments, each conducted over a 24-hour period. Panel (a) shows low-, medium-, and high-Z materials using aluminum, copper, lead, and tungsten blocks. Panel (b) focuses on a hollow tungsten cylinder to study the imaging of hollow structures. Panel (c) combines a hollow tungsten cylinder with three smaller tungsten cylinders to examine the imaging of concealed and small high-Z objects. Finally, Panel (d) features 18 lead blocks arranged on three sides to form a fence, a tungsten block on the fourth side, and a central hollow tungsten cylinder, highlighting the impact of different materials and shapes on imaging performance.
Data acquisition is facilitated by synchronized scintillator signals that act as triggers for muon events. Raw data undergo prepossessing, including noise filtering, normalization, position and trajectory reconstruction, and muon track selection, ensuring high-quality input for the imaging algorithms of the traditional PoCA method and the µRL-enhanced framework. All computing tasks were performed on a platform running Ubuntu 18.04, which was equipped with an NVIDIA GeForce RTX 2080 (8 GB GDDR6) GPU and CUDA 12.4. The computations were implemented using PyTorch 2.4.1, and data processing and visualization were supported by libraries such as NumPy, SciPy, and Matplotlib.
The RL agent employs a Q-learning variant with a learning rate α of 0.001, a discount factor γ of 0.95, and an ε-greedy exploration strategy starting at ε = 1.0. The training process was set to run for up to 1000 episodes; however, empirical results showed that convergence typically occurred within 200 episodes, enabling faster training without sacrificing performance. Each episode consisted of 10 steps. Intermediate checkpoints were saved at each episode to monitor the agent’s learning progress and ensure stability. Multiple trials with different parameters validated the robustness and efficiency of the RL approach, with the agent outperforming the baseline PoCA methods by adaptively selecting the optimal parameters.
Visual comparisons
In Figs. 4, 5, 6, and 7, side-by-side comparisons of the images produced by PoCA, MLSD, and the μRL-enhanced method are illustrated. In the first phantom, which contains four cubes, the μRL version clearly outlines three blocks with crisp rectangular boundaries in the XY view and shows well-separated vertical columns in the XZ and YZ planes, whereas PoCA renders only blurred clusters and MLSD blurs the cube corners. The second and third phantoms feature a hollow ring. PoCA barely hints at a ring, while MLSD recovers a faint but radially blurred ring. In contrast, the μRL-enhanced projections exhibit a continuous ring with a clearly empty center, and the side view locates the ring axis at the correct height as the agent has learned to suppress the vertical streaking. For the final phantom, the μRL method simultaneously preserves the thin outer frame, inner ring, side cube, and central void, whereas MLSD over smooths the frame corners, and PoCA leaves heavy salt-and-pepper noise that obscures the geometry. Along Z, the μRL images maintain the high-density areas distinctly separated, eliminating the patchy artifacts visible in the PoCA and the diffuse haze in the MLSD. Overall, the μRL-enhanced reconstructions present sharper edges, greater contrast between high- and low-density regions, and markedly fewer background artifacts. They produce more faithfully three-dimensional geometries than PoCA and avoid the over-smoothing that attenuates critical details in MLSD, providing visually and quantitatively superior images for downstream interpretation.
The four plots in Fig. 8 illustrate the evolution of the average total reward during training for the four different experimental scenarios. Each plot demonstrates a clear and steady progression of the reward, with a generally smooth and consistent increase over time. This smoothness indicates that the RL algorithm effectively learned and improved during the training process. The steady nature of the curves reflects the stability of the training procedure, suggesting that the model gradually adapts to the environment and makes optimal adjustments to maximize the total reward. The lack of significant fluctuations or sharp changes further confirms the robustness of the training process, where the agent converges toward a reliable policy. This trend is particularly important in the context of RL, where achieving a smooth and consistent improvement in rewards typically signifies that the model is learning in a meaningful way and is not overfitting or getting stuck in suboptimal solutions.
In summary, the μRL-enhanced method provides a substantial improvement in image quality, offering sharper and more defined projections with higher contrast and better detail preservation. These advancements make it a more effective tool for capturing and analyzing complex 3D structures, addressing many of the limitations of traditional PoCA-based methods.
Quantitative analysis
In this study, we also compared the performance of the traditional PoCA and MLSD methods with our μRL-enhanced approach using three quantitative metrics: Peak Signal-to-Noise Ratio (PSNR) [44], Intersection over Union (IoU) [45], and Structural Similarity Index Measure (SSIM). These evaluation metrics were derived from comparisons with the original images shown in Fig. 9, which were generated using Geant4 simulations that produce position point cloud data based on geometric constructs composed of various materials.
PSNR assesses the clarity of reconstructed features by measuring the prominence of the desired signal relative to surrounding noise. The SSIM evaluates image quality from a perceptual standpoint, considering factors such as luminance, contrast, and structural information. The IoU measures the spatial overlap between the reconstructed geometric structures and the original ones, providing a direct assessment of the reconstruction accuracy. By focusing on these metrics, we aim to provide a thorough and objective assessment of how effectively our µRL-enhanced pipeline reconstructs and preserves critical image details compared with the PoCA and MLSD baselines.
Table 1 shows that the μRL-enhanced reconstructions significantly outperform the baseline PoCA algorithm, achieving average gains of 8.4% in PSNR, 307% in IoU, and 79% in SSIM. Relative to the more sophisticated MLSD method, the μRL-enhanced attains a PSNR that is 11% higher on average, while still delivering a 9.8% increase in the IoU and a 29% boost in the SSIM. These numbers already demonstrate a clear advantage, and inspection of representative cases further reveals the basis for this advantage. In Fig. 3a, PoCA achieves a relatively high PSNR, reflecting a low voxel-wise error, yet loses almost all object geometry, whereas MLSD restores the shape but blurs the image; in constrast, μRL-enhanced maintains sharpness while recovering the structure. In the challenging Fig. 3d, μRL-enhanced again surpasses both baselines, offering the clearest contours and the most accurate overlap. Taken together, these concurrent improvements in signal fidelity, structural similarity, and overlap accuracy show that μRL-enhanced preserves both fine details and global structure more faithfully than either reference method, thereby yielding clearer and more reliable reconstructions for downstream material identification.
| Image | Method | PSNR (dB) | IoU | SSIM |
|---|---|---|---|---|
| Fig. 3a | PoCA | 24.29 | 0.06 | 0.50 |
| MLSD | 19.90 | 0.29 | 0.73 | |
| µRL-enhanced | 24.44 | 0.33 | 0.95 | |
| Fig. 3b | PoCA | 19.45 | 0.09 | 0.68 |
| MLSD | 20.15 | 0.44 | 0.62 | |
| μRL-enhanced | 21.87 | 0.47 | 0.96 | |
| Fig. 3c | PoCA | 18.78 | 0.13 | 0.49 |
| MLSD | 18.85 | 0.42 | 0.77 | |
| μRL-enhanced | 21.01 | 0.46 | 0.96 | |
| Fig. 3d | PoCA | 16.34 | 0.17 | 0.46 |
| MLSD | 17.08 | 0.32 | 0.81 | |
| μRL-enhanced | 17.76 | 0.35 | 0.87 |
Conclusion
This study successfully integrated RL with MST, yielding notable gains in reconstruction quality compared to the traditional PoCA and MLSD methods. By dynamically optimizing key parameters, such as the maximum scattering distance, minimum scattering angle, scattering point calculation strategy, and Gaussian smoothing, the μRL-enhanced framework effectively addressed PoCA’s reliance on static settings.
Quantitatively, the μRL-enhanced method improves the average PSNR by 8.4%, boosts the mean IoU by 307%, and raises the SSIM by 79% relative to the PoCA baseline, yielding markedly crisper edges and more faithful structural details. When compared with the more advanced MLSD reconstruction, μRL-enhanced achieves an average improvement of 11% in PSNR, 9.8% in IoU, and 29% in SSIM, confirming its ability to capture fine-scale features without compromising noise performance.
With these encouraging improvements, the current framework exhibits several limitations. First, it relies on a simplified state representation (e.g., density statistics), which restricts its ability to capture the essential spatial features. Second, the fixed step sizes in the action space limit the precision of parameter tuning. Finally, our reward function still relies on manually chosen weights based on experiments. These ad hoc weights can bias the learning process toward specific scenarios, limiting the method’s effectiveness when applied to different detector setups or noise conditions. To further advance the RL framework in MST, future efforts will integrate Convolutional Neural Networks (CNNs) for spatial-aware state encoding, adopt continuous action spaces through policy gradient methods (e.g., DDPG) for finer parameter control, and incorporate adaptive, physics-informed reward functions, leveraging scattering angle distributions to align rewards with theoretical priors, thus eliminating the need for ad hoc weights and ensuring robust generalization across diverse detector configurations and varying noise environments.
Our experimental findings highlight how feedback-driven RL methodologies can adapt MST reconstruction to varying material compositions, noise levels, and geometric complexities. Beyond MST, the fusion of RL and physics-driven imaging principles offers a robust paradigm for muon transmission imaging for geological exploration [46], archaeology, mineral prospecting, and other applications, where increased precision and adaptability are paramount. By addressing these limitations, adaptive learning strategies can unlock unprecedented accuracy in next-generation muon imaging systems.
Detection of high-Z objects using multiple scattering of cosmic ray muons
. Rev. Sci. Instrum. 74, 4294–4297 (2003). https://doi.org/10.1063/1.1606536A novel technique to detect special nuclear material using cosmic rays
. Geosci. Instrum. Methods Data Sys. 1(2), 235–238 (2012). https://doi.org/10.5194/gi-1-235-2012A binned clustering algorithm to detect high-Z material using cosmic muons
. J. Instrum. 8,Tomographic imaging with cosmic ray muons
. Science Global Security 16, 37–53 (2008). https://doi.org/10.1080/08929880802335758Surveillance: radiographic imaging with cosmic-ray muons
. Nature 422, 277 (2003). https://doi.org/10.1038/422277aCosmic ray muon computed tomography of spent nuclear fuel in dry storage casks
. Nucl. Instrum. Meth. Phys. Res. Sect. A 842, 48–53 (2017). https://doi.org/10.1016/j.nima.2016.10.040Analysis of spent nuclear fuel imaging using multiple Coulomb scattering of cosmic muons
. IEEE Trans. Nucl. Sci. 63, 2866–2874 (2016). https://doi.org/10.1109/TNS.2016.2618009Cosmic-ray tomography for border security
. Instruments 7, 13 (2023). https://doi.org/10.3390/instruments7010013Muon tomography imaging algorithms for nuclear threat detection inside large volume containers with the Muon Portal detector
. Nucl. Instrum. Meth. Phys. Res. Sect. A 728, 59–68 (2013). https://doi.org/10.1016/j.nima.2013.06.040The effect of cosmic ray muon momentum measurement for monitoring shielded special nuclear materials
. arXiv:2109.02470 (2021). https://doi.org/10.48550/arxiv.2109.02470Statistical reconstruction for cosmic ray muon tomography
. IEEE Trans. Image Process. 16, 1985-1993 (2007). https://doi.org/10.1109/TIP.2007.901239Research on the acceleration of MLS-EM reconstruction algorithm for cosmic ray muon imaging
. CT Theory and Application Research 16, 1–5 (2007). (in Chinese)MLSD-OSEM reconstruction algorithm for cosmic ray muon radiography
.Bayesian image reconstruction for improving detection performance of muon tomography
. IEEE Trans. Image Process. 18, 1080–1089 (2009). https://doi.org/10.1109/TIP.2009.2014423Statistical image reconstruction for muon tomography using a gaussian scale mixture model
. IEEE Trans. Nucl. Sci. 56, 2480–2486 (2009). https://doi.org/10.1109/TNS.2009.2023518Improvement of machine learning-based vertex reconstruction for large liquid scintillator detectors with multiple types of PMTs
. Nucl. Sci. Tech. 33, 93 (2022). https://doi.org/10.1007/s41365-022-01078-yImproving tomographic reconstruction from limited data using mixed-scale dense convolutional neural networks
. J. Imaging 4, 128 (2018). https://doi.org/10.3390/jimaging4110128Experimental validation of material discrimination ability of muon scattering tomography at the TUMUTY facility
. Nucl. Sci. Tech. 30, 120 (2019). https://doi.org/10.1007/s41365-019-0649-4Introduction to reinforcement learning
. arXiv:2408.07712 (2024). https://doi.org/10.48550/arXiv.2408.07712Reinforcement learning in medical image analysis: Concepts, applications, challenges, and future directions
. J. Appl. Clin. Med. Phys. 24,Deep reinforcement learning in medical imaging: A literature review
. Med. Image Anal. 73,Deep reinforcement learning in computer vision: a comprehensive survey
. Artif. Intell. Rev. 55, 2733–2819 (2022). https://doi.org/10.1007/s10462-021-10061-90A reinforcement learning framework for parameter control in computer vision applications
.Evaluating reinforcement learning agents for anatomical landmark detection
. Med. Image Anal. 53, 156–164 (2019). https://doi.org/10.1016/j.media.2019.02.007Multiple landmark detection using multi-agent reinforcement learning
. arXiv:1907.00318 (2019) https://doi.org/10.48550/arxiv.1907.00318Reinforcement learning for object detection in PET imaging
.A reinforcement learning framework for medical image segmentation
.Multi-step medical image segmentation based on reinforcement learning
. J. Ambient. Intell. Human. Comput. 13, 5011–5022 (2022). https://doi.org/10.1007/s12652-020-01905-3End-to-end multimodal image registration via reinforcement learning
. Med. Image Anal. 68,Intelligent parameter tuning in optimization-based iterative CT reconstruction via deep reinforcement learning
. IEEE T. Med. Imaging. 37, 1430 (2018). https://doi.org/10.1109/TMI.2018.2823679Active object recognition by view integration and reinforcement learning
. Robot. Auton. Syst. 31, 71–86 (2000). https://doi.org/10.1016/S0921-8890(99)00079-2PixelRL: Fully convolutional network with reinforcement learning for image processing
. IEEE T. Multimedia. 22, 1704–1719 (2019). https://doi.org/10.1109/TMM.2019.2960636Boosting denoisers with reinforcement learning for image restoration
. Soft Comput. 26, 3261–3272 (2022). https://doi.org/10.1007/s00500-022-06840-3Graph neural network and multi-agent reinforcement learning for machine-process-system integrated control to optimize production yield
. J. Manuf. Syst. 64, 81–93 (2022). https://doi.org/10.1016/j.jmsy.2022.05.018A new efficient imaging reconstruction method for muon scattering tomography
. Nucl. Instrum. Meth. Phys. Res. Sect. A 1069,Research on rapid imaging with cosmic ray muon scattering tomography
. Sci. Rep.-UK. 13,Q-learning
. Mach. Learn. 8, 279–292 (1992). https://doi.org/10.1007/BF00992698Hybrid model for muon tomography and quantitative analysis of image quality
. Nucl. Sci. Tech. 33, 81 (2022). https://doi.org/10.1007/s41365-022-01070-6A comparison of algebraic reconstruction techniques for a single-detector muon computed tomography system
. Nucl. Instrum. Meth. Phys. Res. Sect. A 987,Imaging internal density structure of the Laoheishan volcanic cone with cosmic ray muon radiography
. Nucl. Sci. Tech. 33, 88 (2022). https://doi.org/10.1007/s41365-022-01072-4Yuan-Yuan Liu is an editorial board member for Nuclear Science and Techniques and was not involved in the editorial review, or the decision to publish this article. All authors declare that there are no competing interests.