Study on analytical noise propagation in convolutional neural network methods used in computed tomography imaging

Study on analytical noise propagation in convolutional neural network methods used in computed tomography imaging 1 2 first-author https://orcid.org/0000-0002-3799-6804 1 2 1 2 corresp xingyx@tsinghua.edu.cn xingyx@tsinghua.edu.cn xingyx@tsinghua.edu.cn Department of Engineering Physics, Tsinghua University, Beijing 100084, China Department of Engineering Physics, Tsinghua University, Beijing 100084, China Key Laboratory of Particle & Radiation Imaging, Tsinghua University, Beijing 100084, China Key Laboratory of Particle & Radiation Imaging, Tsinghua University, Beijing 100084, China Neural network methods have recently emerged as a hot topic in computed tomography (CT) imaging owing to their powerful fitting ability; however, their potential applications still need to be carefully studied because their results are often difficult to interpret and are ambiguous in generalizability. Thus, quality assessments of the results obtained from a neural network are necessary to evaluate the neural network. Assessing the image quality of neural networks using traditional objective measurements is not appropriate because neural networks are nonstationary and nonlinear. In contrast, subjective assessments are trustworthy, although they are time- and energy-consuming for radiologists. Model observers that mimic subjective assessment require the mean and covariance of images, which are calculated from numerous image samples; however, this has not yet been applied to the evaluation of neural networks. In this study, we propose an analytical method for noise propagation from a single projection to efficiently evaluate convolutional neural networks (CNNs) in the CT imaging field. We propagate noise through nonlinear layers in a CNN using the Taylor expansion. Nesting of the linear and nonlinear layer noise propagation constitutes the covariance estimation of the CNN. A commonly used U-net structure is adopted for validation. The results reveal that the covariance estimation obtained from the proposed analytical method agrees well with that obtained from the image samples for different phantoms, noise levels, and activation functions, demonstrating that propagating noise from only a single projection is feasible for CNN methods in CT reconstruction. In addition, we use covariance estimation to provide three measurements for the qualitative and quantitative performance evaluation of U-net. The results indicate that the network cannot be applied to projections with high noise levels and possesses limitations in terms of efficiency for processing low-noise projections. U-net is more effective in improving the image quality of smooth regions compared with that of the edge. LeakyReLU outperforms Swish in terms of noise reduction. Noise propagation Convolutional neural network Image quality assessment Introduction 1 In recent years, neural networks have been applied in computed tomography (CT) imaging. Several studies on the applications of such networks have been published, and their potential in solving several problems in the field of CT imaging has been extensively evaluated. Such networks have been used in applications such as spectral distortion correction [ 1 1 , 2 2 ] for photon-counting X-ray CT, dual-domain learning for two-dimensional [ 3 3 , 4 4 ] and three-dimensional [ 5 5 - 7 7 5-7 ] low-dose CT reconstruction, sparse-view [ 8 8 ] and limited-angle [ 9 9 ] CT reconstruction, noise suppression in the sinogram domain [ 10 10 ] and image domain [ 11 11 ], dual-energy imaging with energy-integrating detectors [ 12 12 ] and photon-counting detectors [ 13 13 ], and CT artifact reduction [ 14 14 , 15 15 ]. However, neural networks have not yet been widely used in practice owing to their confidence scores. Hence, researchers have adopted objective and subjective image quality assessments to evaluate the feasibility of using neural networks in CT imaging [ 16 16 , 17 17 ]. However, traditional metrics are preferred for objective assessments. The commonly used contrast-to-noise ratio measures the region of interest (ROI) clarity, signal-to-noise ratio measures the noise level, noise power spectrum (NPS) measures the noise correlation, and modulation transfer function measures the spatial frequency [ 18 18 , 19 19 ]. Neural networks are normally nonlinear, nonstationary, and unexplainable; however, to the best of our knowledge, limited research on theoretically tractable methods has been conducted. Subjective assessments are commonly used in the field of CT imaging. Radiologists are invited to observe and score images obtained from various methods [ 20 20 , 21 21 ]. Because subjective assessments are time-consuming and laborious, researchers study model observers to simulate the evaluation behavior of radiologists. The assessment of image quality carried out by a radiologist, that is, a human observer, is modeled as a classification problem solved by hypothesis testing. The likelihood ratio is used as the decision variable to obtain an ideal observer (IO). Because the IO is intractable, a linear approximation of the IO is assumed to obtain a Hotelling observer (HO). Combining the human-eye model with HO, researchers have obtained the most widely used channelized Hotelling observer (CHO) [ 22 22 ]. The CHO agrees well with human observers [ 23 23 , 24 24 ]; however, it requires knowledge of the image mean and covariance. Recently, neural networks have been introduced to explore nonlinear model observers that enable better approximation of the human observer or better detectability for auxiliary diagnoses [ 25 25 - 27 27 25-27 ]. However, these methods only target traditional reconstruction methods [ 28 28 , 29 29 ]. The application of these methods to neural network reconstruction has not yet been explored. Noise propagation through a reconstruction neural network is necessary for assessing the performance of the network. Covariance prediction can reveal the uncertainty in inference, thus providing a safer answer to the CT reconstruction problem. Furthermore, it can be used in the calculation of model observers and subjective assessments. The covariance estimation of neural-network outputs is currently of significant interest. Abdelaziz et al. [ 30 30 ] studied the uncertainty propagation of deep neural networks for automatic speech recognition with the assumption that the output was Gaussian. Lee et al. [ 31 31 ] used a Gaussian process equivalent of a deep fully connected neural network to obtain an exact Bayesian inference under the assumption that the parameters and layer outputs follow an independent and identical distribution. Tanno et al.[ 32 32 ] simultaneously estimated the mean and covariance of high-resolution outputs from low-resolution magnetic resonance images based on the assumption of a Gaussian distribution. However, for CT imaging, covariance estimation of the neural network output has not yet been studied. In this study, we propose a new analytical noise propagation method, that is, covariance estimation, particularly for a convolutional neural network (CNN) in CT imaging of noisy projections. With a trained CNN ready for inference, we propagate the noise layer by layer. For linear layers, the output covariance can be calculated accurately. For nonlinear activation layers, we perform a Taylor expansion to obtain a linear approximation, which enables linear propagation of noise through the nonlinear layers. Because a CNN is a stack of linear and nonlinear layers, its covariance estimation is a combination of layer noise propagations. We validate the results of the covariance estimation method by comparing the results with those of statistical estimations using noisy projection and reconstruction samples with different phantoms, noise levels, and activation functions. Methods 1 Physical Model for CT Imaging 2 A simple model for data acquisition in a CT scan is formulated as <math display="block" id="M1"><mrow><mi>I</mi><mo>=</mo><msub><mi>I</mi><mn>0</mn></msub><msup><mi>e</mi><mrow><mo>−</mo><mi>p</mi></mrow></msup><mo>=</mo><msub><mi>I</mi><mn>0</mn></msub><msup><mi>e</mi><mrow><mo>−</mo><mstyle displaystyle="true"><mrow><mo>∫</mo><mrow><mi>μ</mi><mi>d</mi><mi>r</mi></mrow></mrow></mstyle></mrow></msup><mo>,</mo></mrow></math> where <math id="M2"><mi>I</mi></math> represents detected photons, and <math id="M3"><mrow><msub><mi>I</mi><mn>0</mn></msub></mrow></math> represents incident photons. Projection <math id="M4"><mi>p</mi></math> is a line integral of the linear attenuation coefficient <math id="M5"><mi>μ</mi></math> . Usually, the noise distribution of <math id="M6"><mi>I</mi></math> is assumed to be Poisson or Gaussian, and the noise distribution of <math id="M7"><mi>p</mi></math> can be approximated as Gaussian with mean <math id="M8"><mover accent="true"><mi>p</mi><mo>¯</mo></mover></math> and variance <math id="M9"><mrow><mi>exp</mi><mo stretchy="false">(</mo><mover accent="true"><mi>p</mi><mo>¯</mo></mover><mo stretchy="false">)</mo><mo>/</mo><msub><mi>I</mi><mn>0</mn></msub></mrow></math> , that is, <math id="M10"><mrow><mi>p</mi><mo>~</mo><mi mathvariant="script">N</mi><mo stretchy="false">(</mo><mover accent="true"><mi>p</mi><mo>¯</mo></mover><mo>,</mo><mi>e</mi><mi>x</mi><mi>p</mi><mo stretchy="false">(</mo><mover accent="true"><mi>p</mi><mo>¯</mo></mover><mo stretchy="false">)</mo><mo>/</mo><msub><mi>I</mi><mn>0</mn></msub><mo stretchy="false">)</mo></mrow></math> . Analytical, iterative, and neural network methods can be used to reconstruct a linear attenuation coefficient map <math id="M11"><mstyle mathvariant="bold" mathsize="normal"><mi>μ</mi></mstyle></math> from its projection <math id="M12"><mstyle mathvariant="bold" mathsize="normal"><mi>p</mi></mstyle></math> . A CNN is a commonly used reconstruction method for CT imaging. A CNN typically consists of five types of basic layers: convolution, activation, batch normalization, pooling, and up-sampling layers. The overall operation of the network is a cascade of these layers, that is, <math display="block" id="M13"><mrow><mstyle mathvariant="bold" mathsize="normal"><mi>μ</mi></mstyle><mo>=</mo><mi>Φ</mi><mo stretchy="false">(</mo><mstyle mathvariant="bold" mathsize="normal"><mi>p</mi></mstyle><mo stretchy="false">)</mo><mo>=</mo><msub><mi>φ</mi><mi>L</mi></msub><mrow><mo>(</mo><mrow><msub><mi>φ</mi><mrow><mi>L</mi><mo>−</mo><mn>1</mn></mrow></msub><mrow><mo>(</mo><mrow><mo>⋯</mo><msub><mi>φ</mi><mi>l</mi></msub><mo>⋯</mo><mrow><mo>(</mo><mrow><msub><mi>φ</mi><mn>2</mn></msub><mrow><mo>(</mo><mrow><msub><mi>φ</mi><mn>1</mn></msub><mo stretchy="false">(</mo><mstyle mathvariant="bold" mathsize="normal"><mi>p</mi></mstyle><mo stretchy="false">)</mo></mrow><mo>)</mo></mrow></mrow><mo>)</mo></mrow></mrow><mo>)</mo></mrow></mrow><mo>)</mo></mrow><mo>,</mo></mrow></math> where <math id="M14"><mrow><msub><mi>φ</mi><mi>l</mi></msub></mrow></math> denotes the operation function of one layer, and <math id="M15"><mi>Φ</mi></math> denotes the overall function of the neural network. Evidently, the noise in <math id="M16"><mstyle mathvariant="bold" mathsize="normal"><mi>p</mi></mstyle></math> will result in a noisy <math id="M17"><mstyle mathvariant="bold" mathsize="normal"><mi>μ</mi></mstyle></math> , even though several networks are used to reduce noise. We discovered that the noise propagation through <math id="M18"><mrow><msub><mi>φ</mi><mi>l</mi></msub></mrow></math> 's to output <math id="M19"><mstyle mathvariant="bold" mathsize="normal"><mi>μ</mi></mstyle></math> can be studied step-by-step if a network model is ready to serve for inference, that is, network parameters are set. The key lies in sorting out the covariance estimation through the five basic layers constituting the entire CNN. Covariance Propagation through Basic Layers of a CNN 2 Let vector <math id="M20"><mrow><mstyle mathvariant="bold" mathsize="normal"><mi>x</mi></mstyle><mo>∈</mo><msup><mi>ℝ</mi><mrow><mi>M</mi><mo>×</mo><mn>1</mn></mrow></msup></mrow></math> be an arbitrary layer input, and let <math id="M21"><mstyle mathvariant="bold" mathsize="normal"><mi>y</mi></mstyle></math> be the corresponding layer output. This section presents the covariance estimation of <math id="M22"><mstyle mathvariant="bold" mathsize="normal"><mi>y</mi></mstyle></math> from <math id="M23"><mstyle mathvariant="bold" mathsize="normal"><mi>x</mi></mstyle></math> . Convolution Layer 3 For a convolutional layer, its output <math id="M24"><mrow><msub><mstyle mathvariant="bold" mathsize="normal"><mi>y</mi></mstyle><mrow><mtext>conv</mtext></mrow></msub><mo>∈</mo><msup><mi>ℝ</mi><mrow><mi>N</mi><mo>×</mo><mn>1</mn></mrow></msup></mrow></math> can be expressed as a linear combination of inputs: <math display="block" id="M25"><mrow><msub><mstyle mathvariant="bold" mathsize="normal"><mi>y</mi></mstyle><mrow><mtext>conv</mtext></mrow></msub><mo>=</mo><mstyle displaystyle="true"><munderover><mo>∑</mo><mrow><mi>i</mi><mo>=</mo><mn>1</mn></mrow><mi>C</mi></munderover><mrow><msub><mstyle mathvariant="bold" mathsize="normal"><mi>W</mi></mstyle><mi>i</mi></msub><msub><mstyle mathvariant="bold" mathsize="normal"><mi>x</mi></mstyle><mi>i</mi></msub><mo>+</mo><msub><mi>b</mi><mi>i</mi></msub></mrow></mstyle><mo>,</mo></mrow></math> where <math id="M26"><mi>C</mi></math> denotes the number of input channels, <math id="M27"><mrow><mstyle mathvariant="bold" mathsize="normal"><mi>W</mi></mstyle><mo>∈</mo><msup><mi>ℝ</mi><mrow><mi>N</mi><mo>×</mo><mi>M</mi></mrow></msup></mrow></math> represents the convolutional weighting matrix, and <math id="M28"><mrow><msub><mi>b</mi><mi>i</mi></msub></mrow></math> denotes bias. The convolution is linear; hence, it is easy to obtain: <math display="block" id="M29"><mrow><mtext>Cov</mtext><mo stretchy="false">(</mo><msub><mstyle mathvariant="bold" mathsize="normal"><mi>y</mi></mstyle><mrow><mtext>conv</mtext></mrow></msub><mo stretchy="false">)</mo><mo>=</mo><mstyle displaystyle="true"><munderover><mo>∑</mo><mrow><mi>j</mi><mo>=</mo><mn>1</mn></mrow><mi>C</mi></munderover><mrow><mstyle displaystyle="true"><munderover><mo>∑</mo><mrow><mi>i</mi><mo>=</mo><mn>1</mn></mrow><mi>C</mi></munderover><mrow><msub><mstyle mathvariant="bold" mathsize="normal"><mi>W</mi></mstyle><mi>i</mi></msub><mtext>Cov</mtext><mo stretchy="false">(</mo><msub><mstyle mathvariant="bold" mathsize="normal"><mi>x</mi></mstyle><mi>i</mi></msub><mo>,</mo><msub><mstyle mathvariant="bold" mathsize="normal"><mi>x</mi></mstyle><mi>j</mi></msub><mo stretchy="false">)</mo></mrow></mstyle></mrow></mstyle><msubsup><mstyle mathvariant="bold" mathsize="normal"><mi>W</mi></mstyle><mi>j</mi><mi>T</mi></msubsup><mo>.</mo></mrow></math> Activation Layer 3 In an activation layer, an input is normally fed into a nonlinear function, <math id="M30"><mrow><mi>f</mi><mo stretchy="false">(</mo><mo>⋅</mo><mo stretchy="false">)</mo></mrow></math> : <math display="block" id="M31"><mrow><msub><mstyle mathvariant="bold" mathsize="normal"><mi>y</mi></mstyle><mrow><mtext>af</mtext></mrow></msub><mo>=</mo><mi>f</mi><mo stretchy="false">(</mo><mstyle mathvariant="bold" mathsize="normal"><mi>x</mi></mstyle><mo stretchy="false">)</mo><mo>.</mo></mrow></math> Because it is nonlinear, we perform the 1 st order Taylor expansion to obtain its linear approximation: <math display="block" id="M32"><mrow><msub><mstyle mathvariant="bold" mathsize="normal"><mi>y</mi></mstyle><mrow><mtext>af</mtext></mrow></msub><mo>=</mo><mi>f</mi><mo stretchy="false">(</mo><mstyle mathvariant="bold" mathsize="normal"><mi>x</mi></mstyle><mo stretchy="false">)</mo><mo>≃</mo><mi>f</mi><mo stretchy="false">(</mo><mstyle mathvariant="bold" mathsize="normal"><mover accent="true"><mi>x</mi><mo>¯</mo></mover></mstyle><mo stretchy="false">)</mo><mo>+</mo><msup><mi>f</mi><mo>′</mo></msup><mo stretchy="false">(</mo><mstyle mathvariant="bold" mathsize="normal"><mover accent="true"><mi>x</mi><mo>¯</mo></mover></mstyle><mo stretchy="false">)</mo><mo>⋅</mo><mo stretchy="false">(</mo><mstyle mathvariant="bold" mathsize="normal"><mi>x</mi></mstyle><mo>−</mo><mstyle mathvariant="bold" mathsize="normal"><mover accent="true"><mi>x</mi><mo>¯</mo></mover></mstyle><mo stretchy="false">)</mo><mo>=</mo><mstyle mathvariant="bold" mathsize="normal"><mi>F</mi></mstyle><mo stretchy="false">(</mo><mstyle mathvariant="bold" mathsize="normal"><mover accent="true"><mi>x</mi><mo>¯</mo></mover></mstyle><mo stretchy="false">)</mo><mstyle mathvariant="bold" mathsize="normal"><mi>x</mi></mstyle><mo>+</mo><mo stretchy="false">(</mo><mi>f</mi><mo stretchy="false">(</mo><mstyle mathvariant="bold" mathsize="normal"><mover accent="true"><mi>x</mi><mo>¯</mo></mover></mstyle><mo stretchy="false">)</mo><mo>−</mo><mstyle mathvariant="bold" mathsize="normal"><mi>F</mi></mstyle><mo stretchy="false">(</mo><mstyle mathvariant="bold" mathsize="normal"><mover accent="true"><mi>x</mi><mo>¯</mo></mover></mstyle><mo stretchy="false">)</mo><mstyle mathvariant="bold" mathsize="normal"><mover accent="true"><mi>x</mi><mo>¯</mo></mover></mstyle><mo stretchy="false">)</mo></mrow></math> where the Taylor-based coefficient matrix <math id="M33"><mrow><mstyle mathvariant="bold" mathsize="normal"><mi>F</mi></mstyle><mo>∈</mo><msup><mi>ℝ</mi><mrow><mi>M</mi><mo>×</mo><mi>M</mi></mrow></msup></mrow></math> is diagonal to <math id="M34"><mrow><msub><mrow><mo stretchy="false">[</mo><mstyle mathvariant="bold" mathsize="normal"><mi>F</mi></mstyle><mo stretchy="false">]</mo></mrow><mrow><mi>m</mi><mo>,</mo><mi>m</mi></mrow></msub><mo>=</mo><msup><mi>f</mi><mo>′</mo></msup><mo stretchy="false">(</mo><msub><mrow><mo stretchy="false">[</mo><mstyle mathvariant="bold" mathsize="normal"><mover accent="true"><mi>x</mi><mo>¯</mo></mover></mstyle><mo stretchy="false">]</mo></mrow><mi>m</mi></msub><mo stretchy="false">)</mo></mrow></math> . Thus, the covariance of the nonlinear transformation layer can be estimated by <math display="block" id="M35"><mrow><mtext>Cov</mtext><mo stretchy="false">(</mo><msub><mstyle mathvariant="bold" mathsize="normal"><mi>y</mi></mstyle><mrow><mtext>af</mtext></mrow></msub><mo stretchy="false">)</mo><mo>≃</mo><mstyle mathvariant="bold" mathsize="normal"><mi>F</mi></mstyle><mi>C</mi><mi>o</mi><mi>v</mi><mo stretchy="false">(</mo><mstyle mathvariant="bold" mathsize="normal"><mi>x</mi></mstyle><mo stretchy="false">)</mo><msup><mstyle mathvariant="bold" mathsize="normal"><mi>F</mi></mstyle><mi>T</mi></msup><mo>.</mo></mrow></math> Batch Normalization Layer 3 For a batch normalization layer, the input is normalized as <math display="block" id="M36"><mrow><msub><mstyle mathvariant="bold" mathsize="normal"><mi>y</mi></mstyle><mrow><mtext>bn</mtext></mrow></msub><mo>=</mo><mfrac><mi>γ</mi><mrow><msqrt><mrow><msubsup><mi>σ</mi><mi>B</mi><mn>2</mn></msubsup><mo>+</mo><mi>ε</mi></mrow></msqrt></mrow></mfrac><mstyle mathvariant="bold" mathsize="normal"><mi>x</mi></mstyle><mo>+</mo><mrow><mo>(</mo><mrow><mi>β</mi><mo>−</mo><mfrac><mrow><mi>γ</mi><msub><mi>u</mi><mi>B</mi></msub></mrow><mrow><msqrt><mrow><msubsup><mi>σ</mi><mi>B</mi><mn>2</mn></msubsup><mo>+</mo><mi>ε</mi></mrow></msqrt></mrow></mfrac></mrow><mo>)</mo></mrow><mo>.</mo></mrow></math> Here, <math id="M37"><mi>γ</mi></math> and <math id="M38"><mi>β</mi></math> are hyperparameters that are learned during training and frozen when inferencing. <math id="M39"><mrow><msub><mi>u</mi><mi>B</mi></msub></mrow></math> and <math id="M40"><mrow><msubsup><mi>σ</mi><mi>B</mi><mn>2</mn></msubsup></mrow></math> represent the batch mean and variance, respectively, which are also frozen during inference. Hence, the covariance propagating through the batch normalization layer is <math display="block" id="M41"><mrow><mtext>Cov</mtext><mo stretchy="false">(</mo><msub><mstyle mathvariant="bold" mathsize="normal"><mi>y</mi></mstyle><mrow><mtext>bn</mtext></mrow></msub><mo stretchy="false">)</mo><mo>=</mo><mfrac><mrow><msup><mi>γ</mi><mn>2</mn></msup></mrow><mrow><msubsup><mi>σ</mi><mi>B</mi><mn>2</mn></msubsup><mo>+</mo><mi>ε</mi></mrow></mfrac><mtext>Cov</mtext><mo stretchy="false">(</mo><mstyle mathvariant="bold" mathsize="normal"><mi>x</mi></mstyle><mo stretchy="false">)</mo><mo>≡</mo><mi>η</mi><mtext>Cov</mtext><mo stretchy="false">(</mo><mstyle mathvariant="bold" mathsize="normal"><mi>x</mi></mstyle><mo stretchy="false">)</mo></mrow></math> Pooling Layer 3 Average pooling is a widely used method in this field. This can be interpreted as a convolution operation of kernel size <math id="M42"><mi>k</mi></math> with stride <math id="M43"><mi>s</mi></math> , where the convolution kernel is a constant matrix with value <math id="M44"><mrow><mn>1</mn><mo>/</mo><msup><mi>k</mi><mn>2</mn></msup></mrow></math> . Similar to the operation of a convolution layer, its output <math id="M45"><mrow><msub><mstyle mathvariant="bold" mathsize="normal"><mi>y</mi></mstyle><mrow><mtext>ap</mtext></mrow></msub><mo>∈</mo><msup><mi>ℝ</mi><mrow><mi>N</mi><mo>×</mo><mn>1</mn></mrow></msup></mrow></math> can be formulated as a linear transformation of the input as follows: <math display="block" id="M46"><mrow><msub><mstyle mathvariant="bold" mathsize="normal"><mi>y</mi></mstyle><mrow><mtext>ap</mtext></mrow></msub><mo>=</mo><mstyle mathvariant="bold" mathsize="normal"><mi>A</mi><mi>x</mi></mstyle></mrow></math> Here, the average pooling matrix <math id="M47"><mrow><mstyle mathvariant="bold" mathsize="normal"><mi>A</mi></mstyle><mo>∈</mo><msup><mi>ℝ</mi><mrow><mi>N</mi><mo>×</mo><mi>M</mi></mrow></msup></mrow></math> is sparse with <math id="M48"><mrow><mi>N</mi><mo>=</mo><mo stretchy="false">(</mo><mi>M</mi><mo>−</mo><mi>k</mi><mo stretchy="false">)</mo><mo>/</mo><mi>s</mi><mo>+</mo><mn>1</mn></mrow></math> . Thus, <math display="block" id="M49"><mrow><mtext>Cov(</mtext><msub><mstyle mathvariant="bold" mathsize="normal"><mi>y</mi></mstyle><mrow><mtext>ap</mtext></mrow></msub><mo stretchy="false">)</mo><mo>=</mo><mstyle mathvariant="bold" mathsize="normal"><mi>A</mi></mstyle><mtext>Cov</mtext><mo stretchy="false">(</mo><mstyle mathvariant="bold" mathsize="normal"><mi>x</mi></mstyle><mo stretchy="false">)</mo><msup><mstyle mathvariant="bold" mathsize="normal"><mi>A</mi></mstyle><mi>T</mi></msup></mrow></math> Up-sampling Layer 3 For an up-sampling layer, each element of the input is duplicated and can be expressed as a linear combination of the input: <math display="block" id="M50"><mrow><msub><mstyle mathvariant="bold" mathsize="normal"><mi>y</mi></mstyle><mrow><mtext>up</mtext></mrow></msub><mo>=</mo><mstyle mathvariant="bold" mathsize="normal"><mi>U</mi><mi>x</mi></mstyle></mrow></math> where the up sampling matrix <math id="M51"><mrow><mstyle mathvariant="bold" mathsize="normal"><mi>U</mi></mstyle><mo>∈</mo><msup><mi>ℝ</mi><mrow><mi>N</mi><mo>×</mo><mi>M</mi></mrow></msup></mrow></math> is a sparse matrix with only one element in each row and <math id="M52"><mrow><mi>N</mi><mo>=</mo><mn>2</mn><mi>M</mi></mrow></math> . The covariance estimated from an up-sampling layer is <math display="block" id="M53"><mrow><mtext>Cov</mtext><mo stretchy="false">(</mo><msub><mstyle mathvariant="bold" mathsize="normal"><mi>y</mi></mstyle><mrow><mtext>up</mtext></mrow></msub><mo stretchy="false">)</mo><mo>=</mo><mstyle mathvariant="bold" mathsize="normal"><mi>U</mi></mstyle><mtext>Cov</mtext><mo stretchy="false">(</mo><mstyle mathvariant="bold" mathsize="normal"><mi>x</mi></mstyle><mo stretchy="false">)</mo><msup><mstyle mathvariant="bold" mathsize="normal"><mi>U</mi></mstyle><mi>T</mi></msup></mrow></math> Example: U-net 2 We adopt a commonly used U-net structure to denoise the projection, followed by reconstruction with the filter back projection (FBP) method: <math display="block" id="M54"><mrow><mstyle mathvariant="bold" mathsize="normal"><mi>μ</mi></mstyle><mtext>=</mtext><mi>Φ</mi><mo stretchy="false">(</mo><mstyle mathvariant="bold" mathsize="normal"><mi>p</mi></mstyle><mo stretchy="false">)</mo><mtext>=</mtext><mi mathvariant="script">O</mi><mrow><mo>(</mo><mrow><msub><mi>φ</mi><mi>L</mi></msub><mrow><mo>(</mo><mrow><msub><mi>φ</mi><mrow><mi>L</mi><mo>−</mo><mn>1</mn></mrow></msub><mrow><mo>(</mo><mrow><mo>⋯</mo><msub><mi>φ</mi><mi>l</mi></msub><mo>⋯</mo><mrow><mo>(</mo><mrow><msub><mi>φ</mi><mn>2</mn></msub><mrow><mo>(</mo><mrow><msub><mi>φ</mi><mn>1</mn></msub><mo stretchy="false">(</mo><mstyle mathvariant="bold" mathsize="normal"><mi>p</mi></mstyle><mo stretchy="false">)</mo></mrow><mo>)</mo></mrow></mrow><mo>)</mo></mrow></mrow><mo>)</mo></mrow></mrow><mo>)</mo></mrow></mrow><mo>)</mo></mrow></mrow></math> Here, <math id="M55"><mi mathvariant="script">O</mi></math> represents the linear FBP operator. <math id="M56"><mrow><mstyle mathvariant="bold" mathsize="normal"><mi>p</mi></mstyle><mo>∈</mo><msup><mi>ℝ</mi><mrow><mi>M</mi><mo>×</mo><mn>1</mn></mrow></msup></mrow></math> denotes an input projection, and <math id="M57"><mrow><mstyle mathvariant="bold" mathsize="normal"><mi>μ</mi></mstyle><mo>∈</mo><msup><mi>ℝ</mi><mrow><mi>N</mi><mo>×</mo><mn>1</mn></mrow></msup></mrow></math> represents the corresponding reconstruction. The reconstruction flowchart is illustrated in Fig. 1 Fig. 1 . A concatenate layer and residual layer are included in U-net. The concatenated layer merges the feature map in the 1 st layer to the 6 th layer, and the residual layer adds the input projection to the 9 th layer to obtain the output projection. We iteratively estimate the covariance of reconstruction predicted from the trained U-net: <math display="block" id="M58"><mrow><mtext>Cov</mtext><mo stretchy="false">(</mo><msubsup><mstyle mathvariant="bold" mathsize="normal"><mi>z</mi></mstyle><mi>i</mi><mi>l</mi></msubsup><mo>,</mo><msubsup><mstyle mathvariant="bold" mathsize="normal"><mi>z</mi></mstyle><mi>j</mi><mi>l</mi></msubsup><mo stretchy="false">)</mo><mo>≃</mo><mrow><mo>{</mo><mrow><mtable columnalign="left"><mtr columnalign="left"><mtd columnalign="left"><mrow><msubsup><mi>η</mi><mi>i</mi><mi>l</mi></msubsup><msubsup><mstyle mathvariant="bold" mathsize="normal"><mi>F</mi></mstyle><mi>i</mi><mi>l</mi></msubsup><mrow><mo>[</mo><mrow><mstyle displaystyle="true"><munderover><mo>∑</mo><mrow><msup><mi>j</mi><mo>′</mo></msup><mo>=</mo><mn>1</mn></mrow><mrow><msup><mi>C</mi><mrow><mi>l</mi><mo>−</mo><mn>1</mn></mrow></msup></mrow></munderover><mrow><mstyle displaystyle="true"><munderover><mo>∑</mo><mrow><msup><mi>i</mi><mo>′</mo></msup><mo>=</mo><mn>1</mn></mrow><mrow><msup><mi>C</mi><mrow><mi>l</mi><mo>−</mo><mn>1</mn></mrow></msup></mrow></munderover><mrow><msubsup><mstyle mathvariant="bold" mathsize="normal"><mi>W</mi></mstyle><mrow><msup><mi>i</mi><mo>′</mo></msup><mo>,</mo><mi>i</mi></mrow><mi>l</mi></msubsup><mtext>Cov</mtext><mrow><mo>(</mo><mrow><msubsup><mstyle mathvariant="bold" mathsize="normal"><mi>z</mi></mstyle><msup><mi>i</mi><mo>′</mo></msup><mrow><mi>l</mi><mo>−</mo><mn>1</mn></mrow></msubsup><mo>,</mo><msubsup><mstyle mathvariant="bold" mathsize="normal"><mi>z</mi></mstyle><msup><mi>j</mi><mo>′</mo></msup><mrow><mi>l</mi><mo>−</mo><mn>1</mn></mrow></msubsup></mrow><mo>)</mo></mrow><msup><mrow><mrow><mo>(</mo><mrow><msubsup><mstyle mathvariant="bold" mathsize="normal"><mi>W</mi></mstyle><mrow><msup><mi>j</mi><mo>′</mo></msup><mo>,</mo><mi>j</mi></mrow><mi>l</mi></msubsup></mrow><mo>)</mo></mrow></mrow><mi>T</mi></msup></mrow></mstyle></mrow></mstyle></mrow><mo>]</mo></mrow><msup><mrow><mo stretchy="false">(</mo><msubsup><mi>η</mi><mi>j</mi><mi>l</mi></msubsup><msubsup><mstyle mathvariant="bold" mathsize="normal"><mi>F</mi></mstyle><mi>j</mi><mi>l</mi></msubsup><mo stretchy="false">)</mo></mrow><mi>T</mi></msup></mrow></mtd><mtd columnalign="left"><mrow><mo>,</mo><mi>l</mi><mo>=</mo><mn>1</mn><mo>,</mo><mn>3</mn><mo>,</mo><mn>4</mn><mo>,</mo><mn>5</mn><mo>,</mo><mn>8</mn><mo>,</mo><mn>9</mn></mrow></mtd></mtr><mtr columnalign="left"><mtd columnalign="left"><mrow><msub><mstyle mathvariant="bold" mathsize="normal"><mi>A</mi></mstyle><mi>i</mi></msub><mtext>Cov</mtext><mrow><mo>(</mo><mrow><msubsup><mstyle mathvariant="bold" mathsize="normal"><mi>z</mi></mstyle><mi>i</mi><mrow><mi>l</mi><mo>−</mo><mn>1</mn></mrow></msubsup><mo>,</mo><msubsup><mstyle mathvariant="bold" mathsize="normal"><mi>z</mi></mstyle><mi>j</mi><mrow><mi>l</mi><mo>−</mo><mn>1</mn></mrow></msubsup></mrow><mo>)</mo></mrow><msubsup><mstyle mathvariant="bold" mathsize="normal"><mi>A</mi></mstyle><mi>j</mi><mi>T</mi></msubsup></mrow></mtd><mtd columnalign="left"><mrow><mo>,</mo><mi>l</mi><mo>=</mo><mn>2</mn></mrow></mtd></mtr><mtr columnalign="left"><mtd columnalign="left"><mrow><msub><mstyle mathvariant="bold" mathsize="normal"><mi>U</mi></mstyle><mi>i</mi></msub><mtext>Cov</mtext><mrow><mo>(</mo><mrow><msubsup><mstyle mathvariant="bold" mathsize="normal"><mi>z</mi></mstyle><mi>i</mi><mrow><mi>l</mi><mo>−</mo><mn>1</mn></mrow></msubsup><mo>,</mo><msubsup><mstyle mathvariant="bold" mathsize="normal"><mi>z</mi></mstyle><mi>j</mi><mrow><mi>l</mi><mo>−</mo><mn>1</mn></mrow></msubsup></mrow><mo>)</mo></mrow><msubsup><mstyle mathvariant="bold" mathsize="normal"><mi>U</mi></mstyle><mi>j</mi><mi>T</mi></msubsup></mrow></mtd><mtd columnalign="left"><mrow><mo>,</mo><mi>l</mi><mo>=</mo><mn>6</mn></mrow></mtd></mtr></mtable></mrow></mrow></mrow></math> where <math id="M59"><mstyle mathvariant="bold" mathsize="normal"><mi>z</mi></mstyle></math> represents the latent variable in the hidden layer of U-net. <math id="M60"><mrow><msup><mstyle mathvariant="bold" mathsize="normal"><mi>z</mi></mstyle><mn>0</mn></msup><mo>=</mo><mstyle mathvariant="bold" mathsize="normal"><mi>p</mi></mstyle></mrow></math> when <math id="M61"><mrow><mi>l</mi><mo>=</mo><mn>1</mn></mrow></math> and <math id="M62"><mrow><msub><mrow><mtext>[Cov</mtext><mo stretchy="false">(</mo><mstyle mathvariant="bold" mathsize="normal"><mi>p</mi></mstyle><mo stretchy="false">)</mo><mo stretchy="false">]</mo></mrow><mrow><mi>n</mi><mo>,</mo><mi>n</mi></mrow></msub><mo>=</mo><mi>e</mi><mi>x</mi><mi>p</mi><mo stretchy="false">(</mo><mstyle mathvariant="bold" mathsize="normal"><mover accent="true"><mi>p</mi><mo>¯</mo></mover></mstyle><mo stretchy="false">)</mo><mo>/</mo><msub><mi>I</mi><mn>0</mn></msub></mrow></math> . The subscript <math id="M63"><mrow><msubsup><mstyle mathvariant="bold" mathsize="normal"><mi>W</mi></mstyle><mrow><msup><mi>i</mi><mo>′</mo></msup><mo>,</mo><mi>i</mi></mrow><mi>l</mi></msubsup></mrow></math> denotes a convolutional weighting matrix from the <math id="M64"><mrow><msup><msup><mi>i</mi><mo>′</mo></msup><mrow><mtext>th</mtext></mrow></msup></mrow></math> channel input of the <math id="M65"><mrow><msup><mi>l</mi><mrow><mtext>th</mtext></mrow></msup></mrow></math> layer to its <math id="M66"><mrow><msup><mi>i</mi><mrow><mtext>th</mtext></mrow></msup></mrow></math> channel output, where <math id="M67"><mrow><msup><mi>C</mi><mi>l</mi></msup></mrow></math> represents the total number of channels in the <math id="M68"><mrow><msup><mi>l</mi><mrow><mtext>th</mtext></mrow></msup></mrow></math> layer. With a concatenation operation, the 7 th layer contains feature maps from the 1 st and 6 th layers. Thus, three types of covariances for the 7 th layer must be present: 1) covariance between channels in the 6 th layer, 2) covariance between channels in the 1 st layer, and 3) covariance between channels in the 1 st layer and 6 th layer. The covariance of cases 1) and 2) has already been estimated using Eq. (15), and the covariance of case 3) can be estimated as <math display="block" id="M69"><mrow><mtext>Cov</mtext><mrow><mo>(</mo><mrow><msubsup><mstyle mathvariant="bold" mathsize="normal"><mi>z</mi></mstyle><mi>i</mi><mn>6</mn></msubsup><mo>,</mo><msubsup><mstyle mathvariant="bold" mathsize="normal"><mi>z</mi></mstyle><mi>j</mi><mn>1</mn></msubsup></mrow><mo>)</mo></mrow><mo>=</mo><mtext>Cov</mtext><msup><mrow><mrow><mo>(</mo><mrow><msubsup><mstyle mathvariant="bold" mathsize="normal"><mi>z</mi></mstyle><mi>i</mi><mn>1</mn></msubsup><mo>,</mo><msubsup><mstyle mathvariant="bold" mathsize="normal"><mi>z</mi></mstyle><mi>j</mi><mn>6</mn></msubsup></mrow><mo>)</mo></mrow></mrow><mi>T</mi></msup><mo>=</mo><msub><mstyle mathvariant="bold" mathsize="normal"><mi>U</mi></mstyle><mi>i</mi></msub><mtext>Cov</mtext><mrow><mo>(</mo><mrow><msubsup><mstyle mathvariant="bold" mathsize="normal"><mi>z</mi></mstyle><mi>i</mi><mn>5</mn></msubsup><mo>,</mo><msubsup><mstyle mathvariant="bold" mathsize="normal"><mi>z</mi></mstyle><mi>j</mi><mn>1</mn></msubsup></mrow><mo>)</mo></mrow><mtext> </mtext></mrow></math> were <math id="M70"><mrow><mi>C</mi><mi>o</mi><mi>v</mi><mrow><mo>(</mo><mrow><msubsup><mstyle mathvariant="bold" mathsize="normal"><mi>z</mi></mstyle><mi>i</mi><mi>l</mi></msubsup><mo>,</mo><msubsup><mstyle mathvariant="bold" mathsize="normal"><mi>z</mi></mstyle><mi>j</mi><mn>1</mn></msubsup></mrow><mo>)</mo></mrow><mo>=</mo><msub><mstyle mathvariant="bold" mathsize="normal"><mi>A</mi></mstyle><mi>i</mi></msub><mtext>Cov</mtext><mrow><mo>(</mo><mrow><msubsup><mstyle mathvariant="bold" mathsize="normal"><mi>z</mi></mstyle><mi>i</mi><mrow><mi>l</mi><mo>−</mo><mn>1</mn></mrow></msubsup><mo>,</mo><msubsup><mstyle mathvariant="bold" mathsize="normal"><mi>z</mi></mstyle><mi>j</mi><mn>1</mn></msubsup></mrow><mo>)</mo></mrow><mo>,</mo><mtext> </mtext><mi>l</mi><mo>=</mo><mn>2</mn><mo>,</mo></mrow></math> <math display="block" id="M71"><mrow><mtext>Cov</mtext><mrow><mo>(</mo><mrow><msubsup><mstyle mathvariant="bold" mathsize="normal"><mi>z</mi></mstyle><mi>i</mi><mi>l</mi></msubsup><mo>,</mo><msubsup><mstyle mathvariant="bold" mathsize="normal"><mi>z</mi></mstyle><mi>j</mi><mn>1</mn></msubsup></mrow><mo>)</mo></mrow><mo>≃</mo><msubsup><mi>η</mi><mi>i</mi><mi>l</mi></msubsup><msubsup><mstyle mathvariant="bold" mathsize="normal"><mi>F</mi></mstyle><mi>i</mi><mi>l</mi></msubsup><mrow><mo>[</mo><mrow><mstyle displaystyle="true"><munderover><mo>∑</mo><mrow><msup><mi>i</mi><mo>′</mo></msup><mo>=</mo><mn>1</mn></mrow><mrow><msup><mi>C</mi><mrow><mi>l</mi><mo>−</mo><mn>1</mn></mrow></msup></mrow></munderover><mrow><msubsup><mstyle mathvariant="bold" mathsize="normal"><mi>W</mi></mstyle><mrow><msup><mi>i</mi><mo>′</mo></msup><mo>,</mo><mi>i</mi></mrow><mi>l</mi></msubsup><mi>C</mi><mi>o</mi><mi>v</mi><mrow><mo>(</mo><mrow><msubsup><mstyle mathvariant="bold" mathsize="normal"><mi>z</mi></mstyle><msup><mi>i</mi><mo>′</mo></msup><mrow><mi>l</mi><mo>−</mo><mn>1</mn></mrow></msubsup><mo>,</mo><msubsup><mstyle mathvariant="bold" mathsize="normal"><mi>z</mi></mstyle><mi>j</mi><mn>1</mn></msubsup></mrow><mo>)</mo></mrow></mrow></mstyle></mrow><mo>]</mo></mrow><mo>,</mo><mtext> </mtext><mi>l</mi><mo>=</mo><mn>3</mn><mo>,</mo><mn>4</mn><mo>,</mo><mn>5</mn></mrow></math> Therefore, the covariance estimation of the 7 th layer is <math display="block" id="M72"><mrow><mtext>Cov</mtext><mo stretchy="false">(</mo><msubsup><mstyle mathvariant="bold" mathsize="normal"><mi>z</mi></mstyle><mi>i</mi><mn>7</mn></msubsup><mo>,</mo><msubsup><mstyle mathvariant="bold" mathsize="normal"><mi>z</mi></mstyle><mi>j</mi><mn>7</mn></msubsup><mo stretchy="false">)</mo><mo>=</mo><mrow><mo>{</mo><mrow><mtable columnalign="left"><mtr columnalign="left"><mtd columnalign="left"><mrow><mi>C</mi><mi>o</mi><mi>v</mi><mo stretchy="false">(</mo><msubsup><mstyle mathvariant="bold" mathsize="normal"><mi>z</mi></mstyle><mi>i</mi><mn>6</mn></msubsup><mo>,</mo><msubsup><mstyle mathvariant="bold" mathsize="normal"><mi>z</mi></mstyle><mi>j</mi><mn>6</mn></msubsup><mo stretchy="false">)</mo></mrow></mtd><mtd columnalign="left"><mrow><mo>,</mo><mi>i</mi><mo>,</mo><mi>j</mi><mo>=</mo><mn>1</mn><mo>,</mo><mn>2</mn><mo>,</mo><mn>3</mn></mrow></mtd></mtr><mtr columnalign="left"><mtd columnalign="left"><mrow><mi>C</mi><mi>o</mi><mi>v</mi><mo stretchy="false">(</mo><msubsup><mstyle mathvariant="bold" mathsize="normal"><mi>z</mi></mstyle><mi>i</mi><mn>6</mn></msubsup><mo>,</mo><msubsup><mstyle mathvariant="bold" mathsize="normal"><mi>z</mi></mstyle><mi>j</mi><mn>1</mn></msubsup><mo stretchy="false">)</mo></mrow></mtd><mtd columnalign="left"><mrow><mo>,</mo><mi>i</mi><mo>=</mo><mn>1</mn><mo>,</mo><mn>2</mn><mo>,</mo><mn>3</mn><mo>,</mo><mi>j</mi><mo>=</mo><mn>4</mn><mo>,</mo><mn>5</mn><mo>,</mo><mn>6</mn></mrow></mtd></mtr><mtr columnalign="left"><mtd columnalign="left"><mrow><mi>C</mi><mi>o</mi><mi>v</mi><mo stretchy="false">(</mo><msubsup><mstyle mathvariant="bold" mathsize="normal"><mi>z</mi></mstyle><mi>i</mi><mn>1</mn></msubsup><mo>,</mo><msubsup><mstyle mathvariant="bold" mathsize="normal"><mi>z</mi></mstyle><mi>j</mi><mn>6</mn></msubsup><mo stretchy="false">)</mo></mrow></mtd><mtd columnalign="left"><mrow><mo>,</mo><mi>i</mi><mo>=</mo><mn>4</mn><mo>,</mo><mn>5</mn><mo>,</mo><mn>6</mn><mo>,</mo><mi>j</mi><mo>=</mo><mn>1</mn><mo>,</mo><mn>2</mn><mo>,</mo><mn>3</mn></mrow></mtd></mtr><mtr columnalign="left"><mtd columnalign="left"><mrow><mi>C</mi><mi>o</mi><mi>v</mi><mo stretchy="false">(</mo><msubsup><mstyle mathvariant="bold" mathsize="normal"><mi>z</mi></mstyle><mi>i</mi><mn>1</mn></msubsup><mo>,</mo><msubsup><mstyle mathvariant="bold" mathsize="normal"><mi>z</mi></mstyle><mi>j</mi><mn>1</mn></msubsup><mo stretchy="false">)</mo></mrow></mtd><mtd columnalign="left"><mrow><mtext> </mtext><mo>,</mo><mi>i</mi><mo>,</mo><mi>j</mi><mo>=</mo><mn>4</mn><mo>,</mo><mn>5</mn><mo>,</mo><mn>6</mn></mrow></mtd></mtr></mtable></mrow></mrow></mrow></math> With a residual operation, the output projection represents the sum of the input projection and output residue of the 9 th layer. Thus, the covariance of the output projection also consists of three parts: 1) covariance of the input projection, 2) covariance of the output residue of the 9 th layer, and 3) covariance between the input projection and output residue. Only the covariance estimation of case 3) should be calculated because cases 1) and 2) are estimated using Eq. (15): <math display="block" id="M73"><mrow><mtext>Cov(</mtext><msup><mstyle mathvariant="bold" mathsize="normal"><mi>z</mi></mstyle><mn>9</mn></msup><mo>,</mo><mstyle mathvariant="bold" mathsize="normal"><mi>p</mi></mstyle><mo stretchy="false">)</mo><mo>=</mo><mtext>Cov(</mtext><mstyle mathvariant="bold" mathsize="normal"><mi>p</mi></mstyle><mo>,</mo><msup><mstyle mathvariant="bold" mathsize="normal"><mi>z</mi></mstyle><mn>9</mn></msup><msup><mo stretchy="false">)</mo><mi>T</mi></msup><mo>=</mo><mstyle displaystyle="true"><munderover><mo>∑</mo><mrow><mi>i</mi><mo>=</mo><mn>1</mn></mrow><mrow><msup><mi>C</mi><mn>8</mn></msup></mrow></munderover><mrow><msubsup><mstyle mathvariant="bold" mathsize="normal"><mi>W</mi></mstyle><mi>i</mi><mn>9</mn></msubsup><mtext>Cov</mtext><mrow><mo>(</mo><mrow><msubsup><mstyle mathvariant="bold" mathsize="normal"><mi>z</mi></mstyle><mi>i</mi><mn>8</mn></msubsup><mo>,</mo><mstyle mathvariant="bold" mathsize="normal"><mi>p</mi></mstyle></mrow><mo>)</mo></mrow></mrow></mstyle></mrow></math> were <math id="M74"><mrow><mtext>Cov</mtext><mo stretchy="false">(</mo><msubsup><mstyle mathvariant="bold" mathsize="normal"><mi>z</mi></mstyle><mi>i</mi><mi>l</mi></msubsup><mo>,</mo><mstyle mathvariant="bold" mathsize="normal"><mi>p</mi></mstyle><mo stretchy="false">)</mo><mo>≃</mo><msubsup><mi>η</mi><mi>i</mi><mi>l</mi></msubsup><msubsup><mstyle mathvariant="bold" mathsize="normal"><mi>F</mi></mstyle><mi>i</mi><mi>l</mi></msubsup><mrow><mo>[</mo><mrow><mstyle displaystyle="true"><munderover><mo>∑</mo><mrow><msup><mi>i</mi><mo>′</mo></msup><mo>=</mo><mn>1</mn></mrow><mrow><msup><mi>C</mi><mrow><mi>l</mi><mo>−</mo><mn>1</mn></mrow></msup></mrow></munderover><mrow><msubsup><mstyle mathvariant="bold" mathsize="normal"><mi>W</mi></mstyle><mrow><msup><mi>i</mi><mo>′</mo></msup><mo>,</mo><mi>i</mi></mrow><mi>l</mi></msubsup><mtext>Cov</mtext><mrow><mo>(</mo><mrow><msubsup><mstyle mathvariant="bold" mathsize="normal"><mi>z</mi></mstyle><msup><mi>i</mi><mo>′</mo></msup><mrow><mi>l</mi><mo>−</mo><mn>1</mn></mrow></msubsup><mo>,</mo><mstyle mathvariant="bold" mathsize="normal"><mi>p</mi></mstyle></mrow><mo>)</mo></mrow></mrow></mstyle></mrow><mo>]</mo></mrow><mo>,</mo><mtext> </mtext><mi>l</mi><mo>=</mo><mn>1</mn><mo>,</mo><mn>3</mn><mo>,</mo><mn>4</mn><mo>,</mo><mn>5</mn><mo>,</mo><mn>8</mn></mrow></math> <math display="block" id="M75"><mtable columnalign="left"><mtr><mtd><mi>C</mi><mi>o</mi><mi>v</mi><mo stretchy="false">(</mo><msubsup><mstyle mathvariant="bold" mathsize="normal"><mi>z</mi></mstyle><mi>i</mi><mi>l</mi></msubsup><mo>,</mo><mstyle mathvariant="bold" mathsize="normal"><mi>p</mi></mstyle><mo stretchy="false">)</mo><mo>=</mo><msub><mstyle mathvariant="bold" mathsize="normal"><mi>A</mi></mstyle><mi>i</mi></msub><mi>C</mi><mi>o</mi><mi>v</mi><mo stretchy="false">(</mo><msubsup><mstyle mathvariant="bold" mathsize="normal"><mi>z</mi></mstyle><mi>i</mi><mrow><mi>l</mi><mo>−</mo><mn>1</mn></mrow></msubsup><mo>,</mo><mstyle mathvariant="bold" mathsize="normal"><mi>p</mi></mstyle><mo stretchy="false">)</mo><mo>,</mo><mtext> </mtext><mi>l</mi><mo>=</mo><mn>2</mn></mtd></mtr><mtr><mtd><mi>C</mi><mi>o</mi><mi>v</mi><mo stretchy="false">(</mo><msubsup><mstyle mathvariant="bold" mathsize="normal"><mi>z</mi></mstyle><mi>i</mi><mi>l</mi></msubsup><mo>,</mo><mstyle mathvariant="bold" mathsize="normal"><mi>p</mi></mstyle><mo stretchy="false">)</mo><mo>=</mo><msub><mstyle mathvariant="bold" mathsize="normal"><mi>U</mi></mstyle><mi>i</mi></msub><mi>C</mi><mi>o</mi><mi>v</mi><mo stretchy="false">(</mo><msubsup><mstyle mathvariant="bold" mathsize="normal"><mi>z</mi></mstyle><mi>i</mi><mrow><mi>l</mi><mo>−</mo><mn>1</mn></mrow></msubsup><mo>,</mo><mstyle mathvariant="bold" mathsize="normal"><mi>p</mi></mstyle><mo stretchy="false">)</mo><mo>,</mo><mtext> </mtext><mi>l</mi><mo>=</mo><mn>6</mn></mtd></mtr><mtr><mtd><mtext>Cov</mtext><mo stretchy="false">(</mo><msubsup><mstyle mathvariant="bold" mathsize="normal"><mi>z</mi></mstyle><mi>i</mi><mi>l</mi></msubsup><mo>,</mo><mstyle mathvariant="bold" mathsize="normal"><mi>p</mi></mstyle><mo stretchy="false">)</mo><mo>=</mo><mrow><mo>{</mo><mrow><mtable columnalign="left"><mtr columnalign="left"><mtd columnalign="left"><mrow><mtext>Cov</mtext><mo stretchy="false">(</mo><msubsup><mstyle mathvariant="bold" mathsize="normal"><mi>z</mi></mstyle><mi>i</mi><mn>6</mn></msubsup><mo>,</mo><mstyle mathvariant="bold" mathsize="normal"><mi>p</mi></mstyle><mo stretchy="false">)</mo></mrow></mtd><mtd columnalign="left"><mrow><mo>,</mo><mi>i</mi><mo>=</mo><mn>1</mn><mo>,</mo><mn>2</mn><mo>,</mo><mn>3</mn></mrow></mtd></mtr><mtr columnalign="left"><mtd columnalign="left"><mrow><mtext>Cov</mtext><mo stretchy="false">(</mo><msubsup><mstyle mathvariant="bold" mathsize="normal"><mi>z</mi></mstyle><mi>i</mi><mn>1</mn></msubsup><mo>,</mo><mstyle mathvariant="bold" mathsize="normal"><mi>p</mi></mstyle><mo stretchy="false">)</mo></mrow></mtd><mtd columnalign="left"><mrow><mo>,</mo><mi>i</mi><mo>=</mo><mn>4</mn><mo>,</mo><mn>5</mn><mo>,</mo><mn>6</mn></mrow></mtd></mtr></mtable></mrow></mrow><mo>,</mo><mtext> </mtext><mi>l</mi><mo>=</mo><mn>7</mn></mtd></mtr></mtable></math> The covariance estimation of the 10 th layer is then calculated as <math display="block" id="M76"><mrow><mtext>Cov(</mtext><msup><mstyle mathvariant="bold" mathsize="normal"><mi>z</mi></mstyle><mrow><mn>10</mn></mrow></msup><mo stretchy="false">)</mo><mo>=</mo><mtext>Cov</mtext><mo stretchy="false">(</mo><mstyle mathvariant="bold" mathsize="normal"><mi>p</mi></mstyle><mo stretchy="false">)</mo><mo>+</mo><mtext>Cov</mtext><mo stretchy="false">(</mo><msup><mstyle mathvariant="bold" mathsize="normal"><mi>z</mi></mstyle><mn>9</mn></msup><mo stretchy="false">)</mo><mo>+</mo><mtext>Cov</mtext><mo stretchy="false">(</mo><msup><mstyle mathvariant="bold" mathsize="normal"><mi>z</mi></mstyle><mn>9</mn></msup><mo>,</mo><mstyle mathvariant="bold" mathsize="normal"><mi>p</mi></mstyle><mo stretchy="false">)</mo><mo>+</mo><mtext>Cov</mtext><mo stretchy="false">(</mo><mstyle mathvariant="bold" mathsize="normal"><mi>p</mi></mstyle><mo>,</mo><msup><mstyle mathvariant="bold" mathsize="normal"><mi>z</mi></mstyle><mn>9</mn></msup><mo stretchy="false">)</mo></mrow></math> Combining Eqs. (15)–(19), we obtain the final covariance estimation of the reconstruction: <math display="block" id="M77"><mrow><mtext>Cov(</mtext><mstyle mathvariant="bold" mathsize="normal"><mi>μ</mi></mstyle><mo stretchy="false">)</mo><mo>=</mo><mi mathvariant="script">O</mi><mtext>Cov(</mtext><msup><mstyle mathvariant="bold" mathsize="normal"><mi>z</mi></mstyle><mrow><mn>10</mn></mrow></msup><mo stretchy="false">)</mo><msup><mi mathvariant="script">O</mi><mi>T</mi></msup></mrow></math> A gradient discontinuous function, LeakyReLU, and a gradient continuous function, Swish, are chosen as activation functions to detect the influence of activation functions on the covariance estimation of the CNN. LeakyReLU Activation Function 3 The LeakyReLU function and its corresponding gradient are expressed as <math display="block" id="M78"><mtable columnalign="left"><mtr><mtd><msub><mi>f</mi><mrow><mtext>LR</mtext></mrow></msub><mo stretchy="false">(</mo><mi>x</mi><mo stretchy="false">)</mo><mo>=</mo><mrow><mo>{</mo><mrow><mtable><mtr><mtd><mi>x</mi></mtd><mtd><mrow><mo>,</mo><mtext>if</mtext><mtext> </mtext><mi>x</mi><mo>></mo><mn>0</mn></mrow></mtd></mtr><mtr><mtd><mrow><mi>α</mi><mi>x</mi></mrow></mtd><mtd><mrow><mo>,</mo><mtext>if</mtext><mtext> </mtext><mi>x</mi><mo>≤</mo><mn>0</mn></mrow></mtd></mtr></mtable></mrow></mrow></mtd></mtr><mtr><mtd><msub><msup><mi>f</mi><mo>′</mo></msup><mrow><mtext>LR</mtext></mrow></msub><mo stretchy="false">(</mo><mi>x</mi><mo stretchy="false">)</mo><mo>=</mo><mrow><mo>{</mo><mrow><mtable><mtr><mtd><mn>1</mn></mtd><mtd><mrow><mo>,</mo><mtext>if</mtext><mtext> </mtext><mi>x</mi><mo>></mo><mn>0</mn></mrow></mtd></mtr><mtr><mtd><mi>α</mi></mtd><mtd><mrow><mo>,</mo><mtext>if</mtext><mtext> </mtext><mi>x</mi><mo>≤</mo><mn>0</mn></mrow></mtd></mtr></mtable></mrow></mrow></mtd></mtr></mtable></math> The Taylor-based coefficient matrix in Eq. (6) is <math id="M79"><mrow><msub><mrow><mo stretchy="false">[</mo><msub><mstyle mathvariant="bold" mathsize="normal"><mi>F</mi></mstyle><mrow><mtext>LR</mtext></mrow></msub><mo stretchy="false">]</mo></mrow><mrow><mi>m</mi><mo>,</mo><mi>m</mi></mrow></msub><mo>=</mo><msub><msup><mi>f</mi><mo>′</mo></msup><mrow><mtext>LR</mtext></mrow></msub><mo stretchy="false">(</mo><msub><mrow><mo stretchy="false">[</mo><mstyle mathvariant="bold" mathsize="normal"><mover accent="true"><mi>x</mi><mo>¯</mo></mover></mstyle><mo stretchy="false">]</mo></mrow><mi>m</mi></msub><mo stretchy="false">)</mo></mrow></math> . Plugging <math id="M80"><mrow><msub><mstyle mathvariant="bold" mathsize="normal"><mi>F</mi></mstyle><mrow><mtext>LR</mtext></mrow></msub></mrow></math> into Eqs. (15)–(20), we obtain the covariance estimation from U-net with LeakyReLU. Swish Activation Function 3 The Swish function is another commonly used activation function: <math display="block" id="M81"><mrow><msub><mi>f</mi><mrow><mtext>Sw</mtext></mrow></msub><mo stretchy="false">(</mo><mi>x</mi><mo stretchy="false">)</mo><mo>=</mo><mfrac><mi>x</mi><mrow><mn>1</mn><mo>+</mo><mi>exp</mi><mo stretchy="false">(</mo><mo>−</mo><mi>λ</mi><mi>x</mi><mo stretchy="false">)</mo></mrow></mfrac></mrow></math> Its gradient is <math display="block" id="M82"><mrow><msub><msup><mi>f</mi><mo>′</mo></msup><mrow><mtext>Sw</mtext></mrow></msub><mo stretchy="false">(</mo><mover accent="true"><mi>x</mi><mo>¯</mo></mover><mo stretchy="false">)</mo><mo>=</mo><mi>λ</mi><msub><mi>f</mi><mrow><mtext>Sw</mtext></mrow></msub><mo stretchy="false">(</mo><mover accent="true"><mi>x</mi><mo>¯</mo></mover><mo stretchy="false">)</mo><mo>+</mo><mfrac><mrow><mn>1</mn><mo>−</mo><mi>λ</mi><msub><mi>f</mi><mrow><mtext>Sw</mtext></mrow></msub><mo stretchy="false">(</mo><mover accent="true"><mi>x</mi><mo>¯</mo></mover><mo stretchy="false">)</mo></mrow><mrow><mn>1</mn><mo>+</mo><mi>exp</mi><mo stretchy="false">(</mo><mo>−</mo><mi>λ</mi><mover accent="true"><mi>x</mi><mo>¯</mo></mover><mo stretchy="false">)</mo></mrow></mfrac></mrow></math> Thus, the Taylor-based coefficient matrix is <math id="M83"><mrow><msub><mrow><mo stretchy="false">[</mo><msub><mstyle mathvariant="bold" mathsize="normal"><mi>F</mi></mstyle><mrow><mtext>Sw</mtext></mrow></msub><mo stretchy="false">]</mo></mrow><mrow><mi>m</mi><mo>,</mo><mi>m</mi></mrow></msub><mo>=</mo><msub><msup><mi>f</mi><mo>′</mo></msup><mrow><mtext>Sw</mtext></mrow></msub><mo stretchy="false">(</mo><msub><mrow><mo stretchy="false">[</mo><mstyle mathvariant="bold" mathsize="normal"><mover accent="true"><mi>x</mi><mo>¯</mo></mover></mstyle><mo stretchy="false">]</mo></mrow><mi>m</mi></msub><mo stretchy="false">)</mo></mrow></math> . Replacing <math id="M84"><mstyle mathvariant="bold" mathsize="normal"><mi>F</mi></mstyle></math> in Eqs. (15)–(18) with <math id="M85"><mrow><msub><mstyle mathvariant="bold" mathsize="normal"><mi>F</mi></mstyle><mrow><mtext>Sw</mtext></mrow></msub></mrow></math> , we obtain the covariance estimation of the projection by U-net with Swish using Eq. (19) and the covariance estimation of its reconstruction by FBP using Eq. (20). Experiments 1 The projection data used for training are generated from the Grand Challenge dataset of the Mayo Clinic. We randomly choose a reconstruction dataset of one patient and select various ROIs with sizes of 128 × 128 pixels as phantoms. Geometrical parameters of the simulated system are listed in Table 1 Table 1 . By setting the number of incident photons to <math id="M86"><mrow><msub><mi>I</mi><mn>0</mn></msub><mo>=</mo><msup><mrow><mn>10</mn></mrow><mn>4</mn></msup></mrow></math> , we add Poisson noise to the noise-free projections simulated from phantoms to obtain noisy projections. 1 1 Parameter 1 1 Value 1 1 Source to origin distance (mm) 1 1 200 1 1 Origin to detector distance (mm) 1 1 200 1 1 Scanning angle 1 1 <math id="M87"><mrow><mn>2</mn><mi>π</mi></mrow></math> 1 1 Detector pixel size (mm) 1 1 0.5 1 1 Detector pixel number 1 1 240 1 1 Image pixel size (mm) 1 1 0.3 1 1 Image pixel number 1 1 128 Using noise-free projections as labels and noisy projections as inputs, we train U-net by minimizing an L2 norm loss function between labels and outputs. In addition, we fix the hyper-parameter <math id="M88"><mrow><mi>α</mi><mtext>=</mtext><mn>0.1</mn></mrow></math> for LeakyReLU and <math id="M89"><mrow><mi>λ</mi><mtext>=</mtext><mn>0.5</mn></mrow></math> for Swish. We simulate 1792 noisy projections for the study. The dataset is randomly split into a training set and a validation set, where 80% of the dataset represents the training set and 20% represents the validation set. We train the network with Keras on a GPU RTX8000 of 48G.The loss function of U-net with LeakyReLU and Swish decreases to approximately <math id="M90"><mrow><msup><mrow><mn>10</mn></mrow><mrow><mo>−</mo><mn>4</mn></mrow></msup></mrow></math> during converging. Furthermore, we randomly split the dataset into 5 folds to run a 5-fold cross validation on the trained network. The average loss in the 5-fold cross validation is also approximately <math id="M91"><mrow><msup><mrow><mn>10</mn></mrow><mrow><mo>−</mo><mn>4</mn></mrow></msup></mrow></math> , which is similar to the loss of the trained network. Thus, the dataset is sufficient for training the small-size U-net in Fig. 1 Fig. 1 , and the trained network is stable. Noisy projections used for inference are generated from another patient dataset in the same manner. We generate noisy projections of different phantoms and noise levels to validate the proposed analytical noise propagation method and analyze the performance of U-net using the analytically estimated covariance. Information on the noisy projections generated for prediction is presented in Table 2 Table 2 . Note that the number of incident photons increases linearly from 10 3 to 5.05 × 10 4 . The reconstruction of both phantoms using <math id="M92"><mrow><msub><mi>I</mi><mn>0</mn></msub><mo>=</mo><msup><mrow><mn>10</mn></mrow><mn>4</mn></msup></mrow></math> is illustrated in Fig. 2 Fig. 2 . 1 1 Number of generatednoise realizations 1 1 <math id="M93"><mrow><msub><mi>I</mi><mn>0</mn></msub><mo>=</mo><msup><mrow><mn>10</mn></mrow><mn>3</mn></msup></mrow></math> 1 1 <math id="M94"><mrow><msub><mi>I</mi><mn>0</mn></msub><mo>=</mo><mn>5.5</mn><mo>×</mo><msup><mrow><mn>10</mn></mrow><mn>3</mn></msup></mrow></math> 1 1 <math id="M95"><mrow><msub><mi>I</mi><mn>0</mn></msub><mo>=</mo><msup><mrow><mn>10</mn></mrow><mn>4</mn></msup></mrow></math> 1 1 <math id="M96"><mrow><msub><mi>I</mi><mn>0</mn></msub><mo>=</mo><mn>3.25</mn><mo>×</mo><msup><mrow><mn>10</mn></mrow><mn>4</mn></msup></mrow></math> 1 1 <math id="M97"><mrow><msub><mi>I</mi><mn>0</mn></msub><mo>=</mo><mn>5.05</mn><mo>×</mo><msup><mrow><mn>10</mn></mrow><mn>4</mn></msup></mrow></math> 1 1 Test phantom A 1 1 - 1 1 - 1 1 1000 1 1 - 1 1 - 1 1 Test phantom B 1 1 1000 1 1 1000 1 1 1000 1 1 1000 1 1 1000 In addition, we conduct a practical experiment to validate our proposed method. The experimental platform and phantom are presented in Fig. 3 Fig. 3 . The scanning parameters are presented in Table 3 Table 3 . We repeat the scan 450 times at each angle and acquire projection data of 360 views using <math id="M99"><mrow><mn>2</mn><mi>π</mi></mrow></math> . Considering the computational cost, every four pixels of the detector are binned into one to obtain a projection with a smaller size. 1 1 Parameters 1 1 Before binning 1 1 After binning 1 1 Source to origin distance (mm) 1 1 750.2 1 1 750.2 1 1 Origin to detector distance (mm) 1 1 433.84 1 1 433.84 1 1 Detector pixel size (mm) 1 1 0.45 1 1 1.80 1 1 Detector pixel number 1 1 896 1 1 224 1 1 Image pixel size (mm) 1 1 0.48 1 1 1.92 1 1 Image pixel number 1 1 512 1 1 128 1 1 Tube voltage/current 1 1 110 kV/2.5 mA 1 1 110 kV/2.5 mA Covariance estimation from a statistical method is used as a reference in this study. <math display="block" id="M100"><mtable columnalign="left"><mtr><mtd><msup><mtext>Cov</mtext><mo>*</mo></msup><mo stretchy="false">(</mo><mtext> </mtext><mstyle mathvariant="bold" mathsize="normal"><mover accent="true"><mi>p</mi><mo>^</mo></mover></mstyle><mo stretchy="false">)</mo><mo>=</mo><mfrac><mn>1</mn><mrow><mi>K</mi><mo>−</mo><mn>1</mn></mrow></mfrac><mstyle displaystyle="true"><munderover><mo>∑</mo><mrow><mi>k</mi><mo>=</mo><mn>1</mn></mrow><mi>K</mi></munderover><mrow><mo stretchy="false">(</mo><msub><mstyle mathvariant="bold" mathsize="normal"><mover accent="true"><mi>p</mi><mo>^</mo></mover></mstyle><mi>k</mi></msub><mo>−</mo><mstyle mathvariant="bold" mathsize="normal"><mover accent="true"><mover accent="true"><mi>p</mi><mo>^</mo></mover><mo>¯</mo></mover></mstyle><mo stretchy="false">)</mo><msup><mrow><mo stretchy="false">(</mo><msub><mstyle mathvariant="bold" mathsize="normal"><mover accent="true"><mi>p</mi><mo>^</mo></mover></mstyle><mi>k</mi></msub><mo>−</mo><mstyle mathvariant="bold" mathsize="normal"><mover accent="true"><mover accent="true"><mi>p</mi><mo>^</mo></mover><mo>¯</mo></mover></mstyle><mo stretchy="false">)</mo></mrow><mi>T</mi></msup></mrow></mstyle></mtd></mtr><mtr><mtd><msup><mtext>Cov</mtext><mo>*</mo></msup><mo stretchy="false">(</mo><mtext> </mtext><mstyle mathvariant="bold" mathsize="normal"><mover accent="true"><mi>μ</mi><mo>^</mo></mover></mstyle><mo stretchy="false">)</mo><mo>=</mo><mfrac><mn>1</mn><mrow><mi>K</mi><mo>−</mo><mn>1</mn></mrow></mfrac><mstyle displaystyle="true"><munderover><mo>∑</mo><mrow><mi>k</mi><mo>=</mo><mn>1</mn></mrow><mi>K</mi></munderover><mrow><mo stretchy="false">(</mo><msub><mstyle mathvariant="bold" mathsize="normal"><mover accent="true"><mi>μ</mi><mo>^</mo></mover></mstyle><mi>k</mi></msub><mo>−</mo><mstyle mathvariant="bold" mathsize="normal"><mover accent="true"><mover accent="true"><mi>μ</mi><mo>^</mo></mover><mo>¯</mo></mover></mstyle><mo stretchy="false">)</mo><msup><mrow><mo stretchy="false">(</mo><msub><mstyle mathvariant="bold" mathsize="normal"><mover accent="true"><mi>μ</mi><mo>^</mo></mover></mstyle><mi>k</mi></msub><mo>−</mo><mstyle mathvariant="bold" mathsize="normal"><mover accent="true"><mover accent="true"><mi>μ</mi><mo>^</mo></mover><mo>¯</mo></mover></mstyle><mo stretchy="false">)</mo></mrow><mi>T</mi></msup></mrow></mstyle></mtd></mtr></mtable></math> where the total number of noise realizations is <math id="M101"><mrow><mi>K</mi><mo>=</mo><mn>1000</mn></mrow></math> . The generalization error (GE) [ 33 33 ] represents the sum of the bias and variance and measures the generalization ability of a neural network: <math display="block" id="M102"><mrow><msub><mrow><mtext>[GE]</mtext></mrow><mi>n</mi></msub><mo>=</mo><msup><mrow><mo stretchy="false">(</mo><msub><mrow><mo stretchy="false">[</mo><mstyle mathvariant="bold" mathsize="normal"><mover accent="true"><mi>μ</mi><mo>^</mo></mover></mstyle><mo stretchy="false">]</mo></mrow><mi>n</mi></msub><mo>−</mo><msub><mrow><mo stretchy="false">[</mo><msup><mstyle mathvariant="bold" mathsize="normal"><mi>μ</mi></mstyle><mo>*</mo></msup><mo stretchy="false">]</mo></mrow><mi>n</mi></msub><mo stretchy="false">)</mo></mrow><mn>2</mn></msup><mo>+</mo><msub><mrow><mo stretchy="false">[</mo><mtext>Cov</mtext><mo stretchy="false">(</mo><mstyle mathvariant="bold" mathsize="normal"><mover accent="true"><mi>μ</mi><mo>^</mo></mover></mstyle><mo stretchy="false">)</mo><mo stretchy="false">]</mo></mrow><mrow><mi>n</mi><mo>,</mo><mi>n</mi></mrow></msub></mrow></math> where <math id="M103"><mrow><msup><mstyle mathvariant="bold" mathsize="normal"><mi>μ</mi></mstyle><mo>*</mo></msup></mrow></math> represents noise-free reconstruction. A pixel-wise noise reduction percentage (NPR) is calculated to analyze the denoising performance of U-net. <math display="block" id="M104"><mrow><msub><mrow><mtext>[NRP]</mtext></mrow><mi>n</mi></msub><mo>=</mo><mrow><mo>|</mo><mrow><mfrac><mrow><msqrt><mrow><msub><mrow><mo stretchy="false">[</mo><mtext>Cov</mtext><mo stretchy="false">(</mo><mstyle mathvariant="bold" mathsize="normal"><mover accent="true"><mi>μ</mi><mo>^</mo></mover></mstyle><mo stretchy="false">)</mo><mo stretchy="false">]</mo></mrow><mrow><mi>n</mi><mo>,</mo><mi>n</mi></mrow></msub></mrow></msqrt><mo>−</mo><msqrt><mrow><msub><mrow><mo stretchy="false">[</mo><mtext>Cov</mtext><mo stretchy="false">(</mo><mstyle mathvariant="bold" mathsize="normal"><mi>μ</mi></mstyle><mo stretchy="false">)</mo><mo stretchy="false">]</mo></mrow><mrow><mi>n</mi><mo>,</mo><mi>n</mi></mrow></msub></mrow></msqrt></mrow><mrow><msqrt><mrow><msub><mrow><mo stretchy="false">[</mo><mtext>Cov</mtext><mo stretchy="false">(</mo><mstyle mathvariant="bold" mathsize="normal"><mi>μ</mi></mstyle><mo stretchy="false">)</mo><mo stretchy="false">]</mo></mrow><mrow><mi>n</mi><mo>,</mo><mi>n</mi></mrow></msub></mrow></msqrt></mrow></mfrac></mrow><mo>|</mo></mrow><mo>×</mo><mn>100</mn><mi>%</mi></mrow></math> Here, <math id="M105"><mrow><mstyle mathvariant="bold" mathsize="normal"><mi>μ</mi></mstyle><mo>=</mo><mi mathvariant="script">O</mi><mo stretchy="false">(</mo><mstyle mathvariant="bold" mathsize="normal"><mi>p</mi></mstyle><mo stretchy="false">)</mo></mrow></math> and <math id="M106"><mrow><mtext>Cov(</mtext><mstyle mathvariant="bold" mathsize="normal"><mi>μ</mi></mstyle><mo stretchy="false">)</mo><mo>=</mo><mi mathvariant="script">O</mi><mtext>Cov</mtext><mo stretchy="false">(</mo><mstyle mathvariant="bold" mathsize="normal"><mi>p</mi></mstyle><mo stretchy="false">)</mo><msup><mi mathvariant="script">O</mi><mi>T</mi></msup></mrow></math> , according to Eq. (20). We choose one low-attenuation point and one high-attenuation point in test phantom B to present the trend of GE with the noise level; the two points are marked by red dots in Fig. 2(b) Fig. 2(b) . We also calculate the NPS to analyze the noise spatial correlation of U-net in the Fourier domain: <math display="block" id="M107"><mrow><mtext>NPS</mtext><mo>=</mo><mi mathvariant="script">F</mi><mo stretchy="false">[</mo><mtext>Cov</mtext><mo stretchy="false">(</mo><mstyle mathvariant="bold" mathsize="normal"><mover accent="true"><mi>μ</mi><mo>^</mo></mover></mstyle><mo stretchy="false">)</mo><mo stretchy="false">]</mo></mrow></math> where <math id="M108"><mi mathvariant="script">F</mi></math> denotes the Fourier transform operator. Experimental Results 1 Because the linear approximation of nonlinear activation functions requires the mean of the input in Eq. (6), we only use one noise realization as the mean of the input to analytically estimate the covariance of the projection acquired by U-net and its corresponding reconstruction by FBP. Validation of the Proposed Analytical Covariance Estimation Method for U-net 2 For test phantom A, the variance of the projections obtained by U-net is illustrated in Fig. 4(a) Fig. 4(a) , and its covariance at the center of the projections is presented in Fig. 4(b) Fig. 4(b) . The results reveal that the variance estimation is in agreement with the reference for both the activation functions. The error between the variance estimation and reference is not significant compared with the variance itself. We observe that the variance obtained from LeakyReLU varies sharply when approaching the boundary, whereas that from Swish changes smoothly. Meanwhile, the covariance estimation also agrees with the reference, where the error is primarily statistical. The shape of the covariance from the two activation functions is quite different; it is circular for LeakyReLU and elliptical for Swish. For test phantom B, good agreement can still be observed between the variance estimation and reference (as shown in Fig. 4(c) Fig. 4(c) ). Sharp changes also occur near the boundary in the projection variance from LeakyReLU. As shown in Fig. 4(d) Fig. 4(d) , the covariance estimation for both activation functions is in agreement with the reference. The variance and covariance of the reconstructions obtained by FBP from projections denoised by U-net are presented in Fig. 5 Fig. 5 . For both phantoms and activation functions, the variance estimation of the reconstructions agrees with the reference because FBP is linear. Meanwhile, the error that propagates through the FBP is not a concern. We discover that the central areas of variance from the two activation functions have different appearances. The central area appears dark for LeakyReLU and bright for Swish in the same display window. The covariance estimation is yet again in agreement with the reference, leaving only statistical noise in the error map. The profiles of the variance and covariance for the projections and reconstructions are plotted in Fig. 6 Fig. 6 . For both phantoms and activation functions, the profiles of the variance and covariance estimations match those of the references. As shown in Fig. 6(a1)(c1) Fig. 6(a1)(c1) , the profile of the projection variance from Swish appears smooth, whereas that from LeakyReLU appears sharp. The profile of covariance (as shown in Fig. 6(b1) and 6(d1) Fig. 6(b1) and 6(d1) ) from Swish demonstrates a larger spread, whereas that from LeakyReLU exhbits sharper changes. For the profiles of the reconstruction variance, displayed in Fig. 6(a2) and 6(c2) Fig. 6(a2) and 6(c2) , we discover that the noise from LeakyReLU is lower than that from Swish. The variance gradually decreases from the edge of the field of view (FOV) to its center for LeakyReLU, whereas it demonstrates an opposite behavior for Swish. Although the values of the projection covariance are close, the absolute value of the reconstruction covariance from LeakyReLU is much smaller than that from Swish; this demonstrates that the covariance from Swish is more structurally related and difficult to deal with. In addition, we estimate the variance of the projections obtained by U-net with LeakyReLU under different noise levels for test phantom B. As shown in Fig. 7 Fig. 7 , the variance estimation agrees with the reference for different noise levels. Although the error between the variance estimation and reference increases with the noise level, it is still insignificant compared with its corresponding variance. The noise estimation of U-net with LeakyReLU in the practical experiment is illustrated in Fig. 8 Fig. 8 . It is apparent that both the variance and covariance estimations from the analytical method agree well with the references, which strongly demonstrates the feasibility of the proposed analytical noise propagation method in practical usage. Performance Analysis with Analytical Covariance 2 Pixel-wise GE maps for test phantom B are illustrated in Fig. 9 Fig. 9 . For both activation functions, GE increases for each pixel with increasing noise levels, which indicates that U-net is inapplicable for highly noisy projections. When <math id="M112"><mrow><msub><mi>I</mi><mn>0</mn></msub></mrow></math> increases to a certain number, the decrease in GE is not significant. The GE in the smooth region is smaller than that at the edge when the number of incident photons increases to <math id="M113"><mrow><mn>3.25</mn><mo>×</mo><msup><mrow><mn>10</mn></mrow><mn>4</mn></msup></mrow></math> , indicating that U-net is more effective in smooth regions. Compared with the GE for LeakyReLU, the GE for Swish is relatively large in smooth regions, whereas it is almost the same at the edge. Further, noise reduction percentage (NRPs) are listed in Table 4 Table 4 . For both activation functions, the increase in NRP from <math id="M114"><mrow><msub><mi>I</mi><mn>0</mn></msub><mo>=</mo><msup><mrow><mn>10</mn></mrow><mn>3</mn></msup></mrow></math> to <math id="M115"><mrow><msub><mi>I</mi><mn>0</mn></msub><mo>=</mo><mn>5.5</mn><mo>×</mo><msup><mrow><mn>10</mn></mrow><mn>3</mn></msup></mrow></math> is approximately 20%, and this increase quickly slows to approximately 5% or less. For both low- and high-attenuation points, the noise reduction effect of LeakyReLU is stronger than that of Swish at various noise levels. The NRP of LeakyReLU is approximately 10% higher than that of Swish, particularly for the low-attenuation point at <math id="M116"><mrow><msub><mi>I</mi><mn>0</mn></msub><mo>=</mo><msup><mrow><mn>10</mn></mrow><mn>3</mn></msup></mrow></math> , and the difference decreases to approximately 1% for a low noise level with <math id="M117"><mrow><msub><mi>I</mi><mn>0</mn></msub><mo>=</mo><mn>5.05</mn><mo>×</mo><msup><mrow><mn>10</mn></mrow><mn>4</mn></msup></mrow></math> . For the high-attenuation point, the difference in the NRP for both activation functions is smaller than 3% and becomes even smaller when the number of incident photons increases. The NRPs at both points for LeakyReLU are comparable, which suggests that LeakyReLU treats low- and high-attenuation areas equally during noise suppression. The NRP at the low-attenuation point for Swish is slightly smaller than that at the high-attenuation point; however, the difference between the NRPs at the low- and high-attenuation points gradually reduces to approximately 1% with decreasing noise levels. 1 1 NRP (%) 1 1 Activation function 1 1 <math id="M118"><mrow><msub><mi>I</mi><mn>0</mn></msub><mo>=</mo><msup><mrow><mn>10</mn></mrow><mn>3</mn></msup></mrow></math> 1 1 <math id="M119"><mrow><msub><mi>I</mi><mn>0</mn></msub><mo>=</mo><mn>5.5</mn><mo>×</mo><msup><mrow><mn>10</mn></mrow><mn>3</mn></msup></mrow></math> 1 1 <math id="M120"><mrow><msub><mi>I</mi><mn>0</mn></msub><mo>=</mo><msup><mrow><mn>10</mn></mrow><mn>4</mn></msup></mrow></math> 1 1 <math id="M121"><mrow><msub><mi>I</mi><mn>0</mn></msub><mo>=</mo><mn>3.25</mn><mo>×</mo><msup><mrow><mn>10</mn></mrow><mn>4</mn></msup></mrow></math> 1 1 <math id="M122"><mrow><msub><mi>I</mi><mn>0</mn></msub><mo>=</mo><mn>5.05</mn><mo>×</mo><msup><mrow><mn>10</mn></mrow><mn>4</mn></msup></mrow></math> 1 1 Point ① 1 1 LeakyReLU 1 1 64.78 1 1 84.99 1 1 88.87 1 1 93.82 1 1 95.04 1 1   1 1 Swish 1 1 55.66 1 1 81.16 1 1 86.02 1 1 92.25 1 1 93.78 1 1 Point ② 1 1 LeakyReLU 1 1 63.93 1 1 84.61 1 1 88.59 1 1 93.67 1 1 94.93 1 1   1 1 Swish 1 1 60.42 1 1 83.11 1 1 87.48 1 1 93.05 1 1 94.43 The NPS at the center of the reconstruction is illustrated in Fig. 10 Fig. 10 . For each noise level, the NPS at the center of the reconstruction from both activation functions decreases as <math id="M123"><mrow><msup><mrow><mn>10</mn></mrow><mn>3</mn></msup></mrow></math> increases to <math id="M124"><mrow><mn>5.05</mn><mo>×</mo><msup><mrow><mn>10</mn></mrow><mn>4</mn></msup></mrow></math> and drops by approximately an order of magnitude from <math id="M125"><mrow><msup><mrow><mn>10</mn></mrow><mn>3</mn></msup></mrow></math> to <math id="M126"><mrow><mn>5.5</mn><mo>×</mo><msup><mrow><mn>10</mn></mrow><mn>3</mn></msup></mrow></math> . The NPS from LeakyReLU first increases to a maximum and then decreases as the frequency increases, whereas that from Swish continues to increase with increasing frequency. Both LeakyReLU and Swish exhibit similar NPS shapes at low frequencies, indicating that their performance in dealing with low-frequency noise is comparable. The high-frequency noise in the NPS from LeakyReLU gradually reduces; however, it increases considerably for Swish, suggesting that more structures are present in the reconstruction noise propagated through U-net with Swish. Discussion and Conclusion 1 In this study, an analytical noise propagation method for CNNs in CT imaging is proposed. The five basic layers that comprise a typical CNN include the convolution, nonlinear activation, batch normalization, average pooling, and up-sampling layers. Except for the nonlinear activation layer, the other four layers are all linear, which simplifies the estimation of the covariance of their output by linear propagation. The 1 st order Taylor expansion is used to obtain the linear approximation of the nonlinear activation layer for linear propagation of noise. By integrating the noise propagation of both linear and nonlinear layers in the CNN, we can estimate the covariance of reconstruction from the projection in a step-by-step manner. The results indicate that the covariance estimated by the proposed analytical method agrees well with that estimated by the statistical method, regardless of phantoms, noise levels, and activation functions. We demonstrate that it is feasible to propagate noise from only a single projection to image reconstructed from CNN. The covariance of the projection obtained from U-net with the gradient continuous activation function Swish is smooth, whereas that with the discontinuous gradient activation function LeakyReLU exhibits sharp changes near the boundary. The noise in the reconstruction from LeakyReLU is smaller than that in Swish. They demonstrate opposite performances, where the variance from Swish gradually decreases from the FOV edge to its center. Therefore, LeakyReLU and Swish are completely different in terms of noise suppression. The covariance for Swish spreads wider than that for LeakyReLU, which indicates that Swish uses the information of more neighborhood pixels in denoising. We further qualitatively and quantitatively evaluate network performance from three aspects. The GE, which contains bias and variance, is a tradeoff between the accuracy and noise of the network output and measures the generalization of the neural network. Trained with data under the condition of <math id="M132"><mrow><msub><mi>I</mi><mn>0</mn></msub><mo>=</mo><msup><mrow><mn>10</mn></mrow><mn>4</mn></msup></mrow></math> , the network fails to reduce the GE for projections with lower incident photons, which renders it unacceptable for application in projection denoising with high noise levels. This also limits its application to projections with lower noise when <math id="M133"><mrow><msub><mi>I</mi><mn>0</mn></msub></mrow></math> increases to a certain number because the improvement in GE is trivial. A pixel-wise NRP is defined to measure the denoising ability of the network. The effect of noise suppression is strong only when the noise level is sufficiently high; otherwise, it quickly weakens as the noise level decreases. An evident drop in GE can be observed in smooth regions but not at the edge when the number of incident photons increases, although the NPRs for smooth regions and the edge are the same. Therefore, the accuracy of the smooth regions is higher than that of the edges. In addition, the spatial correlation of noise is analyzed using the NPS. Consequently, it is discovered that there is no significant difference in NPS between LeakyReLU and Swish at low frequencies. However, the NPS at high frequencies is completely opposite for these two activation functions, where it weakens with increasing frequencies for LeakyReLU. The variance in projection denoised by the two activation functions is comparable; however, the NRP of the reconstruction from LeakyReLU is larger than that of Swish, and the NPS of the reconstruction from Swish increases as the frequency increases. Thus, both activation functions demonstrate comparable performance in projection denoising; however, their different noise distributions lead to different effects of noise suppression in reconstruction. Swish utilizes information from more adjacent pixels in noise reduction; therefore, its noise is too structural to be handled by FBP. The noise correlation of LeakyReLU is lower and easier to process. Therefore, the image quality of the reconstructions from LeakyReLU is better than that of Swish in terms of noise suppression. In summary, the proposed analytical noise propagation method is capable of providing a reasonable pixel-wise noise property estimation from only a single sample, whereas other noise estimation methods cannot present comparable performance under the same conditions. Our proposed method can be applied to any inference-ready CNN with a fixed structure and weight for noise estimation. Because the convolution, batch normalization, average pooling, and up-sampling operations are all linear, and the nonlinear activation function is linearly approximated, the noise of the network input propagates linearly in the network. Evidently, the error in the noise estimation of the network output results from the linear approximation of the nonlinear activation function. Two activation functions, LeakyReLU and Swish, are validated in this study; hence, the proposed method is applicable to any network with these two activation functions, regardless of the network structure. Moreover, the noise property estimation of the network output can be used to evaluate the performance of the reconstruction methods. We can characterize noise features based on pixel-by-pixel noise estimation, which also enables us to analyze the spatial correlation and structural properties of noise. The experimental results reveal the significant value of this method in evaluating the output from CNN methods. In future studies, we aim to study the application of covariance estimation to a model observer for subjective image quality assessment. However, the computational cost is expected to increase with increasing network complexity and dimensions. Hence, efficient noise propagation methods for complex and high-dimensional networks must be studied. References M.H. Touch , D.P. Clark , W. Barber et al. , A neural network-based method for spectral distortion correction in photon counting x-ray CT . Phys. Med. Biol. 61 ( 16 ), 6132 - 6153 ( 2016 ). doi: 10.1088/0031-9155/61/16/6132 http://doi.org/10.1088/0031-9155/61/16/6132 A neural network-based method for spectral distortion correction in photon counting x-ray CT M.D. Holbrook , D.P. Clark , C.T. Badea , Deep learning based spectral distortion correction and decomposition for photon counting CT using calibration provided by an energy integrated detector , in SPIE Medical Imaging 2021: Physics of Medical Imaging ( 2021 ). doi: 10.1117/12.2581124 http://doi.org/10.1117/12.2581124 SPIE Medical Imaging 2021: Physics of Medical Imaging K.C. Liang , L. Zhang , H.K. Yang et al. , A model-based unsupervised deep learning method for low-dose CT reconstruction . IEEE Access 8 , 159260 - 159273 ( 2020 ). doi: 10.1109/ACCESS.2020.3020406 http://doi.org/10.1109/ACCESS.2020.3020406 A model-based unsupervised deep learning method for low-dose CT reconstruction Y.K. Zhang , D.L. Hu , Q.L. Zhao et al. , CLEAR: comprehensive learning enabled adversarial reconstruction for subtle structure enhanced low-Dose CT imaging . IEEE Trans. Med. Imaging 40 ( 11 ), 3089 - 3101 ( 2021 ). doi: 10.1109/TMI.2021.3097808 http://doi.org/10.1109/TMI.2021.3097808 CLEAR: comprehensive learning enabled adversarial reconstruction for subtle structure enhanced low-Dose CT imaging H.K. Yang , K.C. Liang , K.J. Kang et al. , Slice-wise reconstruction for low-dose cone-beam CT using a deep residual convolutional neural network . Nucl. Sci. Tech. 30 , 59 ( 2019 ). doi: 10.1007/s41365-019-0581-7 http://doi.org/10.1007/s41365-019-0581-7 Slice-wise reconstruction for low-dose cone-beam CT using a deep residual convolutional neural network X.R. Yin , Q.L. Zhao , J. Liu et al. , Domain progressive 3D residual convolution network to improve low-dose CT imaging . IEEE Trans. Med. Imaging 38 ( 12 ), 2903 - 2913 ( 2019 ). doi: 10.1109/TMI.2019.2917258 http://doi.org/10.1109/TMI.2019.2917258 Domain progressive 3D residual convolution network to improve low-dose CT imaging J. Liu , Y. Zhang , Q.L. Zhao et al. , Deep iterative reconstruction estimation (DIRE): approximate iterative reconstruction estimation for low dose CT imaging . Phys. Med. Biol. 64 ( 13 ), 135007 ( 2019 ). doi: 10.1088/1361-6560/ab18db http://doi.org/10.1088/1361-6560/ab18db Deep iterative reconstruction estimation (DIRE): approximate iterative reconstruction estimation for low dose CT imaging D.L. Hu , J. Liu , T.L. Lv et al. , Hybrid-domain neural network processing for sparse-view CT reconstruction . IEEE Trans. Radiat. Plasma. Med. Sci. 5 ( 1 ), 88 - 98 ( 2021 ). doi: 10.1109/TRPMS.2020.3011413 http://doi.org/10.1109/TRPMS.2020.3011413 Hybrid-domain neural network processing for sparse-view CT reconstruction D. Hu , Y. Zhang , J. Liu et al. , DIOR: deep iterative optimization-based residual-learning for limited-angle CT reconstruction . IEEE Trans. Med. Imaging ( 2022 ). doi: 10.1109/TMI.2022.3148110 http://doi.org/10.1109/TMI.2022.3148110 DIOR: deep iterative optimization-based residual-learning for limited-angle CT reconstruction Y.J. Ma , Y. Ren , P. Feng et al. , Sinogram denoising via attention residual dense convolutional neural network for low-dose computed tomography . Nucl. Sci. Tech. 32 , 41 ( 2021 ). doi: 10.1007/s41365-021-00874-2 http://doi.org/10.1007/s41365-021-00874-2 Sinogram denoising via attention residual dense convolutional neural network for low-dose computed tomography W. Fang , D.F. Wu , K. Kim et al. , Iterative material decomposition for spectral CT using self-supervised Noise2Noise prior . Phys. Med. Biol. 66 ( 15 ), 1 - 17 ( 2021 ). doi: 10.1088/1361-6560/ac0afd http://doi.org/10.1088/1361-6560/ac0afd Iterative material decomposition for spectral CT using self-supervised Noise2Noise prior T.L. Lyu , W. Zhao , Y.S. Zhu et al. , Estimating dual-energy CT imaging from single-energy CT data with material decomposition convolutional neural network . Med. Image Anal. 70 , 102001 ( 2021 ). doi: 10.1016/j.media.2021.102001 http://doi.org/10.1016/j.media.2021.102001 Estimating dual-energy CT imaging from single-energy CT data with material decomposition convolutional neural network A. Zheng , H.K. Yang , L. Zhang et al. , Interweaving network: a novel monochromatic image synthesis method for a photon-counting detector CT system . IEEE Access 8 , 217710 ( 2020 ). doi: 10.1109/ACCESS.2020.3041078 http://doi.org/10.1109/ACCESS.2020.3041078 Interweaving network: a novel monochromatic image synthesis method for a photon-counting detector CT system K.C. Liang , L. Zhang , H.K. Yang et al. , Metal artifact reduction for practical dental computed tomography by improving interpolation-based reconstruction with deep learning . Med. Phys. 46 ( 12 ), e823 - e834 ( 2019 ). doi: 10.1002/mp.13644 http://doi.org/10.1002/mp.13644 Metal artifact reduction for practical dental computed tomography by improving interpolation-based reconstruction with deep learning W. Fang , L. Li , Z.Q. Chen , Removing ring artefacts for photon-counting detectors using neural networks in different domains . IEEE Access 8 , 42447 - 42457 ( 2020 ). doi: 10.1109/ACCESS.2020.2977096 http://doi.org/10.1109/ACCESS.2020.2977096 Removing ring artefacts for photon-counting detectors using neural networks in different domains P.J. Liu , M. Wang , Y.N. Wang et al. , Impact of deep learning-based optimization algorithm on image quality of low-dose coronary CT angiography with noise reduction: a prospective study . Acad. Radiol. 27 ( 9 ), 1241 - 1248 ( 2020 ). doi: 10.1016/j.acra.2019.11.010 http://doi.org/10.1016/j.acra.2019.11.010 Impact of deep learning-based optimization algorithm on image quality of low-dose coronary CT angiography with noise reduction: a prospective study A. Steuwe , M. Weber , O.T. Bethge et al. , Influence of a novel deep-learning based reconstruction software on the objective and subjective image quality in low-dose abdominal computed tomography . Br. J. Radiol. 94 ( 1117 ), 1 - 9 ( 2020 ). doi: 10.1259/bjr.20200677 http://doi.org/10.1259/bjr.20200677 Influence of a novel deep-learning based reconstruction software on the objective and subjective image quality in low-dose abdominal computed tomography C. Park , K.S. Choo , Y. Jung et al. , CT iterative vs deep learning reconstruction: comparison of noise and sharpness . Eur. Radiol. 31 ( 5 ), 3156 - 3164 ( 2021 ). doi: 10.1007/s00330-020-07358-8 http://doi.org/10.1007/s00330-020-07358-8 CT iterative vs deep learning reconstruction: comparison of noise and sharpness J. Greffier , A. Hamard , F. Pereira et al. , Image quality and dose reduction opportunity of deep learning image reconstruction algorithm for CT: a phantom study . Eur. Radiol. 30 ( 7 ), 3951 - 3959 ( 2020 ). doi: 10.1007/s00330-020-06724-w http://doi.org/10.1007/s00330-020-06724-w Image quality and dose reduction opportunity of deep learning image reconstruction algorithm for CT: a phantom study C.T. Jensen , X.M. Liu , E.P. Tamm et al. , Image quality assessment of abdominal CT by use of new deep learning image reconstruction: initial experience . AJR 215 ( 1 ), 50 - 57 ( 2020 ). doi: 10.2214/ajr.19.22332 http://doi.org/10.2214/ajr.19.22332 Image quality assessment of abdominal CT by use of new deep learning image reconstruction: initial experience R. Singh , S.R. Digumarthy , V.V. Muse et al. , Image quality and lesion detection on deep learning reconstruction and iterative reconstruction of submillisievert chest and abdominal CT . AJR 214 ( 3 ), 566 - 573 ( 2020 ). doi: 10.2214/AJR.19.21809 http://doi.org/10.2214/AJR.19.21809 Image quality and lesion detection on deep learning reconstruction and iterative reconstruction of submillisievert chest and abdominal CT X. He , S. Park , Model observers in medical imaging research . Theranostics 3 ( 10 ), 774 - 786 ( 2013 ). doi: 10.7150/thno.5138 http://doi.org/10.7150/thno.5138 Model observers in medical imaging research S. Leng , L.Y. Yu , Y. Zhang et al. , Correlation between model observer and human observer performance in CT imaging when lesion location is uncertain . Med. Phys. 40 ( 8 ), 081908 ( 2013 ). doi: 10.1118/1.4812430 http://doi.org/10.1118/1.4812430 Correlation between model observer and human observer performance in CT imaging when lesion location is uncertain L.Y. Yu , B.Y. Chen , J.M. Kofler et al. , Correlation between a 2D channelized Hotelling observer and human observers in a low-contrast detection task with multislice reading in CT . Med. Phys. 44 ( 8 ), 3990 - 3999 ( 2017 ). doi: 10.1002/mp.12380 http://doi.org/10.1002/mp.12380 Correlation between a 2D channelized Hotelling observer and human observers in a low-contrast detection task with multislice reading in CT G. Kim , M. Han , H. Shim et al. , A convolutional neural network-based model observer for breast CT images . Med. Phys. 47 ( 4 ), 1619 - 1632 ( 2020 ). doi: 10.1002/mp.14072 http://doi.org/10.1002/mp.14072 A convolutional neural network-based model observer for breast CT images D. Piccini , R. Demesmaeker , J. Heerfordt et al. , Deep learning to automate reference-free image quality assessment of whole-heart MR images . Radiol. Artif. Intell. 2 ( 3 ), e190123 ( 2020 ). doi: 10.1148/ryai.2020190123 http://doi.org/10.1148/ryai.2020190123 Deep learning to automate reference-free image quality assessment of whole-heart MR images H. Gong , L.Y. Yu , S. Leng et al. , A deep learning- and partial least square regression-based model observer for a low-contrast lesion detection task in CT . Med. Phys. 46 ( 5 ), 2052 - 2063 ( 2019 ). doi: 10.1002/mp.13500 http://doi.org/10.1002/mp.13500 A deep learning- and partial least square regression-based model observer for a low-contrast lesion detection task in CT H. Gong , Q. Hu , A. Walther et al. , Deep-learning-based model observer for a lung nodule detection task in computed tomography . J. Med. Imaging 7 ( 4 ), 042807 ( 2020 ). doi: 10.1117/1.JMI.7.4.042807 http://doi.org/10.1117/1.JMI.7.4.042807 Deep-learning-based model observer for a lung nodule detection task in computed tomography H. Gong , J.G. Fletcher , J.P. Heiken et al. , Deep-learning model observer for a low-contrast hepatic metastases localization task in computed tomography . Med. Phys. 49 ( 1 ), 70 - 83 ( 2021 ). doi: 10.1002/mp.15362 http://doi.org/10.1002/mp.15362 Deep-learning model observer for a low-contrast hepatic metastases localization task in computed tomography A. H. Abdelaziz , S. Watanabe , J. Hershey et al. , Uncertainty propagation through deep neural networks , in InterSpeech ( 2015 ). https://hal.inria.fr/hal-01162550 https://hal.inria.fr/hal-01162550 Uncertainty propagation through deep neural networks J. Lee , Y Bahri , R. Novak et al. , Deep neural networks as Gaussian processes , in the 6th International Conference on Learning Representations (ICRL 2018) ( 2018 ). arXiv:1711.00165 Deep neural networks as Gaussian processes R. Tanno , D.E. Worrall , E. Kaden et al. , Uncertainty modelling in deep learning for safer neuroimage enhancement: demonstration in diffusion MRI . Neuroimage 225 , 117366 ( 2021 ). doi: 10.1016/j.neuroimage.2020.117366 http://doi.org/10.1016/j.neuroimage.2020.117366 Uncertainty modelling in deep learning for safer neuroimage enhancement: demonstration in diffusion MRI N. Ueda , R. Nakano , Generalization error of ensemble estimators , in Proceedings of International Conference on Neural Networks (ICNN’96) , pp. 90 - 95 ( 1996 ). doi: 10.1109/ICNN.1996.548872 http://doi.org/10.1109/ICNN.1996.548872 Generalization error of ensemble estimators All authors contributed to the study conception and design. Material preparation, data collection and analysis were performed by Xiao-Yue Guo, Li Zhang and Yu-Xiang Xing. The first draft of the manuscript was written by Xiao-Yue Guo and all authors commented on previous versions of the manuscript. All authors read and approved the final manuscript. 10.1007/s41365-022-01057-3 This work was supported by the National Natural Science Foundation of China (Nos. 62031020 and 61771279) . 2021-12-17 2022-04-26 2022-04-30 2022-07-08 2022-07-26 © China Science Publishing & Media Ltd. (Science Press), Shanghai Institute of Applied Physics, the Chinese Academy of Sciences, ChineseNuclear Society and Springer Nature Singapore Pte Ltd. 2022 This is a post-peer-review, pre-copyedit version of an article published in Nuclear Science and Techniques. The final authenticated version is available online at: http://dx.doi.org/10.1007/s41365-022-01057-3 http://dx.doi.org/10.1007/s41365-022-01057-3 . 2022 Nuclear Science and Techniques 06 33