Human- and model-observer performance in ramp-spectrum noise: effects of regularization and object variability

A deep learning anthropomorphic model observer for a detection task in PET

BACKGROUND

Lesion detection is one of the most important clinical tasks in positron emission tomography (PET) for oncology. An anthropomorphic model observer (MO) designed to replicate human observers (HOs) in a detection task is an important tool for assessing task-based image quality. The channelized Hotelling observer (CHO) has been the most popular anthropomorphic MO. Recently, deep learning MOs (DLMOs), mostly based on convolutional neural networks (CNNs), have been investigated for various imaging modalities. However, there have been few studies on DLMOs for PET.

PURPOSE

The goal of the study is to investigate whether DLMOs can predict HOs better than conventional MOs such as CHO in a two-alternative forced-choice (2AFC) detection task using PET images with real anatomical variability.

METHODS

Two types of DLMOs were implemented: (1) CNN DLMO, and (2) CNN-SwinT DLMO that combines CNN and Swin Transformer (SwinT) encoders. Lesion-absent PET images were reconstructed from clinical data, and lesion-present images were reconstructed with adding simulated lesion sinogram data. Lesion-present and lesion-absent PET image pairs were labeled by eight HOs consisting of four radiologists and four image scientists in a 2AFC detection task. In total, 2268 pairs of lesion-present and lesion-absent images were used for training, 324 pairs for validation, and 324 pairs for test. CNN DLMO, CNN-SwinT DLMO, CHO with internal noise, and non-prewhitening matched filter (NPWMF) were compared in the same train-test paradigm. For comparison, six quantitative metrics including prediction accuracy, mean squared errors (MSEs) and correlation coefficients, which measure how well a MO predicts HOs, were calculated in a 9-fold cross-validation experiment.

RESULTS

In terms of the accuracy and MSE metrics, CNN DLMO and CNN-SwinT DLMO showed better performance than CHO and NPWMF, and CNN-SwinT DLMO showed the best performance among the MOs evaluated.

CONCLUSIONS

DLMO can predict HOs more accurately than conventional MOs such as CHO in PET lesion detection. Combining SwinT and CNN encoders can improve the DLMO prediction performance compared to using CNN only.

Experimental measurement of local noise power spectrum (NPS) in photon counting detector-CT (PCD-CT) using a single data acquisition

BACKGROUND

Accurate noise power spectra (NPS) measurement in clinical X-ray CT exams is challenging due to the need for repeated scans, which expose patients to high radiation risks. A reliable method for single CT acquisition NPS estimation is thus highly desirable.

PURPOSE

To develop a method for estimating local NPS from a single photon counting detector-CT (PCD-CT) acquisition.

METHODS

A novel nearly statistical bias-free estimator was constructed from the raw counts data of PCD-CT scan to estimate the variance of sinogram projection data. An analytical algorithm is employed to reconstruct point-wise covariancecov ( x i , x j ) $\text{cov}({\bf x}_i,{\bf x}_j)$ between any two image pixel/voxel locationsx i ${\bf x}_i$ andx j ${\bf x_j}$ . A Fourier transform is applied to obtain the desired point-wise NPS for any chosen locationx i ${\bf x}_i$ . The method was validated using experimental data acquired from a benchtop PCD-CT system with various physical phantoms, and the results were compared with the conventional local NPS measurement method using repeated scans and statistical ensemble averaging.

RESULTS

The experimental results demonstrate that (1) the proposed method can achieve pointwise/local NPS measurement for a region of interest (ROI) located at any chosen position, accurately characterizing the NPS with spatial structures resulting from image content heterogeneity; (2) the local NPS measured using the proposed method show a higher precision in the measured NPS compared to the conventional measurement method; (3) spatial averaging of the local NPS yields the conventional NPS for a given local ROI.

CONCLUSION

A new method was developed to enable local NPS from a single PCD-CT acquisition.

DEMIST: A Deep-Learning-Based Detection-Task-Specific Denoising Approach for Myocardial Perfusion SPECT

There is an important need for methods to process myocardial perfusion imaging (MPI) single-photon emission computed tomography (SPECT) images acquired at lower-radiation dose and/or acquisition time such that the processed images improve observer performance on the clinical task of detecting perfusion defects compared to low-dose images. To address this need, we build upon concepts from model-observer theory and our understanding of the human visual system to propose a detection task-specific deep-learning-based approach for denoising MPI SPECT images (DEMIST). The approach, while performing denoising, is designed to preserve features that influence observer performance on detection tasks. We objectively evaluated DEMIST on the task of detecting perfusion defects using a retrospective study with anonymized clinical data in patients who underwent MPI studies across two scanners (N = 338). The evaluation was performed at low-dose levels of 6.25%, 12.5%, and 25% and using an anthropomorphic channelized Hotelling observer. Performance was quantified using area under the receiver operating characteristics curve (AUC). Images denoised with DEMIST yielded significantly higher AUC compared to corresponding low-dose images and images denoised with a commonly used task-agnostic deep learning-based denoising method. Similar results were observed with stratified analysis based on patient sex and defect type. Additionally, DEMIST improved visual fidelity of the low-dose images as quantified using root mean squared error and structural similarity index metric. A mathematical analysis revealed that DEMIST preserved features that assist in detection tasks while improving the noise properties, resulting in improved observer performance. The results provide strong evidence for further clinical evaluation of DEMIST to denoise low-count images in MPI SPECT.

Applying Self-Supervised Learning to Image Quality Assessment in Chest CT Imaging

Many new reconstruction techniques have been deployed to allow low-dose CT examinations. Such reconstruction techniques exhibit nonlinear properties, which strengthen the need for a task-based measure of image quality. The Hotelling observer (HO) is the optimal linear observer and provides a lower bound of the Bayesian ideal observer detection performance. However, its computational complexity impedes its widespread practical usage. To address this issue, we proposed a self-supervised learning (SSL)-based model observer to provide accurate estimates of HO performance in very low-dose chest CT images. Our approach involved a two-stage model combining a convolutional denoising auto-encoder (CDAE) for feature extraction and dimensionality reduction and a support vector machine for classification. To evaluate this approach, we conducted signal detection tasks employing chest CT images with different noise structures generated by computer-based simulations. We compared this approach with two supervised learning-based methods: a single-layer neural network (SLNN) and a convolutional neural network (CNN). The results showed that the CDAE-based model was able to achieve similar detection performance to the HO. In addition, it outperformed both SLNN and CNN when a reduced number of training images was considered. The proposed approach holds promise for optimizing low-dose CT protocols across scanner platforms.

Modeling human observer detection for varying data acquisition in undersampled MRI for two-alternative forced choice (2-AFC) and forced localization tasks

Undersampling in the frequency domain (k-space) in MRI enables faster data acquisition. In this study, we used a fixed 1D undersampling factor of 5x with only 20% of the k-space collected. The fraction of fully acquired low k-space frequencies were varied from 0% (all aliasing) to 20% (all blurring). The images were reconstructed using a multi-coil SENSE algorithm. We used two-alternative forced choice (2-AFC) and the forced localization tasks with a subtle signal to estimate the human observer performance. The 2-AFC average human observer performance remained fairly constant across all imaging conditions. The forced localization task performance improved from the 0% condition to the 2.5% condition and remained fairly constant for the remaining conditions, suggesting that there was a decrease in task performance only in the pure aliasing situation. We modeled the average human performance using a sparse-difference of Gaussians (SDOG) Hotelling observer model. Because the blurring in the undersampling direction makes the mean signal asymmetric, we explored an adaptation for irregular signals that made the SDOG template asymmetric. To improve the observer performance, we also varied the number of SDOG channels from 3 to 4. We found that despite the asymmetry in the mean signal, both the symmetric and asymmetric models reasonably predicted the human performance in the 2-AFC experiments. However, the symmetric model performed slightly better. We also found that a symmetric SDOG model with 4 channels implemented using a spatial domain convolution and constrained to the possible signal locations reasonably modeled the forced localization human observer results.

Pediatric evaluations for deep learning CT denoising

BACKGROUND

Deep learning (DL) CT denoising models have the potential to improve image quality for lower radiation dose exams. These models are generally trained with large quantities of adult patient image data. However, CT, and increasingly DL denoising methods, are used in both adult and pediatric populations. Pediatric body habitus and size can differ significantly from adults and vary dramatically from newborns to adolescents. Ensuring that pediatric subgroups of different body sizes are not disadvantaged by DL methods requires evaluations capable of assessing performance in each subgroup.

PURPOSE

To assess DL CT denoising in pediatric and adult-sized patients, we built a framework of computer simulated image quality (IQ) control phantoms and evaluation methodology.

METHODS

The computer simulated IQ phantoms in the framework featured pediatric-sized versions of standard CatPhan 600 and MITA-LCD phantoms with a range of diameters matching the mean effective diameters of pediatric patients ranging from newborns to 18 years old. These phantoms were used in simulating CT images that were then inputs for a DL denoiser to evaluate performance in different sized patients. Adult CT test images were simulated using standard-sized phantoms scanned with adult scan protocols. Pediatric CT test images were simulated with pediatric-sized phantoms and adjusted pediatric protocols. The framework's evaluation methodology consisted of denoising both adult and pediatric test images then assessing changes in image quality, including noise, image sharpness, CT number accuracy, and low contrast detectability. To demonstrate the use of the framework, a REDCNN denoising model trained on adult patient images was evaluated. To validate that the DL model performance measured with the proposed pediatric IQ phantoms was representative of performance in more realistic patient anatomy, anthropomorphic pediatric XCAT phantoms of the same age range were also used to compare noise reduction performance.

RESULTS

Using the proposed pediatric-sized IQ phantom framework, size differences between adult and pediatric-sized phantoms were observed to substantially influence the adult trained DL denoising model's performance. When applied to adult images, the DL model achieved a 60% reduction in noise standard deviation without substantial loss in sharpness in mid or high spatial frequencies. However, in smaller phantoms the denoising performance dropped due to different image noise textures resulting from the smaller field of view (FOV) between adult and pediatric protocols. In the validation study, noise reduction trends in the pediatric-sized IQ phantoms were found to be consistent with those found in anthropomorphic phantoms.

CONCLUSION

We developed a framework of using pediatric-sized IQ phantoms for pediatric subgroup evaluation of DL denoising models. Using the framework, we found the performance of an adult trained DL denoiser did not generalize well in the smaller diameter phantoms corresponding to younger pediatric patient sizes. Our work suggests noise texture differences from FOV changes between adult and pediatric protocols can contribute to poor generalizability in DL denoising and that the proposed framework is an effective means to identify these performance disparities for a given model.

Dose-efficiency quantification of computed tomography systems using a model-observer

BACKGROUND

Recent advances in computed tomography (CT) technology have considerably improved the quality of CT images and reduced radiation exposure in patients. At present, however, there is no generally accepted figure of merit (FOM) for comparing the dose efficiencies of CT systems.

PURPOSE

(i) To establish an FOM that characterizes the quality of CT images in relation to the radiation dose by means of a mathematical model observer and (ii) to evaluate the new FOM on different CT systems and image reconstruction algorithms.

METHODS

Images of a homogeneous phantom with four low-contrast inserts were acquired using three different CT systems at three dose levels and a representative protocol for CT imaging of low-contrast objects in the abdomen. The images were reconstructed using filtered-back projection and iterative algorithms. A channelized hotelling observer with difference-of-Gaussian channels was applied to compute the detectability (d ' $d^{\prime}$ ). This was done for each insert and each of the considered imaging conditions from square regions of interest (ROIs) that were (semi-)automatically centered on the inserts. The estimated detectabilities (d ' $d^{\prime}$ ) were averaged in the first step over the three dose levels (⟨ d ' ⟩ $\langle {d^{\prime}} \rangle $ ), and subsequently over the four contrast inserts (⟨ d ' ⟩ w ${\langle {d^{\prime}} \rangle _{\rm{w}}}$ ). All calculation steps included a dedicated assessment of the related uncertainties following accepted metrological guidelines.

RESULTS

The determined detectabilities (d ' $d^{\prime}$ ) varied considerably with the contrast and diameter of the four inserts, as well as with the radiation doses and reconstruction algorithms used for image generation (d ' $d^{\prime}\;$ = 1.3-5.5). Thus, the specification of a single detectability as an FOM is not well suited for comprehensively characterizing the dose efficiency of a CT system. A more comprehensive and robust characterization was provided by the averaged detectabilities⟨ d ' ⟩ $\langle {d^{\prime}} \rangle $ and, in particular,⟨ d ' ⟩ w ${\langle {d^{\prime}} \rangle _{\rm{w}}}$ . Our analysis reveals that the model observer analysis is very sensitive to the exact position of the ROIs.

CONCLUSIONS

The presented automatable software approach yielded with the weighted detectability⟨ d ' ⟩ w ${\langle {d^{\prime}} \rangle _{\rm{w}}}$ an objective FOM to benchmark different CT systems and reconstruction algorithms in a robust and reliable manner. An essential advantage of the proposed model-observer approach is that uncertainties in the FOM can be provided, which is an indispensable prerequisite for type testing.

Power-law spectrum-based objective function to train a generative adversarial network with transfer learning for the synthetic breast CT image

Objective.This paper proposes a new objective function to improve the quality of synthesized breast CT images generated by the GAN and compares the GAN performances on transfer learning datasets from different image domains.Approach.The proposed objective function, named beta loss function, is based on the fact that x-ray-based breast images follow the power-law spectrum. Accordingly, the exponent of the power-law spectrum (beta value) for breast CT images is approximately two. The beta loss function is defined in terms of L1 distance between the beta value of synthetic images and validation samples. To compare the GAN performances for transfer learning datasets from different image domains, ImageNet and anatomical noise images are used in the transfer learning dataset. We employ styleGAN2 as the backbone network and add the proposed beta loss function. The patient-derived breast CT dataset is used as the training and validation dataset; 7355 and 212 images are used for network training and validation, respectively. We use the beta value evaluation and Fréchet inception distance (FID) score for quantitative evaluation.Main results.For qualitative assessment, we attempt to replicate the images from the validation dataset using the trained GAN. Our results show that the proposed beta loss function achieves a more similar beta value to real images and a lower FID score. Moreover, we observe that the GAN pretrained with anatomical noise images achieves better equality than ImageNet for beta value evaluation and FID score. Finally, the beta loss function with anatomical noise as the transfer learning dataset achieves the lowest FID score.Significance.Overall, the GAN using the proposed beta loss function with anatomical noise images as the transfer learning dataset provides the lowest FID score among all tested cases. Hence, this work has implications for developing GAN-based breast image synthesis methods for medical imaging applications.

Convolutional neural network-based model observer for signal known statistically task in breast tomosynthesis images

BACKGROUND

Since human observer studies are resource-intensive, mathematical model observers are frequently used to assess task-based image quality. The most common implementation of these model observers assume that the signal information is exactly known. However, these tasks cannot thoroughly represent situations where the signal information is not exactly known in terms of size and shape.

PURPOSE

Considering the limitations of the tasks for which signal information is exactly known, we proposed a convolutional neural network (CNN)-based model observer for signal known statistically (SKS) and background known statistically (BKS) detection tasks in breast tomosynthesis images.

METHODS

A wide parameter search was conducted from six different acquisition angles (i.e., 10°, 20°, 30°, 40°, 50°, and 60°) within the same dose level (i.e., 2.3 mGy) under two separate acquisition schemes: (1) constant total number of projections, and (2) constant angular separation between projections. Two different types of signals: spherical (i.e., SKE tasks) and spiculated (i.e., SKS tasks) were used. The detection performance of the CNN-based model observer was compared with that of the Hotelling observer (HO) instead of the IO. Pixel-wise gradient-weighted class activation mapping (pGrad-CAM) map was extracted from each reconstructed tomosynthesis image to provide an intuitive understanding of the trained CNN-based model observer.

RESULTS

The CNN-based model observer achieved a higher detection performance compared to that of the HO for all tasks. Moreover, the improvement in its detection performance was greater for SKS tasks compared to that for SKE tasks. These results demonstrated that the addition of nonlinearity improved the detection performance owing to the variation of the background and signal. Interestingly, the pGrad-CAM results effectively localized the class-specific discriminative region, further supporting the quantitative evaluation results of the CNN-based model observer. In addition, we verified that the CNN-based model observer required fewer images to achieve the detection performance of the HO.

CONCLUSIONS

In this work, we proposed a CNN-based model observer for SKS and BKS detection tasks in breast tomosynthesis images. Throughout the study, we demonstrated that the detection performance of the proposed CNN-based model observer was superior to that of the HO.

A 2D Synthesized Image Improves the 3D Search for Foveated Visual Systems

Current medical imaging increasingly relies on 3D volumetric data making it difficult for radiologists to thoroughly search all regions of the volume. In some applications (e.g., Digital Breast Tomosynthesis), the volumetric data is typically paired with a synthesized 2D image (2D-S) generated from the corresponding 3D volume. We investigate how this image pairing affects the search for spatially large and small signals. Observers searched for these signals in 3D volumes, 2D-S images, and while viewing both. We hypothesize that lower spatial acuity in the observers' visual periphery hinders the search for the small signals in the 3D images. However, the inclusion of the 2D-S guides eye movements to suspicious locations, improving the observer's ability to find the signals in 3D. Behavioral results show that the 2D-S, used as an adjunct to the volumetric data, improves the localization and detection of the small (but not large) signal compared to 3D alone. There is a concomitant reduction in search errors as well. To understand this process at a computational level, we implement a Foveated Search Model (FSM) that executes human eye movements and then processes points in the image with varying spatial detail based on their eccentricity from fixations. The FSM predicts human performance for both signals and captures the reduction in search errors when the 2D-S supplements the 3D search. Our experimental and modeling results delineate the utility of 2D-S in 3D search-reduce the detrimental impact of low-resolution peripheral processing by guiding attention to regions of interest, effectively reducing errors.

Discrimination tasks in simulated low-dose CT noise

BACKGROUND

This study reports the results of a set of discrimination experiments using simulated images that represent the appearance of subtle lesions in low-dose computed tomography (CT) of the lungs. Noise in these images has a characteristic ramp-spectrum before apodization by noise control filters. We consider three specific diagnostic features that determine whether a lesion is considered malignant or benign, two system-resolution levels, and four apodization levels for a total of 24 experimental conditions.

PURPOSE

The goal of the investigation is to better understand how well human observers perform subtle discrimination tasks like these, and the mechanisms of that performance. We use a forced-choice psychophysical paradigm to estimate observer efficiency and classification images. These measures quantify how effectively subjects can read the images, and how they use images to perform discrimination tasks across the different imaging conditions.

MATERIALS AND METHODS

The simulated CT images used as stimuli in the psychophysical experiments are generated from high-resolution objects passed through a modulation transfer function (MTF) before down-sampling to the image-pixel grid. Acquisition noise is then added with a ramp noise-power spectrum (NPS), with subsequent smoothing through apodization filters. The features considered are lesion size, indistinct lesion boundary, and a nonuniform lesion interior. System resolution is implemented by an MTF with resolution (10% max.) of 0.47 or 0.58 cyc/mm. Apodization is implemented by a Shepp-Logan filter (Sinc profile) with various cutoffs. Six medically naïve subjects participated in the psychophysical studies, entailing training and testing components for each condition. Training consisted of staircase procedures to find the 80% correct threshold for each subject, and testing involved 2000 psychophysical trials at the threshold value for each subject. Human-observer performance is compared to the Ideal Observer to generate estimates of task efficiency. The significance of imaging factors is assessed using ANOVA. Classification images are used to estimate the linear template weights used by subjects to perform these tasks. Classification-image spectra are used to analyze subject weights in the spatial-frequency domain.

RESULTS

Overall, average observer efficiency is relatively low in these experiments (10%-40%) relative to detection and localization studies reported previously. We find significant effects for feature type and apodization level on observer efficiency. Somewhat surprisingly, system resolution is not a significant factor. Efficiency effects of the different features appear to be well explained by the profile of the linear templates in the classification images. Increasingly strong apodization is found to both increase the classification-image weights and to increase the mean-frequency of the classification-image spectra. A secondary analysis of "Unapodized" classification images shows that this is largely due to observers undoing (inverting) the effects of apodization filters.

CONCLUSIONS

These studies demonstrate that human observers can be relatively inefficient at feature-discrimination tasks in ramp-spectrum noise. Observers appear to be adapting to frequency suppression implemented in apodization filters, but there are residual effects that are not explained by spatial weighting patterns. The studies also suggest that the mechanisms for improving performance through the application of noise-control filters may require further investigation.

Deep Learning Algorithm for Differentiating Patients with a Healthy Liver from Patients with Liver Lesions Based on MR Images

The problems in diagnosing the state of a vital organ such as the liver are complex and remain unresolved. These problems are underscored by frequently published studies on this issue. At the same time, demand for imaging diagnostics, preferably using a method that can detect the disease at the earliest possible stage, is constantly increasing. In this paper, we present liver diseases in the context of diagnosis, diagnostic problems, and possible elimination. We discuss the dataset and methods and present the stages of the pipeline we developed, leading to multiclass segmentation of the liver in multiparametric MR image into lesions and normal tissue. Finally, based on the processing results, each case is classified as either a healthy liver or a liver with lesions. For the training set, the AUC ROC is 0.925 (standard error 0.013 and a p-value less than 0.001), and for the test set, the AUC ROC is 0.852 (standard error 0.039 and a p-value less than 0.001). Further refinements to the proposed pipeline are also discussed. The proposed approach could be used in the detection of focal lesions in the liver and the description of liver tumors. Practical application of the developed multi-class segmentation method represents a key step toward standardizing the medical evaluation of focal lesions in the liver.

Dual-Tracer Parathyroid Imaging Using Joint SPECT Reconstruction

Purpose

We assessed the lesion detection performance of the dual-tracer parathyroid SPECT imaging using the joint reconstruction method.

Materials and Methods

Thirty-six noise realizations were created from SPECT projections collected from an in-house neck phantom to emulate 99mTc-pertechnetate/99mTc-sestamibi parathyroid SPECT datasets. Difference images representing parathyroid lesions were reconstructed using the subtraction and the joint methods whose corresponding optimal iteration was defined as the iteration which maximized the channelized Hotelling observer signal-to-noise ratio (CHO-SNR). The joint method whose initial estimate was derived from the subtraction method at optimal iteration (the joint-AltInt method) was also assessed. In a study of 36 patients, a human-observer lesion-detection study was performed using difference images from the three methods at optimal iteration and the subtraction method with four iterations. The area under the receiver operating characteristic curve (AUC) was calculated for each method.

Results

In the phantom study, both the joint-AltInt method and the joint method improved SNR compared to the subtraction method at their optimal iteration by 444% and 81%, respectively. In the patient study, the joint-AltInt method yielded the highest AUC of 0.73 as compared with 0.72, 0.71, and 0.64 from the joint method, the subtraction method at optimal iteration, and the subtraction method at four iterations. At a specificity of at least 0.70, the joint-AltInt method yielded significantly higher sensitivity than the other methods (0.60 vs 0.46, 042, and 0.42; p < 0.05).

Conclusions

The joint reconstruction method yielded higher lesion detectability than the conventional method and holds promise for dual-tracer parathyroid SPECT imaging.

A nonparametric measure of noise in x-ray diagnostic images-mammography

Objective.In x-ray diagnostics, modern image reconstruction or image processing methods may render established methods of image quality assessment inadequate. Task specific quality assessment by using model observers has the disadvantage of being very labour-intensive. Therefore, it appears highly desirable to develop novel image quality parameters that neither rely on the linearity and the shift-invariace of the imaging system nor require the acquisition of hundreds of images as is necessary for the application of model observers, and which can be derived directly from diagnostic images.Approach.A new measure for the noise based on non-maximum-suppression images is defined and its properties are explored using simulated images before it is applied to an exposure series of mammograms of a homogeneous phantom and a 3D-printed breast phantom to demonstrate its usefulness under realistic conditions.Main results.The new noise parameter cannot only be derived from images with a homogeneous background but it can be extracted directly from images containing anatomic structures and is proportional to the standard deviation of the noise. At present, the applicability is restricted to mammography, which satisfies the assumption of short covariance length of the noise.Significance.The new measure of the noise is but a first step of the development of a set of parameters that are required to quantify image quality directly from diagnostic images without relying on the assumption of a linear, shift-invariant system, e.g. by providing measures of sharpness, contrast and structural complexity, in addition to the noise measure. For mammography, a convenient method is now available to quantify noise in processed diagnostic images.

Comparison of supervised-learning approaches for designing a channelized observer for image quality assessment in CT

BACKGROUND

The current paradigm for evaluating computed tomography (CT) system performance relies on a task-based approach. As the Hotelling observer (HO) provides an upper bound of observer performances in specific signal detection tasks, the literature advocates HO use for optimization purposes. However, computing the HO requires calculating the inverse of the image covariance matrix, which is often intractable in medical applications. As an alternative, dimensionality reduction has been extensively investigated to extract the task-relevant features from the raw images. This can be achieved by using channels, which yields the channelized-HO (CHO). The channels are only considered efficient when the channelized observer (CO) can approximate its unconstrained counterpart. Previous work has demonstrated that supervised learning-based methods can usually benefit CO design, either for generating efficient channels using partial least squares (PLS) or for replacing the Hotelling detector with machine-learning (ML) methods.

PURPOSE

Here we investigated the efficiency of a supervised ML-algorithm used to design a CO for predicting the performance of unconstrained HO. The ML-algorithm was applied either (1) in the estimator for dimensionality reduction, or (2) in the detector function.

METHODS

A channelized support vector machine (CSVM) was employed and compared against the CHO in terms of ability to predict HO performances. Both the CSVM and the CHO were estimated with channels derived from the singular value decomposition (SVD) of the system operator, principal component analysis (PCA), and PLS. The huge variety of regularization strategies proposed by CT system vendors for statistical image reconstruction (SIR) make the generalization capability of an observer a key point to consider upfront of implementation in clinical practice. To evaluate the generalization properties of the observers, we adopted a 2-step testing process: (1) achieved with the same regularization strategy (as in the training phase) and (2) performed using different reconstruction properties. We generated simulated- signal-known-exactly/background-known-exactly (SKE/BKE) tasks in which different noise structures were generated using Markov random field (MRF) regularizations using either a Green or a quadratic, function.

RESULTS

The CSVM outperformed the CHO for all types of channels and regularization strategies. Furthermore, even though both COs generalized well to images reconstructed with the same regularization strategy as the images considered in the training phase, the CHO failed to generalize to images reconstructed differently whereas the CSVM managed to successfully generalize. Lastly, the proposed CSVM observer used with PCA channels outperformed the CHO with PLS channels while using a smaller training data set.

CONCLUSION

These results argue for introducing the supervised-learning paradigm in the detector function rather than in the operator of the channels when designing a CO to provide an accurate estimate of HO performance. The CSVM with PCA channels proposed here could be used as a surrogate for HO in image quality assessment.

Modeling human observer detection in undersampled magnetic resonance imaging reconstruction with total variation and wavelet sparsity regularization

Purpose

Task-based assessment of image quality in undersampled magnetic resonance imaging provides a way of evaluating the impact of regularization on task performance. In this work, we evaluated the effect of total variation (TV) and wavelet regularization on human detection of signals with a varying background and validated a model observer in predicting human performance.

Approach

Human observer studies used two-alternative forced choice (2-AFC) trials with a small signal known exactly task but with varying backgrounds for fluid-attenuated inversion recovery images reconstructed from undersampled multi-coil data. We used a 3.48 undersampling factor with TV and a wavelet sparsity constraints. The sparse difference-of-Gaussians (S-DOG) observer with internal noise was used to model human observer detection. The internal noise for the S-DOG was chosen to match the average percent correct (PC) in 2-AFC studies for four observers using no regularization. That S-DOG model was used to predict the PC of human observers for a range of regularization parameters.

Results

We observed a trend that the human observer detection performance remained fairly constant for a broad range of values in the regularization parameter before decreasing at large values. A similar result was found for the normalized ensemble root mean squared error. Without changing the internal noise, the model observer tracked the performance of the human observers as the regularization was increased but overestimated the PC for large amounts of regularization for TV and wavelet sparsity, as well as the combination of both parameters.

Conclusions

For the task we studied, the S-DOG observer was able to reasonably predict human performance with both TV and wavelet sparsity regularizers over a broad range of regularization parameters. We observed a trend that task performance remained fairly constant for a range of regularization parameters before decreasing for large amounts of regularization.

Evaluation of the quantitative performance of non-enhanced dual-energy CT X-map in detecting acute ischemic brain stroke: A model observer study using computer simulation

PURPOSE

A simulation study was performed to evaluate the quantitative performance of X-map images-derived from non-enhanced (NE) dual-energy computed tomography (DECT)-in detecting acute ischemic stroke (AIS) compared with that of NE-DECT mixed images.

METHODS

A virtual phantom, 150 mm in diameter, filled with tissues comprising various gray- and white-matter proportions was used to generate pairs of NE-head images at 80 kV and Sn150 kV at three dose levels (20, 40, and 60 mGy). The phantom included an inserted low-contrast object, 15 mm in diameter, with four densities (0%, 5%, 10%, and 15%) mimicking ischemic edema. Mixed and X-map images were generated from these sets of images and compared in terms of detectability of ischemic edema using a channelized Hotelling observer (CHO). The area under the curve (AUC) of the receiver operating characteristic that generated CHO for each condition was used as a figure of merit.

RESULTS

The AUCs of X-map images were always significantly higher than those of mixed images (P < 0.001). The improvement in AUC for X-map images compared with that for mixed images at edema densities was 9.2%-12.6% at 20 mGy, 10.1%-17.7% at 40 mGy, and 14.0%-19.4% at 60 mGy. At any edema density, X-map images at 20 mGy resulted in higher AUCs than mixed images acquired at any other dose level (P < 0.001), which corresponded to a 66% dose reduction on X-map images.

CONCLUSIONS

The simulation study confirmed that NE-DECT X-map images have superior capability of detecting AIS than NE-DECT mixed images.

Nuclear Medicine and Artificial Intelligence: Best Practices for Evaluation (the RELAINCE Guidelines)

An important need exists for strategies to perform rigorous objective clinical-task-based evaluation of artificial intelligence (AI) algorithms for nuclear medicine. To address this need, we propose a 4-class framework to evaluate AI algorithms for promise, technical task-specific efficacy, clinical decision making, and postdeployment efficacy. We provide best practices to evaluate AI algorithms for each of these classes. Each class of evaluation yields a claim that provides a descriptive performance of the AI algorithm. Key best practices are tabulated as the RELAINCE (Recommendations for EvaLuation of AI for NuClear medicinE) guidelines. The report was prepared by the Society of Nuclear Medicine and Molecular Imaging AI Task Force Evaluation team, which consisted of nuclear-medicine physicians, physicists, computational imaging scientists, and representatives from industry and regulatory agencies.

Iterative reconstruction with multifrequency signal recognition technology to improve low-contrast detectability: A phantom study

Background

Brain CT needs more attention to improve the extremely low image contrast and image texture.

Purpose

To evaluate the performance of iterative progressive reconstruction with visual modeling (IPV) for the improvement of low-contrast detectability (IPV-LCD) compared with filtered backprojection (FBP) and conventional IPV.

Materials and methods

Low-contrast and water phantoms were used. Helical scans were conducted with the use of a CT scanner with 64 detectors. The tube voltage was set at 120 kVp; the tube current was adjusted from 60 to 300 mA with a slice thickness of 0.625 mm and from 20 to 150 mA with a slice thickness of 5.0 mm. Images were reconstructed with the FBP, conventional IPV, and IPV-LCD algorithms. The channelized Hotelling observer (CHO) model was applied in conjunction with the use of low-contrast modules in the low-contrast phantom. The noise power spectrum (NPS) and normalized NPS were calculated.

Results

At the same standard and strong levels, the IPV-LCD method improved low-contrast detectability compared with the conventional IPV, regardless of contrast-rod diameters. The mean CHO values at a slice thickness of 0.625 mm were 1.83, 3.28, 4.40, 4.53, and 5.27 for FBP, IPV STD, IPV-LCD STD, IPV STR, and IPV-LCD STR, respectively. The normalized NPS for the IPV-LCD STD and STR images were slightly shifted to the higher frequency compared with that for the FBP image.

Conclusion

IPV-LCD images further improve the low-contrast detectability compared with FBP and conventional IPV images while maintaining similar FBP image appearances.

Effect of optical blurring of X-ray source on breast tomosynthesis image quality: Modulation transfer function, anatomical noise power spectrum, and signal detectability perspectives

We investigated the effect of the optical blurring of X-ray source on digital breast tomosynthesis (DBT) image quality using well-designed DBT simulator and table-top experimental systems. To measure the in-plane modulation transfer function (MTF), we used simulated sphere phantom and Teflon sphere phantom and generated their projection data using two acquisition modes (i.e., step-and-shoot mode and continuous mode). After reconstruction, we measured the in-plane MTF using reconstructed sphere phantom images. In addition, we measured the anatomical noise power spectrum (aNPS) and signal detectability. We constructed simulated breast phantoms with a 50% volume glandular fraction (VGF) of breast anatomy using the power law spectrum and inserted spherical objects with 1 mm, 2 mm, and 5 mm diameters as breast masses. Projection data were acquired using two acquisition modes, and in-plane breast images were reconstructed using the Feldkamp-Davis-Kress (FDK) algorithm. For the experimental study, we used BR3D breast phantom with 50% VGF and obtained projection data using a table-top experimental system. To compare the detection performance of the two acquisition modes, we calculated the signal detectability using the channelized Hotelling observer (CHO) with Laguerre-Gauss (LG) channels. Our results show that spatial resolution of in-plane image in continuous mode was degraded due to the optical blurring of X-ray source. This blurring effect was reflected in aNPS, resulting in large β values. From a signal detectability perspective, the signal detectability in step-and-shoot mode is higher than that in continuous mode for small spherical signals but not large spherical signals. Although the step-and-shoot mode has disadvantage in terms of scan time compared to the continuous mode, scanning in step-and-shoot mode is better for detecting small signals, indicating that there is a tradeoff between scan time and image quality.

Computer Simulations of Case Difficulty in Digital Breast Tomosynthesis Using Virtual Clinical Trials

PURPOSE

Virtual clinical trials (VCTs) require computer simulations of representative patients and images to evaluate and compare changes in performance of imaging technologies. The simulated images are usually interpreted by model observers whose performance depends upon the selection of imaging cases used in training evaluation models. This work proposes an efficient method to simulate and calibrate soft tissue lesions, which matches the detectability threshold of virtual and human readings.

METHODS

Anthropomorphic breast phantoms were used to evaluate the simulation of four mass models (I-IV) that vary in shape and composition of soft tissue. Ellipsoidal (I) and spiculated (II-IV) masses were simulated using composite voxels with partial volumes. Digital breast tomosynthesis projections and reconstructions of a clinical system were simulated. Channelized Hotelling observers (CHOs) were evaluated using reconstructed slices of masses that varied in shape, composition, and density of surrounded tissue. The detectability threshold of each mass model was evaluated using receiver operating characteristic (ROC) curves calculated with the CHO's scores.

RESULTS

The area under the curve (AUC) of each calibrated mass model were within the 95% confidence interval (mean AUC [95% CI]) reported in a previous reader study (0.93 [0.89, 0.97]). The mean AUC [95% CI] obtained were 0.94 [0.93, 0.96], 0.92 [0.90, 0.93], 0.92 [0.90, 0.94], 0.93 [0.92, 0.95] for models I to IV, respectively. The mean AUC results varied substantially as a function of shape, composition, and density of surrounded tissue.

CONCLUSIONS

For successful VCTs, lesions composed of soft tissue should be calibrated to simulate imaging cases that match the case difficulty predicted by human readers. Lesion composition, shape, and size are parameters that should be carefully selected to calibrate VCTs. This article is protected by copyright. All rights reserved.

Predicting human detection performance in magnetic resonance imaging (MRI) with total variation and wavelet sparsity regularizers

Two common regularization methods in reconstruction of magnetic resonance images are total variation (TV) which restricts the magnitude of the gradient in the reconstructed image and wavelet sparsity which assumes that the object being imaged is sparse in the wavelet domain. These regularization methods have resulted in images with fewer undersampling artifacts and less noise but introduce their own artifacts. In this work, we extend previous results on modeling of human observer performance for images using TV regularization to also predict human detection performance using wavelet regularization and a combination of wavelet and TV regularization. Small lesions were placed in the coil k-space data for fluid-attenuated inversion recovery (FLAIR) brain images from the fastMRI database. The data was undersampled using an acceleration factor of 3.48. The undersampled data was reconstructed using a range of regularization parameters for both the TV and wavelet regularization. The internal noise level for the sparse difference-of-Gaussians (S-DOG) model observer was chosen to match the average human percent correct in two-alternative forced choice (2-AFC) studies with a signal known exactly with variable backgrounds and no regularization. The S-DOG model largely tracked the human observer results except at large values of the regularization parameter where it outperformed the average human observer. We found that the regularization with either constraint or in combination did not improve human observer performance for this task.

Impact of reconstruction parameters on lesion detection and localization in joint ictal/inter-ictal SPECT reconstruction

PURPOSE

Previously, a joint ictal/inter-ictal SPECT reconstruction was proposed to reconstruct a differential image representing the change of brain SPECT image from an inter-ictal to an ictal study. The so-called joint method yielded better performance for epileptic foci localization than the conventional subtraction method. In this study, we evaluated the performance of different reconstruction settings of the joint reconstruction of ictal/inter-ictal SPECT data, which creates a differential image showing the difference between ictal and inter-ictal images, in lesion detection and localization in epilepsy imaging.

METHODS

Differential images reconstructed from phantom data using the joint and the subtraction methods were compared based on lesion detection performance (channelized Hotelling observer signal-to-noise ratio (SNR_CHO) averaged across four lesion-to-background contrast levels) at the optimal iteration. The joint-initial method which was the joint method that was initialized by the subtraction method at optimal iteration was also used to reconstruct differential images. These three methods with respective optimal iteration and the subtraction method with four iterations were applied to epileptic patient datasets. A human observer lesion localization study was performed based on localization receiver operating characteristic (LROC) analysis.

RESULTS

From the phantom study, at their respective optimal iteration, the joint method yielded an improvement in lesion detection performance over the subtraction method of 26%, which increased to 145% when using the joint-initial method. From the patient study, the joint-initial method yielded the highest area under the LROC curve as compared with those of the joint and the subtraction methods with optimal iteration and with 4 iterations (0.44 vs 0.41, 0.39 and 0.36, respectively).

CONCLUSIONS

In lesion detection and localization, the joint method at optimal iteration outperformed the subtraction method at optimal iteration and at iteration typically used in clinical practice. Furthermore, initialization by the subtraction method improved the performance of the joint method.

Objective Task-Based Evaluation of Artificial Intelligence-Based Medical Imaging Methods:: Framework, Strategies, and Role of the Physician

Artificial intelligence-based methods are showing promise in medical imaging applications. There is substantial interest in clinical translation of these methods, requiring that they be evaluated rigorously. We lay out a framework for objective task-based evaluation of artificial intelligence methods. We provide a list of available tools to conduct this evaluation. We outline the important role of physicians in conducting these evaluation studies. The examples in this article are proposed in the context of PET scans with a focus on evaluating neural network-based methods. However, the framework is also applicable to evaluate other medical imaging modalities and other types of artificial intelligence methods.

Assessing the Impact of Deep Neural Network-Based Image Denoising on Binary Signal Detection Tasks

A variety of deep neural network (DNN)-based image denoising methods have been proposed for use with medical images. Traditional measures of image quality (IQ) have been employed to optimize and evaluate these methods. However, the objective evaluation of IQ for the DNN-based denoising methods remains largely lacking. In this work, we evaluate the performance of DNN-based denoising methods by use of task-based IQ measures. Specifically, binary signal detection tasks under signal-known-exactly (SKE) with background-known-statistically (BKS) conditions are considered. The performance of the ideal observer (IO) and common linear numerical observers are quantified and detection efficiencies are computed to assess the impact of the denoising operation on task performance. The numerical results indicate that, in the cases considered, the application of a denoising network can result in a loss of task-relevant information in the image. The impact of the depth of the denoising networks on task performance is also assessed. The presented results highlight the need for the objective evaluation of IQ for DNN-based denoising technologies and may suggest future avenues for improving their effectiveness in medical imaging applications.

Medical image quality metrics for foveated model observers

Purpose: A recently proposed model observer mimics the foveated nature of the human visual system by processing the entire image with varying spatial detail, executing eye movements, and scrolling through slices. The model can predict how human search performance changes with signal type and modality (2D versus 3D), yet its implementation is computationally expensive and time-consuming. Here, we evaluate various image quality metrics using extensions of the classic index of detectability expression and assess foveated model observers for search tasks. Approach: We evaluated foveated extensions of a channelized Hotelling and nonprewhitening matched filter model with an eye filter. The proposed methods involve calculating a model index of detectability ( d ' ) for each retinal eccentricity and combining these with a weighting function into a single detectability metric. We assessed different versions of the weighting function that varied in the required measurements of the human observers' search (no measurements, eye movement patterns, size of the image, and median search times). Results: We show that the index of detectability across eccentricities weighted using the eye movement patterns of observers best predicted human performance in 2D versus 3D search performance for a small microcalcification-like signal and a larger mass-like. The metric with a weighting function based on median search times was the second best predicting human results. Conclusions: The findings provide a set of model observer tools to evaluate image quality in the early stages of imaging system evaluation or design without implementing the more computationally complex foveated search model.

Optimizing constrained reconstruction in magnetic resonance imaging for signal detection

Constrained reconstruction in magnetic resonance imaging (MRI) allows the use of prior information through constraints to improve reconstructed images. These constraints often take the form of regularization terms in the objective function used for reconstruction. Constrained reconstruction leads to images which appear to have fewer artifacts than reconstructions without constraints but because the methods are typically nonlinear, the reconstructed images have artifacts whose structure is hard to predict. In this work, we compared different methods of optimizing the regularization parameter using a total variation (TV) constraint in the spatial domain and sparsity in the wavelet domain for one-dimensional (2.56×) undersampling using variable density undersampling. We compared the mean squared error (MSE), structural similarity (SSIM), L-curve and the area under the receiver operating characteristic (AUC) using a linear discriminant for detecting a small and a large signal. We used a signal-known-exactly task with varying backgrounds in a simulation where the anatomical variation was the major source of clutter for the detection task. Our results show that the AUC dependence on regularization parameters varies with the imaging task (i.e. the signal being detected). The choice of regularization parameters for MSE, SSIM, L-curve and AUC were similar. We also found that a model-based reconstruction including TV and wavelet sparsity did slightly better in terms of AUC than just enforcing data consistency but using these constraints resulted in much better MSE and SSIM. These results suggest that the increased performance in MSE and SSIM over-estimate the improvement in detection performance for the tasks in this paper. The MSE and SSIM metrics show a big difference in performance where the difference in AUC is small. To our knowledge, this is the first time that signal detection with varying backgrounds has been used to optimize constrained reconstruction in MRI.

Impact of iterative reconstruction algorithms on the applicability of Fourier-based detectability index for x-ray CT imaging

PURPOSE

The increasing application of iterative reconstruction algorithms in clinical computed tomography to improve image quality and reduce radiation dose, elicits strong interest, and needs model observers to optimize CT scanning protocols objectively and efficiently. The current paradigm for evaluating imaging system performance relies on Fourier methods, which presuppose a linear, wide-sense stationary system. Long-range correlations introduced by iterative reconstruction algorithms may narrow the applicability of Fourier techniques. Differences in the implementation of reconstruction algorithms between manufacturers add further complexity. The present work set out to quantify the errors entailed by the use of Fourier methods, which can lead to design decisions that do not correlate with detectability.

METHODS

To address this question, we evaluated the noise properties and the detectability index of the ideal linear observer using the spatial approach and the Fourier-based approach. For this purpose, a homogeneous phantom was imaged on two scanners: the Revolution CT (GE Healthcare) and the Somatom Definition AS+ (Siemens Healthcare) at different exposure levels. Images were reconstructed using different strength levels of IR algorithms available on the systems considered: Adaptative Statistical Iterative Reconstruction (ASIR-V) and Sinogram Affirmed Iterative Reconstruction (SAFIRE).

RESULTS

Our findings highlight that the spatial domain estimate of the detectability index is higher than the Fourier domain estimate. This trend is found to be dependent on the specific regularization used by IR algorithms as well as the signal to be detected. The eigenanalysis of the noise covariance matrix and of its circulant approximation yields explanation about the evoked trends. In particular, this analysis suggests that the predictive power of the Fourier-based ideal linear observer depends on the ability of each basis analyzed to be relevant to the signal to be detected.

CONCLUSION

The applicability of Fourier techniques is dependent on the specific regularization used by IR algorithms. These results argue for verifying the assumptions made when using Fourier methods since Fourier-task-based detectability index does not always correlate with signal detectability.

A signal detection model for quantifying overregularization in nonlinear image reconstruction

Many useful image quality metrics for evaluating linear image reconstruction techniques do not apply to or are difficult to interpret for nonlinear image reconstruction. The vast majority of metrics employed for evaluating nonlinear image reconstruction are based on some form of global image fidelity, such as image root mean square error (RMSE). Use of such metrics can lead to overregularization in the sense that they can favor removal of subtle details in the image. To address this shortcoming, we develop an image quality metric based on signal detection that serves as a surrogate to the qualitative loss of fine image details. The metric is demonstrated in the context of a breast CT simulation, where different equal-dose configurations are considered. The configurations differ in the number of projections acquired. Image reconstruction is performed with a nonlinear algorithm based on total variation constrained least-squares (TV-LSQ). The resulting images are studied as a function of three parameters: number of views acquired, total variation constraint value, and number of iterations. The images are evaluated visually, with image RMSE, and with the proposed signal-detection-based metric. The latter uses a small signal, and computes detectability in the sinogram and in the reconstructed image. Loss of signal detectability through the image reconstruction process is taken as a quantitative measure of loss of fine details in the image. Loss of signal detectability is seen to correlate well with the blocky or patchy appearance due to overregularization with TV-LSQ, and this trend runs counter to the image RMSE metric, which tends to favor the over-regularized images. The proposed signal detection-based metric provides an image quality assessment that is complimentary to that of image RMSE. Using the two metrics in concert may yield a useful prescription for determining CT algorithm and configuration parameters when nonlinear image reconstruction is used.

Foveated Model Observers for Visual Search in 3D Medical Images

Model observers have a long history of success in predicting human observer performance in clinically-relevant detection tasks. New 3D image modalities provide more signal information but vastly increase the search space to be scrutinized. Here, we compared standard linear model observers (ideal observers, non-pre-whitening matched filter with eye filter, and various versions of Channelized Hotelling models) to human performance searching in 3D 1/f2.8 filtered noise images and assessed its relationship to the more traditional location known exactly detection tasks and 2D search. We investigated two different signal types that vary in their detectability away from the point of fixation (visual periphery). We show that the influence of 3D search on human performance interacts with the signal's detectability in the visual periphery. Detection performance for signals difficult to detect in the visual periphery deteriorates greatly in 3D search but not in 3D location known exactly and 2D search. Standard model observers do not predict the interaction between 3D search and signal type. A proposed extension of the Channelized Hotelling model (foveated search model) that processes the image with reduced spatial detail away from the point of fixation, explores the image through eye movements, and scrolls across slices can successfully predict the interaction observed in humans and also the types of errors in 3D search. Together, the findings highlight the need for foveated model observers for image quality evaluation with 3D search.

Modeling human observer detection in undersampled magnetic resonance imaging (MRI)

Task-based assessment of image quality in undersampled magnetic resonance imaging (MRI) using constraints is important because of the need to quantify the effect of the artifacts on task performance. Fluid-attenuated inversion recovery (FLAIR) images are used in detection of small metastases in the brain. In this work we carry out two-alternative forced choice (2-AFC) studies with a small signal known exactly (SKE) but with varying background for reconstructed FLAIR images from undersampled multi-coil data. Using a 4x undersampling and a total variation (TV) constraint we found that the human observer detection performance remained fairly constant for a broad range of values in the regularization parameter before decreasing at large values. Using the TV constraint did not improve task performance. The non- prewhitening eye (NPWE) observer and sparse difference-of-Gaussians (S-DOG) observer with internal noise were used to model human observer detection. The parameters for the NPWE and the internal noise for the S-DOG were chosen to match the average percent correct (PC) in 2-AFC studies for three observers using no regularization. The NPWE model observer tracked the performance of the human observers as the regularization was increased but slightly over-estimated the PC for large amounts of regularization. The S-DOG model observer with internal noise tracked human performace for all levels of regularization studied. To our knowledge this is the first time that model observers have been used to track human observer detection for undersampled MRI.

Patient-based Performance Assessment for Pediatric Abdominal CT: An Automated Monitoring System Based on Lesion Detectability and Radiation Dose

RATIONALE AND OBJECTIVE

To deploy an automated tool for evaluating pediatric body computed tomography (CT) performance utilizing metrics of radiation dose and image quality for the task of liver lesion detection.

MATERIALS AND METHODS

This IRB approved retrospective investigation used 507 IV-contrast-enhanced abdominopelvic CT scans of pediatric patients (<18 years) between June 2014 and November 2017 acquired on three scanner models from two manufacturers. The scans were evaluated in terms of radiation metrics (CTDI_vol, DLP, and SSDE) as well as task-based performance based on the clinical task of detecting a 5 mm liver lesion with a 10 HU attenuation difference from background liver. An informatics algorithm extracted a previously-validated quantitative detectability index (d') from each case reflective of the likelihood of detecting a liver lesion. The results were analyzed in terms of the relationship between d' and radiation dose metrics.

RESULTS

There was minimal SSDE variability by age. Median SSDE at 100 kV on one scanner model was 5.2 mGy (5.0-5.4 mGy interquartile range). However, when assessing image quality by applying d', the age groups separated such that the younger patients had higher d' values than older patients. Similar trends were seen in all scanners.

CONCLUSIONS

An automated method to assess clinical image quality for pediatric CT provided a metric of image quality that varied as expected across ages (i.e., higher quality for younger patients). This tool affords the establishment of a quality reference level that, in addition to dose estimations currently available, would allow for enhanced assessment (e.g., facilitated audit) of CT imaging performance.

Interobserver variability in quality assessment of magnetic resonance images

Background

The perceptual quality of magnetic resonance (MR) images influences diagnosis and may compromise the treatment. The purpose of this study was to evaluate how the image quality changes influence the interobserver variability of their assessment.

Methods

For the variability evaluation, a dataset containing distorted MRI images was prepared and then assessed by 31 experienced medical professionals (radiologists). Differences between observers were analyzed using the Fleiss’ kappa. However, since the kappa evaluates the agreement among radiologists taking into account aggregated decisions, a typically employed criterion of the image quality assessment (IQA) performance was used to provide a more thorough analysis. The IQA performance of radiologists was evaluated by comparing the Spearman correlation coefficients, ρ, between individual scores with the mean opinion scores (MOS) composed of the subjective opinions of the remaining professionals.

Results

The experiments show that there is a significant agreement among radiologists (κ=0.12; 95% confidence interval [CI]: 0.118, 0.121; P<0.001) on the quality of the assessed images. The resulted κ is strongly affected by the subjectivity of the assigned scores, separately presenting close scores. Therefore, the ρ was used to identify poor performance cases and to confirm the consistency of the majority of collected scores (ρ_mean = 0.5706). The results for interns (ρ_mean = 0.6868) supports the finding that the quality assessment of MR images can be successfully taught.

Conclusions

The agreement observed among radiologists from different imaging centers confirms the subjectivity of the perception of MR images. It was shown that the image content and severity of distortions affect the IQA. Furthermore, the study highlights the importance of the psychosomatic condition of the observers and their attitude.

Joint Optimization of Collimator and Reconstruction Parameters in X-Ray Fluorescence Computed Tomography Using Analytical Point Spread Function and Model Observer

OBJECTIVE

To jointly optimize collimator design and image reconstruction algorithm in X-ray Fluorescence Computed Tomography (XFCT) for imaging low concentrations of high atomic number (Z) elements in small animal models.

METHODS

Single pinhole (SPH) collimator and three types of multi-pinhole (MPH) collimators were evaluated. MPH collimators with 5, 7, and 9 pinholes using lead, stainless steel and brass were considered. A digital cylindrical phantom (0.5 mm3 voxels) of 25 mm diameter and 25 mm height with a central 5 mm diameter and 12.5 mm height cylindrical insert containing gold nanoparticles (2:1 insert: background concentration) was modeled. A 125 kVp, 2 mm Sn filtered x-ray spectrum (0.5 cGy/projection) for gold K-shell XFCT was considered. System matrices were implemented using analytical point spread functions (PSF) for each pinhole collimator. Poisson noise was added to the projection data (16 equiangular views) before image reconstruction using Maximum-Likelihood Expectation-Maximization (ML-EM) algorithm. Signal-present and signal-absent images were generated for the detection task performed by a channelized Hotelling observer (CHO) with 10 Dense Difference-of-Gaussian channels. The area under the curve (AUC) in receiver operating characteristic was used as the image quality metric.

RESULTS

A stainless steel focusing type MPH with 7 pinholes and 20 iterations of ML-EM provided the highest AUC.

CONCLUSION

MPH collimators outperformed SPH collimators for XFCT and consistently high AUCs were observed with focusing type MPH designs with 7 pinholes.

SIGNIFICANCE

The combinations of collimator design and image reconstruction parameters that maximized AUC were identified, which could improve the performance of XFCT.

Virtual clinical trials in medical imaging: a review

The accelerating complexity and variety of medical imaging devices and methods have outpaced the ability to evaluate and optimize their design and clinical use. This is a significant and increasing challenge for both scientific investigations and clinical applications. Evaluations would ideally be done using clinical imaging trials. These experiments, however, are often not practical due to ethical limitations, expense, time requirements, or lack of ground truth. Virtual clinical trials (VCTs) (also known as in silico imaging trials or virtual imaging trials) offer an alternative means to efficiently evaluate medical imaging technologies virtually. They do so by simulating the patients, imaging systems, and interpreters. The field of VCTs has been constantly advanced over the past decades in multiple areas. We summarize the major developments and current status of the field of VCTs in medical imaging. We review the core components of a VCT: computational phantoms, simulators of different imaging modalities, and interpretation models. We also highlight some of the applications of VCTs across various imaging modalities.

Computational reader design and statistical performance evaluation of an in-silico imaging clinical trial comparing digital breast tomosynthesis with full-field digital mammography

A recent study reported on an in-silico imaging trial that evaluated the performance of digital breast tomosynthesis (DBT) as a replacement for full-field digital mammography (FFDM) for breast cancer screening. In this in-silico trial, the whole imaging chain was simulated, including the breast phantom generation, the x-ray transport process, and computational readers for image interpretation. We focus on the design and performance characteristics of the computational reader in the above-mentioned trial. Location-known lesion (spiculated mass and clustered microcalcifications) detection tasks were used to evaluate the imaging system performance. The computational readers were designed based on the mechanism of a channelized Hotelling observer (CHO), and the reader models were selected to trend human performance. Parameters were tuned to ensure stable lesion detectability. A convolutional CHO that can adapt a round channel function to irregular lesion shapes was compared with the original CHO and was found to be suitable for detecting clustered microcalcifications but was less optimal in detecting spiculated masses. A three-dimensional CHO that operated on the multiple slices was compared with a two-dimensional (2-D) CHO that operated on three versions of 2-D slabs converted from the multiple slices and was found to be optimal in detecting lesions in DBT. Multireader multicase reader output analysis was used to analyze the performance difference between FFDM and DBT for various breast and lesion types. The results showed that DBT was more beneficial in detecting masses than detecting clustered microcalcifications compared with FFDM, consistent with the finding in a clinical imaging trial. Statistical uncertainty smaller than 0.01 standard error for the estimated performance differences was achieved with a dataset containing approximately 3000 breast phantoms. The computational reader design methodology presented provides evidence that model observers can be useful in-silico tools for supporting the performance comparison of breast imaging systems.

CT metal artifact reduction algorithms: Toward a framework for objective performance assessment

Purpose

Although several metal artifact reduction (MAR) algorithms for computed tomography (CT) scanning are commercially available, no quantitative, rigorous, and reproducible method exists for assessing their performance. The lack of assessment methods poses a challenge to regulators, consumers, and industry. We explored a phantom‐based framework for assessing an important aspect of MAR performance: how applying MAR in the presence of metal affects model observer performance at a low‐contrast detectability (LCD) task This work is, to our knowledge, the first model observer–based framework for the evaluation of MAR algorithms in the published literature.

Methods

We designed a numerical head phantom with metal implants. In order to incorporate an element of randomness, the phantom included a rotatable inset with an inhomogeneous background. We generated simulated projection data for the phantom. We applied two variants of a simple MAR algorithm, sinogram inpainting, to the projection data, that we reconstructed using filtered backprojection. To assess how MAR affected observer performance, we examined the detectability of a signal at the center of a region of interest (ROI) by a channelized Hotelling observer (CHO). As a figure of merit, we used the area under the ROC curve (AUC).

Results

We used simulation to test our framework on two variants of the MAR technique of sinogram inpainting. We found that our method was able to resolve the difference in two different MAR algorithms’ effect on LCD task performance, as well as the difference in task performances when MAR was applied, vs not.

Conclusion

We laid out a phantom‐based framework for objective assessment of how MAR impacts low‐contrast detectability, that we tested on two MAR algorithms. Our results demonstrate the importance of testing MAR performance over a range of object and imaging parameters, since applying MAR does not always improve the quality of an image for a given diagnostic task. Our framework is an initial step toward developing a more comprehensive objective assessment method for MAR, which would require developing additional phantoms and methods specific to various clinical applications of MAR, and increasing study efficiency.

RDI[Formula: see text]a regression detectability index for quality assurance in: x-ray imaging

Novel iterative image reconstruction methods can help reduce the required radiation dose in x-ray diagnostics such as computed tomography (CT), while maintaining sufficient image quality. Since some of the established image quality measures are not appropriate for reliably judging the quality of images derived by iterative methods, alternative approaches such as task-specific quality assessment would be highly desirable for acceptance or constancy testing. Task-based image quality methods are also closer to tasks performed by the radiologists, such as lesion detection. However, this approach is usually hampered by a huge workload, since hundreds of images are usually required for its application. It is demonstrated that the proposed approach works reliably on the basis of significantly fewer images, and that it correlates well with results obtained from human observers.

Human observer performance on in-plane digital breast tomosynthesis images: Effects of reconstruction filters and data acquisition angles on signal detection

For digital breast tomosynthesis (DBT) systems, we investigate the effects of the reconstruction filters for different data acquisition angles on signal detection. We simulated a breast phantom with a 30% volume glandular fraction (VGF) of breast anatomy using the power law spectrum and modeled the breast mass as a spherical object with a 1 mm diameter. Projection data were acquired using two different data acquisition angles and numbers of projection view pairs, and in-plane breast images were reconstructed using the Feldkamp-Davis-Kress (FDK) algorithm with three different reconstruction filter schemes. To measure the ability to detect a signal, we conducted the human observer study with a binary detection task and compared the signal detectability of human to that of channelized Hotelling observer (CHO) with Laguerre-Gauss (LG) channels and dense difference-of-Gaussian (D-DOG) channels. We also measured the contrast-to-noise ratio (CNR), signal power spectrum (SPS), and β values of the anatomical noise power spectrum (NPS) to show the association between human observer performance and these traditional metrics. Our results show that using a slice thickness (ST) filter degraded the signal detection performance of human observers at the same data acquisition angle. This could be predicted by D-DOG CHO with internal noise, but the correlation between the traditional metrics and signal detectability was not observed in this work.

Radiation dose reduction and static image quality assessment using a channelized hotelling observer on an angiography system upgraded with clarity IQ

The goal of this paper was the comparison of radiation dose and imaging quality before and after the Clarity IQ technology installation in a Philips AlluraXper FD20/20 angiography system using a Channelized Hotelling Observer model (CHO). The core characteristics of the Allura Clarity IQ technology are its real-time noise reduction algorithms (NRT) combined with state-of-the-art hardware; this technology allows to implement acquisition protocols able to significantly reduce patient entrance dose. To measure the system performances in terms of image quality we used a contrast detail phantom in a clinical scatter condition. A Leeds TO10 phantom has been imaged between two 10 cm thick homogeneous solid water slabs. Fluoroscopy images were acquired using a cerebral protocol at 3 dose levels (low, medium and high) with a field- of view (FOV) of 31 cm. Cineangiography images were acquired using a cerebral protocol at 2 fps. Thus, 4 acquisitions were obtained for the conventional technology and 4 acquisitions were taken after the Clarity IQ upgrade, for a total of 8 different image sets. A validated 40 Gabor channels CHO with an internal noise model compared the image sets. Human observers' studies were carried out to tune the internal noise parameter. We showed that the CHO did not detect any significant difference between any of the image sets acquired using the two technologies. Consequently, this x-ray imaging technology provides a non-inferior image quality with an average patient dose reduction of 57% and 28% respectively in cineangiography and fluoroscopy. The Clarity IQ installation has certainly allowed a considerable improvement in patient and staff safety, while maintaining the same image quality.

Human observer templates for lesion discrimination tasks

We investigate a series of two-alternative forced-choice (2AFC) discrimination tasks based on malignant features of abnormalities in low-dose lung CT scans. A total of 3 tasks are evaluated, and these consist of a size-discrimination task, a boundary-sharpness task, and an irregular-interior task. Target and alternative signal profiles for these tasks are modulated by one of two system transfer functions and embedded in ramp-spectrum noise that has been apodized for noise control in one of 4 different ways. This gives the resulting images statistical properties that are related to weak ground-glass lesions in axial slices of low-dose lung CT images. We investigate observer performance in these tasks using a combination of statistical efficiency and classification images. We report results of 24 2AFC experiments involving the three tasks. A staircase procedure is used to find the approximate 80% correct discrimination threshold in each task, with a subsequent set of 2,000 trials at this threshold. These data are used to estimate statistical efficiency with respect to the ideal observer for each task, and to estimate the observer template using the classification-image methodology. We find efficiency varies between the different tasks with lowest efficiency in the boundary-sharpness task, and highest efficiency in the non-uniform interior task. All three tasks produce clearly visible patterns of positive and negative weighting in the classification images. The spatial frequency plots of classification images show how apodization results in larger weights at higher spatial frequencies.

Numerical observer study of lesion detectability for a long axial field-of-view whole-body PET imager using the PennPET Explorer

This work uses lesion detectability to characterize the performance of long axial field of view (AFOV) PET scanners which have increased sensitivity compared to clinical scanners. Studies were performed using the PennPET Explorer, a 70 cm long AFOV scanner built at the University of Pennsylvania, for small lesions distributed in a uniform water-filled cylinder (simulations and measurements), an anthropomorphic torso phantom (measurement), and a human subject (measurement). The lesion localization and detection task was quantified numerically using a generalized scan statistics methodology. Detectability was studied as a function of background activity distribution, scan duration for a single bed position, and axial location of the lesions. For the cylindrical phantom, the areas under the localization receiver operating curve (ALROCs) of lesions placed at various axial locations in the scanner were greater than 0.8-a value considered to be clinically acceptable (i.e. 80% probability of detecting lesion)-for scan times of 60 s or longer for standard-of-care (SoC) clinical dose levels. 10 mm diameter lesions placed in the anthropomorphic phantom and human subject resulted in ALROCs of 0.8 or greater for scan times longer than 30 s in the lung region and 60 s in the liver region, also for SoC doses. ALROC results from all three activity distributions show similar trends as a function of counts detected per axial location. These results will be used to guide decisions on imaging parameters, such as scan time and patient dose, when imaging patients in a single bed position on long AFOV systems and can also be applied to clinical scanners with consideration of the sensitivity differences.

A performance comparison of convolutional neural network-based image denoising methods: The effect of loss functions on low-dose CT images

PURPOSE

Convolutional neural network (CNN)-based image denoising techniques have shown promising results in low-dose CT denoising. However, CNN often introduces blurring in denoised images when trained with a widely used pixel-level loss function. Perceptual loss and adversarial loss have been proposed recently to further improve the image denoising performance. In this paper, we investigate the effect of different loss functions on image denoising performance using task-based image quality assessment methods for various signals and dose levels.

METHODS

We used a modified version of U-net that was effective at reducing the correlated noise in CT images. The loss functions used for comparison were two pixel-level losses (i.e., the mean-squared error and the mean absolute error), Visual Geometry Group network-based perceptual loss (VGG loss), adversarial loss used to train the Wasserstein generative adversarial network with gradient penalty (WGAN-GP), and their weighted summation. Each image denoising method was applied to reconstructed images and sinogram images independently and validated using the extended cardiac-torso (XCAT) simulation and Mayo Clinic datasets. In the XCAT simulation, we generated fan-beam CT datasets with four different dose levels (25%, 50%, 75%, and 100% of a normal-dose level) using 10 XCAT phantoms and inserted signals in a test set. The signals had two different shapes (spherical and spiculated), sizes (4 and 12 mm), and contrast levels (60 and 160 HU). To evaluate signal detectability, we used a detection task SNR (tSNR) calculated from a non-prewhitening model observer with an eye filter. We also measured the noise power spectrum (NPS) and modulation transfer function (MTF) to compare the noise and signal transfer properties.

RESULTS

Compared to CNNs without VGG loss, VGG-loss-based CNNs achieved a more similar tSNR to that of the normal-dose CT for all signals at different dose levels except for a small signal at the 25% dose level. For a low-contrast signal at 25% or 50% dose, adding other losses to the VGG loss showed more improved performance than only using VGG loss. The NPS shapes from VGG-loss-based CNN closely matched that of normal-dose CT images while CNN without VGG loss overly reduced the mid-high-frequency noise power at all dose levels. MTF also showed VGG-loss-based CNN with better-preserved high resolution for all dose and contrast levels. It is also observed that additional WGAN-GP loss helps improve the noise and signal transfer properties of VGG-loss-based CNN.

CONCLUSIONS

The evaluation results using tSNR, NPS, and MTF indicate that VGG-loss-based CNNs are more effective than those without VGG loss for natural denoising of low-dose images and WGAN-GP loss improves the denoising performance of VGG-loss-based CNNs, which corresponds with the qualitative evaluation.

Optimization of the difference-of-Gaussian channel sets for the channelized Hotelling observer

The channelized-Hotelling observer (CHO) was investigated as a surrogate of human observers in task-based image quality assessment. The CHO with difference-of-Gaussian (DoG) channels has shown potential for the prediction of human detection performance in digital mammography (DM) images. However, the DoG channels employ parameters that describe the shape of each channel. The selection of these parameters influences the performance of the DoG CHO and needs further investigation. The detection performance of the DoG CHO was calculated and correlated with the detection performance of three humans who evaluated DM images in 2-alternative forced-choice experiments. A set of DM images of an anthropomorphic breast phantom with and without calcification-like signals was acquired at four different dose levels. For each dose level, 200 square regions-of-interest (ROIs) with and without signal were extracted. Signal detectability was assessed on ROI basis using the CHO with various DoG channel parameters and it was compared to that of the human observers. It was found that varying these DoG parameter values affects the correlation (r 2 ) of the CHO with human observers for the detection task investigated. In conclusion, it appears that the the optimal DoG channel sets that maximize the prediction ability of the CHO might be dependent on the type of background and signal of ROIs investigated.

A deep learning- and partial least square regression-based model observer for a low-contrast lesion detection task in CT

PURPOSE

This work aims to develop a new framework of image quality assessment using deep learning-based model observer (DL-MO) and to validate it in a low-contrast lesion detection task that involves CT images with patient anatomical background.

METHODS

The DL-MO was developed using the transfer learning strategy to incorporate a pretrained deep convolutional neural network (CNN), a partial least square regression discriminant analysis (PLS-DA) model and an internal noise component. The CNN was previously trained to achieve the state-of-the-art classification accuracy over a natural image database. The earlier layers of the CNN were used as a deep feature extractor, with the assumption that similarity exists between the CNN and the human visual system. The PLSR model was used to further engineer the deep feature for the lesion detection task in CT images. The internal noise component was incorporated to model the inefficiency and variability of human observer (HO) performance, and to generate the ultimate DL-MO test statistics. Seven abdominal CT exams were retrospectively collected from the same type of CT scanners. To compare DL-MO with HO, 12 experimental conditions with varying lesion size, lesion contrast, radiation dose, and reconstruction types were generated, each condition with 154 trials. CT images of a real liver metastatic lesion were numerically modified to generate lesion models with four lesion sizes (5, 7, 9, and 11 mm) and three contrast levels (15, 20, and 25 HU). The lesions were inserted into patient liver images using a projection-based method. A validated noise insertion tool was used to synthesize CT exams with 50% and 25% of routine radiation dose level. CT images were reconstructed using the weighted filtered back projection algorithm and an iterative reconstruction algorithm. Four medical physicists performed a two-alternative forced choice (2AFC) detection task (with multislice scrolling viewing mode) on patient images across the 12 experimental conditions. DL-MO was operated on the same datasets. Statistical analyses were performed to evaluate the correlation and agreement between DL-MO and HO.

RESULTS

A statistically significant positive correlation was observed between DL-MO and HO for the 2AFC low-contrast detection task that involves patient liver background. The corresponding Pearson product moment correlation coefficient was 0.986 [95% confidence interval (0.950, 0.996)]. Bland-Altman agreement analysis did not indicate statistically significant differences.

CONCLUSIONS

The proposed DL-MO is highly correlated with HO in a low-contrast detection task that involves realistic patient liver background. This study demonstrated the potential of the proposed DL-MO to assess image quality directly based on patient images in realistic, clinically relevant CT tasks.