Comparison of Channel Methods and Observer Models for the Task-Based Assessment of Multi-Projection Imaging in the Presence of Structured Anatomical Noise

Comparison of supervised-learning approaches for designing a channelized observer for image quality assessment in CT

BACKGROUND

The current paradigm for evaluating computed tomography (CT) system performance relies on a task-based approach. As the Hotelling observer (HO) provides an upper bound of observer performances in specific signal detection tasks, the literature advocates HO use for optimization purposes. However, computing the HO requires calculating the inverse of the image covariance matrix, which is often intractable in medical applications. As an alternative, dimensionality reduction has been extensively investigated to extract the task-relevant features from the raw images. This can be achieved by using channels, which yields the channelized-HO (CHO). The channels are only considered efficient when the channelized observer (CO) can approximate its unconstrained counterpart. Previous work has demonstrated that supervised learning-based methods can usually benefit CO design, either for generating efficient channels using partial least squares (PLS) or for replacing the Hotelling detector with machine-learning (ML) methods.

PURPOSE

Here we investigated the efficiency of a supervised ML-algorithm used to design a CO for predicting the performance of unconstrained HO. The ML-algorithm was applied either (1) in the estimator for dimensionality reduction, or (2) in the detector function.

METHODS

A channelized support vector machine (CSVM) was employed and compared against the CHO in terms of ability to predict HO performances. Both the CSVM and the CHO were estimated with channels derived from the singular value decomposition (SVD) of the system operator, principal component analysis (PCA), and PLS. The huge variety of regularization strategies proposed by CT system vendors for statistical image reconstruction (SIR) make the generalization capability of an observer a key point to consider upfront of implementation in clinical practice. To evaluate the generalization properties of the observers, we adopted a 2-step testing process: (1) achieved with the same regularization strategy (as in the training phase) and (2) performed using different reconstruction properties. We generated simulated- signal-known-exactly/background-known-exactly (SKE/BKE) tasks in which different noise structures were generated using Markov random field (MRF) regularizations using either a Green or a quadratic, function.

RESULTS

The CSVM outperformed the CHO for all types of channels and regularization strategies. Furthermore, even though both COs generalized well to images reconstructed with the same regularization strategy as the images considered in the training phase, the CHO failed to generalize to images reconstructed differently whereas the CSVM managed to successfully generalize. Lastly, the proposed CSVM observer used with PCA channels outperformed the CHO with PLS channels while using a smaller training data set.

CONCLUSION

These results argue for introducing the supervised-learning paradigm in the detector function rather than in the operator of the channels when designing a CO to provide an accurate estimate of HO performance. The CSVM with PCA channels proposed here could be used as a surrogate for HO in image quality assessment.

Computer Simulations of Case Difficulty in Digital Breast Tomosynthesis Using Virtual Clinical Trials

PURPOSE

Virtual clinical trials (VCTs) require computer simulations of representative patients and images to evaluate and compare changes in performance of imaging technologies. The simulated images are usually interpreted by model observers whose performance depends upon the selection of imaging cases used in training evaluation models. This work proposes an efficient method to simulate and calibrate soft tissue lesions, which matches the detectability threshold of virtual and human readings.

METHODS

Anthropomorphic breast phantoms were used to evaluate the simulation of four mass models (I-IV) that vary in shape and composition of soft tissue. Ellipsoidal (I) and spiculated (II-IV) masses were simulated using composite voxels with partial volumes. Digital breast tomosynthesis projections and reconstructions of a clinical system were simulated. Channelized Hotelling observers (CHOs) were evaluated using reconstructed slices of masses that varied in shape, composition, and density of surrounded tissue. The detectability threshold of each mass model was evaluated using receiver operating characteristic (ROC) curves calculated with the CHO's scores.

RESULTS

The area under the curve (AUC) of each calibrated mass model were within the 95% confidence interval (mean AUC [95% CI]) reported in a previous reader study (0.93 [0.89, 0.97]). The mean AUC [95% CI] obtained were 0.94 [0.93, 0.96], 0.92 [0.90, 0.93], 0.92 [0.90, 0.94], 0.93 [0.92, 0.95] for models I to IV, respectively. The mean AUC results varied substantially as a function of shape, composition, and density of surrounded tissue.

CONCLUSIONS

For successful VCTs, lesions composed of soft tissue should be calibrated to simulate imaging cases that match the case difficulty predicted by human readers. Lesion composition, shape, and size are parameters that should be carefully selected to calibrate VCTs. This article is protected by copyright. All rights reserved.

Virtual clinical trials in medical imaging: a review

The accelerating complexity and variety of medical imaging devices and methods have outpaced the ability to evaluate and optimize their design and clinical use. This is a significant and increasing challenge for both scientific investigations and clinical applications. Evaluations would ideally be done using clinical imaging trials. These experiments, however, are often not practical due to ethical limitations, expense, time requirements, or lack of ground truth. Virtual clinical trials (VCTs) (also known as in silico imaging trials or virtual imaging trials) offer an alternative means to efficiently evaluate medical imaging technologies virtually. They do so by simulating the patients, imaging systems, and interpreters. The field of VCTs has been constantly advanced over the past decades in multiple areas. We summarize the major developments and current status of the field of VCTs in medical imaging. We review the core components of a VCT: computational phantoms, simulators of different imaging modalities, and interpretation models. We also highlight some of the applications of VCTs across various imaging modalities.

Optimization of the difference-of-Gaussian channel sets for the channelized Hotelling observer

The channelized-Hotelling observer (CHO) was investigated as a surrogate of human observers in task-based image quality assessment. The CHO with difference-of-Gaussian (DoG) channels has shown potential for the prediction of human detection performance in digital mammography (DM) images. However, the DoG channels employ parameters that describe the shape of each channel. The selection of these parameters influences the performance of the DoG CHO and needs further investigation. The detection performance of the DoG CHO was calculated and correlated with the detection performance of three humans who evaluated DM images in 2-alternative forced-choice experiments. A set of DM images of an anthropomorphic breast phantom with and without calcification-like signals was acquired at four different dose levels. For each dose level, 200 square regions-of-interest (ROIs) with and without signal were extracted. Signal detectability was assessed on ROI basis using the CHO with various DoG channel parameters and it was compared to that of the human observers. It was found that varying these DoG parameter values affects the correlation (r 2 ) of the CHO with human observers for the detection task investigated. In conclusion, it appears that the the optimal DoG channel sets that maximize the prediction ability of the CHO might be dependent on the type of background and signal of ROIs investigated.

Systematic approach to a channelized Hotelling model observer implementation for a physical phantom containing mass-like lesions: Application to digital breast tomosynthesis

PURPOSE

to develop a channelized model observer (CHO) that matches human reader (HR) scoring of a physical phantom containing breast simulating structure and mass lesion-like targets for use in quality control of digital breast tomosynthesis (DBT) imaging systems.

METHODS

A total of 108 DBT scans of the phantom was acquired using a Siemens Inspiration DBT system. The detectability of mass-like targets was evaluated by human readers using a 4-alternative forced choice (4-AFC) method. The percentage correct (PC) values were then used as the benchmark for CHO tuning, again using a 4-AFC method. Three different channel functions were considered: Gabor, Laguerre-Gauss and Difference of Gaussian. With regard to the observer template, various methods for generating the expected signal were studied along with the influence of the number of training images used to form the covariance matrix for the observer template. Impact of bias in the training process on the observer template was evaluated next, as well as HR and CHO reproducibility.

RESULTS

HR performance was most closely matched by 8 Gabor channels with tuned phase, orientation and frequency, using an observer template generated from training image data. Just 24 DBT image stacks were required to give robust CHO performance with 0% bias, although a bias of up to 33% in the training images also gave acceptable performance. CHO and HR reproducibility were similar (on average 3.2 PC versus 3.4 PC).

CONCLUSIONS

The CHO algorithm developed matches human reader performance and is therefore a promising candidate for automated readout of phantom studies.

Virtual assessment of stereoscopic viewing of digital breast tomosynthesis projection images

Digital breast tomosynthesis (DBT) acquires a series of projection images from different angles as an x-ray source rotates around the breast. Such imaging geometry lends DBT naturally to stereoscopic viewing as two projection images with a reasonable separation angle can easily form a stereo pair. This simulation study assessed the efficacy of stereo viewing of DBT projection images. Three-dimensional computational breast phantoms with realistically shaped synthetic lesions were scanned by three simulated DBT systems. The projection images were combined into a sequence of stereo pairs and presented to a stereomatching-based model observer for deciding lesion presence. Signal-to-noise ratio was estimated, and the detection performance with stack viewing of reconstructed slices was the benchmark. We have shown that: (1) stereo viewing of projection images may underperform stack viewing of reconstructed slices for current DBT geometries; (2) DBT geometries may impact the efficacy of the two viewing modes differently: narrow-arc and wide-arc geometries may be better for stereo viewing and stack viewing, respectively; (3) the efficacy of stereo viewing may be more robust than stack viewing to reductions in dose. While in principle stereo viewing is potentially effective for visualizing DBT data, effective stereo viewing may require specifically optimized DBT image acquisition.

Synthetic breast phantoms from patient based eigenbreasts

PURPOSE

The limited number of 3D patient-based breast phantoms available could be augmented by synthetic breast phantoms in order to facilitate virtual clinical trials (VCTs) using model observers for breast imaging optimization and evaluation.

METHODS

These synthetic breast phantoms were developed using Principal Component Analysis (PCA) to reduce the number of dimensions needed to describe a training set of images. PCA decomposed a training set of M breast CT volumes (with millions of voxels each) into an M-1-dimensional space of eigenvectors, which we call eigenbreasts. Each of the training breast phantoms was compactly represented by the mean image plus a weighted sum of eigenbreasts. The distribution of weights observed from training was then sampled to create new synthesized breast phantoms.

RESULTS

The resulting synthesized breast phantoms demonstrated a high degree of realism, as supported by an observer study. Two out of three experienced physicist observers were unable to distinguish between the synthesized breast phantoms and the patient-based phantoms. The fibroglandular density and noise power law exponent of the synthesized breast phantoms agreed well with the training data.

CONCLUSIONS

Our method extends our series of digital breast phantoms based on breast CT data, providing the capability to generate new, statistically varying ensembles consisting of tens of thousands of virtual subjects. This work represents an important step toward conducting future virtual trials for task-based assessment of breast imaging, where it is vital to have a large ensemble of realistic phantoms for statistical power as well as clinical relevance.

Task-Based Modeling of a 5k Ultra-High-Resolution Medical Imaging System for Digital Breast Tomosynthesis

High-resolution, low-noise X-ray detectors based on CMOS active pixel sensor (APS) technology have demonstrated superior imaging performance for digital breast tomosynthesis (DBT). This paper presents a task-based model for a high-resolution medical imaging system to evaluate its ability to detect simulated microcalcifications and masses as lesions for breast cancer. A 3-D cascaded system analysis for a 50- [Formula: see text] pixel pitch CMOS APS X-ray detector was integrated with an object task function, a medical imaging display model, and the human eye contrast sensitivity function to calculate the detectability index and area under the ROC curve (AUC). It was demonstrated that the display pixel pitch and zoom factor should be optimized to improve the AUC for detecting small microcalcifications. In addition, detector electronic noise of smaller than 300 e^- and a high display maximum luminance (>1000 cd/cm ²) are desirable to distinguish microcalcifications of [Formula: see text] in size. For low contrast mass detection, a medical imaging display with a minimum of 12-bit gray levels is recommended to realize accurate luminance levels. A wide projection angle range of greater than ±30° in combination with the image gray level magnification could improve the mass detectability especially when the anatomical background noise is high. On the other hand, a narrower projection angle range below ±20° can improve the small, high contrast object detection. Due to the low mass contrast and luminance, the ambient luminance should be controlled below 5 cd/ [Formula: see text]. Task-based modeling provides important firsthand imaging performance of the high-resolution CMOS-based medical imaging system that is still at early stage development for DBT. The modeling results could guide the prototype design and clinical studies in the future.

Lack of agreement between radiologists: implications for image-based model observers

We tested the agreement of radiologists' rankings of different reconstructions of breast computed tomography images based on their diagnostic (classification) performance and on their subjective image quality assessments. We used 102 pathology proven cases (62 malignant, 40 benign), and an iterative image reconstruction (IIR) algorithm to obtain 24 reconstructions per case with different image appearances. Using image feature analysis, we selected 3 IIRs and 1 clinical reconstruction and 50 lesions. The reconstructions produced a range of image quality from smooth/low-noise to sharp/high-noise, which had a range in classifier performance corresponding to AUCs of 0.62 to 0.96. Six experienced Mammography Quality Standards Act (MQSA) radiologists rated the likelihood of malignancy for each lesion. We conducted an additional reader study with the same radiologists and a subset of 30 lesions. Radiologists ranked each reconstruction according to their preference. There was disagreement among the six radiologists on which reconstruction produced images with the highest diagnostic content, but they preferred the midsharp/noise image appearance over the others. However, the reconstruction they preferred most did not match with their performance. Due to these disagreements, it may be difficult to develop a single image-based model observer that is representative of a population of radiologists for this particular imaging task.

Can the channelized Hotelling observer including aspects of the human visual system predict human observer performance in mammography?

PURPOSE

In mammography, images are processed prior to display. Model observers (MO) are candidates to objectively evaluate processed images if they can predict human observer performance for detail detection. The aim of this study was to investigate if the channelized Hotelling observer (CHO) can be configured to predict human observer performance in mammography like images.

METHODS

The performance correlation between human observers and CHO has been evaluated using different channel-sets and by including aspects of the human visual system (HVS). The correlation was investigated for the detection of disk-shaped details in simulated white noise (WN) and clustered lumpy backgrounds (CLB) images, representing respectively quantum noise limited and mammography like images. The images were scored by the MO and five human observers in 2-alternative forced choice experiments.

RESULTS

For WN images the most useful formulation of the CHO to predict human observer performance was obtained using three difference of Gaussian channels without adding HVS aspects (RLR2=0.62). For CLB images the most useful formulation was the partial least square channel-set without adding HVS aspects (RLR2=0.71). The correlation was affected by detail size and background.

CONCLUSIONS

This study has shown that the CHO can predict human observer performance. Due to object size and background dependency it is important that the range of object sizes and allowed variability in background are specified and validated carefully before the CHO can be implemented for objective image quality assessment.