Toward image quality assessment in mammography using model observers: Detection of a calcification-like object

Modeling Radiologists' Assessments to Explore Pairing Strategies for Optimized Double Reading of Screening Mammograms

PURPOSE

To develop a model that simulates radiologist assessments and use it to explore whether pairing readers based on their individual performance characteristics could optimize screening performance.

METHODS

Logistic regression models were designed and used to model individual radiologist assessments. For model evaluation, model-predicted individual performance metrics and paired disagreement rates were compared against the observed data using Pearson correlation coefficients. The logistic regression models were subsequently used to simulate different screening programs with reader pairing based on individual true-positive rates (TPR) and/or false-positive rates (FPR). For this, retrospective results from breast cancer screening programs employing double reading in Sweden, England, and Norway were used. Outcomes of random pairing were compared against those composed of readers with similar and opposite TPRs/FPRs, with positive assessments defined by either reader flagging an examination as abnormal.

RESULTS

The analysis data sets consisted of 936,621 (Sweden), 435,281 (England), and 1,820,053 (Norway) examinations. There was good agreement between the model-predicted and observed radiologists' TPR and FPR (r ≥ 0.969). Model-predicted negative-case disagreement rates showed high correlations (r ≥ 0.709), whereas positive-case disagreement rates had lower correlation levels due to sparse data (r ≥ 0.532). Pairing radiologists with similar FPR characteristics (Sweden: 4.50% [95% confidence interval: 4.46%-4.54%], England: 5.51% [5.47%-5.56%], Norway: 8.03% [7.99%-8.07%]) resulted in significantly lower FPR than with random pairing (Sweden: 4.74% [4.70%-4.78%], England: 5.76% [5.71%-5.80%], Norway: 8.30% [8.26%-8.34%]), reducing examinations sent to consensus/arbitration while the TPR did not change significantly. Other pairing strategies resulted in equal or worse performance than random pairing.

CONCLUSIONS

Logistic regression models accurately predicted screening mammography assessments and helped explore different radiologist pairing strategies. Pairing readers with similar modeled FPR characteristics reduced the number of examinations unnecessarily sent to consensus/arbitration without significantly compromising the TPR.

HIGHLIGHTS

A logistic-regression model can be derived that accurately predicts individual and paired reader performance during mammography screening reading.Pairing screening mammography radiologists with similar false-positive characteristics reduced false-positive rates with no significant loss in true positives and may reduce the number of examinations unnecessarily sent to consensus/arbitration.

Performance evaluation of digital breast tomosynthesis systems: physical methods and experimental data

Digital breast tomosynthesis (DBT) has become a well-established breast imaging technique, whose performance has been investigated in many clinical studies, including a number of prospective clinical trials. Results from these studies generally point to non-inferiority in terms of microcalcification detection and superior mass-lesion detection for DBT imaging compared to digital mammography (DM). This modality has become an essential tool in the clinic for assessment and ad-hoc screening but is not yet implemented in most breast screening programmes at a state or national level. While evidence on the clinical utility of DBT has been accumulating, there has also been progress in the development of methods for technical performance assessment and quality control of these imaging systems. DBT is a relatively complicated ‘pseudo-3D’ modality whose technical assessment poses a number of difficulties. This paper reviews methods for the technical performance assessment of DBT devices, starting at the component level in part one and leading up to discussion of system evaluation with physical test objects in part two. We provide some historical and basic theoretical perspective, often starting from methods developed for DM imaging. Data from a multi-vendor comparison are also included, acquired under the medical physics quality control protocol developed by EUREF and currently being consolidated by a European Federation of Organisations for Medical Physics working group. These data and associated methods can serve as a reference for the development of reference data and provide some context for clinical studies.

The regression detectability index RDI for mammography images of breast phantoms with calcification-like objects and anatomical background

Currently, quality assurance measurements in mammography are performed on unprocessed images. For diagnosis, however, radiologists are provided with processed images. This image processing is optimised for images of human anatomy and therefore does not always perform satisfactorily with technical phantoms. To overcome this problem, it may be possible to use anthropomorphic phantoms reflecting the anatomic structure of the human breast in place of technical phantoms when carrying out task-specific quality assessment using model observers. However, the use of model observers is hampered by the fact that a large number of images needs to be acquired. A recently published novel observer called the regression detectability index (RDI) needs significantly fewer images, but requires the background of the images to be flat. Therefore, to be able to apply the RDI to images of anthropomorphic phantoms, the anatomic background needs to be removed. For this, a procedure in which the anatomical structures are fitted by thin plate spline (TPS) interpolation has been developed. When the object to be detected is small, such as a calcification-like lesion, it is shown that the anatomic background can be removed successfully by subtracting the TPS interpolation, which makes the background-free image accessible to the RDI. We have compared the detectability obtained by the RDI with TPS background subtraction to results of the channelized Hotelling observer (CHO) and human observers. With the RDI, results for the detectabilityd'can be obtained using 75% fewer images compared to the CHO, while the same uncertainty ofd'is achieved. Furthermore, the correlation ofd'(RDI) with the results of human observers is at least as good as that ofd'(CHO) with human observers.

Image texture, low contrast liver lesion detectability and impact on dose: Deep learning algorithm compared to partial model-based iterative reconstruction

OBJECTIVES

To compare deep learning (True Fidelity, TF) and partial model based Iterative Reconstruction (ASiR-V) algorithm for image texture, low contrast lesion detectability and potential dose reduction.

METHODS

Anthropomorphic phantoms (mimicking non-overweight and overweight patient), containing lesions of 6 mm in diameter with 20HU contrast, were scanned at five different dose levels (2,6,10,15,20 mGy) on a CT system, using clinical routine protocols for liver lesion detection. Images were reconstructed using ASiR-V 0% (surrogate for FBP), 60 % and TF at low, medium and high strength. Noise texture was characterized by computing a normalized Noise Power Spectrum filtered by an eye filter. The similarity against FBP texture was evaluated using peak frequency difference (PFD) and root mean square deviation (RMSD). Low contrast detectability was assessed using a channelized Hotelling observer and the area under the ROC curve (AUC) was used as figure of merit. Potential dose reduction was calculated to obtain the same AUC for TF and ASiR-V.

RESULTS

FBP-like noise texture was more preserved with TF (PFD from -0.043mm-1 to -0.09mm-1, RMSD from 0.12mm-1 to 0.21mm-1) than with ASiR-V (PFD equal to 0.12 mm-1, RMSD equal to 0.53mm-1), resulting in a sharper image. AUC was always higher with TF than ASIR-V. In average, TF compared to ASiR-V, enabled a radiation dose reduction potential of 7%, 25 % and 33 % for low, medium and high strength respectively.

CONCLUSION

Compared to ASIR-V, TF at high strength does not impact noise texture and maintains low contrast liver lesions detectability at significant lower dose.

RDI[Formula: see text]a regression detectability index for quality assurance in: x-ray imaging

Novel iterative image reconstruction methods can help reduce the required radiation dose in x-ray diagnostics such as computed tomography (CT), while maintaining sufficient image quality. Since some of the established image quality measures are not appropriate for reliably judging the quality of images derived by iterative methods, alternative approaches such as task-specific quality assessment would be highly desirable for acceptance or constancy testing. Task-based image quality methods are also closer to tasks performed by the radiologists, such as lesion detection. However, this approach is usually hampered by a huge workload, since hundreds of images are usually required for its application. It is demonstrated that the proposed approach works reliably on the basis of significantly fewer images, and that it correlates well with results obtained from human observers.

Breast phantoms for 2D digital mammography with realistic anatomical structures and attenuation characteristics based on clinical images using 3D printing

The aim of this work was to develop a production process for breast phantoms for 2D digital mammography (DM) with realistic anatomical structures and attenuation characteristics based on clinical images using 3D printing. The presented production process is based on PolyJet 3D printing technology using a polypropylene like printing material. First, an attenuation calibration function for this material and the achievable lateral resolution of the printing process of about 200 µm was determined. Subsequently, to generate the digital 3D model of the breast phantom, the pixel intensities of the unprocessed clinical image that are related to the attenuation along the z-axis of the breast, were converted to corresponding phantom heights using the calibration function. To validate the process, an image of the 3D printed breast phantom was acquired on the full field digital mammography (FFDM) system used for calibration and compared with the clinical image in terms of anatomical structures and associated attenuation characteristics. The exposure parameters and image impression of the phantom were evaluated using five other FFDM systems of different manufacturers and types. Results demonstrated that the anatomical structures in the images and the attenuation characteristics of a female breast and the derived phantom agreed on the FFDM system used for calibration. The automatic exposure control segmentation, the automatically selected exposure parameters and the image postprocessing of the clinical and phantom image indicated a high level of conformity. As shown, the phantom is also suitable for other FFDM systems. In conclusion, an approach to produce anthropomorphic breast phantoms for DM offering realistic anatomical structures and attenuation characteristics based on clinical images was successfully developed. As shown, the phantom realistically simulated the original female breast. Therefore, it is expected that such phantoms are promising to support bridging the gap between physical-technical and diagnostic image quality assessment. In addition, they enable a variety of practical and scientific applications for which present technical phantoms are not suitable.

Determination of contrast-detail curves in mammography image quality assessment by a parametric model observer

A novel approach is proposed for the determination of contrast-detail curves in mammography image quality assessment. The approach is compared with current practice using virtual mammography. A binary parametric model observer is applied to images of the CDMAM phantom. The observer accounts for the simple disc shaped objects in the phantom and is applied separately to each cell of the phantom. For each of these applications, the area under the ROC curve (AUC) of the model observer is determined. The different AUCs, calculated from different applications of the parametric model observer, are then combined to a single contrast-detail curve quantifying the ability of the observer to detect details in the images. Virtual mammography is developed as a tool to simulate X-ray images of single CDMAM cells and to quantitatively assess the approach in comparison with current practice. It is shown that the proposed approach can lead to similar contrast-detail curves as current practice. The precision of the estimated contrast-detail curves is increased, i.e. using 5 images yields about the same precision for the proposed approach as 16 images when applying current practice. We conclude that contrast-detail curves in mammography image quality assessment can also be determined through the AUC of a binary parametric model observer. Since the proposed approach has higher precision than current practice, it is a promising candidate for contrast-detail analysis in mammography image quality assessment.

Can a channelized Hotelling observer assess image quality in acquired mammographic images of an anthropomorphic breast phantom including image processing?

PURPOSE

To study the feasibility of a channelized Hotelling observer (CHO) to predict human observer performance in detecting calcification-like signals in mammography images of an anthropomorphic breast phantom, as part of a quality control (QC) framework.

METHODS

A prototype anthropomorphic breast phantom with inserted gold disks of 0.25 mm diameter was imaged with two different digital mammography x-ray systems at four different dose levels. Regions of interest (ROIs) were extracted from the acquired processed and unprocessed images, signal-present and signal-absent. The ROIs were evaluated by a CHO using four different formulations of the difference of Gaussian (DoG) channel sets. Three human observers scored the ROIs in a two-alternative forced-choice experiment. We compared the human and the CHO performance on the simple task to detect calcification-like disks in ROIs with and without postprocessing. The proportion of correct responses of the human reader (PC_H ) and the CHO (PC_CHO ) was calculated and the correlation between the two was analyzed using a mixed-effect regression model. To address the signal location uncertainty, the impact of shifting the DoG channel sets in all directions up to two pixels was evaluated. Correlation results including the goodness of fit (r² ) of PC_H and PC_CHO for all different parameters were evaluated.

RESULTS

Subanalysis by system yielded strong correlations between PC_H and PC_CHO , with r² between PC_H and PC_CHO was found to be between 0.926 and 0.958 for the unshifted and between 0.759 and 0.938 for the shifted channel sets, respectively. However, the linear fit suggested a slight system dependence. PC_CHO with shifted channel sets increased CHO performance but the correlation with humans was decreased. These correlations were not considerably affected by of the DoG channel set used.

CONCLUSIONS

There is potential for the CHO to be used in QC for the evaluation of detectability of calcification-like signals. The CHO can predict the PC of humans in images of calcification-like signals of two different systems. However, a global model to be used for all systems requires further investigation.

Task-Based Model Observer Assessment of A Partial Model-Based Iterative Reconstruction Algorithm in Thoracic Oncologic Multidetector CT

To investigate the impact of a partial model-based iterative reconstruction (ASiR-V) on image quality in thoracic oncologic multidetector computed tomography (MDCT), using human and mathematical model observers. Twenty cancer patients examined with regular-dose thoracic-abdominal-pelvic MDCT were retrospectively included. Thoracic images reconstructed using a sharp kernel and filtered back-projection (reference) or ASiR-V (0-100%, 20% increments; follow-up) were analysed by three thoracic radiologists. Advanced quantitative physical metrics, including detectability indexes of simulated 4-mm-diameter solid non-calcified nodules and ground-glass opacities, were computed at regular and reduced doses using a custom-designed phantom. All three radiologists preferred higher ASiR-V levels (best = 80%). Increasing ASiR-V substantially decreased noise magnitude, with slight changes in noise texture. For high-contrast objects, changing the ASiR-V level had no major effect on spatial resolution; whereas for lower-contrast objects, increasing ASiR-V substantially decreased spatial resolution, more markedly at reduced dose. For both high- and lower-contrast pulmonary lesions, detectability remained excellent, regardless of ASiR-V and dose levels, and increased significantly with increasing ASiR-V levels (all p < 0.001). While high ASiR-V levels (80%) are recommended to detect solid non-calcified nodules and ground-glass opacities in regular-dose thoracic oncologic MDCT, care must be taken because, for lower-contrast pulmonary lesions, high ASiR-V levels slightly change noise texture and substantially decrease spatial resolution, more markedly at reduced dose.

A simple parametric model observer for quality assurance in computer tomography

Model observers are mathematical classifiers that are used for the quality assessment of imaging systems such as computer tomography. The quality of the imaging system is quantified by means of the performance of a selected model observer. For binary classification tasks, the performance of the model observer is defined by the area under its ROC curve (AUC). Typically, the AUC is estimated by applying the model observer to a large set of training and test data. However, the recording of these large data sets is not always practical for routine quality assurance. In this paper we propose as an alternative a parametric model observer that is based on a simple phantom, and we provide a Bayesian estimation of its AUC. It is shown that a limited number of repeatedly recorded images (10-15) is already sufficient to obtain results suitable for the quality assessment of an imaging system. A MATLAB® function is provided for the calculation of the results. The performance of the proposed model observer is compared to that of the established channelized Hotelling observer and the nonprewhitening matched filter for simulated images as well as for images obtained from a low-contrast phantom on an x-ray tomography scanner. The results suggest that the proposed parametric model observer, along with its Bayesian treatment, can provide an efficient, practical alternative for the quality assessment of CT imaging systems.