Use of computer-aided detection (CAD) tools in screening mammography: a multidisciplinary investigation

Improving diagnostic decision support through deliberate reflection: a proposal

Digital decision support (DDS) is expected to play an important role in improving a physician's diagnostic performance and reducing the burden of diagnostic error. Studies with currently available DDS systems indicate that they lead to modest gains in diagnostic accuracy, and these systems are expected to evolve to become more effective and user-friendly in the future. In this position paper, we propose that a way towards this future is to rethink DDS systems based on deliberate reflection, a strategy by which physicians systematically review the clinical findings observed in a patient in the light of an initial diagnosis. Deliberate reflection has been demonstrated to improve diagnostic accuracy in several contexts. In this paper, we first describe the deliberate reflection strategy, including the crucial element that would make it useful in the interaction with a DDS system. We examine the nature of conventional DDS systems and their shortcomings. Finally, we propose what DDS based on deliberate reflection might look like, and consider why it would overcome downsides of conventional DDS.

Hierarchical Cluster Analysis to Aid Diagnostic Image Data Visualization of MS and Other Medical Imaging Modalities

Perceiving abnormal regions in the images of different medical modalities plays a crucial role in diagnosis and subsequent treatment planning. In medical images to visually perceive abnormalities' extent and boundaries requires substantial experience. Consequently, manually drawn region of interest (ROI) to outline boundaries of abnormalities suffers from limitations of human perception leading to inter-observer variability. As an alternative to human drawn ROI, it is proposed the use of a computer-based segmentation algorithm to segment digital medical image data.Hierarchical Clustering-based Segmentation (HCS) process is a generic unsupervised segmentation process that can be used to segment dissimilar regions in digital images. HCS process generates a hierarchy of segmented images by partitioning an image into its constituent regions at hierarchical levels of allowable dissimilarity between its different regions. The hierarchy represents the continuous merging of similar, spatially adjacent, and/or disjoint regions as the allowable threshold value of dissimilarity between regions, for merging, is gradually increased.This chapter discusses in detail first the implementation of the HCS process, second the implementation details of how the HCS process is used for the presentation of multi-modal imaging data (MALDI and MRI) of a biological sample, third the implementation details of how the process is used as a perception aid for X-ray mammogram readers, and finally the implementation details of how it is used as an interpretation aid for the interpretation of Multi-parametric Magnetic Resonance Imaging (mpMRI) of the Prostate.

Analog Computer-Aided Detection (CAD) information can be more effective than binary marks

In socially important visual search tasks, such as baggage screening and diagnostic radiology, experts miss more targets than is desirable. Computer-aided detection (CAD) programs have been developed specifically to improve performance in these professional search tasks. For example, in breast cancer screening, many CAD systems are capable of detecting approximately 90% of breast cancer, with approximately 0.5 false-positive detections per image. Nevertheless, benefits of CAD in clinical settings tend to be small (Birdwell, 2009) or even absent (Meziane et al., 2011; Philpotts, 2009). The marks made by a CAD system can be "binary," giving the same signal to any location where the signal is above some threshold. Alternatively, a CAD system presents an analog signal that reflects strength of the signal at a location. In the experiments reported, we compare analog and binary CAD presentations using nonexpert observers and artificial stimuli defined by two noisy signals: a visible color signal and an "invisible" signal that informed our simulated CAD system. We found that analog CAD generally yielded better overall performance than binary CAD. The analog benefit is similar at high and low target prevalence. Our data suggest that the form of the CAD signal can directly influence performance. Analog CAD may allow the computer to be more helpful to the searcher.

How to discriminate between computer-aided and computer-hindered decisions: a case study in mammography

BACKGROUND

Computer aids can affect decisions in complex ways, potentially even making them worse; common assessment methods may miss these effects. We developed a method for estimating the quality of decisions, as well as how computer aids affect it, and applied it to computer-aided detection (CAD) of cancer, reanalyzing data from a published study where 50 professionals ("readers") interpreted 180 mammograms, both with and without computer support.

METHOD

We used stepwise regression to estimate how CAD affected the probability of a reader making a correct screening decision on a patient with cancer (sensitivity), thereby taking into account the effects of the difficulty of the cancer (proportion of readers who missed it) and the reader's discriminating ability (Youden's determinant). Using regression estimates, we obtained thresholds for classifying a posteriori the cases (by difficulty) and the readers (by discriminating ability).

RESULTS

Use of CAD was associated with a 0.016 increase in sensitivity (95% confidence interval [CI], 0.003-0.028) for the 44 least discriminating radiologists for 45 relatively easy, mostly CAD-detected cancers. However, for the 6 most discriminating radiologists, with CAD, sensitivity decreased by 0.145 (95% CI, 0.034-0.257) for the 15 relatively difficult cancers.

CONCLUSIONS

Our exploratory analysis method reveals unexpected effects. It indicates that, despite the original study detecting no significant average effect, CAD helped the less discriminating readers but hindered the more discriminating readers. Such differential effects, although subtle, may be clinically significant and important for improving both computer algorithms and protocols for their use. They should be assessed when evaluating CAD and similar warning systems.

Effectiveness of computer-aided detection in community mammography practice

BACKGROUND

Computer-aided detection (CAD) is applied during screening mammography for millions of US women annually, although it is uncertain whether CAD improves breast cancer detection when used by community radiologists.

METHODS

We investigated the association between CAD use during film-screen screening mammography and specificity, sensitivity, positive predictive value, cancer detection rates, and prognostic characteristics of breast cancers (stage, size, and node involvement). Records from 684 956 women who received more than 1.6 million film-screen mammograms at Breast Cancer Surveillance Consortium facilities in seven states in the United States from 1998 to 2006 were analyzed. We used random-effects logistic regression to estimate associations between CAD and specificity (true-negative examinations among women without breast cancer), sensitivity (true-positive examinations among women with breast cancer diagnosed within 1 year of mammography), and positive predictive value (breast cancer diagnosed after positive mammograms) while adjusting for mammography registry, patient age, time since previous mammography, breast density, use of hormone replacement therapy, and year of examination (1998-2002 vs 2003-2006). All statistical tests were two-sided.

RESULTS

Of 90 total facilities, 25 (27.8%) adopted CAD and used it for an average of 27.5 study months. In adjusted analyses, CAD use was associated with statistically significantly lower specificity (OR = 0.87, 95% confidence interval [CI] = 0.85 to 0.89, P < .001) and positive predictive value (OR = 0.89, 95% CI = 0.80 to 0.99, P = .03). A non-statistically significant increase in overall sensitivity with CAD (OR = 1.06, 95% CI = 0.84 to 1.33, P = .62) was attributed to increased sensitivity for ductal carcinoma in situ (OR = 1.55, 95% CI = 0.83 to 2.91; P = .17), although sensitivity for invasive cancer was similar with or without CAD (OR = 0.96, 95% CI = 0.75 to 1.24; P = .77). CAD was not associated with higher breast cancer detection rates or more favorable stage, size, or lymph node status of invasive breast cancer.

CONCLUSION

CAD use during film-screen screening mammography in the United States is associated with decreased specificity but not with improvement in the detection rate or prognostic characteristics of invasive breast cancer.

Studying technology use as social practice: the untapped potential of ethnography

Information and communications technologies (ICTs) in healthcare are often introduced with expectations of higher-quality, more efficient, and safer care. Many fail to meet these expectations. We argue here that the well-documented failures of ICTs in healthcare are partly attributable to the philosophical foundations of much health informatics research. Positivistic assumptions underpinning the design, implementation and evaluation of ICTs (in particular the notion that technology X has an impact which can be measured and reproduced in new settings), and the deterministic experimental and quasi-experimental study designs which follow from these assumptions, have inherent limitations when ICTs are part of complex social practices involving multiple human actors. We suggest that while experimental and quasi-experimental studies have an important place in health informatics research overall, ethnography is the preferred methodological approach for studying ICTs introduced into complex social systems. But for ethnographic approaches to be accepted and used to their full potential, many in the health informatics community will need to revisit their philosophical assumptions about what counts as research rigor.

Exploring the potential of context-sensitive CADe in screening mammography

PURPOSE

Conventional computer-assisted detection (CADe) systems in screening mammography provide the same decision support to all users. The aim of this study was to investigate the potential of a context-sensitive CADe system which provides decision support guided by each user's focus of attention during visual search and reporting patterns for a specific case.

METHODS

An observer study for the detection of malignant masses in screening mammograms was conducted in which six radiologists evaluated 20 mammograms while wearing an eye-tracking device. Eye-position data and diagnostic decisions were collected for each radiologist and case they reviewed. These cases were subsequently analyzed with an in-house knowledge-based CADe system using two different modes: Conventional mode with a globally fixed decision threshold and context-sensitive mode with a location-variable decision threshold based on the radiologists' eye dwelling data and reporting information.

RESULTS

The CADe system operating in conventional mode had 85.7% per-image malignant mass sensitivity at 3.15 false positives per image (FPsI). The same system operating in context-sensitive mode provided personalized decision support at 85.7%-100% sensitivity and 0.35-0.40 FPsI to all six radiologists. Furthermore, context-sensitive CADe system could improve the radiologists' sensitivity and reduce their performance gap more effectively than conventional CADe.

CONCLUSIONS

Context-sensitive CADe support shows promise in delineating and reducing the radiologists' perceptual and cognitive errors in the diagnostic interpretation of screening mammograms more effectively than conventional CADe.

Using computer-aided detection in mammography as a decision support

Objective:

To evaluate an interactive computer-aided detection (CAD) system for reading mammograms to improve decision making.

Methods:

A dedicated mammographic workstation has been developed in which readers can probe image locations for the presence of CAD information. If present, CAD findings are displayed with the computed malignancy rating. A reader study was conducted in which four screening radiologists and five non-radiologists participated to study the effect of this system on detection performance. The participants read 120 cases of which 40 cases had a malignant mass that was missed at the original screening. The readers read each mammogram both with and without CAD in separate sessions. Each reader reported localized findings and assigned a malignancy score per finding. Mean sensitivity was computed in an interval of false-positive fractions less than 10%.

Results:

Mean sensitivity was 25.1% in the sessions without CAD and 34.8% in the CAD-assisted sessions. The increase in detection performance was significant (p = 0.012). Average reading time was 84.7 ± 61.5 s/case in the unaided sessions and was not significantly higher when interactive CAD was used (85.9 ± 57.8 s/case).

Conclusion:

Interactive use of CAD in mammography may be more effective than traditional CAD for improving mass detection without affecting reading time.

Variable size computer-aided detection prompts and mammography film reader decisions

Introduction

The purpose of the present study was to investigate the effect of computer-aided detection (CAD) prompts on reader behaviour in a large sample of breast screening mammograms by analysing the relationship of the presence and size of prompts to the recall decision.

Methods

Local research ethics committee approval was obtained; informed consent was not required. Mammograms were obtained from women attending routine mammography at two breast screening centres in 1996. Films, previously double read, were re-read by a different reader using CAD. The study material included 315 cancer cases comprising all screen-detected cancer cases, all subsequent interval cancers and 861 normal cases randomly selected from 10,267 cases. Ground truth data were used to assess the efficacy of CAD prompting. Associations between prompt attributes and tumour features or reader recall decisions were assessed by chi-squared tests.

Results

There was a highly significant relationship between prompting and a decision to recall for cancer cases and for a random sample of normal cases (P < 0.001). Sixty-four per cent of all cases contained at least one CAD prompt. In cancer cases, larger prompts were more likely to be recalled (P = 0.02) for masses but there was no such association for calcifications (P = 0.9). In a random sample of 861 normal cases, larger prompts were more likely to be recalled (P = 0.02) for both mass and calcification prompts. Significant associations were observed with prompting and breast density (p = 0.009) for cancer cases but not for normal cases (P = 0.05).

Conclusions

For both normal cases and cancer cases, prompted mammograms were more likely to be recalled and the prompt size was also associated with a recall decision.

Improving mammographic interpretation: double reading and computer-aided diagnosis

This article discusses two commonly used techniques advocated to improve screening mammography performance: double reading (DR) and computer-aided detection (CAD). Analysis of these methods is incomplete because no randomized controlled trials have been performed to assess changes in survival. Although DR and CAD have shown improvement in sensitivity, specificity often has decreased. Balancing which parameter is more important involves health care policy, costs, cultural factors, legal risk, and patient preference.

Computer-assisted detection for CT colonography: external validation

AIM

To externally validate a computer-assisted detection (CAD) system for computed tomography (CT) colonography, using data from a single centre uninvolved with the software development.

MATERIALS AND METHODS

Twenty-five multi-detector CT colonography examinations of patients with validated polyps accumulated at a single centre were examined by two readers who used endoscopic and histopathological data to identify polyp coordinates. A CAD system that had been developed using data from elsewhere, and had not previously encountered the present data, was then applied to the data at sphericity filter settings of 0.75 and 0.50 and identified potential polyps. True-positive, false-negative, and false-positive counts were determined by comparison with the known polyp coordinates.

RESULTS

Twenty-five patients had 57 polyps, median size 6mm (range 1-15mm). Per-patient sensitivity for the CAD system was 96% (24 of 25). The CAD system detected 44 (77%) polyps at sphericity setting 0.75 and 49 (86%) polyps at sphericity 0.50: the additional five polyps detected all measured 5mm or less. Sphericity of 0.75 resulted in a median of 10 (one to 34) easily dismissed false-positive prompts per patient and a median of 4 (zero to 15) that needed three-dimensional rendering before dismissal. This rose to 32 (16 to 99) and 11 (three to 35), respectively, at sphericity 0.5.

CONCLUSIONS

A per-patient sensitivity of 96% was found for the CAD system (in patients with a median polyp diameter of 6mm) using external validation, a more stringent test than either internal cross-validation or temporal validation. Decreasing sphericity increases sensitivity for small polyps at the expense of decreased specificity.