Computer-aided diagnosis for the detection and classification of lung cancers on chest radiographs ROC analysis of radiologists' performance

Effect of a comprehensive deep-learning model on the accuracy of chest x-ray interpretation by radiologists: a retrospective, multireader multicase study

BACKGROUND

Chest x-rays are widely used in clinical practice; however, interpretation can be hindered by human error and a lack of experienced thoracic radiologists. Deep learning has the potential to improve the accuracy of chest x-ray interpretation. We therefore aimed to assess the accuracy of radiologists with and without the assistance of a deep-learning model.

METHODS

In this retrospective study, a deep-learning model was trained on 821 681 images (284 649 patients) from five data sets from Australia, Europe, and the USA. 2568 enriched chest x-ray cases from adult patients (≥16 years) who had at least one frontal chest x-ray were included in the test dataset; cases were representative of inpatient, outpatient, and emergency settings. 20 radiologists reviewed cases with and without the assistance of the deep-learning model with a 3-month washout period. We assessed the change in accuracy of chest x-ray interpretation across 127 clinical findings when the deep-learning model was used as a decision support by calculating area under the receiver operating characteristic curve (AUC) for each radiologist with and without the deep-learning model. We also compared AUCs for the model alone with those of unassisted radiologists. If the lower bound of the adjusted 95% CI of the difference in AUC between the model and the unassisted radiologists was more than -0·05, the model was considered to be non-inferior for that finding. If the lower bound exceeded 0, the model was considered to be superior.

FINDINGS

Unassisted radiologists had a macroaveraged AUC of 0·713 (95% CI 0·645-0·785) across the 127 clinical findings, compared with 0·808 (0·763-0·839) when assisted by the model. The deep-learning model statistically significantly improved the classification accuracy of radiologists for 102 (80%) of 127 clinical findings, was statistically non-inferior for 19 (15%) findings, and no findings showed a decrease in accuracy when radiologists used the deep-learning model. Unassisted radiologists had a macroaveraged mean AUC of 0·713 (0·645-0·785) across all findings, compared with 0·957 (0·954-0·959) for the model alone. Model classification alone was significantly more accurate than unassisted radiologists for 117 (94%) of 124 clinical findings predicted by the model and was non-inferior to unassisted radiologists for all other clinical findings.

INTERPRETATION

This study shows the potential of a comprehensive deep-learning model to improve chest x-ray interpretation across a large breadth of clinical practice.

FUNDING

Annalise.ai.

Clinical Implementation of Deep Learning in Thoracic Radiology: Potential Applications and Challenges

Chest X-ray radiography and computed tomography, the two mainstay modalities in thoracic radiology, are under active investigation with deep learning technology, which has shown promising performance in various tasks, including detection, classification, segmentation, and image synthesis, outperforming conventional methods and suggesting its potential for clinical implementation. However, the implementation of deep learning in daily clinical practice is in its infancy and facing several challenges, such as its limited ability to explain the output results, uncertain benefits regarding patient outcomes, and incomplete integration in daily workflow. In this review article, we will introduce the potential clinical applications of deep learning technology in thoracic radiology and discuss several challenges for its implementation in daily clinical practice.

Feature selection using ant colony optimization with tandem-run recruitment to diagnose bronchitis from CT scan images

BACKGROUND AND OBJECTIVES

Computer-aided diagnosis (CAD) plays a vital role in the routine clinical activity for the detection of lung disorders using computed tomography (CT) images. It serves as a source of second opinion that radiologists may consider in order to interpret CT images. In this work, the purpose of CAD is to improve the diagnostic accuracy of pulmonary bronchitis from CT images of the lung.

METHODS

Left and right lung fields are segmented using optimal thresholding from the lung CT images. Texture and shape features are extracted from the pathology bearing regions. A hybrid feature selection approach based on ant colony optimization (ACO) combining cosine similarity and support vector machine (SVM) classifier is used to select relevant features. Additionally, tandem run recruitment strategy is included in the selection activity to choose the promising features. The SVM classifier is trained using the selected features and the performance of the trained classifier is evaluated using trivial performance evaluation measures.

RESULTS

The training and testing datasets used in building the classifier model are disjoint and contains 200 CT slices affected with bronchitis, 50 normal slices and 300 slices with cancer. Out of 100 features extracted from each CT slice, a subset of 60 features is used for classification. ACO with tandem run strategy yielded 81.66% of accuracy whereas ACO without tandem run yielded an accuracy of 77.52%. When all the features are used for classifier training without feature selection algorithm, an accuracy of 75.14% is achieved.

CONCLUSION

From the results, it is inferred that identifying relevant features to train the classifier has a definite impact on the classifier performance.

After Detection: The Improved Accuracy of Lung Cancer Assessment Using Radiologic Computer-aided Diagnosis

RATIONALE AND OBJECTIVES

The aim of this study was to evaluate the improved accuracy of radiologic assessment of lung cancer afforded by computer-aided diagnosis (CADx).

MATERIALS AND METHODS

Inclusion/exclusion criteria were formulated, and a systematic inquiry of research databases was conducted. Following title and abstract review, an in-depth review of 149 surviving articles was performed with accepted articles undergoing a Quality Assessment of Diagnostic Accuracy Studies (QUADAS)-based quality review and data abstraction.

RESULTS

A total of 14 articles, representing 1868 scans, passed the review. Increases in the receiver operating characteristic (ROC) area under the curve of .8 or higher were seen in all nine studies that reported it, except for one that employed subspecialized radiologists.

CONCLUSIONS

This systematic review demonstrated improved accuracy of lung cancer assessment using CADx over manual review, in eight high-quality observer-performance studies. The improved accuracy afforded by radiologic lung-CADx suggests the need to explore its use in screening and regular clinical workflow.

Potential clinical impact of advanced imaging and computer-aided diagnosis in chest radiology: importance of radiologist's role and successful observer study

This review paper is based on our research experience in the past 30 years. The importance of radiologists' role is discussed in the development or evaluation of new medical images and of computer-aided detection (CAD) schemes in chest radiology. The four main topics include (1) introducing what diseases can be included in a research database for different imaging techniques or CAD systems and what imaging database can be built by radiologists, (2) understanding how radiologists' subjective judgment can be combined with technical objective features to improve CAD performance, (3) sharing our experience in the design of successful observer performance studies, and (4) finally, discussing whether the new images and CAD systems can improve radiologists' diagnostic ability in chest radiology. In conclusion, advanced imaging techniques and detection/classification of CAD systems have a potential clinical impact on improvement of radiologists' diagnostic ability, for both the detection and the differential diagnosis of various lung diseases, in chest radiology.

[Detection of lung nodules. New opportunities in chest radiography]

BACKGROUND

Chest radiography still represents the most commonly performed X-ray examination because it is readily available, requires low radiation doses and is relatively inexpensive. However, as previously published, many initially undetected lung nodules are retrospectively visible in chest radiographs.

STANDARD RADIOLOGICAL METHODS

The great improvements in detector technology with the increasing dose efficiency and improved contrast resolution provide a better image quality and reduced dose needs.

METHODICAL INNOVATIONS

The dual energy acquisition technique and advanced image processing methods (e.g. digital bone subtraction and temporal subtraction) reduce the anatomical background noise by reduction of overlapping structures in chest radiography. Computer-aided detection (CAD) schemes increase the awareness of radiologists for suspicious areas.

RESULTS

The advanced image processing methods show clear improvements for the detection of pulmonary lung nodules in chest radiography and strengthen the role of this method in comparison to 3D acquisition techniques, such as computed tomography (CT).

ASSESSMENT

Many of these methods will probably be integrated into standard clinical treatment in the near future. Digital software solutions offer advantages as they can be easily incorporated into radiology departments and are often more affordable as compared to hardware solutions.

Improving the radiologist-CAD interaction: designing for appropriate trust

Computer-aided diagnosis (CAD) has great potential to improve radiologists' diagnostic performance. However, the reported performance of the radiologist-CAD team is lower than what might be expected based on the performance of the radiologist and the CAD system in isolation. This indicates that the interaction between radiologists and the CAD system is not optimal. An important factor in the interaction between humans and automated aids (such as CAD) is trust. Suboptimal performance of the human-automation team is often caused by an inappropriate level of trust in the automation. In this review, we examine the role of trust in the radiologist-CAD interaction and suggest ways to improve the output of the CAD system so that it allows radiologists to calibrate their trust in the CAD system more effectively. Observer studies of the CAD systems show that radiologists often have an inappropriate level of trust in the CAD system. They sometimes under-trust CAD, thereby reducing its potential benefits, and sometimes over-trust it, leading to diagnostic errors they would not have made without CAD. Based on the literature on trust in human-automation interaction and the results of CAD observer studies, we have identified four ways to improve the output of CAD so that it allows radiologists to form a more appropriate level of trust in CAD. Designing CAD systems for appropriate trust is important and can improve the performance of the radiologist-CAD team. Future CAD research and development should acknowledge the importance of the radiologist-CAD interaction, and specifically the role of trust therein, in order to create the perfect artificial partner for the radiologist. This review focuses on the role of trust in the radiologist-CAD interaction. The aim of the review is to encourage CAD developers to design for appropriate trust and thereby improve the performance of the radiologist-CAD team.

Automatic tuberculosis screening using chest radiographs

Tuberculosis is a major health threat in many regions of the world. Opportunistic infections in immunocompromised HIV/AIDS patients and multi-drug-resistant bacterial strains have exacerbated the problem, while diagnosing tuberculosis still remains a challenge. When left undiagnosed and thus untreated, mortality rates of patients with tuberculosis are high. Standard diagnostics still rely on methods developed in the last century. They are slow and often unreliable. In an effort to reduce the burden of the disease, this paper presents our automated approach for detecting tuberculosis in conventional posteroanterior chest radiographs. We first extract the lung region using a graph cut segmentation method. For this lung region, we compute a set of texture and shape features, which enable the X-rays to be classified as normal or abnormal using a binary classifier. We measure the performance of our system on two datasets: a set collected by the tuberculosis control program of our local county's health department in the United States, and a set collected by Shenzhen Hospital, China. The proposed computer-aided diagnostic system for TB screening, which is ready for field deployment, achieves a performance that approaches the performance of human experts. We achieve an area under the ROC curve (AUC) of 87% (78.3% accuracy) for the first set, and an AUC of 90% (84% accuracy) for the second set. For the first set, we compare our system performance with the performance of radiologists. When trying not to miss any positive cases, radiologists achieve an accuracy of about 82% on this set, and their false positive rate is about half of our system's rate.

Potential of computer-aided diagnosis to improve CT lung cancer screening

The development of low-dose spiral computed tomography (CT) has rekindled hope that effective lung cancer screening might yet be found. Screening is justified when there is evidence that it will extend lives at reasonable cost and acceptable levels of risk. A screening test should detect all extant cancers while avoiding unnecessary workups. Thus optimal screening modalities have both high sensitivity and specificity. Due to the present state of technology, radiologists must opt to increase sensitivity and rely on follow-up diagnostic procedures to rule out the incurred false positives. There is evidence in published reports that computer-aided diagnosis technology may help radiologists alter the benefit-cost calculus of CT sensitivity and specificity in lung cancer screening protocols. This review will provide insight into the current discussion of the effectiveness of lung cancer screening and assesses the potential of state-of-the-art computer-aided design developments.

A computerized scheme for lung nodule detection in multiprojection chest radiography

PURPOSE

Our previous study indicated that multiprojection chest radiography could significantly improve radiologists' performance for lung nodule detection in clinical practice. In this study, the authors further verify that multiprojection chest radiography can greatly improve the performance of a computer-aided diagnostic (CAD) scheme.

METHODS

Our database consisted of 59 subjects, including 43 subjects with 45 nodules and 16 subjects without nodules. The 45 nodules included 7 real and 38 simulated ones. The authors developed a conventional CAD scheme and a new fusion CAD scheme to detect lung nodules. The conventional CAD scheme consisted of four steps for (1) identification of initial nodule candidates inside lungs, (2) nodule candidate segmentation based on dynamic programming, (3) extraction of 33 features from nodule candidates, and (4) false positive reduction using a piecewise linear classifier. The conventional CAD scheme processed each of the three projection images of a subject independently and discarded the correlation information between the three images. The fusion CAD scheme included the four steps in the conventional CAD scheme and two additional steps for (5) registration of all candidates in the three images of a subject, and (6) integration of correlation information between the registered candidates in the three images. The integration step retained all candidates detected at least twice in the three images of a subject and removed those detected only once in the three images as false positives. A leave-one-subject-out testing method was used for evaluation of the performance levels of the two CAD schemes.

RESULTS

At the sensitivities of 70%, 65%, and 60%, our conventional CAD scheme reported 14.7, 11.3, and 8.6 false positives per image, respectively, whereas our fusion CAD scheme reported 3.9, 1.9, and 1.2 false positives per image, and 5.5, 2.8, and 1.7 false positives per patient, respectively. The low performance of the conventional CAD scheme may be attributed to the high noise level in chest radiography, and the small size and low contrast of most nodules.

CONCLUSIONS

This study indicated that the fusion of correlation information in multiprojection chest radiography can markedly improve the performance of CAD scheme for lung nodule detection.

Computer-aided detection of small pulmonary nodules in chest radiographs: an observer study

RATIONALE AND OBJECTIVES

To evaluate the impact of computer-aided detection (CAD, IQQA-Chest; EDDA Technology, Princeton Junction, NJ) used as second reader on the detection of small pulmonary nodules in chest radiography (CXR).

MATERIALS AND METHODS

A total of 113 patients (mean age 62 years) with CT and CXR within 6 weeks were selected. Fifty-nine patients showed 101 pulmonary nodules (diameter 5-15mm); the remaining 54 patients served as negative controls. Six readers of varying experience individually evaluated the CXR without and with CAD as second reader in two separate reading sessions. The sensitivity per lesion, figure of merit (FOM), and mean false positive per image (mFP) were calculated. Institutional review board approval was waived.

RESULTS

With CAD, the sensitivity increased for inexperienced readers (39% vs. 45%, P < .05) and remained unchanged for experienced readers (50% vs. 51%). The mFP nonsignificantly increased for both inexperienced and experienced readers (0.27 vs. 0.34 and 0.16 vs. 0.21). The mean FOM did not significantly differ for readings without and with CAD irrespective of reader experience (0.71 vs. 0.71 and 0.84 vs. 0.87). All readers together dismissed 33% of true-positive CAD candidates. False-positive candidates by CAD provoked 40% of all false-positive marks made by the readers.

CONCLUSION

CAD improves the sensitivity of inexperienced readers for the detection of small nodules at the expense of loss of specificity. Overall performance by means of FOM was therefore not affected. To use CAD more beneficial, readers need to improve their ability to differentiate true from false-positive CAD candidates.

Predicting readers' diagnostic accuracy with a new CAD algorithm

RATIONALE AND OBJECTIVES

Before computer-aided detection (CAD) algorithms can be used in clinical practice, they must be shown to improve readers' diagnostic accuracy over their unaided performance. This is usually accomplished through a large multireader, multicase (MRMC) clinical trial. It is burdensome, however, for an MRMC study to be performed with each new release of a CAD algorithm. The aim of this report is to present an approach for building models to predict readers' accuracy with a new CAD algorithm.

MATERIALS AND METHODS

A modeling approach for predicting readers' results with a new CAD algorithm is described. Multiple-variable logistic regression was used to build models for readers' sensitivity and false-positive rate, given the results of an MRMC study with an older CAD algorithm and the stand-alone performance results of a new CAD algorithm. Data from a large lung MRMC CAD trial are used to illustrate the modeling approach and test the ability of the models to predict readers' accuracy with the new CAD algorithm.

RESULTS

The model overestimated the readers' actual sensitivity with the new CAD algorithm, but this did not reach statistical significance (0.621 vs 0.603, P = .147). The observed and predicted false-positive rates also did not differ significantly (0.275 vs 0.285, P = .250).

CONCLUSIONS

Using one clinical study as a test case, it is shown that the modeling approach is feasible. More testing of the approach is needed to determine if and under what circumstances it can be used as an alternative to a full-scale MRMC study. Meanwhile, the approach can be used to determine if a new CAD algorithm is likely to improve readers' accuracy before embarking on a full-scale MRMC study.

A comparison of follow-up recommendations by chest radiologists, general radiologists, and pulmonologists using computer-aided detection to assess radiographs for actionable pulmonary nodules

OBJECTIVE

The primary objective of our study was to compare the effect of a chest radiography computer-aided detection (CAD) system on the follow-up recommendations of chest radiologists, general radiologists, and pulmonologists.

MATERIALS AND METHODS

A chest radiography CAD system (RapidScreen 1.1) that has been approved by the U.S. Food and Drug Administration (FDA) and a second-generation version of the system (OnGuard 3.0) not yet approved by the FDA were applied to single frontal radiographs of 200 patients at high risk for lung cancer. One hundred patients had actionable nodules (mean size, 16.9 mm) and 100 patients did not. Six chest radiologists, six general radiologists, and six pulmonologists independently interpreted each image first without CAD and then with CAD during blinded reading sessions. The frequency with which readers correctly referred patients for follow-up tests was measured. Differential effects based on nodule size, shape, location, density, and subtlety were tested with multiplevariable logistic regression.

RESULTS

For patients without actionable lesions, pulmonologists showed an increase in their recommendations for follow-up from 0.46 unaided to 0.52 with CAD (p = 0.001), whereas chest and general radiologists had much lower average rates and were not affected by CAD's false marks (0.26 without CAD vs 0.25 with RapidScreen 1.1 and 0.26 with OnGuard 3.0, p ≥ 0.734). CAD improved all readers' detection of moderately subtle lesions (p = 0.013) but did not significantly increase follow-up rates overall for patients with actionable nodules (0.63 unaided vs 0.63 with RapidScreen 1.1, p = 0.795; and 0.63 unaided vs 0.64 with OnGuard 3.0, p = 0.187).

CONCLUSION

The effect of CAD on readers' clinical decisions varies depending on the training of the reader. CAD did not improve the performance of chest or general radiologists. Nonradiologists are particularly vulnerable to CAD's false-positive marks.

Neural network ensemble-based computer-aided diagnosis for differentiation of lung nodules on CT images: clinical evaluation

RATIONALE AND OBJECTIVES

To evaluate the diagnostic performance of a neural network ensemble-based computer-aided diagnosis (CAD) scheme for classifying lung nodules on thin-section computed tomography (CT).

MATERIALS AND METHODS

Thirty-two CT images that depicted 19 malignant nodules and 13 benign nodules were used. One of three possible classifications (probably benign, uncertain, and probably malignant) for each nodule was determined by using a neural network ensemble-based CAD scheme. The images were presented to three senior radiologists (each with more than 10 years of thoracic radiology experience) who were asked to determine the classification for each nodule blindly. The radiologists made their diagnostic decisions solely based on images and excluded any external data. The performance of the CAD scheme and of the radiologists was evaluated with receiver operating characteristic (ROC) analysis and agreement analysis.

RESULTS

Areas under the ROC curve (Az values) for the CAD scheme and the radiologist group were 0.79 and 0.82, respectively, and the partial areas under the ROC curves at a range of sensitivity values greater than or equal to 90% were 0.051 and 0.020 (P = .203), respectively. The weighted Kappa coefficients between the CAD scheme and each radiologist were 0.657, 0.431, and 0.606, respectively. For the diagnosis of the 11 small nodules (with diameters not greater than 10 mm), areas under the ROC curves of the CAD scheme and the radiologist group were 0.915 and 0.683 (P = .227), respectively.

CONCLUSIONS

The diagnostic performance of the neural network ensemble-based CAD scheme is similar to that of senior radiologists for classifying lung nodules on thin-section CT. Furthermore, the CAD scheme has certain advantages in diagnosing small lung nodules.

Computer-aided detection (CAD) of lung nodules and small tumours on chest radiographs

Detection of focal pulmonary lesions is limited by quantum and anatomic noise and highly influenced by variable perception capacity of the reader. Multiple studies have proven that lesions - missed at time of primary interpretation - were visible on the chest radiographs in retrospect. Computer-aided diagnosis (CAD) schemes do not alter the anatomic noise but aim at decreasing the intrinsic limitations and variations of human perception by alerting the reader to suspicious areas in a chest radiograph when used as a 'second reader'. Multiple studies have shown that the detection performance can be improved using CAD especially for less experienced readers at a variable amount of decreased specificity. There seem to be a substantial learning process for both, experienced and inexperienced readers, to be able to optimally differentiate between false positive and true positive lesions and to build up sufficient trust in the capabilities of these systems to be able to use them at their full advantage. Studies so far focussed on stand-alone performance of the CAD schemes to reveal the magnitude of potential impact or on retrospective evaluation of CAD as a second reader for selected study groups. Further research is needed to assess the performance of these systems in clinical routine and to determine the trade-off between performance increase in terms of increased sensitivity and decreased inter-reader variability and loss of specificity and secondary indicated follow-up examinations for further diagnostic workup.

Enhancement of visual perception with use of dynamic cues

Institutional review board approval and signed informed consent were not needed, as medical images included in public databases were used in this study. The purpose of this study was to improve the detection of microcalcifications on mammograms and lung nodules on chest radiographs by using the dynamic cues algorithm and the motion and flickering sensitivity of the human visual system (HVS). Different sets of mammograms from the Mammographic Image Analysis Society database and chest radiographs from the Japanese Society of Radiological Technology database were presented statically, as is standard, and in a video sequence generated with the dynamic cues algorithm. Nine observers were asked to rate the presence of abnormalities with a five-point scale (1, definitely not present; 5, definitely present). The data were analyzed with receiver operating characteristic (ROC) techniques and the Dorfman-Berbaum-Metz method. The video sequence generated with the dynamic cues algorithm increased the rate of detection of microcalcifications by 10.2% (P = .002) compared with that obtained with the standard static method, as measured by the area under the ROC curve. Similar results were obtained for lung nodules, with an increase of 12.3% (P = .0054). The increase in the rate of correct detection did not come just from the image contrast change produced by the algorithm but also from the fact that image frames generated with the dynamic cues algorithm were put together in a video sequence so that the motion sensitivity of the HVS could be used to facilitate the detection of low-contrast objects.

SUPPLEMENTAL MATERIAL

http://radiology.rsnajnls.org/cgi/content/full/250/2/551/DC1.

Agreement of the order of overall performance levels under different reading paradigms

RATIONALE AND OBJECTIVES

To investigate consistency of the orders of performance levels when interpreting mammograms under three different reading paradigms.

MATERIALS AND METHODS

We performed a retrospective observer study in which nine experienced radiologists rated an enriched set of mammography examinations that they personally had read in the clinic ("individualized") mixed with a set that none of them had read in the clinic ("common set"). Examinations were interpreted under three different reading paradigms: binary using screening Breast Imaging Reporting and Data System (BI-RADS), receiver-operating characteristic (ROC), and free-response ROC (FROC). The performance in discriminating between cancer and noncancer findings under each of the paradigms was summarized using Youden's index/2+0.5 (Binary), nonparameteric area under the ROC curve (AUC), and an overall FROC index (JAFROC-2). Pearson correlation coefficients were then computed to assess consistency in the ordering of observers' performance levels. Statistical significance of the computed correlation coefficients was assessed using bootstrap confidence intervals obtained by resampling sets of examination-specific observations.

RESULTS

All but one of the computed pair-wise correlation coefficients were larger than 0.66 and were significantly different from zero. The correlation between the overall performance measures under the Binary and ROC paradigms was the lowest (0.43) and was not significantly different from zero (95% confidence interval -0.078 to 0.733).

CONCLUSION

The use of different evaluation paradigms in the laboratory tends to lead to consistent ordering of the overall performance levels of observers. However, one should recognize that conceptually similar performance indexes resulting from different paradigms often measure different performance characteristics and thus disagreements are not only possible but frequently quite natural.

Feature Reduction in Graph Analysis

A common approach to improve medical image classification is to add more features to the classifiers; however, this increases the time required for preprocessing raw data and training the classifiers, and the increase in features is not always beneficial. The number of commonly used features in the literature for training of image feature classifiers is over 50. Existing algorithms for selecting a subset of available features for image analysis fail to adequately eliminate redundant features. This paper presents a new selection algorithm based on graph analysis of interactions among features and between features to classifier decision. A modification of path analysis is done by applying regression analysis, multiple logistic and posterior Bayesian inference in order to eliminate features that provide the same contributions. A database of 113 mammograms from the Mammographic Image Analysis Society was used in the experiments. Tested on two classifiers - ANN and logistic regression - cancer detection accuracy (true positive and false-positive rates) using a 13-feature set selected by our algorithm yielded substantially similar accuracy as using a 26-feature set selected by SFS and results using all 50-features. However, the 13-feature greatly reduced the amount of computation needed.

The "laboratory" effect: comparing radiologists' performance and variability during prospective clinical and laboratory mammography interpretations

PURPOSE

To compare radiologists' performance during interpretation of screening mammograms in the clinic with their performance when reading the same mammograms in a retrospective laboratory study.

MATERIALS AND METHODS

This study was conducted under an institutional review board-approved, HIPAA-compliant protocol; the need for informed consent was waived. Nine experienced radiologists rated an enriched set of mammograms that they had personally read in the clinic (the "reader-specific" set) mixed with an enriched "common" set of mammograms that none of the participants had previously read in the clinic by using a screening Breast Imaging Reporting and Data System (BI-RADS) rating scale. The original clinical recommendations to recall the women for a diagnostic work-up, for both reader-specific and common sets, were compared with their recommendations during the retrospective experiment. The results are presented in terms of reader-specific and group-averaged sensitivity and specificity levels and the dispersion (spread) of reader-specific performance estimates.

RESULTS

On average, the radiologists' performance was significantly better in the clinic than in the laboratory (P = .035). Interreader dispersion of the computed performance levels was significantly lower during the clinical interpretations (P < .01).

CONCLUSION

Retrospective laboratory experiments may not represent either expected performance levels or interreader variability during clinical interpretations of the same set of mammograms in the clinical environment well.

Usefulness of computer-aided diagnosis schemes for vertebral fractures and lung nodules on chest radiographs

OBJECTIVE

We retrospectively evaluated the usefulness of computer-aided diagnosis (CAD) schemes to radiologist performance in the simultaneous detection of vertebral fractures and lung nodules on chest radiographs.

MATERIALS AND METHODS

We evaluated posteroanterior and lateral chest images of 21 patients with vertebral fractures, 31 patients with lung nodules, and 10 persons acting as controls. The total number of subjects was 60 because both lesions were present in four patients. Eighteen radiologists were asked to detect vertebral fractures and nodules simultaneously on posteroanterior and lateral images. The radiologists indicated their confidence level ratings regarding the presence or absence of lesions and the most likely location of each lesion on either posteroanterior or lateral images, first without and then with CAD output. The observers' performance was evaluated with use of receiver operating characteristic (ROC) and jackknife free-response ROC curves.

RESULTS

With the CAD scheme, the average area under the ROC curve for detection of vertebral fractures improved from 0.906 to 0.951 (p = 0.002). That for lung nodules also improved, but the improvement was not statistically significant (0.804-0.816, p = 0.297). The figure-of-merit values obtained with the jackknife free-response ROC program improved from 0.585 to 0.680 (p < 0.001) for vertebral fractures and from 0.622 to 0.650 (p = 0.017) for nodules, both results having statistical significance. Average sensitivity in the detection of lesions improved from 59.8% to 69.3% for vertebral fractures and from 64.9% to 67.6% for nodules.

CONCLUSION

In the detection of vertebral fractures and lung nodules on chest images, diagnostic accuracy among radiologists improves with the use of CAD.

Automatic volumetry on MR brain images can support diagnostic decision making

Background

Diagnostic decisions in clinical imaging currently rely almost exclusively on visual image interpretation. This can lead to uncertainty, for example in dementia disease, where some of the changes resemble those of normal ageing. We hypothesized that extracting volumetric data from patients' MR brain images, relating them to reference data and presenting the results as a colour overlay on the grey scale data would aid diagnostic readers in classifying dementia disease versus normal ageing.

Methods

A proof-of-concept forced-choice reader study was designed using MR brain images from 36 subjects. Images were segmented into 43 regions using an automatic atlas registration-based label propagation procedure. Seven subjects had clinically probable AD, the remaining 29 of a similar age range were used as controls. Seven of the control subject data sets were selected at random to be presented along with the seven AD datasets to two readers, who were blinded to all clinical and demographic information except age and gender. Readers were asked to review the grey scale MR images and to record their choice of diagnosis (AD or non-AD) along with their confidence in this decision. Afterwards, readers were given the option to switch on a false-colour overlay representing the relative size of the segmented structures. Colorization was based on the size rank of the test subject when compared with a reference group consisting of the 22 control subjects who were not used as review subjects. The readers were then asked to record whether and how the additional information had an impact on their diagnostic confidence.

Results

The size rank colour overlays were useful in 18 of 28 diagnoses, as determined by their impact on readers' diagnostic confidence. A not useful result was found in 6 of 28 cases. The impact of the additional information on diagnostic confidence was significant (p < 0.02).

Conclusion

Volumetric anatomical information extracted from brain images using automatic segmentation and presented as colour overlays can support diagnostic decision making.

Computer-aided diagnosis of lung cancer and pulmonary embolism in computed tomography-a review

Computer-aided detection (CADe) and computer-aided diagnosis (CADx) have been important areas of research in the last two decades. Significant progress has been made in the area of breast cancer detection, and CAD techniques are being developed in many other areas. Recent advances in multidetector row computed tomography have made it an increasingly common modality for imaging of lung diseases. A thoracic examination using thin-section computed tomography contains hundreds of images. Detection of lung cancer and pulmonary embolism on computed tomographic (CT) examinations are demanding tasks for radiologists because they have to search for abnormalities in a large number of images, and the lesions can be subtle. If successfully developed, CAD can be a useful second opinion to radiologists in thoracic CT interpretation. In this review, we summarize the studies that have been reported in these areas, discuss some challenges in the development of CAD, and identify areas that deserve particular attention in future research.

Comparing areas under receiver operating characteristic curves: potential impact of the "Last" experimentally measured operating point

A specific issue related to the selection of the analytic tool used when comparing the estimated performance of systems within the receiver operating characteristic (ROC) paradigm is reviewed. This issue is related to the possible effect of the last experimentally ascertained ROC data point in terms of highest true-positive and false-positive fractions. An example of a case is presented where the selection of a specific analysis approach could affect the study conclusion from being not statistically significant for parametric analysis and significant for nonparametric analysis. This is followed by recommendations that should help prevent misinterpretation of results.

Recent advances in cancer stem/progenitor cell research: therapeutic implications for overcoming resistance to the most aggressive cancers

Overcoming intrinsic and acquired resistance of cancer stem/progenitor cells to current clinical treatments represents a major challenge in treating and curing the most aggressive and metastatic cancers. This review summarizes recent advances in our understanding of the cellular origin and molecular mechanisms at the basis of cancer initiation and progression as well as the heterogeneity of cancers arising from the malignant transformation of adult stem/progenitor cells. We describe the critical functions provided by several growth factor cascades, including epidermal growth factor receptor (EGFR), platelet-derived growth factor receptor (PDGFR), stem cell factor (SCF) receptor (KIT), hedgehog and Wnt/beta-catenin signalling pathways that are frequently activated in cancer progenitor cells and are involved in their sustained growth, survival, invasion and drug resistance. Of therapeutic interest, we also discuss recent progress in the development of new drug combinations to treat the highly aggressive and metastatic cancers including refractory/relapsed leukaemias, melanoma and head and neck, brain, lung, breast, ovary, prostate, pancreas and gastrointestinal cancers which remain incurable in the clinics. The emphasis is on new therapeutic strategies consisting of molecular targeting of distinct oncogenic signalling elements activated in the cancer progenitor cells and their local microenvironment during cancer progression. These new targeted therapies should improve the efficacy of current therapeutic treatments against aggressive cancers, and thereby preventing disease relapse and enhancing patient survival.

Recent advances on the molecular mechanisms involved in the drug resistance of cancer cells and novel targeting therapies

This review summarizes the recent knowledge obtained on the molecular mechanisms involved in the intrinsic and acquired resistance of cancer cells to current cancer therapies. We describe the cascades that are often altered in cancer cells during cancer progression that may contribute in a crucial manner to drug resistance and disease relapse. The emphasis is on the implication of ATP-binding cassette (ABC) multidrug efflux transporters in drug disposition and antiapoptotic factors, including epidermal growth factor receptor cascades and deregulated enzymes in ceramide metabolic pathways. The altered expression and activity of these signaling elements may have a critical role in the resistance of cancer cells to cytotoxic effects induced by diverse chemotherapeutic drugs and cancer recurrence. Of therapeutic interest, new strategies for reversing the multidrug resistance and developing more effective clinical treatments against the highly aggressive, metastatic, and recurrent cancers, based on the molecular targeting of the cancer progenitor cells and their further differentiated progeny, are also described.

Does computer-aided diagnosis for lung tumors change satisfaction of search in chest radiography?

RATIONALE AND OBJECTIVES

Computer-aided diagnosis (CAD) has been developed to ensure that the radiologist considers suspect focal opacities that may represent cancer in chest radiography. Although CAD was not developed to counteract the satisfaction of search (SOS) effect, it may be an effective intervention to do so. The objective of this study is to determine whether an idealized CAD can reduce SOS effects in chest radiography.

MATERIALS AND METHODS

Fifty-seven chest radiographs, half of which demonstrated diverse, native abnormalities were read twice by 16 observers, once with and once without the addition of a simulated pulmonary nodule. Simulated CAD prompts were provided during the interpretation, which unerringly pointed to the added simulated nodule. Area under the ROC curve for detecting the native abnormalities was estimated for each observer in each treatment condition. In addition to testing for the SOS effect in the presence of CAD prompts, results were compared to those of a previous SOS study.

RESULTS

Significantly more nodules were reported in the SOS with CAD experiment than in the original SOS experiment (49 versus 43, P < .01). An SOS effect was found even when CAD prompts were provided; ROC areas for detecting native abnormalities were reduced with added nodules [0.68 versus 0.65, P (one-tailed) < .05]. Comparison of the current experiment with CAD and the previous SOS experiments failed to show a significant difference of the magnitude of the SOS effect (P = .52). The threshold for reporting was more conservative with CAD prompts than in SOS studies (P = .052).

CONCLUSION

Our results indicate that the CAD prompts, even those that always point to their target lesion without false-positive error, fail to counteract SOS in chest radiography. The stricter decision thresholds with CAD prompts may indicate less visual search for native abnormalities.

"Binary" and "non-binary" detection tasks: are current performance measures optimal?

We have observed that a very large fraction of responses for several detection tasks during the performance of observer studies are in the extreme ranges of lower than 11% or higher than 89% regardless of the actual presence or absence of the abnormality in question or its subjectively rated "subtleness." This observation raises questions regarding the validity and appropriateness of using multicategory rating scales for such detection tasks. Monte Carlo simulation of binary and multicategory ratings for these tasks demonstrate that the use of the former (binary) often results in a less biased and more precise summary index and hence may lead to a higher statistical power for determining differences between modalities.

Interplay of distinct growth factors during epithelial mesenchymal transition of cancer progenitor cells and molecular targeting as novel cancer therapies

In this review, we describe the critical functions assumed by the interplay of epidermal growth factor, hedgehog, Wnt/beta-catenin, tumor growth factor-beta and integrin signaling cascades in tumorigenic and migrating cancer progenitor cells and activated stromal cells during carcinogenesis. These growth factors provide an important role for the sustained growth and survival of tumorigenic cancer progenitor cells and their progeny by up-regulating numerous mitotic and antiapoptotic signaling cascades. Furthermore, these potent morphogens may cooperate for inducing the molecular events associated with the epithelial-mesenchymal program in cancer cells including the alterations in epithelial cell shape and motility through the dissociation of intercellular adherens junctions. Of therapeutic interest, new strategies for the development of more effective clinical treatments against the locally aggressive and invasive cancers based on the molecular targeting of deregulated signaling elements in tumorigenic and migrating cancer cells and their local microenvironment are also described.