Effect of a computer-aided diagnosis scheme on radiologists' performance in detection of lung nodules on radiographs

Comparison of Mammography and Mammography with Supplemental Whole-Breast US Tomography for Cancer Detection in Patients with Dense Breasts

BACKGROUND

Women with dense breasts benefit from supplemental cancer screening with US, but US has low specificity.

PURPOSE

To evaluate the performance of breast US tomography (UST) combined with full-field digital mammography (FFDM) compared with FFDM alone for breast cancer screening in women with dense breasts.

MATERIALS AND METHODS

This retrospective multireader multicase study included women with dense breasts who underwent FFDM and UST at 10 centers between August 2017 and October 2019 as part of a prospective case collection registry. All patients in the registry with cancer were included; patients with benign biopsy or negative follow-up imaging findings were randomly selected for inclusion. Thirty-two Mammography Quality Standards Act-qualified radiologists independently evaluated FFDM followed immediately by FFDM plus UST for suspicious findings and assigned a Breast Imaging Reporting and Data System (BI-RADS) category. The superiority of FFDM plus UST versus FFDM alone for cancer detection (assessed with area under the receiver operating characteristic curve [AUC]), BI-RADS 4 sensitivity, and BI-RADS 3 sensitivity and specificity were evaluated using the two-sided significance level of α = .05. Noninferiority of BI-RADS 4 specificity was evaluated at the one-sided significance level of α = .025 with a -10% margin.

RESULTS

Among 140 women (mean age, 56 years ±10 [SD]; 36 with cancer, 104 without), FFDM plus UST achieved superior performance compared with FFDM alone (AUC, 0.60 [95% CI: 0.51, 0.69] vs 0.54 [95% CI: 0.45, 0.64]; P = .03). For FFDM plus UST versus FFDM alone, BI-RADS 4 mean sensitivity was superior (37% [428 of 1152] vs 30% [343 of 1152]; P = .03) and BI-RADS 4 mean specificity was noninferior (82% [2741 of 3328] vs 88% [2916 of 3328]; P = .004). For FFDM plus UST versus FFDM, no difference in BI-RADS 3 mean sensitivity was observed (40% [461 of 1152] vs 33% [385 of 1152]; P = .08), but BI-RADS 3 mean specificity was superior (75% [2491 of 3328] vs 69% [2299 of 3328]; P = .04).

CONCLUSION

In women with dense breasts, FFDM plus UST improved cancer detection by radiologists versus FFDM alone. Clinical trial registration nos. NCT03257839 and NCT04260620 Published under a CC BY 4.0 license. Supplemental material is available for this article. See also the editorial by Mann in this issue.

Impact of Artificial Intelligence Assistance on Chest CT Interpretation Times: A Prospective Randomized Study

BACKGROUND. Deep learning-based convolutional neural networks have enabled major advances in development of artificial intelligence (AI) software applications. Modern AI applications offer comprehensive multiorgan evaluation. OBJECTIVE. The purpose of this article was to evaluate the impact of an automated AI platform integrated into clinical workflow for chest CT interpretation on radiologists' interpretation times when evaluated in a real-world clinical setting. METHODS. In this prospective single-center study, a commercial AI software solution was integrated into clinical workflow for chest CT interpretation. The software provided automated analysis of cardiac, pulmonary, and musculoskeletal findings, including labeling, segmenting, and measuring normal structures as well as detecting, labeling, and measuring abnormalities. AI-annotated images and autogenerated summary results were stored in the PACS and available to interpreting radiologists. A total of 390 patients (204 women, 186 men; mean age, 62.8 ± 13.3 [SD] years) who underwent out-patient chest CT between January 19, 2021, and January 28, 2021, were included. Scans were randomized using 1:1 allocation between AI-assisted and non-AI-assisted arms and were clinically interpreted by one of three cardiothoracic radiologists (65 scans per arm per radiologist; total of 195 scans per arm) who recorded interpretation times using a stopwatch. Findings were categorized according to review of report impressions. Interpretation times were compared between arms. RESULTS. Mean interpretation times were significantly shorter in the AI-assisted than in the non-AI-assisted arm for all three readers (289 ± 89 vs 344 ± 129 seconds, p < .001; 449 ± 110 vs 649 ± 82 seconds, p < .001; 281 ± 114 vs 348 ± 93 seconds, p = .01) and for readers combined (328 ± 122 vs 421 ± 175 seconds, p < .001). For readers combined, the mean difference was 93 seconds (95% CI, 63-123 seconds), corresponding with a 22.1% reduction in the AI-assisted arm. Mean interpretation time was also shorter in the AI-assisted arm compared with the non-AI-assisted arm for contrast-enhanced scans (83 seconds), noncontrast scans (104 seconds), negative scans (84 seconds), positive scans without significant new findings (117 seconds), and positive scans with significant new findings (92 seconds). CONCLUSION. Cardiothoracic radiologists exhibited a 22.1% reduction in chest CT interpretations times when they had access to results from an automated AI support platform during real-world clinical practice. CLINICAL IMPACT. Integration of the AI support platform into clinical workflow improved radiologist efficiency.

Effective Utilization of Multiple Convolutional Neural Networks for Chest X-Ray Classification

Out of the numerous types of Medical Imaging modalities available, radiography stands out a bit more than others due to its capabilities of diagnosing diseases and conditions, including life-threatening conditions. Its affordability is another main reason for its prevalence. Chest Radiography holds even higher importance, as it focuses a critical area of the human body. However, interpreting a Chest Radiography image can be challenging and usually done by an experienced Radiologist for accurate results. There are two main issues related to this. One is that in some countries, experienced Radiologists are scarce. The other issue is that the inevitability of human errors in diagnoses. Researchers attempt to use Artificial Intelligence to address these two issues. Most of the existing work incorporates Convolutional Neural Networks for this purpose. This paper presents a novel way of parallelizing multiple architectures of Convolutional Neural Networks focusing on Chest X-ray classification. The paper further presents a comprehensive evaluation of the existing architectures with the parallelized results of them using our method. We used four large-scale datasets, including a non-medical one, for the evaluation of our models. We managed to achieve better accuracy for 9 out 13 and 11 out of 14 labels on our two main evaluation datasets. The paper concludes by presenting the limitations and future improvements possible for the system.

The Added Effect of Artificial Intelligence on Physicians' Performance in Detecting Thoracic Pathologies on CT and Chest X-ray: A Systematic Review

Our systematic review investigated the additional effect of artificial intelligence-based devices on human observers when diagnosing and/or detecting thoracic pathologies using different diagnostic imaging modalities, such as chest X-ray and CT. Peer-reviewed, original research articles from EMBASE, PubMed, Cochrane library, SCOPUS, and Web of Science were retrieved. Included articles were published within the last 20 years and used a device based on artificial intelligence (AI) technology to detect or diagnose pulmonary findings. The AI-based device had to be used in an observer test where the performance of human observers with and without addition of the device was measured as sensitivity, specificity, accuracy, AUC, or time spent on image reading. A total of 38 studies were included for final assessment. The quality assessment tool for diagnostic accuracy studies (QUADAS-2) was used for bias assessment. The average sensitivity increased from 67.8% to 74.6%; specificity from 82.2% to 85.4%; accuracy from 75.4% to 81.7%; and Area Under the ROC Curve (AUC) from 0.75 to 0.80. Generally, a faster reading time was reported when radiologists were aided by AI-based devices. Our systematic review showed that performance generally improved for the physicians when assisted by AI-based devices compared to unaided interpretation.

Added Value of Deep Learning-based Detection System for Multiple Major Findings on Chest Radiographs: A Randomized Crossover Study

Background Previous studies assessing the effects of computer-aided detection on observer performance in the reading of chest radiographs used a sequential reading design that may have biased the results because of reading order or recall bias. Purpose To compare observer performance in detecting and localizing major abnormal findings including nodules, consolidation, interstitial opacity, pleural effusion, and pneumothorax on chest radiographs without versus with deep learning-based detection (DLD) system assistance in a randomized crossover design. Materials and Methods This study included retrospectively collected normal and abnormal chest radiographs between January 2016 and December 2017 (https://cris.nih.go.kr/; registration no. KCT0004147). The radiographs were randomized into two groups, and six observers, including thoracic radiologists, interpreted each radiograph without and with use of a commercially available DLD system by using a crossover design with a washout period. Jackknife alternative free-response receiver operating characteristic (JAFROC) figure of merit (FOM), area under the receiver operating characteristic curve (AUC), sensitivity, specificity, false-positive findings per image, and reading times of observers with and without the DLD system were compared by using McNemar and paired t tests. Results A total of 114 normal (mean patient age ± standard deviation, 51 years ± 11; 58 men) and 114 abnormal (mean patient age, 60 years ± 15; 75 men) chest radiographs were evaluated. The radiographs were randomized to two groups: group A (n = 114) and group B (n = 114). Use of the DLD system improved the observers' JAFROC FOM (from 0.90 to 0.95, P = .002), AUC (from 0.93 to 0.98, P = .002), per-lesion sensitivity (from 83% [822 of 990 lesions] to 89.1% [882 of 990 lesions], P = .009), per-image sensitivity (from 80% [548 of 684 radiographs] to 89% [608 of 684 radiographs], P = .009), and specificity (from 89.3% [611 of 684 radiographs] to 96.6% [661 of 684 radiographs], P = .01) and reduced the reading time (from 10-65 seconds to 6-27 seconds, P < .001). The DLD system alone outperformed the pooled observers (JAFROC FOM: 0.96 vs 0.90, respectively, P = .007; AUC: 0.98 vs 0.93, P = .003). Conclusion Observers including thoracic radiologists showed improved performance in the detection and localization of major abnormal findings on chest radiographs and reduced reading time with use of a deep learning-based detection system. © RSNA, 2021 Online supplemental material is available for this article.

The detection of lung cancer using massive artificial neural network based on soft tissue technique

BACKGROUND

A proposed computer aided detection (CAD) scheme faces major issues during subtle nodule recognition. However, radiologists have not noticed subtle nodules in beginning stage of lung cancer while a proposed CAD scheme recognizes non subtle nodules using x-ray images.

METHOD

Such an issue has been resolved by creating MANN (Massive Artificial Neural Network) based soft tissue technique from the lung segmented x-ray image. A soft tissue image recognizes nodule candidate for feature extortion and classification. X-ray images are downloaded using Japanese society of radiological technology (JSRT) image set. This image set includes 233 images (140 nodule x-ray images and 93 normal x-ray images). A mean size for a nodule is 17.8 mm and it is validated with computed tomography (CT) image. Thirty percent (42/140) abnormal represents subtle nodules and it is split into five stages (tremendously subtle, very subtle, subtle, observable, relatively observable) by radiologists.

RESULT

A proposed CAD scheme without soft tissue technique attained 66.42% (93/140) sensitivity and 66.76% accuracy having 2.5 false positives per image. Utilizing soft tissue technique, many nodules superimposed by ribs as well as clavicles have identified (sensitivity is 72.85% (102/140) and accuracy is 72.96% at one false positive rate).

CONCLUSION

In particular, a proposed CAD system determine sensitivity and accuracy in support of subtle nodules (sensitivity is 14/42 = 33.33% and accuracy is 33.66%) is statistically higher than CAD (sensitivity is 13/42 = 30.95% and accuracy is 30.97%) scheme without soft tissue technique. A proposed CAD scheme attained tremendously minimum false positive rate and it is a promising technique in support of cancerous recognition due to improved sensitivity and specificity.

Usefulness of deep learning-assisted identification of hyperdense MCA sign in acute ischemic stroke: comparison with readers' performance

PURPOSE

To evaluate the usefulness of deep learning-assisted diagnosis for identifying hyperdense middle cerebral artery sign (HMCAS) on non-contrast computed tomography in comparison with the diagnostic performance of neuroradiologists.

MATERIALS AND METHODS

We obtained 46 HMCAS-positive and 52 HMCAS-negative test samples extracted using 50-pixel-diameter circular regions of interest. Five neuroradiologists undertook an initial diagnostic performance test by describing the HMCAS-positive prediction rate in each sample. Their diagnostic performance was compared with that of a deep convolutional neural network (DCNN) model that had been trained using another dataset in our previous study. In the second test, readers could reference the prediction rate of the DCNN model in each sample.

RESULTS

The diagnostic performance of the DCNN for HMCAS showed an accuracy of 81.6% and area under the receiver-operating characteristic curve (AUC) of 0.869, whereas the initial diagnostic performance of neuroradiologists showed an accuracy of 78.8% and AUC of 0.882. The second diagnostic test of neuroradiologists with reference to the results of the DCNN model showed an accuracy of 84.7% and AUC of 0.932. In all readers, AUC values were higher in the second test than the initial test.

CONCLUSION

The ability of DCNN to identify HMCAS is comparable with the diagnostic performance of neuroradiologists.

Radiomics: from qualitative to quantitative imaging

Historically, medical imaging has been a qualitative or semi-quantitative modality. It is difficult to quantify what can be seen in an image, and to turn it into valuable predictive outcomes. As a result of advances in both computational hardware and machine learning algorithms, computers are making great strides in obtaining quantitative information from imaging and correlating it with outcomes. Radiomics, in its two forms "handcrafted and deep," is an emerging field that translates medical images into quantitative data to yield biological information and enable radiologic phenotypic profiling for diagnosis, theragnosis, decision support, and monitoring. Handcrafted radiomics is a multistage process in which features based on shape, pixel intensities, and texture are extracted from radiographs. Within this review, we describe the steps: starting with quantitative imaging data, how it can be extracted, how to correlate it with clinical and biological outcomes, resulting in models that can be used to make predictions, such as survival, or for detection and classification used in diagnostics. The application of deep learning, the second arm of radiomics, and its place in the radiomics workflow is discussed, along with its advantages and disadvantages. To better illustrate the technologies being used, we provide real-world clinical applications of radiomics in oncology, showcasing research on the applications of radiomics, as well as covering its limitations and its future direction.

Bone suppression for chest X-ray image using a convolutional neural filter

Chest X-rays are used for mass screening for the early detection of lung cancer. However, lung nodules are often overlooked because of bones overlapping the lung fields. Bone suppression techniques based on artificial intelligence have been developed to solve this problem. However, bone suppression accuracy needs improvement. In this study, we propose a convolutional neural filter (CNF) for bone suppression based on a convolutional neural network which is frequently used in the medical field and has excellent performance in image processing. CNF outputs a value for the bone component of the target pixel by inputting pixel values in the neighborhood of the target pixel. By processing all positions in the input image, a bone-extracted image is generated. Finally, bone-suppressed image is obtained by subtracting the bone-extracted image from the original chest X-ray image. Bone suppression was most accurate when using CNF with six convolutional layers, yielding bone suppression of 89.2%. In addition, abnormalities, if present, were effectively imaged by suppressing only bone components and maintaining soft-tissue. These results suggest that the chances of missing abnormalities may be reduced by using the proposed method. The proposed method is useful for bone suppression in chest X-ray images.

Multi-resolution convolutional networks for chest X-ray radiograph based lung nodule detection

Lung cancer is the leading cause of cancer death worldwide. Early detection of lung cancer is helpful to provide the best possible clinical treatment for patients. Due to the limited number of radiologist and the huge number of chest x-ray radiographs (CXR) available for observation, a computer-aided detection scheme should be developed to assist radiologists in decision-making. While deep learning showed state-of-the-art performance in several computer vision applications, it has not been used for lung nodule detection on CXR. In this paper, a deep learning-based lung nodule detection method was proposed. We employed patch-based multi-resolution convolutional networks to extract the features and employed four different fusion methods for classification. The proposed method shows much better performance and is much more robust than those previously reported researches. For publicly available Japanese Society of Radiological Technology (JSRT) database, more than 99% of lung nodules can be detected when the false positives per image (FPs/image) was 0.2. The FAUC and R-CPM of the proposed method were 0.982 and 0.987, respectively. The proposed approach has the potential of applications in clinical practice.

Feasibility of lung cancer screening in developing countries: challenges, opportunities and way forward

Lung cancer is the leading cause of all cancer deaths worldwide, comprising 18.4% of all cancer deaths. Low-dose computed tomography (LDCT) has shown mortality benefit in various trials and now a standard tool for lung cancer screening. Most researches have been carried out in developed countries where lung cancer incidence and mortality is very high. There is an increasing trend in lung cancer incidence in developing countries attributed to tobacco smoking and various environmental and occupational risk factors. Implementation of lung cancer screening is challenging, so organised lung cancer screening is practically non-existent. There are numerous challenges in implementing such programs ranging from infrastructure, trained human resources, referral algorithm to cost and psychological trauma due to over-diagnosis. Pulmonary tuberculosis and other chest infections are important issues to be addressed while planning for lung cancer screening in developing countries. Burden of these diseases is very high and can lead to over-diagnosis in view of cut off of lung nodule size in various studies. Assessment of high risk cases for lung cancer is difficult as various forms of smoking make quantification non-uniform and difficult. Lung cancer screening targets only high risk population unlike screening programs for other cancers where entire population is targeted. There is a need of lung cancer screening for high risk cases as it saves life. Tobacco control and smoking cessation remain the most important long term intervention to decrease morbidity and mortality from lung cancer in developing countries. There is no sufficient evidence supporting the introduction of population-based screening for lung cancer in public health services.

Reliability of diagnosing acute ischemic cerebrovascular on magnetic resonance imaging disorders using iPads

PURPOSE

The use of tablet terminals has been explored in various medical settings; however, caution should be exercised when performing image diagnosis using this technology. The present study examined the characteristics of an iPad Air™ monitor and assessed radiographic image interpretations to verify the reliability of the telediagnosis of acute cerebral infarction based on magnetic resonance imaging (MRI) using a tablet terminal.

MATERIALS AND METHODS

The luminance of the iPad Air™ was measured using a UA-10 analyzer, and radiographic image interpretation experiments were performed in 100 patients who underwent MRI within 6 h of symptom onset. Ten physicians viewed the images on the iPad Air™ and a medical monitor, with an interval of 2 months between each interpretation.

RESULTS

When the iPad Air™ screen was pure white, the contour lines revealed nonuniform luminance distribution. In the reading experiment, the areas under the curve of the medical monitor and the iPad Air™ were 0.9311 and 0.9431, respectively. No significant difference was observed between the medical monitor and the iPad Air™ (p = 0.113).

CONCLUSION

The results of the observer performance studies for detecting acute ischemic cerebrovascular disorders on an iPad Air™ were found to be similar to those on a medical monitor.

Computer-aided detection in chest radiography based on artificial intelligence: a survey

As the most common examination tool in medical practice, chest radiography has important clinical value in the diagnosis of disease. Thus, the automatic detection of chest disease based on chest radiography has become one of the hot topics in medical imaging research. Based on the clinical applications, the study conducts a comprehensive survey on computer-aided detection (CAD) systems, and especially focuses on the artificial intelligence technology applied in chest radiography. The paper presents several common chest X-ray datasets and briefly introduces general image preprocessing procedures, such as contrast enhancement and segmentation, and bone suppression techniques that are applied to chest radiography. Then, the CAD system in the detection of specific disease (pulmonary nodules, tuberculosis, and interstitial lung diseases) and multiple diseases is described, focusing on the basic principles of the algorithm, the data used in the study, the evaluation measures, and the results. Finally, the paper summarizes the CAD system in chest radiography based on artificial intelligence and discusses the existing problems and trends.

Computer-aided detection of brain metastasis on 3D MR imaging: Observer performance study

PURPOSE

To assess the effect of computer-aided detection (CAD) of brain metastasis (BM) on radiologists' diagnostic performance in interpreting three-dimensional brain magnetic resonance (MR) imaging using follow-up imaging and consensus as the reference standard.

MATERIALS AND METHODS

The institutional review board approved this retrospective study. The study cohort consisted of 110 consecutive patients with BM and 30 patients without BM. The training data set included MR images of 80 patients with 450 BM nodules. The test set included MR images of 30 patients with 134 BM nodules and 30 patients without BM. We developed a CAD system for BM detection using template-matching and K-means clustering algorithms for candidate detection and an artificial neural network for false-positive reduction. Four reviewers (two neuroradiologists and two radiology residents) interpreted the test set images before and after the use of CAD in a sequential manner. The sensitivity, false positive (FP) per case, and reading time were analyzed. A jackknife free-response receiver operating characteristic (JAFROC) method was used to determine the improvement in the diagnostic accuracy.

RESULTS

The sensitivity of CAD was 87.3% with an FP per case of 302.4. CAD significantly improved the diagnostic performance of the four reviewers with a figure-of-merit (FOM) of 0.874 (without CAD) vs. 0.898 (with CAD) according to JAFROC analysis (p < 0.01). Statistically significant improvement was noted only for less-experienced reviewers (FOM without vs. with CAD, 0.834 vs. 0.877, p < 0.01). The additional time required to review the CAD results was approximately 72 sec (40% of the total review time).

CONCLUSION

CAD as a second reader helps radiologists improve their diagnostic performance in the detection of BM on MR imaging, particularly for less-experienced reviewers.

The usefulness of a computer-aided diagnosis scheme for improving the performance of clinicians to diagnose non-mass lesions on breast ultrasonographic images

PURPOSE

The purpose of this study was to evaluate the usefulness of a computer-aided diagnosis (CAD) scheme for improving the performance of clinicians to diagnose non-mass lesions appearing as hypoechoic areas on breast ultrasonographic images.

METHODS

The database included 97 ultrasonographic images with hypoechoic areas: 48 benign cases [benign lesion with benign mammary tissue or fibrocystic disease (n = 20), fibroadenoma (n = 11), and intraductal papilloma (n = 17)] and 49 malignant cases [ductal carcinoma in situ (n = 17) and invasive ductal carcinoma (n = 32)]. Seven clinicians, three expert breast surgeons, and four general surgeons participated in the observer study. They were asked their confidence level concerning the possibility of malignancy in all 97 cases with and without the use of the CAD scheme. Receiver operating characteristic (ROC) analysis was performed to evaluate the usefulness of the CAD scheme.

RESULTS

The areas under the ROC curve (AUC) improved for all observers when they used the CAD scheme and increased from 0.649 to 0.783 (P = 0.0167). Notably, the AUC for the general surgeon group increased from 0.625 to 0.793 (P = 0.045).

CONCLUSIONS

This study showed that the performance of clinicians to diagnose non-mass lesions appearing as hypoechoic areas on breast ultrasonographic images was improved by the use of a CAD scheme.

Unsupervised segmentation of lung fields in chest radiographs using multiresolution fractal feature vector and deformable models

Segmenting lung fields in a chest radiograph is essential for automatically analyzing an image. We present an unsupervised method based on multiresolution fractal feature vector. The feature vector characterizes the lung field region effectively. A fuzzy c-means clustering algorithm is then applied to obtain a satisfactory initial contour. The final contour is obtained by deformable models. The results show the feasibility and high performance of the proposed method. Furthermore, based on the segmentation of lung fields, the cardiothoracic ratio (CTR) can be measured. The CTR is a simple index for evaluating cardiac hypertrophy. After identifying a suspicious symptom based on the estimated CTR, a physician can suggest that the patient undergoes additional extensive tests before a treatment plan is finalized.

Potential clinical impact of advanced imaging and computer-aided diagnosis in chest radiology: importance of radiologist's role and successful observer study

This review paper is based on our research experience in the past 30 years. The importance of radiologists' role is discussed in the development or evaluation of new medical images and of computer-aided detection (CAD) schemes in chest radiology. The four main topics include (1) introducing what diseases can be included in a research database for different imaging techniques or CAD systems and what imaging database can be built by radiologists, (2) understanding how radiologists' subjective judgment can be combined with technical objective features to improve CAD performance, (3) sharing our experience in the design of successful observer performance studies, and (4) finally, discussing whether the new images and CAD systems can improve radiologists' diagnostic ability in chest radiology. In conclusion, advanced imaging techniques and detection/classification of CAD systems have a potential clinical impact on improvement of radiologists' diagnostic ability, for both the detection and the differential diagnosis of various lung diseases, in chest radiology.

Automatic lung segmentation using control feedback system: morphology and texture paradigm

Interstitial Lung Disease (ILD) encompasses a wide array of diseases that share some common radiologic characteristics. When diagnosing such diseases, radiologists can be affected by heavy workload and fatigue thus decreasing diagnostic accuracy. Automatic segmentation is the first step in implementing a Computer Aided Diagnosis (CAD) that will help radiologists to improve diagnostic accuracy thereby reducing manual interpretation. Automatic segmentation proposed uses an initial thresholding and morphology based segmentation coupled with feedback that detects large deviations with a corrective segmentation. This feedback is analogous to a control system which allows detection of abnormal or severe lung disease and provides a feedback to an online segmentation improving the overall performance of the system. This feedback system encompasses a texture paradigm. In this study we studied 48 males and 48 female patients consisting of 15 normal and 81 abnormal patients. A senior radiologist chose the five levels needed for ILD diagnosis. The results of segmentation were displayed by showing the comparison of the automated and ground truth boundaries (courtesy of ImgTracer™ 1.0, AtheroPoint™ LLC, Roseville, CA, USA). The left lung's performance of segmentation was 96.52% for Jaccard Index and 98.21% for Dice Similarity, 0.61 mm for Polyline Distance Metric (PDM), -1.15% for Relative Area Error and 4.09% Area Overlap Error. The right lung's performance of segmentation was 97.24% for Jaccard Index, 98.58% for Dice Similarity, 0.61 mm for PDM, -0.03% for Relative Area Error and 3.53% for Area Overlap Error. The segmentation overall has an overall similarity of 98.4%. The segmentation proposed is an accurate and fully automated system.

Improving the radiologist-CAD interaction: designing for appropriate trust

Computer-aided diagnosis (CAD) has great potential to improve radiologists' diagnostic performance. However, the reported performance of the radiologist-CAD team is lower than what might be expected based on the performance of the radiologist and the CAD system in isolation. This indicates that the interaction between radiologists and the CAD system is not optimal. An important factor in the interaction between humans and automated aids (such as CAD) is trust. Suboptimal performance of the human-automation team is often caused by an inappropriate level of trust in the automation. In this review, we examine the role of trust in the radiologist-CAD interaction and suggest ways to improve the output of the CAD system so that it allows radiologists to calibrate their trust in the CAD system more effectively. Observer studies of the CAD systems show that radiologists often have an inappropriate level of trust in the CAD system. They sometimes under-trust CAD, thereby reducing its potential benefits, and sometimes over-trust it, leading to diagnostic errors they would not have made without CAD. Based on the literature on trust in human-automation interaction and the results of CAD observer studies, we have identified four ways to improve the output of CAD so that it allows radiologists to form a more appropriate level of trust in CAD. Designing CAD systems for appropriate trust is important and can improve the performance of the radiologist-CAD team. Future CAD research and development should acknowledge the importance of the radiologist-CAD interaction, and specifically the role of trust therein, in order to create the perfect artificial partner for the radiologist. This review focuses on the role of trust in the radiologist-CAD interaction. The aim of the review is to encourage CAD developers to design for appropriate trust and thereby improve the performance of the radiologist-CAD team.

Influence of study design in receiver operating characteristics studies: sequential versus independent reading

Observer studies to assess new image processing devices or computer-aided diagnosis techniques are often performed, but little is known about the effect of the study design on observer performance results. We investigated the effect of the sequential and independent reading design on observer study results with respect to reader performance and their statistical power. For this we performed an observer study for the detection of lung nodules with bone-suppressed images (BSIs) compared with original chest radiographs. In a fully crossed observer study, eight observers assessed a series of 300 radiographs four times, including one assessment of the original radiograph with sequential BSI and two independent reading sessions with BSI. Observer performance was compared using multireader multicase receiver operating characteristics. No significant difference between the effect of BSI in the sequential and the independent reading sessions could be found ([Formula: see text]; [Formula: see text]). Compared with the original radiographs, increased performance with BSI was significant in the sequential and one of the independent reading sessions ([Formula: see text]; [Formula: see text]), and nonsignificant in the other independent reading session ([Formula: see text]). A strong increase of uncorrelated variance components was found in the independent reading sessions, masking the ability to demonstrate differences in observer performance across modalities. Therefore, the sequential reading design is the preferred design because it is less burdensome and has more statistical power.

A comparison of computer-aided detection (CAD) effectiveness in pulmonary nodule identification using different methods of bone suppression in chest radiographs

This study aimed to compare the diagnostic effectiveness of computer-aided detection (CAD) software (OnGuard™ 5.2) in combination with hardware-based bone suppression (dual-energy subtraction radiography (DESR)), software-based bone suppression (SoftView™, version 2.4), and standard posteroanterior images with no bone suppression. A retrospective pilot study compared the diagnostic performance of two commercially available methods of bone suppression when used with commercially available CAD software. Chest images from 27 patients with computed tomography (CT) and pathology-proven malignant pulmonary nodules (8-34 mm) and 25 CT-negative patient controls were used for analysis. The Friedman, McNemar, and chi-square tests were used to compare diagnostic performance and the kappa statistic was used to evaluate method agreement. The average number of regions of interest and false-positives per image identified by CAD were not found to be significantly different regardless of the bone suppression methods evaluated. Similarly, the sensitivity, specificity, and test efficiency were not found to be significantly different. Agreement between the methods was between poor and excellent. The accuracy of CAD (OnGuard™, version 5.2) is not statistically different with either DESR or SoftView™ (version 2.4) bone suppression technology in digital chest images for pulmonary nodule identification. Low values for sensitivity (<80 %) and specificity (<50 %) may limit their utility for clinical radiology.

Separation of bones from chest radiographs by means of anatomically specific multiple massive-training ANNs combined with total variation minimization smoothing

Most lung nodules that are missed by radiologists as well as computer-aided detection (CADe) schemes overlap with ribs or clavicles in chest radiographs (CXRs). The purpose of this study was to separate bony structures such as ribs and clavicles from soft tissue in CXRs. To achieve this, we developed anatomically specific multiple massive-training artificial neural networks (MTANNs) combined with total variation (TV) minimization smoothing and a histogram-matching-based consistency improvement method. The anatomically specific multiple MTANNs were designed to separate bones from soft tissue in different anatomic segments of the lungs. Each of the MTANNs was trained with the corresponding anatomic segment in the teaching bone images. The output segmental images from the multiple MTANNs were merged to produce an entire bone image. TV minimization smoothing was applied to the bone image for reduction of noise while preserving edges. This bone image was then subtracted from the original CXR to produce a soft-tissue image where bones were separated out. This new method was compared with conventional MTANNs with a database of 110 CXRs with nodules. Our new anatomically specific MTANNs separated rib edges, ribs close to the lung wall, and the clavicles from soft tissue in CXRs to a substantially higher level than did the conventional MTANNs, while the conspicuity of lung nodules and vessels was maintained. Thus, our technique for bone-soft-tissue separation by means of our new MTANNs would be potentially useful for radiologists as well as CADe schemes in detection of lung nodules on CXRs.

Comparison of liquid crystal display monitors calibrated with gray-scale standard display function and with γ 2.2 and iPad: observer performance in detection of cerebral infarction on brain CT

OBJECTIVE

The purpose of the study was to compare observer performance in the detection of cerebral infarction on a brain CT using medical-grade liquid crystal display (LCD) monitors calibrated with the gray-scale standard display function and with γ 2.2 and using an iPad with a simulated screen setting.

MATERIALS AND METHODS

We amassed 97 sample sets, from 47 patients with proven cerebral infarction and 50 healthy control subjects. Nine radiologists independently assessed brain CT on a gray-scale standard display function LCD, a γ 2.2 LCD, and an iPad in random order over 4-week intervals. Receiver operating characteristic (ROC) analysis was performed by using the continuous scale, and the area under the ROC curve (A(z)) was calculated for each monitor.

RESULTS

The A(z) values for gray-scale standard display function LCD, γ 2.2 LCD, and iPad were 0.875, 0.884, and 0.839, respectively. The difference among the three monitors was very small. There was no significant difference between gray-scale standard display function LCD and γ 2.2 LCD. However, the A(z) value was statistically significantly smaller for the iPad than the γ 2.2 LCD (p < 0.05).

CONCLUSION

Observer performance for detecting cerebral infarction on the LCD with γ 2.2 calibration was found to be similar to the LCD with gray-scale standard display function calibration. Although observer performance using the iPad was poorer than that using the other LCDs, the difference was small. Therefore, the iPad could not substitute for other LCD monitors. However, owing to the promising potential advantages of tablet PCs, such as portability, further examination is needed into the clinical use of tablet PCs.

Comparison of dual-energy subtraction and electronic bone suppression combined with computer-aided detection on chest radiographs: effect on human observers' performance in nodule detection

OBJECTIVE

The objective of our study was to compare the effect of dual-energy subtraction and bone suppression software alone and in combination with computer-aided detection (CAD) on the performance of human observers in lung nodule detection.

MATERIALS AND METHODS

One hundred one patients with from one to five lung nodules measuring 5-29 mm and 42 subjects with no nodules were retrospectively selected and randomized. Three independent radiologists marked suspicious-appearing lesions on the original chest radiographs, dual-energy subtraction images, and bone-suppressed images before and after postprocessing with CAD. Marks of the observers and CAD marks were compared with CT as the reference standard. Data were analyzed using nonparametric tests and the jackknife alternative free-response receiver operating characteristic (JAFROC) method.

RESULTS

Using dual-energy subtraction alone (p = 0.0198) or CAD alone (p = 0.0095) improved the detection rate compared with using the original conventional chest radiograph. The combination of bone suppression and CAD provided the highest sensitivity (51.6%) and the original nonenhanced conventional chest radiograph alone provided the lowest (46.9%; p = 0.0049). Dual-energy subtraction and bone suppression provided the same false-positive (p = 0.2702) and true-positive (p = 0.8451) rates. Up to 22.9% of lesions were found only by the CAD program and were missed by the readers. JAFROC showed no difference in the performance between modalities (p = 0.2742-0.5442).

CONCLUSION

Dual-energy subtraction and the electronic bone suppression program used in this study provided similar detection rates for pulmonary nodules. Additionally, CAD alone or combined with bone suppression can significantly improve the sensitivity of human observers for pulmonary nodule detection.

Improved differential diagnosis of breast masses on ultrasonographic images with a computer-aided diagnosis scheme for determining histological classifications

OBJECTIVES

A computer-aided diagnosis (CAD) scheme for determining histological classifications of breast masses is expected to be useful for clinicians in making a differential diagnosis. The purpose of this study was to evaluate the usefulness of using the CAD scheme on ultrasonographic images.

METHODS

The database consisted of 390 breast ultrasonographic images with masses. Three experienced clinicians independently provided subjective ratings on the likelihood of malignancy for each of the 390 masses. Fifty benign masses (25 cysts and 25 fibroadenomas) and 50 malignant masses (25 noninvasive ductal carcinomas and 25 invasive ductal carcinomas) were selected as unknown cases for an observer study based on a stratified randomization method with the ratings. The likelihood of the histological classification in each unknown case was evaluated by the CAD scheme with image features that clinicians commonly use for describing masses. In the observer study, seven observers provided their confidence levels regarding the malignancy of the unknown case before and after viewing the likelihood of the histological classification. The usefulness of the CAD scheme was evaluated with a multireader multicase receiver operating characteristic (ROC) analysis.

RESULTS

The areas under the ROC curves (AUCs) for all observers were improved by use of the CAD scheme. The average AUC increased from 0.716 without to 0.864 with the CAD scheme (P = .006).

CONCLUSION

The presentation of the likelihood of the histological classification evaluated by the CAD scheme improved the clinicians' performance and therefore would be useful in making a differential diagnosis of masses on ultrasonographic images.

Quantitative analysis of multiparametric prostate MR images: differentiation between prostate cancer and normal tissue and correlation with Gleason score--a computer-aided diagnosis development study

PURPOSE

To evaluate the potential utility of a number of parameters obtained at T2-weighted, diffusion-weighted, and dynamic contrast material-enhanced multiparametric magnetic resonance (MR) imaging for computer-aided diagnosis (CAD) of prostate cancer and assessment of cancer aggressiveness.

MATERIALS AND METHODS

In this institutional review board-approved HIPAA-compliant study, multiparametric MR images were acquired with an endorectal coil in 48 patients with prostate cancer (median age, 62.5 years; age range, 44-73 years) who subsequently underwent prostatectomy. A radiologist and a pathologist identified 104 regions of interest (ROIs) (61 cancer ROIs, 43 normal ROIs) based on correlation of histologic and MR findings. The 10th percentile and average apparent diffusion coefficient (ADC) values, T2-weighted signal intensity histogram skewness, and Tofts K(trans) were analyzed, both individually and combined, via linear discriminant analysis, with receiver operating characteristic curve analysis with area under the curve (AUC) as figure of merit, to distinguish cancer foci from normal foci. Spearman rank-order correlation (ρ) was calculated between cancer foci Gleason score (GS) and image features.

RESULTS

AUC (maximum likelihood estimate ± standard error) values in the differentiation of prostate cancer from normal foci of 10th percentile ADC, average ADC, T2-weighted skewness, and K(trans) were 0.92 ± 0.03, 0.89 ± 0.03, 0.86 ± 0.04, and 0.69 ± 0.04, respectively. The combination of 10th percentile ADC, average ADC, and T2-weighted skewness yielded an AUC value for the same task of 0.95 ± 0.02. GS correlated moderately with 10th percentile ADC (ρ = -0.34, P = .008), average ADC (ρ = -0.30, P = .02), and K(trans) (ρ = 0.38, P = .004).

CONCLUSION

The combination of 10th percentile ADC, average ADC, and T2-weighted skewness with CAD is promising in the differentiation of prostate cancer from normal tissue. ADC image features and K(trans) moderately correlate with GS.

Computer-aided diagnosis for detection of lacunar infarcts on MR images: ROC analysis of radiologists' performance

The purpose of this study was to retrospectively evaluate radiologist performance in detection of lacunar infarcts on T1- and T2-weighted images, without and with the use of a computer-aided diagnosis (CAD) scheme. Thirty T1-weighted and 30 T2-weighted MR images obtained from 30 patients were used for assessing observer performance. These images were acquired using the fast spin-echo sequence with a 1.5-T MR imaging scanner. The group included 15 patients (age range, 48-83 years; mean age, 67.2 years; 10 men and five women) with a lacunar infarct and 15 patients (age range, 39-76 years; mean age, 64.0 years; eight men and seven women) without lacunar infarcts. Nine radiologists participated in the study. The radiologists initially interpreted the T1- and T2-weighted images without and then with the use of CAD, which indicated their confidence levels regarding the presence (or absence) of lacunar infarcts and the most likely position of a lesion on each MR scan. The observers' performance without and with the computer output was evaluated by performing receiver operating characteristic analysis. For the nine radiologists, the mean area under the best-fit binormal receiver operating characteristic curve plotted for unit square values of radiologists who interpreted the images without and with the scheme were 0.891 and 0.937, respectively. The performance of the radiologists improved significantly when they used the computer output (p=0.032). The CAD scheme has potential to improve the accuracy of radiologists' performance in detection of lacunar infarcts.

Statistical methods for predicting mortality in patients diagnosed with acute pulmonary embolism

RATIONALE AND OBJECTIVES

Risk stratification in pulmonary embolism (PE) guides patient management. The purpose of this study was to develop and test novel mortality risk prediction models for subjects with acute PE diagnosed using computed tomographic pulmonary angiography in a large cohort with comprehensive clinical data.

MATERIALS AND METHODS

Retrospective analyses of 1596 consecutive subjects diagnosed with acute PE from a single, large, urban teaching hospital included two modern statistical methods to predict survival in patients with acute PE. Landmark analysis was used for 90-day mortality. Adaptive least absolute shrinkage and selection operator (aLASSO), a penalization method, was used to select variables important for prediction and to estimate model coefficients. Receiver-operating characteristic analysis was used to evaluate the resulting prediction rules.

RESULTS

Using 30-day all-cause mortality outcome, three of the 16 clinical risk factors (the presence of a known malignancy, coronary artery disease, and increased age) were associated with high risk, while subjects treated with anticoagulation had lower risk. For 90-day landmark mortality, subjects with recent operations had a lower risk for death. Both prediction rules developed using aLASSO performed well compared to standard logistic regression.

CONCLUSIONS

The aLASSO regression approach combined with landmark analysis provides a novel tool for large patient populations and can be applied for clinical risk stratification among subjects diagnosed with acute PE. After positive results on computed tomographic pulmonary angiography, the presence of a known malignancy, coronary artery disease, and advanced age increase 30-day mortality. Additional risk stratification can be simplified with these methods, and future work will place imaging-based prediction of mortality in perspective with other clinical data.

Diagnostic Role of Tumour Markers CEA, CA15-3, CA19-9 and CA125 in Lung Cancer

The aim of this study was to assess the diagnostic yield of the tumour markers carcinoembryonic antigen, carbohydrate antigen 15-3, carbohydrate antigen 19-9 and carbohydrate antigen 125, in serum and bronchoalveolar lavage fluid in a group of patients with bronchogenic carcinoma. Serum and bronchoalveolar lavage fluid samples were collected in a group of 90 patients with benign or malignant pulmonary diseases. After appropriate processing, tumour markers were determined by enzyme immunoassay. The diagnostic yields (sensitivity, specificity and predictive values) in each environment (serum and bronchoalveolar lavage fluid) were obtained by using "Receivers operating characteristic" curve. Determined individually, carcinoembryonic antigen, carbohydrate antigen 19-9 and carbohydrate antigen 125, showed the greatest diagnostic accuracy in bronchoalveolar lavage fluid. Carbohydrate antigen 15-3 did so in serum. Carcinoembryonic antigen was the most relevant marker in bronchoalveolar lavage fluid. For the factors evaluated in this study, determination of carcinoembryonic antigen, carbohydrate antigen 19-9 and carbohydrate antigen 125 in bronchoalveolar lavage fluid were clinically more useful markers in comparison with serum, although the latter may also be helpful in certain situations. Although there is no specific tumour marker for lung cancer, the combination of several can be used to diagnose most patients with lung cancer and also to rule out false positive and negative cases.

Comparison of Kupffer-phase Sonazoid-enhanced sonography and hepatobiliary-phase gadoxetic acid-enhanced magnetic resonance imaging of hepatocellular carcinoma and correlation with histologic grading

OBJECTIVES

To determine the relative wash-out of hepatocellular carcinomas and dysplastic nodules using Kupffer-phase sonography with Sonazoid (Daiichi-Sankyo, Tokyo, Japan) enhancement and hepatobiliary-phase gadoxetic acid-enhanced magnetic resonance imaging (MRI) in the evaluation of the histopathologic grades of individual nodules.

METHODS

This retrospective study included 66 consecutive patients with 78 histologically confirmed hepatocellular carcinomas and dysplastic nodules. In patients with carcinomas, 33 were well differentiated; 29 were moderately differentiated; and 11 were poorly differentiated; and there were 5 dysplastic nodules. All patients underwent both gadoxetic acid-enhanced MRI and Sonazoid-enhanced sonography. The interval between the two imaging examinations was less than 30 days. Six radiologists independently reviewed both images and rated the degree of relative wash-out between the tumorous and nontumorous areas on Kupffer- and hepatobiliary-phase images using a continuous rating scale. We compared these results with the histopathologic grade of each nodule, and the results were then analyzed with multireader multicase receiver operating characteristic analysis.

RESULTS

The average Kupffer-phase (P < .001) and hepatobiliary-phase (P = .004) rating scores increased as the carcinomas became less differentiated (Kruskal-Wallis test). The diagnostic accuracies of the average area under the receiver operating characteristic curve, which were estimated using the confidence levels of the relative wash-out of the Kupffer- and hepatobiliary-phase images, were 0.705 and 0.785 for dysplastic nodules versus well-, moderately, and poorly differentiated carcinomas (P = .517), 0.791 and 0.687 for dysplastic nodules and well-differentiated carcinomas versus moderately and poorly differentiated carcinomas (P = .093), and 0.871 and 0.716 for dysplastic nodules and well-and moderately differentiated carcinomas versus poorly differentiated carcinomas (P = .005), respectively.

CONCLUSIONS

Kupffer-phase Sonazoid-enhanced sonography and hepatobiliary-phase gadoxetic acid-enhanced MRI may be useful in estimating the histologic grade, although Kupffer-phase Sonazoid-enhanced sonography may be more accurate in distinguishing hepatocellular carcinomas, especially moderately and poorly differentiated types.

Evaluating imaging and computer-aided detection and diagnosis devices at the FDA

This report summarizes the Joint FDA-MIPS Workshop on Methods for the Evaluation of Imaging and Computer-Assist Devices. The purpose of the workshop was to gather information on the current state of the science and facilitate consensus development on statistical methods and study designs for the evaluation of imaging devices to support US Food and Drug Administration submissions. Additionally, participants expected to identify gaps in knowledge and unmet needs that should be addressed in future research. This summary is intended to document the topics that were discussed at the meeting and disseminate the lessons that have been learned through past studies of imaging and computer-aided detection and diagnosis device performance.

Sample size tables for computer-aided detection studies

OBJECTIVE

Calculating the sample size for a multireader, multicase study of readers' diagnostic accuracy is complicated. Studies in which patients can have multiple findings, as is common in many computer-aided detection (CAD) studies, are particularly challenging to design.

MATERIALS AND METHODS

We modified existing methods for sample size estimation for multireader, multicase studies to accommodate multiple findings on the same case. We use data from two large multireader, multicase CAD studies as ballpark estimates of parameter values.

RESULTS

Sample size tables are presented to provide an estimate of the number of patients and readers required for a multireader, multicase study with multiple findings per case; these estimates may be conservative for many CAD studies. Two figures can be used to adjust the number of readers when there is some data on the between-reader variability.

CONCLUSION

The sample size tables are useful in determining whether a proposed study is feasible with the available resources; however, it is important that investigators compute sample size for their particular study using any available pilot data.

Small lung cancers: improved detection by use of bone suppression imaging--comparison with dual-energy subtraction chest radiography

PURPOSE

To determine whether use of bone suppression (BS) imaging, used together with a standard radiograph, could improve radiologists' performance for detection of small lung cancers compared with use of standard chest radiographs alone and whether BS imaging would provide accuracy equivalent to that of dual-energy subtraction (DES) radiography.

MATERIALS AND METHODS

Institutional review board approval was obtained. The requirement for informed consent was waived. The study was HIPAA compliant. Standard and DES chest radiographs of 50 patients with 55 confirmed primary nodular cancers (mean diameter, 20 mm) as well as 30 patients without cancers were included in the observer study. A new BS imaging processing system that can suppress the conspicuity of bones was applied to the standard radiographs to create corresponding BS images. Ten observers, including six experienced radiologists and four radiology residents, indicated their confidence levels regarding the presence or absence of a lung cancer for each lung, first by using a standard image, then a BS image, and finally DES soft-tissue and bone images. Receiver operating characteristic (ROC) analysis was used to evaluate observer performance.

RESULTS

The average area under the ROC curve (AUC) for all observers was significantly improved from 0.807 to 0.867 with BS imaging and to 0.916 with DES (both P < .001). The average AUC for the six experienced radiologists was significantly improved from 0.846 with standard images to 0.894 with BS images (P < .001) and from 0.894 to 0.945 with DES images (P = .001).

CONCLUSION

Use of BS imaging together with a standard radiograph can improve radiologists' accuracy for detection of small lung cancers on chest radiographs. Further improvements can be achieved by use of DES radiography but with the requirement for special equipment and a potential small increase in radiation dose.

Sensitivity and specificity of a CAD solution for lung nodule detection on chest radiograph with CTA correlation

The objective of this research was to determine the sensitivity and specificity of a commercially available computer-aided detection (CAD) system for detection of lung nodule on posterior-anterior (PA) chest radiograph in a varied patient population who are referred to computed tomographic angiogram (CTA) of the chest as a reference standard. Patients who had a PA chest radiograph with concomitant CTA of the chest were included in this retrospective study. The PA chest radiograph was analyzed by a CAD device, and results were recorded. A qualitative assessment of the CAD results was performed using a 5-point Likert scale. The CTA was then reviewed to determine if there were correlative nodules. The presence of a correlative nodule between 0.5 cm and 1.5 cm was considered a positive result. The baseline sensitivity of the system was determined to be 0.707 (95% CI = 0.52-0.86), with a specificity of 0.50 (95% CI = 0.38-0.76). Positive predictive value was 0.30 (95% CI = 0.24-0.49), with a negative predictive value of 0.858 (95% CI = 0.82-0.95), and accuracy of 0.555 (95% CI = 0.40-0.66). When excluding nodules that were qualitatively determined by a thoracic radiologist to be false positives, the specificity was 0.781 (95% CI = 0.764-0.839), the positive predictive value was 0.564 (95% CI = 0.491-0.654), the negative predictive value was 0.829 (95% CI = 0.819-0.878), and the accuracy was 0.737 (95% CI = 0.721-0.801). The use of CAD for lung nodule detection on chest radiograph, when used in conjunction with an experienced radiologist, has a very good sensitivity, specificity, and accuracy.

Usefulness of computerized method for lung nodule detection in digital chest radiographs using temporal subtraction images

RATIONALE AND OBJECTIVES

The aim of this study was to evaluate the usefulness of a novel computerized method for lung nodule detection on digital chest radiographs using temporal subtraction images.

MATERIALS AND METHODS

To significantly reduce the number of false-positive results while maintaining high sensitivity, temporal subtraction images, which can enhance interval changes on sequential chest radiographs, were used. Fifty-one cases with lung nodules <3 cm and 51 cases without lung nodules were selected for an observer performance test. Twelve radiologists participated in this observer performance test. The radiologists' performance was evaluated using receiver-operating characteristic analysis, on a continuous rating scale. To estimate the numbers of cases affected beneficially and those affected detrimentally using this computerized method, the computer output was assumed to have an effect on an observer's diagnosis when there was a difference in rating score of ≥30% between the first and second ratings.

RESULTS

The average area under the curve for all radiologists increased significantly from 0.849 to 0.950 with the computerized method (P < .001). The mean number of cases affected beneficially was significantly higher than that of cases affected detrimentally (8.92 vs 1.25, P < .001).

CONCLUSIONS

The novel computerized method using temporal subtraction images would be useful in detecting lung nodules on digital chest radiographs.

Optimizing analysis, visualization, and navigation of large image data sets: one 5000-section CT scan can ruin your whole day

The technology revolution in image acquisition, instrumentation, and methods has resulted in vast data sets that far outstrip the human observers' ability to view, digest, and interpret modern medical images by using traditional methods. This may require a paradigm shift in the radiologic interpretation process. As human observers, radiologists must search for, detect, and interpret targets. Potential interventions should be based on an understanding of human perceptual and attentional abilities and limitations. New technologies and tools already in use in other fields can be adapted to the health care environment to improve medical image analysis, visualization, and navigation through large data sets. This historical psychophysical and technical review touches on a broad range of disciplines but focuses mainly on the analysis, visualization, and navigation of image data performed during the interpretive process. Advanced postprocessing, including three-dimensional image display, multimodality image fusion, quantitative measures, and incorporation of innovative human-machine interfaces, will likely be the future. Successful new paradigms will integrate image and nonimage data, incorporate workflow considerations, and be informed by evidence-based practices. This overview is meant to heighten the awareness of the complexities and limitations of how radiologists interact with images, particularly the large image sets generated today. Also addressed is how human-machine interface and informatics technologies could combine to transform the interpretation process in the future to achieve safer and better quality care for patients and a more efficient and effective work environment for radiologists.

SUPPLEMENTAL MATERIAL

http://radiology.rsna.org/lookup/suppl/doi:10.1148/radiol.11091276/-/DC1.

A comparison of follow-up recommendations by chest radiologists, general radiologists, and pulmonologists using computer-aided detection to assess radiographs for actionable pulmonary nodules

OBJECTIVE

The primary objective of our study was to compare the effect of a chest radiography computer-aided detection (CAD) system on the follow-up recommendations of chest radiologists, general radiologists, and pulmonologists.

MATERIALS AND METHODS

A chest radiography CAD system (RapidScreen 1.1) that has been approved by the U.S. Food and Drug Administration (FDA) and a second-generation version of the system (OnGuard 3.0) not yet approved by the FDA were applied to single frontal radiographs of 200 patients at high risk for lung cancer. One hundred patients had actionable nodules (mean size, 16.9 mm) and 100 patients did not. Six chest radiologists, six general radiologists, and six pulmonologists independently interpreted each image first without CAD and then with CAD during blinded reading sessions. The frequency with which readers correctly referred patients for follow-up tests was measured. Differential effects based on nodule size, shape, location, density, and subtlety were tested with multiplevariable logistic regression.

RESULTS

For patients without actionable lesions, pulmonologists showed an increase in their recommendations for follow-up from 0.46 unaided to 0.52 with CAD (p = 0.001), whereas chest and general radiologists had much lower average rates and were not affected by CAD's false marks (0.26 without CAD vs 0.25 with RapidScreen 1.1 and 0.26 with OnGuard 3.0, p ≥ 0.734). CAD improved all readers' detection of moderately subtle lesions (p = 0.013) but did not significantly increase follow-up rates overall for patients with actionable nodules (0.63 unaided vs 0.63 with RapidScreen 1.1, p = 0.795; and 0.63 unaided vs 0.64 with OnGuard 3.0, p = 0.187).

CONCLUSION

The effect of CAD on readers' clinical decisions varies depending on the training of the reader. CAD did not improve the performance of chest or general radiologists. Nonradiologists are particularly vulnerable to CAD's false-positive marks.