Pulmonary nodules: estimation of malignancy at thin-section helical CT--effect of computer-aided diagnosis on performance of radiologists

Evaluating ChatGPT-4V in chest CT diagnostics: a critical image interpretation assessment

PURPOSE

To assess the diagnostic accuracy of ChatGPT-4V in interpreting a set of four chest CT slices for each case of COVID-19, non-small cell lung cancer (NSCLC), and control cases, thereby evaluating its potential as an AI tool in radiological diagnostics.

MATERIALS AND METHODS

In this retrospective study, 60 CT scans from The Cancer Imaging Archive, covering COVID-19, NSCLC, and control cases were analyzed using ChatGPT-4V. A radiologist selected four CT slices from each scan for evaluation. ChatGPT-4V's interpretations were compared against the gold standard diagnoses and assessed by two radiologists. Statistical analyses focused on accuracy, sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV), along with an examination of the impact of pathology location and lobe involvement.

RESULTS

ChatGPT-4V showed an overall diagnostic accuracy of 56.76%. For NSCLC, sensitivity was 27.27% and specificity was 60.47%. In COVID-19 detection, sensitivity was 13.64% and specificity of 64.29%. For control cases, the sensitivity was 31.82%, with a specificity of 95.24%. The highest sensitivity (83.33%) was observed in cases involving all lung lobes. The chi-squared statistical analysis indicated significant differences in Sensitivity across categories and in relation to the location and lobar involvement of pathologies.

CONCLUSION

ChatGPT-4V demonstrated variable diagnostic performance in chest CT interpretation, with notable proficiency in specific scenarios. This underscores the challenges of cross-modal AI models like ChatGPT-4V in radiology, pointing toward significant areas for improvement to ensure dependability. The study emphasizes the importance of enhancing these models for broader, more reliable medical use.

Visual Knowledge Domain of Artificial Intelligence in Computed Tomography: A Review Based on Bibliometric Analysis

ABSTRACT

Artificial intelligence (AI)-assisted medical imaging technology is a new research area of great interest that has developed rapidly over the last decade. However, there has been no bibliometric analysis of published studies in this field. The present review focuses on AI-related studies on computed tomography imaging in the Web of Science database and uses CiteSpace and VOSviewer to generate a knowledge map and conduct the basic information analysis, co-word analysis, and co-citation analysis. A total of 7265 documents were included and the number of documents published had an overall upward trend. Scholars from the United States and China have made outstanding achievements, and there is a general lack of extensive cooperation in this field. In recent years, the research areas of great interest and difficulty have been the optimization and upgrading of algorithms, and the application of theoretical models to practical clinical applications. This review will help researchers understand the developments, research areas of great interest, and research frontiers in this field and provide reference and guidance for future studies.

Deep-learning for automated detection of MSU deposits on DECT: evaluating impact on efficiency and reader confidence

Introduction

Dual-energy CT (DECT) is a non-invasive way to determine the presence of monosodium urate (MSU) crystals in the workup of gout. Color-coding distinguishes MSU from calcium following material decomposition and post-processing. Manually identifying these foci (most commonly labeled green) is tedious, and an automated detection system could streamline the process. This study aims to evaluate the impact of a deep-learning (DL) algorithm developed for detecting green pixelations on DECT on reader time, accuracy, and confidence.

Methods

We collected a sample of positive and negative DECTs, reviewed twice-once with and once without the DL tool-with a 2-week washout period. An attending musculoskeletal radiologist and a fellow separately reviewed the cases, simulating clinical workflow. Metrics such as time taken, confidence in diagnosis, and the tool's helpfulness were recorded and statistically analyzed.

Results

We included thirty DECTs from different patients. The DL tool significantly reduced the reading time for the trainee radiologist (p = 0.02), but not for the attending radiologist (p = 0.15). Diagnostic confidence remained unchanged for both (p = 0.45). However, the DL model identified tiny MSU deposits that led to a change in diagnosis in two cases for the in-training radiologist and one case for the attending radiologist. In 3/3 of these cases, the diagnosis was correct when using DL.

Conclusions

The implementation of the developed DL model slightly reduced reading time for our less experienced reader and led to improved diagnostic accuracy. There was no statistically significant difference in diagnostic confidence when studies were interpreted without and with the DL model.

Robust and efficient abdominal CT segmentation using shape constrained multi-scale attention network

PURPOSE

Although many deep learning-based abdominal multi-organ segmentation networks have been proposed, the various intensity distributions and organ shapes of the CT images from multi-center, multi-phase with various diseases introduce new challenges for robust abdominal CT segmentation. To achieve robust and efficient abdominal multi-organ segmentation, a new two-stage method is presented in this study.

METHODS

A binary segmentation network is used for coarse localization, followed by a multi-scale attention network for the fine segmentation of liver, kidney, spleen, and pancreas. To constrain the organ shapes produced by the fine segmentation network, an additional network is pre-trained to learn the shape features of the organs with serious diseases and then employed to constrain the training of the fine segmentation network.

RESULTS

The performance of the presented segmentation method was extensively evaluated on the multi-center data set from the Fast and Low GPU Memory Abdominal oRgan sEgmentation (FLARE) challenge, which was held in conjunction with International Conference on Medical Image Computing and Computer Assisted Intervention (MICCAI) 2021. Dice Similarity Coefficient (DSC) and Normalized Surface Dice (NSD) were calculated to quantitatively evaluate the segmentation accuracy and efficiency. An average DSC and NSD of 83.7% and 64.4% were achieved, and our method finally won the second place among more than 90 participating teams.

CONCLUSIONS

The evaluation results on the public challenge demonstrate that our method shows promising performance in robustness and efficiency, which may promote the clinical application of the automatic abdominal multi-organ segmentation.

Performance analyses of five neural network classifiers on nodule classification in lung CT images using WEKA: a comparative study

In this report, we are presenting our work on performance analyses of five different neural network classifiers viz. MLP, DL4JMLP, logistic regression, SGD and simple logistic classifier in lung nodule detection using WEKA interface. To the best of our knowledge, this report demonstrates first use of WEKA for comparative performance analyses of neural network classifiers in identifying lung nodules from lung CT-images. A total of 624 handcrafted features from 52 numbers of lung CT-images collected randomly from Lung Image Database Consortium (LIDC) were fed into WEKA to evaluate the performances of the classifiers under four different categories of computation. Performances of the classifiers were observed in terms of 11 important parameters viz. accuracy, kappa statistic, root mean squared error, TPR, FPR, precision, sensitivity, F-measurement, MCC, ROC area and PRC area. Results show 86.53%, 77.77%, 55.55%, 94.44% & 88.88% accuracy as well as 0.91, 0.86, 0.68, 0.91 & 0.93 ROC area for MLP, DL4JMLP, logistic, SGD and simple logistic classifier respectively at tenfold cross-validation by taking 66% of the data set for training and 34% for testing and validation purpose. SGDClassifier has been found the best performing followed by simple logistic classifier for the purpose.

Fast and Low-GPU-memory abdomen CT organ segmentation: The FLARE challenge

Automatic segmentation of abdominal organs in CT scans plays an important role in clinical practice. However, most existing benchmarks and datasets only focus on segmentation accuracy, while the model efficiency and its accuracy on the testing cases from different medical centers have not been evaluated. To comprehensively benchmark abdominal organ segmentation methods, we organized the first Fast and Low GPU memory Abdominal oRgan sEgmentation (FLARE) challenge, where the segmentation methods were encouraged to achieve high accuracy on the testing cases from different medical centers, fast inference speed, and low GPU memory consumption, simultaneously. The winning method surpassed the existing state-of-the-art method, achieving a 19× faster inference speed and reducing the GPU memory consumption by 60% with comparable accuracy. We provide a summary of the top methods, make their code and Docker containers publicly available, and give practical suggestions on building accurate and efficient abdominal organ segmentation models. The FLARE challenge remains open for future submissions through a live platform for benchmarking further methodology developments at https://flare.grand-challenge.org/.

Radiologists with and without deep learning-based computer-aided diagnosis: comparison of performance and interobserver agreement for characterizing and diagnosing pulmonary nodules/masses

OBJECTIVES

To compare the performance of radiologists in characterizing and diagnosing pulmonary nodules/masses with and without deep learning (DL)-based computer-aided diagnosis (CAD).

METHODS

We studied a total of 101 nodules/masses detected on CT performed between January and March 2018 at Osaka University Hospital (malignancy: 55 cases). SYNAPSE SAI Viewer V1.4 was used to analyze the nodules/masses. In total, 15 independent radiologists were grouped (n = 5 each) according to their experience: L (< 3 years), M (3-5 years), and H (> 5 years). The likelihoods of 15 characteristics, such as cavitation and calcification, and the diagnosis (malignancy) were evaluated by each radiologist with and without CAD, and the assessment time was recorded. The AUCs compared with the reference standard set by two board-certified chest radiologists were analyzed following the multi-reader multi-case method. Furthermore, interobserver agreement was compared using intraclass correlation coefficients (ICCs).

RESULTS

The AUCs for ill-defined boundary, irregular margin, irregular shape, calcification, pleural contact, and malignancy in all 15 radiologists, irregular margin and irregular shape in L and ill-defined boundary and irregular margin in M improved significantly (p < 0.05); no significant improvements were found in H. L showed the greatest increase in the AUC for malignancy (not significant). The ICCs improved in all groups and for nearly all items. The median assessment time was not prolonged by CAD.

CONCLUSIONS

DL-based CAD helps radiologists, particularly those with < 5 years of experience, to accurately characterize and diagnose pulmonary nodules/masses, and improves the reproducibility of findings among radiologists.

KEY POINTS

• Deep learning-based computer-aided diagnosis improves the accuracy of characterizing nodules/masses and diagnosing malignancy, particularly by radiologists with < 5 years of experience. • Computer-aided diagnosis increases not only the accuracy but also the reproducibility of the findings across radiologists.

The Added Effect of Artificial Intelligence on Physicians' Performance in Detecting Thoracic Pathologies on CT and Chest X-ray: A Systematic Review

Our systematic review investigated the additional effect of artificial intelligence-based devices on human observers when diagnosing and/or detecting thoracic pathologies using different diagnostic imaging modalities, such as chest X-ray and CT. Peer-reviewed, original research articles from EMBASE, PubMed, Cochrane library, SCOPUS, and Web of Science were retrieved. Included articles were published within the last 20 years and used a device based on artificial intelligence (AI) technology to detect or diagnose pulmonary findings. The AI-based device had to be used in an observer test where the performance of human observers with and without addition of the device was measured as sensitivity, specificity, accuracy, AUC, or time spent on image reading. A total of 38 studies were included for final assessment. The quality assessment tool for diagnostic accuracy studies (QUADAS-2) was used for bias assessment. The average sensitivity increased from 67.8% to 74.6%; specificity from 82.2% to 85.4%; accuracy from 75.4% to 81.7%; and Area Under the ROC Curve (AUC) from 0.75 to 0.80. Generally, a faster reading time was reported when radiologists were aided by AI-based devices. Our systematic review showed that performance generally improved for the physicians when assisted by AI-based devices compared to unaided interpretation.

Estimation of malignancy of pulmonary nodules at CT scans: Effect of computer-aided diagnosis on diagnostic performance of radiologists

OBJECTIVES

To develop a computer-aided diagnosis (CAD) system for distinguishing malignant from benign pulmonary nodules on computed tomography (CT) scans, and to assess whether the diagnostic performance of radiologists with different experiences can be improved with the assistant of CAD.

MATERIALS AND METHODS

A total of 857 malignant nodules from 601 patients and 426 benign nodules from 278 patients were retrospectively collected from four hospitals. In this study, we exploited convolutional neural network in the framework of deep learning to classify whether a nodule was benign or malignant. A total of 745 malignant nodules and 370 benign nodules were used as the training data of our CAD system. The remaining 112 malignant nodules and 56 benign nodules were used as the test data. The participants were two senior chest radiologists, two secondary chest radiologists, and two junior radiology residents. The readers estimated the likelihood of malignancy of pulmonary nodules first without and then with CAD output. Receiver-operating characteristic (ROC) curve was used to evaluate readers' diagnostic performance.

RESULTS

When a threshold level of 58% was used to estimate the likelihood of malignancy, the sensitivity, specificity, and diagnostic accuracy values of our CAD scheme alone were 93.8%, 83.9%, and 90.5%, respectively. For all six readers, the mean area under the ROC curve (A_z ) values without and with CAD system were 0.913 and 0.938, respectively. For each reader, there is a large difference in A_z values that assessed without and with CAD system. With CAD output, the readers made correct changes an average of 15.7 times and incorrect changes an average of 2 times.

CONCLUSION

Our CAD system significantly improved the diagnostic performance of readers regardless of their experience levels for assessment of the likelihood of malignancy of pulmonary nodules.

Classification of lung nodules based on CT images using squeeze-and-excitation network and aggregated residual transformations

Lung cancer is pointed as a leading cause of cancer death worldwide. Early lung nodule diagnosis has great significance for treating lung cancer and increasing patient survival. In this paper, we present a novel method to classify the malignant from benign lung nodules based on CT images using squeeze-and-excitation network and aggregated residual transformations (SE-ResNeXt). The state-of-the-art SE-ResNeXt module, which integrates the advantages of SENet for feature recalibration and ResNeXt for feature reuse, has great ability in boosting feature discriminability on imaging pattern recognition. The method is evaluated on the public available LUng Nodule Analysis 2016 (LUNA16) database with 1004 (450 malignant and 554 benign) nodules, achieving an area under the receiver operating characteristic curve (AUC) of 0. 9563 and accuracy of 91.67%. The promising results demonstrate that our method has strong robustness in the classification of nodules. The method has the potential to help radiologists better interpret diagnostic data and differentiate the benign from malignant lung nodules on CT images in clinical practice. To our best knowledge, the effectiveness of SE-ResNeXt on lung nodule classification has not been extensively explored.

Risk-Aware Machine Learning Classifier for Skin Lesion Diagnosis

Knowing when a machine learning system is not confident about its prediction is crucial in medical domains where safety is critical. Ideally, a machine learning algorithm should make a prediction only when it is highly certain about its competency, and refer the case to physicians otherwise. In this paper, we investigate how Bayesian deep learning can improve the performance of the machine-physician team in the skin lesion classification task. We used the publicly available HAM10000 dataset, which includes samples from seven common skin lesion categories: Melanoma (MEL), Melanocytic Nevi (NV), Basal Cell Carcinoma (BCC), Actinic Keratoses and Intraepithelial Carcinoma (AKIEC), Benign Keratosis (BKL), Dermatofibroma (DF), and Vascular (VASC) lesions. Our experimental results show that Bayesian deep networks can boost the diagnostic performance of the standard DenseNet-169 model from 81.35% to 83.59% without incurring additional parameters or heavy computation. More importantly, a hybrid physician-machine workflow reaches a classification accuracy of 90 % while only referring 35 % of the cases to physicians. The findings are expected to generalize to other medical diagnosis applications. We believe that the availability of risk-aware machine learning methods will enable a wider adoption of machine learning technology in clinical settings.

Integration of fully automated computer-aided pulmonary nodule detection into CT pulmonary angiography studies in the emergency department: effect on workflow and diagnostic accuracy

PURPOSE

To assess the feasibility of implementing fully automated computer-aided diagnosis (CAD) for detection of pulmonary nodules on CT pulmonary angiography (CTPA) studies in emergency setting.

MATERIALS AND METHODS

CTPA of 48 emergency patients was retrospectively reviewed. Fully automated CAD nodule detection was performed at the scanner and results were automatically submitted to PACS. A third-year radiology resident (RAD1) and a cardiothoracic radiologist with 6 years' experience (RAD2) reviewed the scans independently to detect pulmonary nodules in two different sessions 8 weeks apart: session 1, CAD was reviewed first and then all images were reviewed; session 2, CAD was reviewed last after all images were reviewed. Time spent by RAD to evaluate image sets was measured for each case. Fisher's exact test and t test were used.

RESULTS

There were 17 male and 31 female patients with mean ± SD age of 48.7 ± 16.4 years. Using CAD at the beginning was associated with lower average reading time for both readers. However, difference in reading time did not reach statistical significance for RAD1 (RAD1 94.6 s vs. 102.7 s, P > 0.05; RAD2 61.1 s vs. 76.5 s, P < 0.05). Using CAD at the end significantly increased rate of RAD1 and RAD2 nodule detection by 34% (2.52 vs. 2.12 nodule/scan, P < 0.05) and 27% (2.23 vs. 1.81 nodule/scan, P < 0.05), respectively.

CONCLUSION

Routine utilization of CAD in emergency setting is feasible and can improve detection rate of pulmonary nodules significantly. Different methods of incorporating CAD in detecting pulmonary nodules can improve both the rate of detection and interpretation speed.

Incorporating automatically learned pulmonary nodule attributes into a convolutional neural network to improve accuracy of benign-malignant nodule classification

Existing deep-learning-based pulmonary nodule classification models usually use images and benign-malignant labels as inputs for training. Image attributes of the nodules, as human-nameable high-level semantic labels, are rarely used to build a convolutional neural network (CNN). In this paper, a new method is proposed to combine the advantages of two classifications, which are pulmonary nodule benign-malignant classification and pulmonary nodule image attributes classification, into a deep learning network to improve the accuracy of pulmonary nodule classification. For this purpose, a unique 3D CNN is built to learn image attribute and benign-malignant classification simultaneously. A novel loss function is designed to balance the influence of two different kinds of classifications. The CNN is trained by a publicly available lung image database consortium (LIDC) dataset and is tested by a cross-validation method to predict the risk of a pulmonary nodule being malignant. This proposed method generates the accuracy of 91.47%, which is better than many existing models. Experimental findings show that if the CNN is built properly, the nodule attributes classification and benign-malignant classification can benefit from each other. By using nodule attribute learning as a control factor in a deep learning scheme, the accuracy of pulmonary nodule classification can be significantly improved by using a deep learning scheme.

Automatic inference model construction for computer-aided diagnosis of lung nodule: Explanation adequacy, inference accuracy, and experts' knowledge

We aimed to describe the development of an inference model for computer-aided diagnosis of lung nodules that could provide valid reasoning for any inferences, thereby improving the interpretability and performance of the system. An automatic construction method was used that considered explanation adequacy and inference accuracy. In addition, we evaluated the usefulness of prior experts’ (radiologists’) knowledge while constructing the models. In total, 179 patients with lung nodules were included and divided into 79 and 100 cases for training and test data, respectively. F-measure and accuracy were used to assess explanation adequacy and inference accuracy, respectively. For F-measure, reasons were defined as proper subsets of Evidence that had a strong influence on the inference result. The inference models were automatically constructed using the Bayesian network and Markov chain Monte Carlo methods, selecting only those models that met the predefined criteria. During model constructions, we examined the effect of including radiologist’s knowledge in the initial Bayesian network models. Performance of the best models in terms of F-measure, accuracy, and evaluation metric were as follows: 0.411, 72.0%, and 0.566, respectively, with prior knowledge, and 0.274, 65.0%, and 0.462, respectively, without prior knowledge. The best models with prior knowledge were then subjectively and independently evaluated by two radiologists using a 5-point scale, with 5, 3, and 1 representing beneficial, appropriate, and detrimental, respectively. The average scores by the two radiologists were 3.97 and 3.76 for the test data, indicating that the proposed computer-aided diagnosis system was acceptable to them. In conclusion, the proposed method incorporating radiologists’ knowledge could help in eliminating radiologists’ distrust of computer-aided diagnosis and improving its performance.

Characterization of Pulmonary Nodules Based on Features of Margin Sharpness and Texture

Lung cancer is the leading cause of cancer-related deaths in the world, and one of its manifestations occurs with the appearance of pulmonary nodules. The classification of pulmonary nodules may be a complex task to specialists due to temporal, subjective, and qualitative aspects. Therefore, it is important to integrate computational tools to the early pulmonary nodule classification process, since they have the potential to characterize objectively and quantitatively the lesions. In this context, the goal of this work is to perform the classification of pulmonary nodules based on image features of texture and margin sharpness. Computed tomography scans were obtained from a publicly available image database. Texture attributes were extracted from a co-occurrence matrix obtained from the nodule volume. Margin sharpness attributes were extracted from perpendicular lines drawn over the borders on all nodule slices. Feature selection was performed by different algorithms. Classification was performed by several machine learning classifiers and assessed by the area under the receiver operating characteristic curve, sensitivity, specificity, and accuracy. Highest classification performance was obtained by a random forest algorithm with all 48 extracted features. However, a decision tree using only two selected features obtained statistically equivalent performance on sensitivity and specificity.

A deep learning framework for supporting the classification of breast lesions in ultrasound images

In this research, we exploited the deep learning framework to differentiate the distinctive types of lesions and nodules in breast acquired with ultrasound imaging. A biopsy-proven benchmarking dataset was built from 5151 patients cases containing a total of 7408 ultrasound breast images, representative of semi-automatically segmented lesions associated with masses. The dataset comprised 4254 benign and 3154 malignant lesions. The developed method includes histogram equalization, image cropping and margin augmentation. The GoogLeNet convolutionary neural network was trained to the database to differentiate benign and malignant tumors. The networks were trained on the data with augmentation and the data without augmentation. Both of them showed an area under the curve of over 0.9. The networks showed an accuracy of about 0.9 (90%), a sensitivity of 0.86 and a specificity of 0.96. Although target regions of interest (ROIs) were selected by radiologists, meaning that radiologists still have to point out the location of the ROI, the classification of malignant lesions showed promising results. If this method is used by radiologists in clinical situations it can classify malignant lesions in a short time and support the diagnosis of radiologists in discriminating malignant lesions. Therefore, the proposed method can work in tandem with human radiologists to improve performance, which is a fundamental purpose of computer-aided diagnosis.

Automatic Categorization and Scoring of Solid, Part-Solid and Non-Solid Pulmonary Nodules in CT Images with Convolutional Neural Network

We present a computer-aided diagnosis system (CADx) for the automatic categorization of solid, part-solid and non-solid nodules in pulmonary computerized tomography images using a Convolutional Neural Network (CNN). Provided with only a two-dimensional region of interest (ROI) surrounding each nodule, our CNN automatically reasons from image context to discover informative computational features. As a result, no image segmentation processing is needed for further analysis of nodule attenuation, allowing our system to avoid potential errors caused by inaccurate image processing. We implemented two computerized texture analysis schemes, classification and regression, to automatically categorize solid, part-solid and non-solid nodules in CT scans, with hierarchical features in each case learned directly by the CNN model. To show the effectiveness of our CNN-based CADx, an established method based on histogram analysis (HIST) was implemented for comparison. The experimental results show significant performance improvement by the CNN model over HIST in both classification and regression tasks, yielding nodule classification and rating performance concordant with those of practicing radiologists. Adoption of CNN-based CADx systems may reduce the inter-observer variation among screening radiologists and provide a quantitative reference for further nodule analysis.

Three-dimensional substructure measurements for the differential diagnosis of ground glass nodules

BACKGROUND

We analyzed the differences between maximum and peak computed tomography (CT) numbers (M-P), respectively representing the densities of the solid center and the main periphery of ground-glass nodules (GGNs), and the average change in M-P velocity (V(M-P)) during follow-up to differentiate between pre-invasive (PIA) and invasive adenocarcinoma (IAC).

METHODS

Data of 102 patients were retrospectively collected and analyzed in our study including 43 PIAs and 59 IACs. Diameters, total volumes, and the maximum and peak CT numbers in CT number histograms were measured and followed for at least 3 months. This study was registered retrospectively.

RESULTS

The M-P values for IACs were higher than those for PIAs (p = 0.001), with an area under the curve (AUC) of 0.810 and a threshold of 489.5 Hounsfield units (HU) in ROC analysis. The V(M-P) values for IACs were smaller than those for PIAs (p = 0.04), with an AUC of 0.805 and a threshold of 11.01 HU/day.

CONCLUSIONS

M-P and V(M-P) values may help distinguish IACs from PIAs by representing the changes in the sub-structural densities of GGNs during follow-up.

A study of computer-aided diagnosis for pulmonary nodule: comparison between classification accuracies using calculated image features and imaging findings annotated by radiologists

PURPOSE

In our previous study, we developed a computer-aided diagnosis (CADx) system using imaging findings annotated by radiologists. The system, however, requires radiologists to input many imaging findings. In order to reduce such an interaction of radiologists, we further developed a CADx system using derived imaging findings based on calculated image features, in which the system only requires few user operations. The purpose of this study is to check whether calculated image features (CFT) or derived imaging findings (DFD) can represent information in imaging findings annotated by radiologists (AFD).

METHODS

We calculate 2282 image features and derive 39 imaging findings by using information on a nodule position and its type (solid or ground-glass). These image features are categorized into shape features, texture features and imaging findings-specific features. Each imaging finding is derived based on each corresponding classifier using random forest. To check whether CFT or DFD can represent information in AFD, under an assumption that the accuracies of classifiers are the same if information included in input is the same, we constructed classifiers by using various types of information (CTT, DFD and AFD) and compared accuracies on an inferred diagnosis of a nodule. We employ SVM with RBF kernel as classifier to infer a diagnosis name.

RESULTS

Accuracies of classifiers using DFD, CFT, AFD and CFT [Formula: see text] AFD were 0.613, 0.577, 0.773 and 0.790, respectively. Concordance rates between DFD and AFD of shape findings, texture findings and surrounding findings were 0.644, 0.871 and 0.768, respectively.

CONCLUSIONS

The results suggest that CFT and AFD are similar information and CFT represent only a portion of AFD. Particularly, CFT did not contain shape information in AFD. In order to decrease an interaction of radiologists, a development of a method which overcomes these problems is necessary.

Technical Note: Image filtering to make computer-aided detection robust to image reconstruction kernel choice in lung cancer CT screening

PURPOSE

In lung cancer computed tomography (CT) screening, the performance of a computer-aided detection (CAD) system depends on the selection of the image reconstruction kernel. To reduce this dependence on reconstruction kernels, the authors propose a novel application of an image filtering method previously proposed by their group.

METHODS

The proposed filtering process uses the ratio of modulation transfer functions (MTFs) of two reconstruction kernels as a filtering function in the spatial-frequency domain. This method is referred to as MTFratio filtering. Test image data were obtained from CT screening scans of 67 subjects who each had one nodule. Images were reconstructed using two kernels: fSTD (for standard lung imaging) and fSHARP (for sharp edge-enhancement lung imaging). The MTFratio filtering was implemented using the MTFs measured for those kernels and was applied to the reconstructed fSHARP images to obtain images that were similar to the fSTD images. A mean filter and a median filter were applied (separately) for comparison. All reconstructed and filtered images were processed using their prototype CAD system.

RESULTS

The MTFratio filtered images showed excellent agreement with the fSTD images. The standard deviation for the difference between these images was very small, ∼6.0 Hounsfield units (HU). However, the mean and median filtered images showed larger differences of ∼48.1 and ∼57.9 HU from the fSTD images, respectively. The free-response receiver operating characteristic (FROC) curve for the fSHARP images indicated poorer performance compared with the FROC curve for the fSTD images. The FROC curve for the MTFratio filtered images was equivalent to the curve for the fSTD images. However, this similarity was not achieved by using the mean filter or median filter.

CONCLUSIONS

The accuracy of MTFratio image filtering was verified and the method was demonstrated to be effective for reducing the kernel dependence of CAD performance.

Automatic Scoring of Multiple Semantic Attributes With Multi-Task Feature Leverage: A Study on Pulmonary Nodules in CT Images

The gap between the computational and semantic features is the one of major factors that bottlenecks the computer-aided diagnosis (CAD) performance from clinical usage. To bridge this gap, we exploit three multi-task learning (MTL) schemes to leverage heterogeneous computational features derived from deep learning models of stacked denoising autoencoder (SDAE) and convolutional neural network (CNN), as well as hand-crafted Haar-like and HoG features, for the description of 9 semantic features for lung nodules in CT images. We regard that there may exist relations among the semantic features of "spiculation", "texture", "margin", etc., that can be explored with the MTL. The Lung Image Database Consortium (LIDC) data is adopted in this study for the rich annotation resources. The LIDC nodules were quantitatively scored w.r.t. 9 semantic features from 12 radiologists of several institutes in U.S.A. By treating each semantic feature as an individual task, the MTL schemes select and map the heterogeneous computational features toward the radiologists' ratings with cross validation evaluation schemes on the randomly selected 2400 nodules from the LIDC dataset. The experimental results suggest that the predicted semantic scores from the three MTL schemes are closer to the radiologists' ratings than the scores from single-task LASSO and elastic net regression methods. The proposed semantic attribute scoring scheme may provide richer quantitative assessments of nodules for better support of diagnostic decision and management. Meanwhile, the capability of the automatic association of medical image contents with the clinical semantic terms by our method may also assist the development of medical search engine.

Computer-aided Diagnosis for Lung Cancer: Usefulness of Nodule Heterogeneity

RATIONALE AND OBJECTIVES

To develop a computer-aided diagnosis system to differentiate between malignant and benign nodules.

MATERIALS AND METHODS

Seventy-three lung nodules revealed on 60 sets of computed tomography (CT) images were analyzed. Contrast-enhanced CT was performed in 46 CT examinations. The images were provided by the LUNGx Challenge, and the ground truth of the lung nodules was unavailable; a surrogate ground truth was, therefore, constructed by radiological evaluation. Our proposed method involved novel patch-based feature extraction using principal component analysis, image convolution, and pooling operations. This method was compared to three other systems for the extraction of nodule features: histogram of CT density, local binary pattern on three orthogonal planes, and three-dimensional random local binary pattern. The probabilistic outputs of the systems and surrogate ground truth were analyzed using receiver operating characteristic analysis and area under the curve. The LUNGx Challenge team also calculated the area under the curve of our proposed method based on the actual ground truth of their dataset.

RESULTS

Based on the surrogate ground truth, the areas under the curve were as follows: histogram of CT density, 0.640; local binary pattern on three orthogonal planes, 0.688; three-dimensional random local binary pattern, 0.725; and the proposed method, 0.837. Based on the actual ground truth, the area under the curve of the proposed method was 0.81.

CONCLUSIONS

The proposed method could capture discriminative characteristics of lung nodules and was useful for the differentiation between malignant and benign nodules.

Volumes Learned: It Takes More Than Size to "Size Up" Pulmonary Lesions

RATIONALE AND OBJECTIVES

This study aimed to review the current understanding and capabilities regarding use of imaging for noninvasive lesion characterization and its relationship to lung cancer screening and treatment.

MATERIALS AND METHODS

Our review of the state of the art was broken down into questions about the different lung cancer image phenotypes being characterized, the role of imaging and requirements for increasing its value with respect to increasing diagnostic confidence and quantitative assessment, and a review of the current capabilities with respect to those needs.

RESULTS

The preponderance of the literature has so far been focused on the measurement of lesion size, with increasing contributions being made to determine the formal performance of scanners, measurement tools, and human operators in terms of bias and variability. Concurrently, an increasing number of investigators are reporting utility and predictive value of measures other than size, and sensitivity and specificity is being reported. Relatively little has been documented on quantitative measurement of non-size features with corresponding estimation of measurement performance and reproducibility.

CONCLUSIONS

The weight of the evidence suggests characterization of pulmonary lesions built on quantitative measures adds value to the screening for, and treatment of, lung cancer. Advanced image analysis techniques may identify patterns or biomarkers not readily assessed by eye and may also facilitate management of multidimensional imaging data in such a way as to efficiently integrate it into the clinical workflow.

Computer-Aided Diagnosis with Deep Learning Architecture: Applications to Breast Lesions in US Images and Pulmonary Nodules in CT Scans

This paper performs a comprehensive study on the deep-learning-based computer-aided diagnosis (CADx) for the differential diagnosis of benign and malignant nodules/lesions by avoiding the potential errors caused by inaccurate image processing results (e.g., boundary segmentation), as well as the classification bias resulting from a less robust feature set, as involved in most conventional CADx algorithms. Specifically, the stacked denoising auto-encoder (SDAE) is exploited on the two CADx applications for the differentiation of breast ultrasound lesions and lung CT nodules. The SDAE architecture is well equipped with the automatic feature exploration mechanism and noise tolerance advantage, and hence may be suitable to deal with the intrinsically noisy property of medical image data from various imaging modalities. To show the outperformance of SDAE-based CADx over the conventional scheme, two latest conventional CADx algorithms are implemented for comparison. 10 times of 10-fold cross-validations are conducted to illustrate the efficacy of the SDAE-based CADx algorithm. The experimental results show the significant performance boost by the SDAE-based CADx algorithm over the two conventional methods, suggesting that deep learning techniques can potentially change the design paradigm of the CADx systems without the need of explicit design and selection of problem-oriented features.

After Detection: The Improved Accuracy of Lung Cancer Assessment Using Radiologic Computer-aided Diagnosis

RATIONALE AND OBJECTIVES

The aim of this study was to evaluate the improved accuracy of radiologic assessment of lung cancer afforded by computer-aided diagnosis (CADx).

MATERIALS AND METHODS

Inclusion/exclusion criteria were formulated, and a systematic inquiry of research databases was conducted. Following title and abstract review, an in-depth review of 149 surviving articles was performed with accepted articles undergoing a Quality Assessment of Diagnostic Accuracy Studies (QUADAS)-based quality review and data abstraction.

RESULTS

A total of 14 articles, representing 1868 scans, passed the review. Increases in the receiver operating characteristic (ROC) area under the curve of .8 or higher were seen in all nine studies that reported it, except for one that employed subspecialized radiologists.

CONCLUSIONS

This systematic review demonstrated improved accuracy of lung cancer assessment using CADx over manual review, in eight high-quality observer-performance studies. The improved accuracy afforded by radiologic lung-CADx suggests the need to explore its use in screening and regular clinical workflow.

Noninvasive risk stratification of lung adenocarcinoma using quantitative computed tomography

INTRODUCTION

Lung cancer remains the leading cause of cancer-related deaths in the United States and worldwide. Adenocarcinoma is the most common type of lung cancer and encompasses lesions with widely variable clinical outcomes. In the absence of noninvasive risk stratification, individualized patient management remains challenging. Consequently a subgroup of pulmonary nodules of the lung adenocarcinoma spectrum is likely treated more aggressively than necessary.

METHODS

Consecutive patients with surgically resected pulmonary nodules of the lung adenocarcinoma spectrum (lesion size ≤3 cm, 2006-2009) and available presurgical high-resolution computed tomography (HRCT) imaging were identified at Mayo Clinic Rochester. All cases were classified using an unbiased Computer-Aided Nodule Assessment and Risk Yield (CANARY) approach based on the quantification of presurgical HRCT characteristics. CANARY-based classification was independently correlated to postsurgical progression-free survival.

RESULTS

CANARY analysis of 264 consecutive patients identified three distinct subgroups. Independent comparisons of 5-year disease-free survival (DFS) between these subgroups demonstrated statistically significant differences in 5-year DFS, 100%, 72.7%, and 51.4%, respectively (p = 0.0005).

CONCLUSIONS

Noninvasive CANARY-based risk stratification identifies subgroups of patients with pulmonary nodules of the adenocarcinoma spectrum characterized by distinct clinical outcomes. This technique may ultimately improve the current expert opinion-based approach to the management of these lesions by facilitating individualized patient management.

Pulmonary ground-glass nodules diagnosis: mean change rate of peak CT number as a discriminative factor of pathology during a follow-up

OBJECTIVE

We aimed to analyse the peak CT number (PEAK) in CT number histogram of ground-glass nodules (GGN), meaning the most frequent density of pixels in the image of pulmonary nodule, based on three-dimensional (3D) reconstructive model pre-operatively, and the mean rate of PEAK change (V-PEAK) during a follow-up of GGN for differential diagnosis between pre-invasive adenocarcinoma (PIA) and invasive adenocarcinoma (IAC).

METHODS

CT number histogram of pixels in GGN was made automatically by 3D measurement software. Diameter, total volume, PEAK and V-PEAK were measured from CT data sets of different groups classified by pathology, subtype and number of GGN, respectively.

RESULTS

Among all 102 cases, 47 were PIA, including atypical adenomatous hyperplasia (n = 29) and adenocarcinoma in situ (n = 18), and 55 were IAC, including minimally IAC (MIA, n = 4). By Wilcoxon test, PEAK of IAC was significantly higher than that of PIA (p < 0.001). By receiver operating curve analysis, area under the curve (AUC) was 0.857 and threshold -820.50 Hounsfield units (HU) for differentiation between PIA and IAC. V-PEAK of IAC was unexpectedly remarkably smaller than that of PIA (p < 0.001) with AUC and threshold being 0.810 and -0.829 HU day(-1), respectively.

CONCLUSION

Pre-operative PEAK and V-PEAK, which interpret and evaluate the change of volume and density of pulmonary nodule simultaneously from both exterior and interior perspectives, can help to distinguish IAC from PIA.

ADVANCES IN KNOWLEDGE

This study provided researchers of GGN another perspective, taking both volume and density of nodules into consideration for pathological evaluation.

Radiologic Predictors for Clinical Stage IA Lung Adenocarcinoma with Ground Glass Components: A Multi-Center Study of Long-Term Outcomes

Objective

This study was to define preoperative predictors from radiologic findings for the pathologic risk groups based on long-term surgical outcomes, in the aim to help guide individualized patient management.

Methods

We retrospectively reviewed 321 consecutive patients with clinical stage IA lung adenocarcinoma with ground glass component on computed tomography (CT) scanning. Pathologic diagnosis for resection specimens was based on the 2011 IASLC/ATS/ERS classification of lung adenocarcinoma. Patients were classified into different pathologic risk grading groups based on their lymph node status, local regional recurrence and overall survival. Radiologic characteristics of the pulmonary nodules were re-evaluated by reconstructed three-dimension CT (3D-CT). Univariate and multivariate analysis identifies independent radiologic predictors from tumor diameter, total volume (TV), average CT value (AVG), and solid-to-tumor (S/T) ratio. Receiver operating characteristic curves (ROC) studies were carried out to determine the cutoff value(s) for the predictor(s). Univariate cox regression model was used to determine the clinical significance of the above findings.

Results

A total of 321 patients with clinical stage IA lung adenocarcinoma with ground glass components were included in our study. Patients were classified into two pathologic low- and high- risk groups based on their distinguished surgical outcomes. A total of 134 patients fell into the low-risk group. Univariate and multivariate analyses identified AVG (HR: 32.210, 95% CI: 3.020–79.689, P<0.001) and S/T ratio (HR: 12.212, 95% CI: 5.441–27.408, P<0.001) as independent predictors for pathologic risk grading. ROC curves studies suggested the optimal cut-off values for AVG and S/T ratio were-198 (area under the curve [AUC] 0.921), 2.9 (AUC 0.996) and 54% (AUC 0.907), respectively. The tumor diameter and TV were excluded for the low AUCs (0.778 and 0.767). Both the cutoff values of AVG and S/T ratio were correlated with pathologic risk classification (p<0.001). Univariate Cox regression model identified clinical risk classification (RR: 3.011, 95%CI: 0.796–7.882, P = 0.095) as a good predictor for recurrence-free survival (RFS) in patients with clinical stage IA lung adenocarcinoma. Statistical significance of 5-year OS and RFS was noted among clinical low-, moderate- and high-risk groups (log-rank, p = 0.024 and 0.010).

Conclusions

The AVG and the S/T ratio by reconstructed 3D-CT are important preoperative radiologic predictors for pathologic risk grading. The two cutoff values of AVG and S/T ratio are recommended in decision-making for patients with clinical stage IA lung adenocarcinoma with ground glass components.

Preoperative Prediction of Ki-67 Labeling Index By Three-dimensional CT Image Parameters for Differential Diagnosis Of Ground-Glass Opacity (GGO)

The aim of this study was to predict Ki-67 labeling index (LI) preoperatively by three-dimensional (3D) CT image parameters for pathologic assessment of GGO nodules. Diameter, total volume (TV), the maximum CT number (MAX), average CT number (AVG) and standard deviation of CT number within the whole GGO nodule (STD) were measured by 3D CT workstation. By detection of immunohistochemistry and Image Software Pro Plus 6.0, different Ki-67 LI were measured and statistically analyzed among preinvasive adenocarcinoma (PIA), minimally invasive adenocarcinoma (MIA) and invasive adenocarcinoma (IAC). Receiver operating characteristic (ROC) curve, Spearman correlation analysis and multiple linear regression analysis with cross-validation were performed to further research a quantitative correlation between Ki-67 labeling index and radiological parameters. Diameter, TV, MAX, AVG and STD increased along with PIA, MIA and IAC significantly and consecutively. In the multiple linear regression model by a stepwise way, we obtained an equation: prediction of Ki-67 LI=0.022*STD+0.001* TV+2.137 (R=0.595, R’s square=0.354, p<0.001), which can predict Ki-67 LI as a proliferative marker preoperatively. Diameter, TV, MAX, AVG and STD could discriminate pathologic categories of GGO nodules significantly. Ki-67 LI of early lung adenocarcinoma presenting GGO can be predicted by radiologic parameters based on 3D CT for differential diagnosis.

Improving the radiologist-CAD interaction: designing for appropriate trust

Computer-aided diagnosis (CAD) has great potential to improve radiologists' diagnostic performance. However, the reported performance of the radiologist-CAD team is lower than what might be expected based on the performance of the radiologist and the CAD system in isolation. This indicates that the interaction between radiologists and the CAD system is not optimal. An important factor in the interaction between humans and automated aids (such as CAD) is trust. Suboptimal performance of the human-automation team is often caused by an inappropriate level of trust in the automation. In this review, we examine the role of trust in the radiologist-CAD interaction and suggest ways to improve the output of the CAD system so that it allows radiologists to calibrate their trust in the CAD system more effectively. Observer studies of the CAD systems show that radiologists often have an inappropriate level of trust in the CAD system. They sometimes under-trust CAD, thereby reducing its potential benefits, and sometimes over-trust it, leading to diagnostic errors they would not have made without CAD. Based on the literature on trust in human-automation interaction and the results of CAD observer studies, we have identified four ways to improve the output of CAD so that it allows radiologists to form a more appropriate level of trust in CAD. Designing CAD systems for appropriate trust is important and can improve the performance of the radiologist-CAD team. Future CAD research and development should acknowledge the importance of the radiologist-CAD interaction, and specifically the role of trust therein, in order to create the perfect artificial partner for the radiologist. This review focuses on the role of trust in the radiologist-CAD interaction. The aim of the review is to encourage CAD developers to design for appropriate trust and thereby improve the performance of the radiologist-CAD team.

Fate of pulmonary nodules detected by computer-aided diagnosis and physician review on the computed tomography simulation images for hepatocellular carcinoma

Purpose

To investigate the frequency and clinical significance of detected incidental lung nodules found on computed tomography (CT) simulation images for hepatocellular carcinoma (HCC) using computer-aided diagnosis (CAD) and a physician review.

Materials and Methods

Sixty-seven treatment-naïve HCC patients treated with transcatheter arterial chemoembolization and radiotherapy (RT) were included for the study. Portal phase of simulation CT images was used for CAD analysis and a physician review for lung nodule detection. For automated nodule detection, a commercially available CAD system was used. To assess the performance of lung nodule detection for lung metastasis, the sensitivity, negative predictive value (NPV), and positive predictive value (PPV) were calculated.

Results

Forty-six patients had incidental nodules detected by CAD with a total of 109 nodules. Only 20 (18.3%) nodules were considered to be significant nodules by a physician review. The number of significant nodules detected by both of CAD or a physician review was 24 in 9 patients. Lung metastases developed in 11 of 46 patients who had any type of nodule. The sensitivities were 58.3% and 100% based on patient number and on the number of nodules, respectively. The NPVs were 91.4% and 100%, respectively. And the PPVs were 77.8% and 91.7%, respectively.

Conclusion

Incidental detection of metastatic nodules was not an uncommon event. From our study, CAD could be applied to CT simulation images allowing for an increase in detection of metastatic nodules.

Evaluation and management of pulmonary nodules: state-of-the-art and future perspectives

INTRODUCTION

The imaging evaluation of pulmonary nodules, often incidentally detected on imaging examinations performed for other clinical reasons, is a frequently encountered clinical circumstance. With advances in imaging modalities, both the detection and characterization of pulmonary nodules continue to evolve and improve.

AREAS COVERED

This article will review the imaging modalities used to detect and diagnose benign and malignant pulmonary nodules, with a focus on computed tomography (CT), which continues to be the mainstay for evaluation. The authors discuss recent advances in the lung nodule management, and an algorithm for the management of indeterminate pulmonary nodules.

EXPERT OPINION

There are set of criteria that define a benign nodule, the most important of which are the lack of temporal change for 2 years or more, and certain benign imaging criteria, including specific patterns of calcification or the presence of fat. Although some indeterminate pulmonary nodules are immediately actionable, generally those approaching 1 cm or larger in diameter, at which size the diagnostic accuracy of tools such as positron emission tomography (PET)/CT, single photon emission CT (SPECT) and biopsy techniques are sufficient to warrant their use. The majority of indeterminate pulmonary nodules are under 1 cm, for which serial CT examinations through at least 2 years for solid nodules and 3 years for ground-glass nodules, are used to demonstrate either benign biologic behavior or otherwise. The management of incidental pulmonary nodules involves a multidisciplinary approach in which radiology plays a pivotal role. Newer imaging and postprocessing techniques have made this a more accurate technique eliminating ambiguity and unnecessary follow-up.

Missed lung cancer on chest radiography and computed tomography

Missed lung cancer raises an important medicolegal issue and contributes to one of the most common causes for malpractice actions against radiologists. Lung cancer may be missed on either chest radiography or computed tomography. Although most malpractice cases involve lesions overlooked on the former, a small and increasing portion of cases are related to chest computed tomography scan. Factors contributing to overlooked lung cancer can be attributed to observer performance, lesion characteristics, and technical considerations.

Clinical decision support systems and computer-aided diagnosis in otology

OBJECTIVES

We reviewed the progress of the implementation of expert diagnostic systems in the field of otology.

STUDY DESIGN AND SETTING

We conducted a review of the literature at a research institute.

RESULTS

The utilization of expert diagnostic systems in otology is very limited. Previous applications focused primarily upon the diagnosis of vertiginous disorders with the use of deterministic algorithms and, more recently, with adaptive algorithms such as neural networks.

CONCLUSION

Expert systems provide greater diagnostic accuracy to physicians across a wide range of medical specialties. The success of such a system depends upon the strength of its reasoning algorithm, the validity of its knowledge base, and its ease of use.

SIGNIFICANCE

There have been no attempts to develop an adaptive expert system for the full range of otological conditions. Such a tool may be of great use to physicians as a diagnostic aid and educational resource, particularly for those located in isolated sites.

A computer-aided diagnosis for evaluating lung nodules on chest CT: the current status and perspective

As the detection and characterization of lung nodules are of paramount importance in thoracic radiology, various tools for making a computer-aided diagnosis (CAD) have been developed to improve the diagnostic performance of radiologists in clinical practice. Numerous studies over the years have shown that the CAD system can effectively help readers identify more nodules. Moreover, nodule malignancy and the response of malignant lung tumors to treatment can also be assessed using nodule volumetry. CAD also has the potential to objectively analyze the morphology of nodules and enhance the workflow during the assessment of follow-up studies. Therefore, understanding the current status and limitations of CAD for evaluating lung nodules is essential to effectively apply CAD in clinical practice.

Volume-doubling time of pulmonary nodules with ground glass opacity at multidetector CT: Assessment with computer-aided three-dimensional volumetry

RATIONALE AND OBJECTIVES

To investigate the volume-doubling time (VDT) of histologically proved pulmonary nodules showing ground glass opacity (GGO) at multidetector CT (MDCT) using computer-aided three-dimensional volumetry.

MATERIALS AND METHODS

We retrospectively evaluated 47 GGO nodules (mixed n = 28, pure n = 19) that had been examined by thin-section helical CT more than once. They were histologically confirmed as atypical adenomatous hyperplasia (AAH, n = 13), bronchioloalveolar carcinoma (BAC, n = 22), and adenocarcinoma (AC, n = 12). Using computer-aided three-dimensional volumetry software, two radiologists independently performed volumetry of GGO nodules and calculated the VDT using data acquired from the initial and final CT study. We compared VDT among the three pathologies and also compared the VDT of mixed and pure GGO nodules.

RESULTS

The mean VDT of all GGO nodules was 486.4 ± 368.6 days (range 89.0-1583.0 days). The mean VDT for AAH, BAC, and AC was 859.2 ± 428.9, 421.2 ± 228.4, and 202.1 ± 84.3 days, respectively; there were statistically significant differences for all comparative combinations of AAH, BAC, and AC (Steel-Dwass test, P < .01). The mean VDT for pure and mixed GGO nodules was 628.5 ± 404.2 and 276.9 ± 155.9 days, respectively; it was significantly shorter for mixed than pure GGO nodules (Mann-Whitney U-test, P < .01).

CONCLUSION

The evaluation of VDT using computer-aided volumetry may be helpful in assessing the histological entities of GGO nodules.

An open-source framework of neural networks for diagnosis of coronary artery disease from myocardial perfusion SPECT

BACKGROUND

The purpose of this study is to develop and analyze an open-source artificial intelligence program built on artificial neural networks that can participate in and support the decision making of nuclear medicine physicians in detecting coronary artery disease from myocardial perfusion SPECT (MPS).

METHODS AND RESULTS

Two hundred and forty-three patients, who had MPS and coronary angiography within three months, were selected to train neural networks. Six nuclear medicine residents, one experienced nuclear medicine physician, and neural networks evaluated images of 65 patients for presence of coronary artery stenosis. Area under the curve (AUC) of receiver operating characteristics analysis for networks and expert was .74 and .84, respectively. The AUC of the other physicians ranged from .67 to .80. There were no significant differences between expert, neural networks, and standard quantitative values, summed stress score and total stress defect extent.

CONCLUSIONS

The open-source neural networks developed in this study may provide a framework for further testing, development, and integration of artificial intelligence into nuclear cardiology environment.

Is an ROC-type response truly always better than a binary response in observer performance studies?

RATIONALE AND OBJECTIVES

The aim of this study was to assess similarities and differences between methods of performance comparisons under binary (yes or no) and receiver-operating characteristic (ROC)-type pseudocontinuous (0-100) rating data ascertained during an observer performance study of interpretation of full-field digital mammography (FFDM) versus FFDM plus digital breast tomosynthesis.

MATERIALS AND METHODS

Rating data consisted of ROC-type pseudocontinuous and binary ratings generated by eight radiologists evaluating 77 digital mammographic examinations. Overall performance levels were summarized with a conventionally used probability of correct discrimination or, equivalently, the area under the ROC curve (AUC), which under a binary scale is related to Youden's index. Magnitudes of differences in the reader-averaged empirical AUCs between FFDM alone and FFDM plus digital breast tomosynthesis were compared in the context of fixed-reader and random-reader variability of the estimates.

RESULTS

The absolute differences between modes using the empirical AUCs were larger on average for the binary scale (0.12 vs 0.07) and for the majority of individual readers (six of eight). Standardized differences were consistent with this finding (2.32 vs 1.63 on average). Reader-averaged differences in AUCs standardized by fixed-reader and random-reader variances were also smaller under the binary rating paradigm. The discrepancy between AUC differences depended on the location of the reader-specific binary operating points.

CONCLUSIONS

The human observer's operating point should be a primary consideration in designing an observer performance study. Although in general, the ROC-type rating paradigm provides more detailed information on the characteristics of different modes, it does not reflect the actual operating point adopted by human observers. There are application-driven scenarios in which analysis based on binary responses may provide statistical advantages.

Neural network ensemble-based computer-aided diagnosis for differentiation of lung nodules on CT images: clinical evaluation

RATIONALE AND OBJECTIVES

To evaluate the diagnostic performance of a neural network ensemble-based computer-aided diagnosis (CAD) scheme for classifying lung nodules on thin-section computed tomography (CT).

MATERIALS AND METHODS

Thirty-two CT images that depicted 19 malignant nodules and 13 benign nodules were used. One of three possible classifications (probably benign, uncertain, and probably malignant) for each nodule was determined by using a neural network ensemble-based CAD scheme. The images were presented to three senior radiologists (each with more than 10 years of thoracic radiology experience) who were asked to determine the classification for each nodule blindly. The radiologists made their diagnostic decisions solely based on images and excluded any external data. The performance of the CAD scheme and of the radiologists was evaluated with receiver operating characteristic (ROC) analysis and agreement analysis.

RESULTS

Areas under the ROC curve (Az values) for the CAD scheme and the radiologist group were 0.79 and 0.82, respectively, and the partial areas under the ROC curves at a range of sensitivity values greater than or equal to 90% were 0.051 and 0.020 (P = .203), respectively. The weighted Kappa coefficients between the CAD scheme and each radiologist were 0.657, 0.431, and 0.606, respectively. For the diagnosis of the 11 small nodules (with diameters not greater than 10 mm), areas under the ROC curves of the CAD scheme and the radiologist group were 0.915 and 0.683 (P = .227), respectively.

CONCLUSIONS

The diagnostic performance of the neural network ensemble-based CAD scheme is similar to that of senior radiologists for classifying lung nodules on thin-section CT. Furthermore, the CAD scheme has certain advantages in diagnosing small lung nodules.

Computer-aided diagnosis of lung nodules on CT scans: ROC study of its effect on radiologists' performance

RATIONALE AND OBJECTIVES

The aim of this study was to evaluate the effect of computer-aided diagnosis (CAD) on radiologists' estimates of the likelihood of malignancy of lung nodules on computed tomographic (CT) imaging.

METHODS AND MATERIALS

A total of 256 lung nodules (124 malignant, 132 benign) were retrospectively collected from the thoracic CT scans of 152 patients. An automated CAD system was developed to characterize and provide malignancy ratings for lung nodules on CT volumetric images. An observer study was conducted using receiver-operating characteristic analysis to evaluate the effect of CAD on radiologists' characterization of lung nodules. Six fellowship-trained thoracic radiologists served as readers. The readers rated the likelihood of malignancy on a scale of 0% to 100% and recommended appropriate action first without CAD and then with CAD. The observer ratings were analyzed using the Dorfman-Berbaum-Metz multireader, multicase method.

RESULTS

The CAD system achieved a test area under the receiver-operating characteristic curve (A(z)) of 0.857 +/- 0.023 using the perimeter, two nodule radii measures, two texture features, and two gradient field features. All six radiologists obtained improved performance with CAD. The average A(z) of the radiologists improved significantly (P < .01) from 0.833 (range, 0.817-0.847) to 0.853 (range, 0.834-0.887).

CONCLUSION

CAD has the potential to increase radiologists' accuracy in assessing the likelihood of malignancy of lung nodules on CT imaging.

Computer-aided volumetry of pulmonary nodules exhibiting ground-glass opacity at MDCT

OBJECTIVE

The purpose of this study was to investigate the accuracy and reproducibility of results acquired with computer-aided volumetry software during MDCT of pulmonary nodules exhibiting ground-glass opacity.

MATERIALS AND METHODS

To evaluate the accuracy of computer-aided volumetry software, we performed thin-section helical CT of a chest phantom that included simulated 3-, 5-, 8-, 10-, and 12-mm-diameter ground-glass opacity nodules with attenuation of -800, -630, and -450 HU. Three radiologists measured the volume of the nodules and calculated the relative volume measurement error, which was defined as follows: (measured nodule volume minus assumed nodule volume / assumed nodule volume) x 100. Two radiologists performed two independent measurements of 59 nodules in humans. Intraobserver and interobserver agreement was evaluated with Bland-Altman methods.

RESULTS

The relative volume measurement error for simulated ground-glass opacity nodules measuring 3 mm ranged from 51.1% to 85.2% and for nodules measuring 5 mm or more in diameter ranged from -4.1% to 7.1%. In the clinical study, for intraobserver agreement, the 95% limits of agreement were -14.9% and -13.7% and -16.6% to 15.7% for observers A and B. For interobserver agreement, these values were -16.3% to 23.7% for nodules 8 mm in diameter or larger.

CONCLUSION

With computer-aided volumetry of ground-glass opacity nodules, the relative volume measurement error was small for nodules 5 mm in diameter or larger. Intraobserver and interobserver agreement was relatively high for nodules 8 mm in diameter or larger.

Virtual slide telepathology workstation of the future: lessons learned from teleradiology

The clinical reading environment for the 21st century pathologist looks very different than it did even a few short years ago. Glass slides are quickly being replaced by digital "virtual slides," and the traditional light microscope is being replaced by the computer display. There are numerous questions that arise however when deciding exactly what this new digital display viewing environment will be like. Choosing a workstation for daily use in the interpretation of digital pathology images can be a very daunting task. Radiology went digital nearly 20 years ago and faced many of the same challenges so there are lessons to be learned from these experiences. One major lesson is that there is no "one size fits all" workstation so users must consider a variety of factors when choosing a workstation. In this article, we summarize some of the potentially critical elements in a pathology workstation and the characteristics one should be aware of and look for in the selection of one. Issues pertaining to both hardware and software aspects of medical workstations will be reviewed particularly as they may impact the interpretation process.