Investigation of new psychophysical measures for evaluation of similar images on thoracic computed tomography for distinction between benign and malignant nodules

New solutions for automated image recognition and identification: challenges to radiologic technology and forensic pathology

This paper outlines the history of biometrics for personal identification, the current status of the initial biological fingerprint techniques for digital chest radiography, and patient verification during medical imaging, such as computed tomography and magnetic resonance imaging. Automated image recognition and identification developed for clinical images without metadata could also be applied to the identification of victims in mass disasters or other unidentified individuals. The development of methods that are adaptive to a wide range of recent imaging modalities in the fields of radiologic technology, patient safety, forensic pathology, and forensic odontology is still in its early stages. However, its importance in practice will continue to increase in the future.

Development of a CT image case database and content-based image retrieval system for non-cancerous respiratory diseases: Method and preliminary assessment

BACKGROUND

To evaluate the benefits of using a CT image case database (DB) with content-based image retrieval system for the diagnosis of typical non-cancerous respiratory diseases.

METHODS

Using this DB, which comprised data on 191 cases covering 69 diseases, 933 imaging findings that contributed to differential diagnoses were annotated. Ten test cases were selected. Image similarity between each marked test case lesion and the lesions of the top 10 retrieved cases were assessed and classified as similar, somewhat similar, or dissimilar by two physicians in consensus. Additionally, the accuracy of five internal medicine residents' abilities to interpret CT findings and provide disease diagnoses with and without the proposed system was evaluated by image interpretation experiments involving five test cases. The rates of concordance between the subjects' interpretations and the correct answers prepared in advance by two specialists in consensus were converted into scores.

RESULTS

The mean (± SD) of image similarity among the 10 test cases was as follows: 5.1 ± 2.7 (similar), 2.9 ± 1.0 (somewhat similar), and 2.0 ± 2.4 (dissimilar). Using the proposed system, the subjects' mean score for the correct interpretation of CT findings improved from 15.1 to 28.2 points (p = 0.131) and for the correct disease diagnoses, from 9.3 to 28.2 points (p = 0.034).

CONCLUSIONS

Although this was a preliminary small-scale assessment, the results suggest that this system may contribute to an improved interpretation of CT findings and differential diagnosis of non-cancerous respiratory diseases, which are difficult to diagnose for inexperienced physicians.

Overview on subjective similarity of images for content-based medical image retrieval

Computer-aided diagnosis systems for assisting the classification of various diseases have the potential to improve radiologists' diagnostic accuracy and efficiency, as reported in several studies. Conventional systems generally provide the probabilities of disease types in terms of numerical values, a method that may not be efficient for radiologists who are trained by reading a large number of images. Presentation of reference images similar to those of a new case being diagnosed can supplement the probability outputs based on computerized analysis as an intuitive guide, and it can assist radiologists in their diagnosis, reporting, and treatment planning. Many studies on content-based medical image retrievals have been reported on. For retrieval of perceptually similar and diagnostically relevant images, incorporation of perceptual similarity data by radiologists has been suggested. In this paper, studies on image retrieval methods are reviewed with a special focus on quantification, utilization, and the evaluation of subjective similarities between pairs of images.

Determination of subjective similarity for pairs of masses and pairs of clustered microcalcifications on mammograms: comparison of similarity ranking scores and absolute similarity ratings

The presentation of images that are similar to that of an unknown lesion seen on a mammogram may be helpful for radiologists to correctly diagnose that lesion. For similar images to be useful, they must be quite similar from the radiologists' point of view. We have been trying to quantify the radiologists' impression of similarity for pairs of lesions and to establish a "gold standard" for development and evaluation of a computerized scheme for selecting such similar images. However, it is considered difficult to reliably and accurately determine similarity ratings, because they are subjective. In this study, we compared the subjective similarities obtained by two different methods, an absolute rating method and a 2-alternative forced-choice (2AFC) method, to demonstrate that reliable similarity ratings can be determined by the responses of a group of radiologists. The absolute similarity ratings were previously obtained for pairs of masses and pairs of microcalcifications from five and nine radiologists, respectively. In this study, similarity ranking scores for eight pairs of masses and eight pairs of microcalcifications were determined by use of the 2AFC method. In the first session, the eight pairs of masses and eight pairs of microcalcifications were grouped and compared separately for determining the similarity ranking scores. In the second session, another similarity ranking score was determined by use of mixed pairs, i.e., by comparison of the similarity of a mass pair with that of a calcification pair. Four pairs of masses and four pairs of microcalcifications were grouped together to create two sets of eight pairs. The average absolute similarity ratings and the average similarity ranking scores showed very good correlations in the first study (Pearson's correlation coefficients: 0.94 and 0.98 for masses and microcalcifications, respectively). Moreover, in the second study, the correlations between the absolute ratings and the ranking scores were also very high (0.92 and 0.96), which implies that the observers were able to compare the similarity of a mass pair with that of a calcification pair consistently. These results provide evidence that the concept of similarity for pairs of images is robust, even across different lesion types, and that radiologists are able to reliably determine subjective similarity for pairs of breast lesions.

Texture feature analysis for computer-aided diagnosis on pulmonary nodules

Differentiation of malignant and benign pulmonary nodules is of paramount clinical importance. Texture features of pulmonary nodules in CT images reflect a powerful character of the malignancy in addition to the geometry-related measures. This study first compared three well-known types of two-dimensional (2D) texture features (Haralick, Gabor, and local binary patterns or local binary pattern features) on CADx of lung nodules using the largest public database founded by Lung Image Database Consortium and Image Database Resource Initiative and then investigated extension from 2D to three-dimensional (3D) space. Quantitative comparison measures were made by the well-established support vector machine (SVM) classifier, the area under the receiver operating characteristic curves (AUC) and the p values from hypothesis t tests. While the three feature types showed about 90% differentiation rate, the Haralick features achieved the highest AUC value of 92.70% at an adequate image slice thickness, where a thinner or thicker thickness will deteriorate the performance due to excessive image noise or loss of axial details. Gain was observed when calculating 2D features on all image slices as compared to the single largest slice. The 3D extension revealed potential gain when an optimal number of directions can be found. All the observations from this systematic investigation study on the three feature types can lead to the conclusions that the Haralick feature type is a better choice, the use of the full 3D data is beneficial, and an adequate tradeoff between image thickness and noise is desired for an optimal CADx performance. These conclusions provide a guideline for further research on lung nodule differentiation using CT imaging.

Potential clinical impact of advanced imaging and computer-aided diagnosis in chest radiology: importance of radiologist's role and successful observer study

This review paper is based on our research experience in the past 30 years. The importance of radiologists' role is discussed in the development or evaluation of new medical images and of computer-aided detection (CAD) schemes in chest radiology. The four main topics include (1) introducing what diseases can be included in a research database for different imaging techniques or CAD systems and what imaging database can be built by radiologists, (2) understanding how radiologists' subjective judgment can be combined with technical objective features to improve CAD performance, (3) sharing our experience in the design of successful observer performance studies, and (4) finally, discussing whether the new images and CAD systems can improve radiologists' diagnostic ability in chest radiology. In conclusion, advanced imaging techniques and detection/classification of CAD systems have a potential clinical impact on improvement of radiologists' diagnostic ability, for both the detection and the differential diagnosis of various lung diseases, in chest radiology.

Content-based image retrieval in radiology: analysis of variability in human perception of similarity

We aim to develop a better understanding of perception of similarity in focal computed tomography (CT) liver images to determine the feasibility of techniques for developing reference sets for training and validating content-based image retrieval systems. In an observer study, four radiologists and six nonradiologists assessed overall similarity and similarity in 5 image features in 136 pairs of focal CT liver lesions. We computed intra- and inter-reader agreements in these similarity ratings and viewed the distributions of the ratings. The readers' ratings of overall similarity and similarity in each feature primarily appeared to be bimodally distributed. Median Kappa scores for intra-reader agreement ranged from 0.57 to 0.86 in the five features and from 0.72 to 0.82 for overall similarity. Median Kappa scores for inter-reader agreement ranged from 0.24 to 0.58 in the five features and were 0.39 for overall similarity. There was no significant difference in agreement for radiologists and nonradiologists. Our results show that developing perceptual similarity reference standards is a complex task. Moderate to high inter-reader variability precludes ease of dividing up the workload of rating perceptual similarity among many readers, while low intra-reader variability may make it possible to acquire large volumes of data by asking readers to view image pairs over many sessions.

Modeling perceptual similarity measures in CT images of focal liver lesions

MOTIVATION

A gold standard for perceptual similarity in medical images is vital to content-based image retrieval, but inter-reader variability complicates development. Our objective was to develop a statistical model that predicts the number of readers (N) necessary to achieve acceptable levels of variability.

MATERIALS AND METHODS

We collected 3 radiologists' ratings of the perceptual similarity of 171 pairs of CT images of focal liver lesions rated on a 9-point scale. We modeled the readers' scores as bimodal distributions in additive Gaussian noise and estimated the distribution parameters from the scores using an expectation maximization algorithm. We (a) sampled 171 similarity scores to simulate a ground truth and (b) simulated readers by adding noise, with standard deviation between 0 and 5 for each reader. We computed the mean values of 2-50 readers' scores and calculated the agreement (AGT) between these means and the simulated ground truth, and the inter-reader agreement (IRA), using Cohen's Kappa metric.

RESULTS

IRA for the empirical data ranged from =0.41 to 0.66. For between 1.5 and 2.5, IRA between three simulated readers was comparable to agreement in the empirical data. For these values , AGT ranged from =0.81 to 0.91. As expected, AGT increased with N, ranging from =0.83 to 0.92 for N = 2 to 50, respectively, with =2.

CONCLUSION

Our simulations demonstrated that for moderate to good IRA, excellent AGT could nonetheless be obtained. This model may be used to predict the required N to accurately evaluate similarity in arbitrary size datasets.

CURVATURE-BASED CORRECTION ALGORITHM FOR AUTOMATIC LUNG SEGMENTATION ON CHEST CT IMAGES

In order to ameliorate the lung defects caused by missed juxtapleural nodules in lung segmentation on chest computed tomography (CT) images, we develop a Newton–Cotes-based smoothing algorithm (NCBS) which is used as a preliminary step to remove noises as many as possible. Next considering the crescent outline features of the lung, we propose a curvature-based correction algorithm (CBC) for the determination of the correction threshold. The application of the proposed algorithms is demonstrated in the process of lung segmentation and the experimental results on 25 real datasets are illustrated. Furthermore, some experiments are conducted to investigate the effects of the key parameters in CBC on the performances of lung segmentation so as to decide their optimal values. In addition, the CBC is compared with other methods analytically and experimentally. The overall results show that our proposed algorithm in lung segmentation excels the related methods on the capability of automatic selection of the correction threshold, as well as the performances of accuracy, efficiency and feasibility.

Representation of lesion similarity by use of multidimensional scaling for breast masses on mammograms

Presentation of similar reference images can be useful for diagnosis of new lesions. A similarity map which can visually present the overview of the relationship between the lesions with different types may provide the supplemental information to the reference images. A new method for constructing the similarity map by multidimensional scaling (MDS) for breast masses on mammograms was investigated. Nine pathologic types were included; three regions of interests each from the nine groups were employed in this study. Subjective similarity ratings by expert readers were obtained for all possible 351 pairs of masses. Using the average ratings, MDS similarity map was created. Each axis of the MDS configuration was fitted by the linear model with 13 image features to reconstruct the similarity map. Dissimilarity based on the distance in the reconstructed space was determined and compared with the subjective rating. The MDS map consistently represented the similarity between cysts and fibroadenomas, invasive lobular carcinomas and scirrhous carcinomas, and ductal carcinomas in situ, solid-tubular carcinomas, and papillotubular carcinomas with the experts' data. The correlation between the average subjective ratings and the dissimilarities based on the distance in the reconstructed feature space was much greater (-0.87) than that of the dissimilarities based on the distance in the conventional feature space (-0.65). The new similarity map by MDS can be useful for visualizing the relationship between breast masses with different pathologic types. It has potential usefulness in selecting the similarity measures and providing the supplemental information.

Computer-aided beam arrangement based on similar cases in radiation treatment-planning databases for stereotactic lung radiation therapy

The purpose of this study was to develop a computer-aided method for determination of beam arrangements based on similar cases in a radiotherapy treatment-planning database for stereotactic lung radiation therapy. Similar-case-based beam arrangements were automatically determined based on the following two steps. First, the five most similar cases were searched, based on geometrical features related to the location, size and shape of the planning target volume, lung and spinal cord. Second, five beam arrangements of an objective case were automatically determined by registering five similar cases with the objective case, with respect to lung regions, by means of a linear registration technique. For evaluation of the beam arrangements five treatment plans were manually created by applying the beam arrangements determined in the second step to the objective case. The most usable beam arrangement was selected by sorting the five treatment plans based on eight plan evaluation indices, including the D95, mean lung dose and spinal cord maximum dose. We applied the proposed method to 10 test cases, by using an RTP database of 81 cases with lung cancer, and compared the eight plan evaluation indices between the original treatment plan and the corresponding most usable similar-case-based treatment plan. As a result, the proposed method may provide usable beam arrangements, which have no statistically significant differences from the original beam arrangements (P > 0.05) in terms of the eight plan evaluation indices. Therefore, the proposed method could be employed as an educational tool for less experienced treatment planners.

Generic integration of content-based image retrieval in computer-aided diagnosis

Content-based image retrieval (CBIR) offers approved benefits for computer-aided diagnosis (CAD), but is still not well established in radiological routine yet. An essential factor is the integration gap between CBIR systems and clinical information systems. The international initiative Integrating the Healthcare Enterprise (IHE) aims at improving interoperability of medical computer systems. We took into account deficiencies in IHE compliance of current picture archiving and communication systems (PACS), and developed an intermediate integration scheme based on the IHE post-processing workflow integration profile (PWF) adapted to CBIR in CAD. The Image Retrieval in Medical Applications (IRMA) framework was used to apply our integration scheme exemplarily, resulting in the application called IRMAcon. The novel IRMAcon scheme provides a generic, convenient and reliable integration of CBIR systems into clinical systems and workflows. Based on the IHE PWF and designed to grow at a pace with the IHE compliance of the particular PACS, it provides sustainability and fosters CBIR in CAD.

Computer-aided detection of malignant lung nodules on chest radiographs: effect on observers' performance

Objective

To evaluate the effect of computer-aided detection (CAD) system on observer performance in the detection of malignant lung nodules on chest radiograph.

Materials and Methods

Two hundred chest radiographs (100 normal and 100 abnormal with malignant solitary lung nodules) were evaluated. With CT and histological confirmation serving as a reference, the mean nodule size was 15.4 mm (range, 7-20 mm). Five chest radiologists and five radiology residents independently interpreted both the original radiographs and CAD output images using the sequential testing method. The performances of the observers for the detection of malignant nodules with and without CAD were compared using the jackknife free-response receiver operating characteristic analysis.

Results

Fifty-nine nodules were detected by the CAD system with a false positive rate of 1.9 nodules per case. The detection of malignant lung nodules significantly increased from 0.90 to 0.92 for a group of observers, excluding one first-year resident (p = 0.04). When lowering the confidence score was not allowed, the average figure of merit also increased from 0.90 to 0.91 (p = 0.04) for all observers after a CAD review. On average, the sensitivities with and without CAD were 87% and 84%, respectively; the false positive rates per case with and without CAD were 0.19 and 0.17, respectively. The number of additional malignancies detected following true positive CAD marks ranged from zero to seven for the various observers.

Conclusion

The CAD system may help improve observer performance in detecting malignant lung nodules on chest radiographs and contribute to a decrease in missed lung cancer.

Usefulness of presentation of similar images in the diagnosis of breast masses on mammograms: comparison of observer performances in Japan and the USA

Computer-aided diagnosis has potential in improving radiologists' diagnosis, and presentation of similar images as a reference may provide additional useful information for distinction between benign and malignant lesions. In this study, we evaluated the usefulness of presentation of reference images in observer performance studies and compared the results obtained by groups of observers practicing in the United States and Japan. The results showed that the presentation of the reference images was generally effective for both groups, as the areas under the receiver operating characteristic curves improved from 0.915 to 0.924 for the group in the US and from 0.913 to 0.925 for the group in Japan, although the differences were marginally (p = 0.047) and not (p = 0.13) statistically significant, respectively. There was a slight difference between the two groups in the way that the observers reacted to some benign cases, which might be due to differences in the population of screenees and in the socio-clinical environment. In the future, it may be worthwhile to investigate the development of a customized system for physicians in different socio-clinical environments.

Content-based image-retrieval system in chest computed tomography for a solitary pulmonary nodule: method and preliminary experiments

PURPOSE

The aim of this study was to develop a new diagnostic support system using content-based image-retrieval technology. In this article, we describe the mechanism and preliminary evaluation of this system for use with CT images of solitary pulmonary nodules.

MATERIALS AND METHODS

With the approval of the institutional review board of Shizuoka Cancer Center, we built a database that included CT images of 461 solitary pulmonary nodules. With this database, we developed a system that automatically extracts the pulmonary nodule when the nodule area is clicked, retrieves previous cases based on an image analysis of the extracted lesion, and generates reports of the pulmonary nodule semi-automatically. We compared the percentage of correct diagnoses with and without the system using 30 solitary pulmonary nodules, which were not included in the database, with one radiologist and two residents. As a per-user evaluation, the number of clicks required to extract the nodule region and the extracted regions was compared, and presented candidate cases were evaluated. As an evaluation of the retrieval results, the presented candidate cases were evaluated by comparing the number of diagnostic matches (benign/malignant) between the queries and four presented cases. Additionally, to evaluate the validity of the retrieval technology, the radiologist selected the most similar cases presented by the system and evaluated the visual similarity on a five-point scale.

RESULTS

With this system, the percentage of correct diagnoses for the radiologist improved from 80 to 93%. For the two residents, the diagnostic accuracy improved from 66.7 to 80% and from 76.7 to 90%, respectively. The evaluation of the number of clicks required indicated that for 19 cases with the radiologist and 12 and 11 cases with the two residents, respectively, only one click was required to extract the region. When the extracted regions were compared between the radiologist and the residents, 22 and 19 cases had a Dice's Coefficient of 0.85 or higher, respectively. For the radiologist, the number of cases that matched the diagnosis (benign/malignant) averaged 3.7 ± 0.5 among 23 malignant cases and 1.7 ± 1.4 among 7 benign cases, while for the residents, these values were 3.6 ± 0.5 and 1.1 ± 0.9, and 3.4 ± 0.8 and 1.1 ± 1.3, respectively. With regard to visual evaluations by the radiologist, there were 15 similar cases and 11 somewhat similar cases.

CONCLUSION

These results suggest that, despite some differences in the search results among the users, this system has been confirmed that it can improve the accuracy of diagnosis as it displays similar cases at high probability. In addition, with the use of this system, past cases and their reports can be effectively referred to. Therefore, this diagnostic-assistant system has the potential to improve the efficiency of the CT image-reading workflow.

Evaluation of objective similarity measures for selecting similar images of mammographic lesions

The purpose of this study was to investigate four objective similarity measures as an image retrieval tool for selecting lesions similar to unknown lesions on mammograms. Measures A and B were based on the Euclidean distance in feature space and the psychophysical similarity measure, respectively. Measure C was the sequential combination of B and A, whereas measure D was the sequential combination of A and B. In this study, we selected 100 lesions each for masses and clustered microcalcifications randomly from our database, and we selected five pairs of lesions from 4,950 pairs based on all combinations of the 100 lesions by use of each measure. In two observer studies for 20 mass pairs and 20 calcification pairs, six radiologists compared all combinations of 20 pairs by using a two-alternative forced-choice method to determine the subjective similarity ranking score which was obtained from the frequency with which a pair was considered as more similar than the other 19 pairs. In both mass and calcification pairs, pairs selected by use of measure D had the highest mean value of the average subjective similarity ranking scores. The difference between measures D and A (P = 0.008 and 0.024), as well as that between measures D and B (P = 0.018 and 0.028) were statistically significant for masses and microcalcifications, respectively. The sequential combination of the objective similarity measure based on the Euclidean distance and the psychophysical similarity measure would be useful in the selection of images similar to those of unknown lesions.

Presentation of similar images as a reference for distinction between benign and malignant masses on mammograms: analysis of initial observer study

The effect of the presentation of similar images for distinction between benign and malignant masses on mammograms was evaluated in the observer performance study. Images of masses were obtained from the Digital Database for Screening Mammography. We selected 50 benign and 50 malignant masses by a stratified randomization method. For each case, similar images were selected based on the size of masses and the similarity measures. Radiologists were shown images with unknown masses and asked to provide their confidence level that the lesions were malignant before and after the presentation of the similar images. Eleven observers, including three attending breast radiologists, three breast imaging fellows, and five residents, participated. The average areas under the receiver operating characteristic curves without and with the presentation of the similar images were almost equivalent. However, there were many cases in which the similar images caused beneficial effects to the observers, whereas there were a small number of cases in which the similar images had detrimental effects. From a detailed analysis of the reasons for these detrimental effects, we found that the similar images would not be useful for diagnosis of rare and very difficult cases, i.e., benign-looking malignant and malignant-looking benign cases. In addition, these cases should not be included in the reference database, because radiologists would be confused by these unusual cases. The results of this study could be very important and useful for the future development and improvement of a computer-aided diagnosis system.

Computer-aided diagnosis of lung nodules on CT scans: ROC study of its effect on radiologists' performance

RATIONALE AND OBJECTIVES

The aim of this study was to evaluate the effect of computer-aided diagnosis (CAD) on radiologists' estimates of the likelihood of malignancy of lung nodules on computed tomographic (CT) imaging.

METHODS AND MATERIALS

A total of 256 lung nodules (124 malignant, 132 benign) were retrospectively collected from the thoracic CT scans of 152 patients. An automated CAD system was developed to characterize and provide malignancy ratings for lung nodules on CT volumetric images. An observer study was conducted using receiver-operating characteristic analysis to evaluate the effect of CAD on radiologists' characterization of lung nodules. Six fellowship-trained thoracic radiologists served as readers. The readers rated the likelihood of malignancy on a scale of 0% to 100% and recommended appropriate action first without CAD and then with CAD. The observer ratings were analyzed using the Dorfman-Berbaum-Metz multireader, multicase method.

RESULTS

The CAD system achieved a test area under the receiver-operating characteristic curve (A(z)) of 0.857 +/- 0.023 using the perimeter, two nodule radii measures, two texture features, and two gradient field features. All six radiologists obtained improved performance with CAD. The average A(z) of the radiologists improved significantly (P < .01) from 0.833 (range, 0.817-0.847) to 0.853 (range, 0.834-0.887).

CONCLUSION

CAD has the potential to increase radiologists' accuracy in assessing the likelihood of malignancy of lung nodules on CT imaging.

Potential usefulness of similar images in the differential diagnosis of clustered microcalcifications on mammograms

PURPOSE

To evaluate the potential usefulness of images of lesions of a known disease that have a similar appearance to lesions of an unknown disease in distinguishing between benign and malignant clustered microcalcifications on mammograms by removing the unusual cases from the database.

MATERIALS AND METHODS

Institutional review board approval for this retrospective HIPAA-compliant study of images from a publicly available database was obtained. Unusual lesions, such as malignant-looking benign lesions and benign-looking malignant lesions, were removed from the database. A total of 20 benign and 20 malignant lesions were selected with a stratified randomization method, and it was these lesions that served as unknown cases in this observer study. For each unknown case, eight similar images of benign lesions and eight similar images of malignant lesions were preselected with a computerized scheme. From these preselected images, a breast radiologist subjectively selected the four most similar images of benign lesions and the four most similar images of malignant lesions. Five attending breast radiologists and three breast-imaging fellows participated in the observer study. Observers provided their confidence level regarding malignancy of the unknown case before and after they viewed the similar images. The results were evaluated with multireader multicase receiver operating characteristic (ROC) analysis.

RESULTS

For all observers, the areas under the ROC curves (AUCs) were improved when similar images were used. The average AUC for all observers increased from 0.692 without use of similar images to 0.790 with use of similar images (P = .0009).

CONCLUSION

The presentation of similar images can improve radiologists' performance in the differential diagnosis of clustered microcalcifications on mammograms.

Using relevance feedback to reduce the semantic gap in content-based image retrieval of mammographic masses

This paper presents the use of relevance feedback (RFb) to reduce the semantic gap in content-based image retrieval (CBIR) of mammographic masses. Tests were conducted where the radiologists' classification of the lesions based on the BI-RADS categories were used with techniques of query-point movement to incorporate RFb. The measures of similarity of images used for CBIR were based upon Zernike moments. The performance of CBIR was measured in terms of precision and recall of retrieval. The results indicate improvement due to RFb of up to 41.6% in precision. In our experiments, the gain in the performance of CBIR with RFb was associated with the BI-RADS category of the query mammographic image, with large improvement in cases of lesions belonging to categories 4 and 5. The proposed method could find applications in computer-aided diagnosis (CAD) of breast cancer.

Determination of similarity measures for pairs of mass lesions on mammograms by use of BI-RADS lesion descriptors and image features

RATIONALE AND OBJECTIVES

To determine similarity measures for selection of pathology-known similar images that would be useful for radiologists as a reference guide in the diagnosis of new breast lesions on mammograms.

MATERIALS AND METHODS

The images were obtained from the Digital Database for Screening Mammography developed by the University of South Florida. For determination and evaluation of similarity measures, the "gold standard" of similarities for 300 pairs of masses was determined by 10 breast radiologists. For determining similarity measures that would agree with radiologists' similarity determination, an artificial neural network (ANN) was trained with the radiologists' subjective similarity ratings and the image features. The image features were determined subjectively using the Breast Imaging Reporting and Data System (BI-RADS) lesion descriptors and objectively by computerized image analysis. The similarity measures determined by the ANN were compared to the gold standard and evaluated in terms of the correlation coefficient.

RESULTS

The similarity measures determined using the BI-RADS descriptors only were not as useful as those determined by use of the image features only. When the BI-RADS margin ratings were combined with the image features, the correlation coefficient between the subjective ratings and the objective measures improved slightly (r = 0.76) compared to those based on the image features alone (r = 0.74).

CONCLUSIONS

The inclusion of the BI-RADS margin descriptors may be useful for determination of similarity measures, especially when it is difficult to obtain the manual outlines of the masses and if the BI-RADS descriptors were provided consistently by radiologists.

Subjective similarity of patterns of diffuse interstitial lung disease on thin-section CT: an observer performance study

RATIONALE AND OBJECTIVES

The aim of this study was to investigate the subjective similarity for pairs of images with various abnormal patterns of diffuse interstitial lung disease on thin-section computed tomography by experienced radiologists to explore a basis for selecting similar images to assist radiologists' interpretation.

MATERIALS AND METHODS

Four major patterns (ground-glass opacity, nodular opacity, reticular opacity, and honeycombing) on thin-section computed tomographic images were identified by at least two of three radiologists. One radiologist manually selected 104 image pairs, in which the images in each pair had the same pattern and were similar in appearance. An additional 208 image pairs were randomly selected and evenly divided among the four patterns. These pairs were then rated for subjective similarity (on a continuous scale ranging from 0 = not similar at all to 1.0 = almost identical) by 12 radiologists.

RESULTS

For radiologist-selected pairs, the mean similarity rated by the 12 radiologists was 0.72. For randomly selected pairs, the mean similarity was higher for the same pattern (0.47) than for the varying patterns (0.27) (P < .001), and among the same pattern, the mean similarity was 0.63 for ground-glass opacity, 0.58 for honeycombing, 0.45 for nodular opacity, and 0.32 for reticular opacity. The mean standard deviation for similarity ratings on all pairs given by the 12 radiologists was 0.05 (rang, 0.01-0.09).

CONCLUSION

Subjective similarity ratings for pairs of abnormal images can be measured reliably and reproducibly by radiologists and will provide a basis for the selection of similar images to assist radiologists' interpretation.

Investigation of psychophysical similarity measures for selection of similar images in the diagnosis of clustered microcalcifications on mammograms

The presentation of images with lesions of known pathology that are similar to an unknown lesion may be helpful to radiologists in the diagnosis of challenging cases for improving the diagnostic accuracy and also for reducing variation among different radiologists. The authors have been developing a computerized scheme for automatically selecting similar images with clustered microcalcifications on mammograms from a large database. For similar images to be useful, they must be similar from the point of view of the diagnosing radiologists. In order to select such images, subjective similarity ratings were obtained for a number of pairs of clustered microcalcifications by breast radiologists for establishment of a "gold standard" of image similarity, and the gold standard was employed for determination and evaluation of the selection of similar images. The images used in this study were obtained from the Digital Database for Screening Mammography developed by the University of South Florida. The subjective similarity ratings for 300 pairs of images with clustered microcalcifications were determined by ten breast radiologists. The authors determined a number of image features which represent the characteristics of clustered microcalcifications that radiologists would use in their diagnosis. For determination of objective similarity measures, an artificial neural network (ANN) was employed. The ANN was trained with the average subjective similarity ratings as teacher and selected image features as input data. The ANN was trained to learn the relationship between the image features and the radiologists' similarity ratings; therefore, once the training was completed, the ANN was able to determine the similarity, called a psychophysical similarity measure, which was expected to be close to radiologists' impressions, for an unknown pair of clustered microcalcifications. By use of a leave-one-out test method, the best combination of features was selected. The correlation coefficient between the gold standard and the psychophysical similarity measure through the use of seven features was relatively high (r=0.71) and was comparable to the correlation coefficients between the ratings by one radiologist and the average ratings by nine radiologists (r=0.69 +/- 0.07). The correlation coefficient was improved compared to that of a distance-based method (r=0.58). The result indicated that similar images selected by the psychophysical similarity measure may be useful to radiologists in the diagnosis of clustered microcalcifications on mammograms.

Anniversary paper: History and status of CAD and quantitative image analysis: the role of Medical Physics and AAPM

The roles of physicists in medical imaging have expanded over the years, from the study of imaging systems (sources and detectors) and dose to the assessment of image quality and perception, the development of image processing techniques, and the development of image analysis methods to assist in detection and diagnosis. The latter is a natural extension of medical physicists' goals in developing imaging techniques to help physicians acquire diagnostic information and improve clinical decisions. Studies indicate that radiologists do not detect all abnormalities on images that are visible on retrospective review, and they do not always correctly characterize abnormalities that are found. Since the 1950s, the potential use of computers had been considered for analysis of radiographic abnormalities. In the mid-1980s, however, medical physicists and radiologists began major research efforts for computer-aided detection or computer-aided diagnosis (CAD), that is, using the computer output as an aid to radiologists-as opposed to a completely automatic computer interpretation-focusing initially on methods for the detection of lesions on chest radiographs and mammograms. Since then, extensive investigations of computerized image analysis for detection or diagnosis of abnormalities in a variety of 2D and 3D medical images have been conducted. The growth of CAD over the past 20 years has been tremendous-from the early days of time-consuming film digitization and CPU-intensive computations on a limited number of cases to its current status in which developed CAD approaches are evaluated rigorously on large clinically relevant databases. CAD research by medical physicists includes many aspects-collecting relevant normal and pathological cases; developing computer algorithms appropriate for the medical interpretation task including those for segmentation, feature extraction, and classifier design; developing methodology for assessing CAD performance; validating the algorithms using appropriate cases to measure performance and robustness; conducting observer studies with which to evaluate radiologists in the diagnostic task without and with the use of the computer aid; and ultimately assessing performance with a clinical trial. Medical physicists also have an important role in quantitative imaging, by validating the quantitative integrity of scanners and developing imaging techniques, and image analysis tools that extract quantitative data in a more accurate and automated fashion. As imaging systems become more complex and the need for better quantitative information from images grows, the future includes the combined research efforts from physicists working in CAD with those working on quantitative imaging systems to readily yield information on morphology, function, molecular structure, and more-from animal imaging research to clinical patient care. A historical review of CAD and a discussion of challenges for the future are presented here, along with the extension to quantitative image analysis.

An investigation of radiologists' perception of lesion similarity: observations with paired breast masses on mammograms and paired lung nodules on CT images

RATIONALE AND OBJECTIVES

We conducted an observer study to investigate whether radiologists can judge similarities in pairs of breast masses and lung nodules consistently and reproducibly.

MATERIALS AND METHODS

Institutional review board approval and informed observer consent were obtained. This study was compliant with the Health Insurance Portability and Accountability Act. We used eight pairs of breast masses on mammograms and eight pairs of lung nodules on computed tomographic images, for which subjective similarity ratings ranging from 0 to 1 were determined in our previous studies. From these, four sets of image pairs were created (ie, a set of eight mass pairs, a set of eight nodule pairs, and two mixed sets of four mass and four nodule pairs). Eight radiologists, including four breast radiologists and four chest radiologists, compared all combinations of the eight pairs in each set using a two-alternative forced-choice (2AFC) method to determine the similarity ranking scores by identifying which pair was more similar than the other pair based on the overall impression for diagnosis.

RESULTS

In the mass set and nodule set, the relationship between the average subjective similarity ratings and the average similarity ranking scores by 2AFC indicated very high correlations (r = 0.91 and 0.88). Moreover, in the two mixed sets, the correlations between the average subjective similarity ratings and the average similarity ranking scores were also very high (r = 0.90 and 0.98). Thus, radiologists were able to compare the similarities for pairs of lesions consistently, even in the unusual comparison of pairs of completely different types of lesions.

CONCLUSION

The subjective similarity of a pair of lesions in medical images can be quantified consistently by a group of radiologists. The concept of similarity of lesions in medical images can be subjected to rigorous scientific research and investigation in the future.

Computer-aided diagnosis in medical imaging: historical review, current status and future potential

Computer-aided diagnosis (CAD) has become one of the major research subjects in medical imaging and diagnostic radiology. In this article, the motivation and philosophy for early development of CAD schemes are presented together with the current status and future potential of CAD in a PACS environment. With CAD, radiologists use the computer output as a "second opinion" and make the final decisions. CAD is a concept established by taking into account equally the roles of physicians and computers, whereas automated computer diagnosis is a concept based on computer algorithms only. With CAD, the performance by computers does not have to be comparable to or better than that by physicians, but needs to be complementary to that by physicians. In fact, a large number of CAD systems have been employed for assisting physicians in the early detection of breast cancers on mammograms. A CAD scheme that makes use of lateral chest images has the potential to improve the overall performance in the detection of lung nodules when combined with another CAD scheme for PA chest images. Because vertebral fractures can be detected reliably by computer on lateral chest radiographs, radiologists' accuracy in the detection of vertebral fractures would be improved by the use of CAD, and thus early diagnosis of osteoporosis would become possible. In MRA, a CAD system has been developed for assisting radiologists in the detection of intracranial aneurysms. On successive bone scan images, a CAD scheme for detection of interval changes has been developed by use of temporal subtraction images. In the future, many CAD schemes could be assembled as packages and implemented as a part of PACS. For example, the package for chest CAD may include the computerized detection of lung nodules, interstitial opacities, cardiomegaly, vertebral fractures, and interval changes in chest radiographs as well as the computerized classification of benign and malignant nodules and the differential diagnosis of interstitial lung diseases. In order to assist in the differential diagnosis, it would be possible to search for and retrieve images (or lesions) with known pathology, which would be very similar to a new unknown case, from PACS when a reliable and useful method has been developed for quantifying the similarity of a pair of images for visual comparison by radiologists.

Content-based retrieval of mammograms using visual features related to breast density patterns

This paper describes part of content-based image retrieval (CBIR) system that has been developed for mammograms. Details are presented of methods implemented to derive measures of similarity based upon structural characteristics and distributions of density of the fibroglandular tissue, as well as the anatomical size and shape of the breast region as seen on the mammogram. Well-known features related to shape, size, and texture (statistics of the gray-level histogram, Haralick's texture features, and moment-based features) were applied, as well as less-explored features based in the Radon domain and granulometric measures. The Kohonen self-organizing map (SOM) neural network was used to perform the retrieval operation. Performance evaluation was done using precision and recall curves obtained from comparison between the query and retrieved images. The proposed methodology was tested with 1,080 mammograms, including craniocaudal and mediolateral-oblique views. Precision rates obtained are in the range from 79% to 83% considering the total image set. Considering the first 50% of the retrieved mages, the precision rates are in the range from 78% to 83%; the rates are in the range from 79% to 86% considering the first 25% of the retrieved images. Results obtained indicate the potential of the implemented methodology to serve as a part of a CBIR system for mammography.

Experimental determination of subjective similarity for pairs of clustered microcalcifications on mammograms: observer study results

Presentation of images of lesions similar to that of an unknown lesion might be useful to radiologists in distinguishing between benign and malignant clustered microcalcifications on mammograms. Investigators have been developing computerized schemes to select similar images from large databases. However, whether selected images are really similar in appearance is not examined for most of the schemes. In order to retrieve images that are useful to radiologists, the selected images must be similar from radiologists' diagnostic points of view. Therefore, in this study, the data of radiologists' subjective similarity for pairs of clustered microcalcification images were obtained from a number of observers, and the intra- and inter-observer variations and the intergroup correlations were determined to investigate whether reliable similarity ratings by human observers can be determined. Nineteen images of clustered microcalcifications, each of which was paired with six other images, were selected for the observer study. Thus, subjective similarity ratings for 114 pairs of clustered microcalcifications were determined by each observer. Thirteen breast, ten general, and ten nonradiologists participated in the observer study; some of them completed the study multiple times. Although the intraobserver variations for the individual readings and the interobserver variations for pairs of observers were not small, the interobserver agreements were improved by taking the average of readings by the same observers. When the similarity ratings by a number of observers were averaged among the groups of breast, general, and nonradiologists, the mean differences of the ratings between the groups decreased, and good concordance correlations (0.846, 0.817, and 0.785) between the groups were obtained. The result indicates that reliable similarity ratings can be determined by use of this method, and the average similarity ratings by breast radiologists can be considered meaningful and useful for the development and evaluation of a computerized scheme for selection of similar images.

Computer analysis of computed tomography scans of the lung: a survey

Current computed tomography (CT) technology allows for near isotropic, submillimeter resolution acquisition of the complete chest in a single breath hold. These thin-slice chest scans have become indispensable in thoracic radiology, but have also substantially increased the data load for radiologists. Automating the analysis of such data is, therefore, a necessity and this has created a rapidly developing research area in medical imaging. This paper presents a review of the literature on computer analysis of the lungs in CT scans and addresses segmentation of various pulmonary structures, registration of chest scans, and applications aimed at detection, classification and quantification of chest abnormalities. In addition, research trends and challenges are identified and directions for future research are discussed.

A method to improve visual similarity of breast masses for an interactive computer-aided diagnosis environment

The purpose of this study was to develop and test a method for selecting "visually similar" regions of interest depicting breast masses from a reference library to be used in an interactive computer-aided diagnosis (CAD) environment. A reference library including 1000 malignant mass regions and 2000 benign and CAD-generated false-positive regions was established. When a suspicious mass region is identified, the scheme segments the region and searches for similar regions from the reference library using a multifeature based k-nearest neighbor (KNN) algorithm. To improve selection of reference images, we added an interactive step. All actual masses in the reference library were subjectively rated on a scale from 1 to 9 as to their "visual margins speculations". When an observer identifies a suspected mass region during a case interpretation he/she first rates the margins and the computerized search is then limited only to regions rated as having similar levels of spiculation (within +/-1 scale difference). In an observer preference study including 85 test regions, two sets of the six "similar" reference regions selected by the KNN with and without the interactive step were displayed side by side with each test region. Four radiologists and five nonclinician observers selected the more appropriate ("similar") reference set in a two alternative forced choice preference experiment. All four radiologists and five nonclinician observers preferred the sets of regions selected by the interactive method with an average frequency of 76.8% and 74.6%, respectively. The overall preference for the interactive method was highly significant (p < 0.001). The study demonstrated that a simple interactive approach that includes subjectively perceived ratings of one feature alone namely, a rating of margin "spiculation," could substantially improve the selection of "visually similar" reference images.

Investigation of psychophysical measure for evaluation of similar images for mammographic masses: preliminary results

We investigated a psychophysical similarity measure for selection of images similar to those of unknown masses on mammograms, which may assist radiologists in the distinction between benign and malignant masses. Sixty pairs of masses were selected from 1445 mass images prepared for this study, which were obtained from the Digital Database for Screening Mammography by the University of South Florida. Five radiologists provided subjective similarity ratings for these 60 pairs of masses based on the overall impression for diagnosis. Radiologists' subjective ratings were marked on a continuous rating scale and quantified between 0 and 1, which correspond to pairs not similar at all and pairs almost identical, respectively. By use of the subjective ratings as "gold standard," similarity measures based on the Euclidean distance between pairs in feature space and the psychophysical measure were determined. For determination of the psychophysical similarity measure, an artificial neural network (ANN) was employed to learn the relationship between radiologists' average subjective similarity ratings and computer-extracted image features. To evaluate the usefulness of the similarity measures, the agreement with the radiologists' subjective similarity ratings was assessed in terms of correlation coefficients between the average subjective ratings and the similarity measures. A commonly used similarity measure based on the Euclidean distance was moderately correlated (r=0.644) with the radiologists' average subjective ratings, whereas the psychophysical measure by use of the ANN was highly correlated (r=0.798). The preliminary result indicates that a psychophysical similarity measure would be useful in the selection of images similar to those of unknown masses on mammograms.

Computer-Aided Diagnosis in Thoracic CT

Computer-aided diagnosis (CAD) provides a computerized diagnostic result as a "second opinion" to assist radiologists in the diagnosis of various diseases by use of medical images. CAD has become a practical clinical approach in diagnostic radiology, although, at present, primarily in the area of detection of breast cancer in mammograms. Currently, a large research effort has been devoted to the detection and classification of various lung diseases in thoracic computed tomography (CT) images. We describe in this article the current status of the development of CAD schemes in thoracic CT, including nodule detection, distinction between benign and malignant nodules, and detection, characterization, and differential diagnosis of diffuse lung disease. Observer performance studies indicate that these CAD schemes would be useful in clinical practice by providing radiologists with computer output as a "second opinion."

Computer aided characterization of the solitary pulmonary nodule using volumetric and contrast enhancement features

RATIONALE AND OBJECTIVES

To investigate the utility of a computer-aided diagnosis (CAD) in the task of differentiating malignant nodules from benign nodules based on quantitative features extracted from volumetric thin section CT image data acquired before and after the injection of contrast media.

MATERIALS AND METHODS

35 volumetric CT datasets of solitary pulmonary nodules (SPN) with proven diagnoses (19 malignant/16 benign) were contoured by a thoracic radiologist. All studies had at least a baseline series obtained without contrast media and at least one series following an intravenous contrast injection at 45, 90, 180, and 360 seconds. Two separate contours were created for each nodule: one including only the solid portion and another including the ground-glass component, if any, of the nodule. For each contour 31 features were calculated that measured the attenuation, shape, and enhancement of the nodule due to the injection of contrast. These features were input into a feature selection step and three different classifiers to determine if the diagnosis could be predicted from the resulting feature vector. In addition, observer input was introduced to two of the classifiers as an a priori probability of malignancy and the resulting performance was compared. Training and testing was conducted in a resubstitution and leave-one-out fashion and performance was evaluated using ROC analysis.

RESULTS

In a leave-one-out testing methodology, the classifiers achieved areas under the ROC curves AZ that ranged from 0.69 to 0.92. A classifier based on logistic regression performed the best with an AZ of 0.92 while a classifier based on quadratic discriminant analysis performed the poorest (AZ, 0.69). The AZ increased when using a priori observer input in most cases reaching a maximum of 0.95.

CONCLUSION

Based on this initial work with a limited number of nodules in our dataset, it appears that CAD using volumetric and contrast-enhanced data has the potential to assist radiologists in the task of differentiating solitary pulmonary nodules and in the management of these patients. Further studies with an increased number of patients are required to validate these results.

Computer-aided diagnostic scheme for distinction between benign and malignant nodules in thoracic low-dose CT by use of massive training artificial neural network

Low-dose helical computed tomography (LDCT) is being applied as a modality for lung cancer screening. It may be difficult, however, for radiologists to distinguish malignant from benign nodules in LDCT. Our purpose in this study was to develop a computer-aided diagnostic (CAD) scheme for distinction between benign and malignant nodules in LDCT scans by use of a massive training artificial neural network (MTANN). The MTANN is a trainable, highly nonlinear filter based on an artificial neural network. To distinguish malignant nodules from six different types of benign nodules, we developed multiple MTANNs (multi-MTANN) consisting of six expert MTANNs that are arranged in parallel. Each of the MTANNs was trained by use of input CT images and teaching images containing the estimate of the distribution for the "likelihood of being a malignant nodule," i.e., the teaching image for a malignant nodule contains a two-dimensional Gaussian distribution and that for a benign nodule contains zero. Each MTANN was trained independently with ten typical malignant nodules and ten benign nodules from each of the six types. The outputs of the six MTANNs were combined by use of an integration ANN such that the six types of benign nodules could be distinguished from malignant nodules. After training of the integration ANN, our scheme provided a value related to the "likelihood of malignancy" of a nodule, i.e., a higher value indicates a malignant nodule, and a lower value indicates a benign nodule. Our database consisted of 76 primary lung cancers in 73 patients and 413 benign nodules in 342 patients, which were obtained from a lung cancer screening program on 7847 screenees with LDCT for three years in Nagano, Japan. The performance of our scheme for distinction between benign and malignant nodules was evaluated by use of receiver operating characteristic (ROC) analysis. Our scheme achieved an Az (area under the ROC curve) value of 0.882 in a round-robin test. Our scheme correctly identified 100% (76/76) of malignant nodules as malignant, whereas 48% (200/413) of benign nodules were identified correctly as benign. Therefore, our scheme may be useful in assisting radiologists in the diagnosis of lung nodules in LDCT.

Toward automated segmentation of the pathological lung in CT

Conventional methods of lung segmentation rely on a large gray value contrast between lung fields and surrounding tissues. These methods fail on scans with lungs that contain dense pathologies, and such scans occur frequently in clinical practice. We propose a segmentation-by-registration scheme in which a scan with normal lungs is elastically registered to a scan containing pathology. When the resulting transformation is applied to a mask of the normal lungs, a segmentation is found for the pathological lungs. As a mask of the normal lungs, a probabilistic segmentation built up out of the segmentations of 15 registered normal scans is used. To refine the segmentation, voxel classification is applied to a certain volume around the borders of the transformed probabilistic mask. Performance of this scheme is compared to that of three other algorithms: a conventional, a user-interactive and a voxel classification method. The algorithms are tested on 10 three-dimensional thin-slice computed tomography volumes containing high-density pathology. The resulting segmentations are evaluated by comparing them to manual segmentations in terms of volumetric overlap and border positioning measures. The conventional and user-interactive methods that start off with thresholding techniques fail to segment the pathologies and are outperformed by both voxel classification and the refined segmentation-by-registration. The refined registration scheme enjoys the additional benefit that it does not require pathological (hand-segmented) training data.

Current status and future potential of computer-aided diagnosis in medical imaging

Computer-aided diagnosis (CAD) has become one of the major research subjects in medical imaging and diagnostic radiology. The basic concept of CAD is to provide a computer output as a second opinion to assist radiologists' image interpretation by improving the accuracy and consistency of radiological diagnosis and also by reducing the image reading time. In this article, a number of CAD schemes are presented, with emphasis on potential clinical applications. These schemes include: (1) detection and classification of lung nodules on digital chest radiographs; (2) detection of nodules in low dose CT; (3) distinction between benign and malignant nodules on high resolution CT; (4) usefulness of similar images for distinction between benign and malignant lesions; (5) quantitative analysis of diffuse lung diseases on high resolution CT; and (6) detection of intracranial aneurysms in magnetic resonance angiography. Because CAD can be applied to all imaging modalities, all body parts and all kinds of examinations, it is likely that CAD will have a major impact on medical imaging and diagnostic radiology in the 21st century.

Computer-aided diagnosis of the solitary pulmonary nodule

RATIONALE AND OBJECTIVES

We sought to investigate the utility of a computer-aided diagnosis in the task of differentiating malignant nodules from benign nodules based on single thin-section computed tomography image data.

MATERIALS AND METHODS

Eighty-one thin-section computed tomography data sets of solitary pulmonary nodules with proven diagnoses (48 malignant and 33 benign) were contoured manually on a single representative slice by a thoracic radiologist (>10 years of experience). Two separate contours were created for each nodule, one including only the solid portion of the nodule and one including any ground-glass components. For each contour 75 features were calculated that measured the attenuation, shape, and texture of the nodule. These features were than input into a feature selection step and four different classifiers to determine if the diagnosis could be predicted from the feature vector. Training and testing was conducted in a resubstitution and leave-one-out fashion and performance was evaluated using ROC techniques.

RESULTS

In a leave-one-out testing methodology the classifiers resulted with areas under the ROC curve (A(Z)) that ranged from 0.68 to 0.92. When evaluating with resubstitution the A(Z) ranged from 0.93 to 1.00.

CONCLUSION

Computer-aided diagnosis has the potential to assist radiologists in the task of differentiating solitary pulmonary nodules and in the management of these patients.

Solitary pulmonary nodule diagnosis on CT: results of an observer study

RATIONALE AND OBJECTIVES

To investigate the performance of observers with different levels of experience in distinguishing between benign and malignant solitary pulmonary nodules (SPN) on CT, and to determine the effects on interpretation of three different conditions: image data alone, the addition of clinical data, and the addition of output from a computer-aided diagnosis (CAD) system.

MATERIALS AND METHODS

28 thin-section CT datasets of SPNs with proven diagnoses (15 malignant and 13 benign) were used to measure observer performance. Readers were categorized according to their experience and read the cases in random order. For each case readers were asked to assign a level of confidence on a scale from 0.0-1.0 (0.0 benign, 1.0 malignant) for the diagnosis of the nodule. Each reader scored the cases based on review of image data alone (phase 1), then with limited clinical data (phase 2), and finally with CAD output (phase 3). To assess performance, multiple reader multiple case (MRMC) receiver operating characteristic (ROC) analysis was used.

RESULTS

2 thoracic radiologists, 1 thoracic radiology fellow, 2 nonthoracic radiologists, and 3 radiology residents read the cases. The average area under the ROC curve for all readers (A(z)) at each stage was 0.68, 0.75, and 0.81, for image data alone, with clinical data, and with CAD output respectively. The difference in performance between phases (2 and 3) and (1 and 3) was significantly different (P = 0.018 and P = 0.020). However, the difference between phases (1 and 2) was not significantly different (P = 0.155).

CONCLUSION

Diagnostic performance increased significantly with the addition of CAD output. With further validation CAD output may play a significant role in SPN management.

Overview on research and development of computer-aided diagnostic schemes

Computer-aided diagnosis (CAD) has become a practical clinical approach in diagnostic radiology, although at present only in the area of detection of breast cancer in mammograms. Current research efforts have been focused on detection and classification of images of many different types of lesions in a number of organs, obtained with various imaging modalities. It is likely that the present results of CAD are only at the tip of the iceberg. Although automated computer diagnosis is a concept based on computer algorithms only, CAD is a concept established by taking into account equally the roles of physicians and computers. The effect of CAD on differential diagnosis has already indicated that the performance level is high, and that CAD would be ready for clinical trials and commercialization efforts. The presentation of images similar to those of an unknown case may be useful as a supplemental tool for CAD in the differential diagnosis.

Radiologists' performance for differentiating benign from malignant lung nodules on high-resolution CT using computer-estimated likelihood of malignancy

OBJECTIVE

The purpose of our study was to evaluate whether a computer-aided diagnosis (CAD) scheme can assist radiologists in distinguishing small benign from malignant lung nodules on high-resolution CT (HRCT).

MATERIALS AND METHODS

We developed an automated computerized scheme for determining the likelihood of malignancy of lung nodules on multiple HRCT slices; the likelihood estimate was obtained from various objective features of the nodules using linear discriminant analysis. The data set used in this observer study consisted of 28 primary lung cancers (6-20 mm) and 28 benign nodules. Cancer cases included nodules with pure ground-glass opacity, mixed ground-glass opacity, and solid opacity. Benign nodules were selected by matching their size and pattern to the malignant nodules. Consecutive region-of-interest images for each nodule on HRCT were displayed for interpretation in stacked mode on a cathode ray tube monitor. The images were presented to 16 radiologists-first without and then with the computer output-who were asked to indicate their confidence level regarding the malignancy of a nodule. Performance was evaluated by receiver operating characteristic (ROC) analysis.

RESULTS

The area under the ROC curve (Az value) of the CAD scheme alone was 0.831 for distinguishing benign from malignant nodules. The average Az value for radiologists was improved with the aid of the CAD scheme from 0.785 to 0.853 by a statistically significant level (p = 0.016). The radiologists' diagnostic performance with the CAD scheme was more accurate than that of the CAD scheme alone (p < 0.05) and also that of radiologists alone.

CONCLUSION

CAD has the potential to improve radiologists' diagnostic accuracy in distinguishing small benign nodules from malignant ones on HRCT.