An "intelligent" workstation for computer-aided diagnosis

AI in medical imaging grand challenges: translation from competition to research benefit and patient care

Artificial intelligence (AI), in one form or another, has been a part of medical imaging for decades. The recent evolution of AI into approaches such as deep learning has dramatically accelerated the application of AI across a wide range of radiologic settings. Despite the promises of AI, developers and users of AI technology must be fully aware of its potential biases and pitfalls, and this knowledge must be incorporated throughout the AI system development pipeline that involves training, validation, and testing. Grand challenges offer an opportunity to advance the development of AI methods for targeted applications and provide a mechanism for both directing and facilitating the development of AI systems. In the process, a grand challenge centralizes (with the challenge organizers) the burden of providing a valid benchmark test set to assess performance and generalizability of participants' models and the collection and curation of image metadata, clinical/demographic information, and the required reference standard. The most relevant grand challenges are those designed to maximize the open-science nature of the competition, with code and trained models deposited for future public access. The ultimate goal of AI grand challenges is to foster the translation of AI systems from competition to research benefit and patient care. Rather than reference the many medical imaging grand challenges that have been organized by groups such as MICCAI, RSNA, AAPM, and grand-challenge.org, this review assesses the role of grand challenges in promoting AI technologies for research advancement and for eventual clinical implementation, including their promises and limitations.

Past, Present, and Future of Machine Learning and Artificial Intelligence for Breast Cancer Screening

Breast cancer screening has evolved substantially over the past few decades because of advancements in new image acquisition systems and novel artificial intelligence (AI) algorithms. This review provides a brief overview of the history, current state, and future of AI in breast cancer screening and diagnosis along with challenges involved in the development of AI systems. Although AI has been developing for interpretation tasks associated with breast cancer screening for decades, its potential to combat the subjective nature and improve the efficiency of human image interpretation is always expanding. The rapid advancement of computational power and deep learning has increased greatly in AI research, with promising performance in detection and classification tasks across imaging modalities. Most AI systems, based on human-engineered or deep learning methods, serve as concurrent or secondary readers, that is, as aids to radiologists for a specific, well-defined task. In the future, AI may be able to perform multiple integrated tasks, making decisions at the level of or surpassing the ability of humans. Artificial intelligence may also serve as a partial primary reader to streamline ancillary tasks, triaging cases or ruling out obvious normal cases. However, before AI is used as an independent, autonomous reader, various challenges need to be addressed, including explainability and interpretability, in addition to repeatability and generalizability, to ensure that AI will provide a significant clinical benefit to breast cancer screening across all populations.

Potential clinical impact of advanced imaging and computer-aided diagnosis in chest radiology: importance of radiologist's role and successful observer study

This review paper is based on our research experience in the past 30 years. The importance of radiologists' role is discussed in the development or evaluation of new medical images and of computer-aided detection (CAD) schemes in chest radiology. The four main topics include (1) introducing what diseases can be included in a research database for different imaging techniques or CAD systems and what imaging database can be built by radiologists, (2) understanding how radiologists' subjective judgment can be combined with technical objective features to improve CAD performance, (3) sharing our experience in the design of successful observer performance studies, and (4) finally, discussing whether the new images and CAD systems can improve radiologists' diagnostic ability in chest radiology. In conclusion, advanced imaging techniques and detection/classification of CAD systems have a potential clinical impact on improvement of radiologists' diagnostic ability, for both the detection and the differential diagnosis of various lung diseases, in chest radiology.

Computer-aided diagnosis in chest radiography

We have developed computer-aided diagnosis (CAD) schemes for the detection of lung nodules, interstitial lung diseases, interval changes, and asymmetric opacities, and also for the differential diagnosis of lung nodules and interstitial lung diseases on chest radiographs. Observer performance studies indicate clearly that radiologists' diagnostic accuracy was improved significantly when radiologists used a computer output in their interpretations of chest radiographs. In addition, the automated recognition methods for the patient and the projection view by use of chest radiographs were useful for integrating the chest CAD schemes into the picture-archiving and communication system (PACS).

A ranklet-based image representation for mass classification in digital mammograms

Regions of interest (ROIs) found on breast radiographic images are classified as either tumoral mass or normal tissue by means of a support vector machine classifier. Classification features are the coefficients resulting from the specific image representation used to encode each ROI. Pixel and wavelet image representations have already been discussed in one of our previous works. To investigate the possibility of improving classification performances, a novel nonparametric, orientation-selective, and multiresolution image representation is developed and evaluated, namely a ranklet image representation. A dataset consisting of 1000 ROIs representing biopsy-proven tumoral masses (either benign or malignant) and 5000 ROIs representing normal breast tissue is used. ROIs are extracted from the digital database for screening mammography collected by the University of South Florida. Classification performances are evaluated using the area Az under the receiver operating characteristic curve. By achieving Az values of 0.978 +/- 0.003 and 90% sensitivity with a false positive fraction value of 4.5%, experiments demonstrate classification results higher than those reached by the previous image representations. In particular, the improvement on the Az value over that achieved by the wavelet image representation is statistically relevant with the two-tailed p value <0.0001. Besides, owing to the tolerance that the ranklet image representation reveals to variations in the ROIs' gray-level intensity histogram, this approach discloses to be robust also when tested on radiographic images having gray-level intensity histogram remarkably different from that used for training.

Computerized analysis of mesothelioma on CT scans

Computed tomography (CT) has become an essential tool for monitoring the progression of mesothelioma tumor or a patient's response to treatment. Despite the accepted role of CT for the evaluation of mesothelioma, however, the current standard practice remains the manual measurement of tumor extent on CT scans. The process of manual measurement acquisition is tedious and subject to inconsistency. We have quantitatively assessed the variability of manual mesothelioma measurements both in baseline CT scans and in the context of tumor response classification across temporally sequential scans. To facilitate the acquisition of mesothelioma measurements, we have developed and evaluated computerized methods for the assessment of mesothelioma tumor thickness on CT scans; we anticipate that such computer-assisted approaches will make the radiologic assessment of mesothelioma more efficient, thus facilitating the establishment of improved clinical protocols.

Comprendre et utiliser le CAD (Aide informatisée au diagnostic) en mammographie

Generalization of breast screening programs requires efficient double reading of mammograms, which allows reduction of false negative interpretations, but it may be difficult to achieve. CAD (Computed Aided Detection) systems are dramatically improving and can now assist in the detection of suspicious mammographic lesions, either suspicious microcalcifications, masses or architectural distortion. Characterization of the lesions is improving as well. CAD mammography might complete or substitute to "human" double reading. The aim of this review is to present the main CAD systems commercially available, review the principles of CAD and discuss the results of CAD mammography. Specifically, the role of CAD within breast screening program, according to the results of recent prospective studies will be discussed.

Automated detection of lung nodules in CT scans: effect of image reconstruction algorithm

We have investigated the effect of computed tomography (CT) image reconstruction algorithm on the performance of our automated lung nodule detection method. Commercial CT scanners offer a choice of several algorithms for the reconstruction of projection data into transaxial images. Different algorithms produce images with substantially different properties that are apparent not only quantitatively, but also through visual assessment. During some clinical thoracic CT examinations, patient scans are reconstructed with multiple reconstruction algorithms. Thirty-eight such cases were collected to form two databases: one with patient projection data reconstructed with the "standard" reconstruction algorithm and the other with the same patient projection data reconstructed with the "lung" reconstruction algorithm. The automated nodule detection method was applied to both databases. This method is based on gray-level-thresholding techniques to segment the lung regions from each CT section to create a segmented lung volume. Further gray-level-thresholding techniques are applied within the segmented lung volume to identify a set of lung nodule candidates. Rule-based and linear discriminant classifiers are used to differentiate between lung nodule candidates that correspond to actual nodules and those that correspond to non-nodules. The automated method that was applied to both databases was exactly the same, except that the classifiers were calibrated separately for each database. For comparison, the classifier then was trained on one database and tested independently on the other database. When applied to the databases in this manner, the automated method demonstrated overall a similar level of performance, indicating an encouraging degree of robustness.

Use of artificial neural networks (computer analysis) in the diagnosis of microcalcifications on mammography

INTRODUCTION/OBJECTIVE

The purpose of this study was to evaluate a computer based method for differentiating malignant from benign clustered microcalcifications, comparing it with the performance of three physicians.

METHODS AND MATERIAL

Materials for the study are 240 suspicious microcalcifications on mammograms from 220 female patients who underwent breast biopsy, following hook wire localization under mammographic guidance. The histologic findings were malignant in 108 cases (45%) and benign in 132 cases (55%). Those clusters were analyzed by a computer program and eight features of the calcifications (density, number, area, brightness, diameter average, distance average, proximity average, perimeter compacity average) were quantitatively estimated by a specific artificial neural network. Human input was limited to initial identification of the calcifications. Three physicians-observers were also evaluated for the malignant or benign nature of the clustered microcalcifications.

RESULTS

The performance of the artificial network was evaluated by receiver operating characteristics (ROC) curves. ROC curves were also generated for the performance of each observer and for the three observers as a group. The ROC curves for the computer and for the physicians were compared and the results are:area under the curve (AUC) value for computer is 0.937, for physician-1 is 0.746, for physician-2 is 0.785, for physician-3 is 0.835 and for physicians as a group is 0.810. The results of the Student's t-test for paired data showed statistically significant difference between the artificial neural network and the physicians' performance, independently and as a group.

DISCUSSION AND CONCLUSION

Our study showed that computer analysis achieves statistically significantly better performance than that of physicians in the classification of malignant and benign calcifications. This method, after further evaluation and improvement, may help radiologists and breast surgeons in better predictive estimation of suspicious clustered microcalcifications and reduce the number of biopsies for non-palpable benign lesions.

Knowledge-based computer-aided detection of masses on digitized mammograms: a preliminary assessment

The purpose of this work was to develop and evaluate a computer-aided detection (CAD) scheme for the improvement of mass identification on digitized mammograms using a knowledge-based approach. Three hundred pathologically verified masses and 300 negative, but suspicious, regions, as initially identified by a rule-based CAD scheme, were randomly selected from a large clinical database for development purposes. In addition, 500 different positive and 500 negative regions were used to test the scheme. This suspicious region pruning scheme includes a learning process to establish a knowledge base that is then used to determine whether a previously identified suspicious region is likely to depict a true mass. This is accomplished by quantitatively characterizing the set of known masses, measuring "similarity" between a suspicious region and a "known" mass, then deriving a composite "likelihood" measure based on all "known" masses to determine the state of the suspicious region. To assess the performance of this method, receiver-operating characteristic (ROC) analyses were employed. Using a leave-one-out validation method with the development set of 600 regions, the knowledge-based CAD scheme achieved an area under the ROC curve of 0.83. Fifty-one percent of the previously identified false-positive regions were eliminated, while maintaining 90% sensitivity. During testing of the 1,000 independent regions, an area under the ROC curve as high as 0.80 was achieved. Knowledge-based approaches can yield a significant reduction in false-positive detections while maintaining reasonable sensitivity. This approach has the potential of improving the performance of other rule-based CAD schemes.

Radiologist detection of microcalcifications with and without computer-aided detection: a comparative study

AIM

To compare the sensitivity and specificity of microcalcification detection by radiologists alone and assisted by a computer-aided detection (CAD) system.

MATERIALS AND METHODS

Films of 106 patients were masked, randomized, digitized and analysed by the CAD-system. Five readers interpreted the original mammograms and were blinded to demographics, medical history and earlier films. Forty-two mammograms with malignant microcalcifications, 40 with benign microcalcifications and 24 normal mammograms were included. Results were recorded on a standardized image interpretation form. The mammograms with suspicious areas flagged by the CAD-system were displayed on mini-monitors and immediately re-reviewed. The interpretation was again recorded on a new copy of the standard form and classified according to six groups.

RESULTS

Forty-one out of 42 (98%) malignant microcalcifications and 32 of 40 (80%) benign microcalcifications were flagged by the CAD-system. There was an average of 1.2 markers per image. The sensitivity for malignant microcalcifications detection by mammographers without and with the CAD-system ranged from 81% to 98% and from 88% to 98%, respectively. The mean difference without and with CAD-system was 2.2% (range 0-7%).

CONCLUSION

No statistically significant changes in sensitivity were found when experienced mammographers were assisted by the CAD-system, with no significant compromise in specificity.

Computer-aided detection and diagnosis of breast cancer

The limitations of radiologists when interpreting mammogram examinations provides a reasonable, if not compelling, basis for application of computer techniques that have the potential to improve diagnostic performance. Computer algorithms, at their present state of development, show great promise for clinical use. It can be expected that such use will only improve as computer technology and computer methods continue to become more formidable. The eventual role of computers in mammographic detection and diagnosis has not been fully defined, but their effect on practice may one day be very significant.

Computerized detection of pulmonary nodules on CT scans

Helical computed tomography (CT) is the most sensitive imaging modality for detection of pulmonary nodules. However, a single CT examination produces a large quantity of image data. Therefore, a computerized scheme has been developed to automatically detect pulmonary nodules on CT images. This scheme includes both two- and three-dimensional analyses. Within each section, gray-level thresholding methods are used to segment the thorax from the background and then the lungs from the thorax. A rolling ball algorithm is applied to the lung segmentation contours to avoid the loss of juxtapleural nodules. Multiple gray-level thresholds are applied to the volumetric lung regions to identify nodule candidates. These candidates represent both nodules and normal pulmonary structures. For each candidate, two- and three-dimensional geometric and gray-level features are computed. These features are merged with linear discriminant analysis to reduce the number of candidates that correspond to normal structures. This method was applied to a 17-case database. Receiver operating characteristic (ROC) analysis was used to evaluate the automated classifier. Results yielded an area under the ROC curve of 0.93 in the task of classifying candidates detected during thresholding as nodules or nonnodules.

Computer-aided diagnosis in radiology: potential and pitfalls

Computer-aided diagnosis (CAD) may be defined as a diagnosis made by a physician who takes into account the computer output as a second opinion. The purpose of CAD is to improve the diagnostic accuracy and the consistency of the radiologists' image interpretation. This article is to provide a brief overview of some of CAD schemes for detection and differential diagnosis of pulmonary nodules and interstitial opacities in chest radiographs as well as clustered micro-calcifications and masses in mammograms. ROC analysis clearly indicated that the radiologists' performances were significantly improved when the computer output was available. An intelligent CAD workstation was developed for detection of breast lesions in mammograms. Results obtained from the first 10,000 cases indicated the potential of CAD in detecting approximately one-half of 'missed' breast cancer.

Automated seeded lesion segmentation on digital mammograms

Segmenting lesions is a vital step in many computerized mass-detection schemes for digital (or digitized) mammograms. We have developed two novel lesion segmentation techniques-one based on a single feature called the radial gradient index (RGI) and one based on simple probabilistic models to segment mass lesions, or other similar nodular structures, from surrounding background. In both methods a series of image partitions is created using gray-level information as well as prior knowledge of the shape of typical mass lesions. With the former method the partition that maximizes the RGI is selected. In the latter method, probability distributions for gray-levels inside and outside the partitions are estimated, and subsequently used to determine the probability that the image occurred for each given partition. The partition that maximizes this probability is selected as the final lesion partition (contour). We tested these methods against a conventional region growing algorithm using a database of biopsy-proven, malignant lesions and found that the new lesion segmentation algorithms more closely match radiologists' outlines of these lesions. At an overlap threshold of 0.30, gray level region growing correctly delineates 62% of the lesions in our database while the RGI and probabilistic segmentation algorithms correctly segment 92% and 96% of the lesions, respectively.

Automated computerized classification of malignant and benign masses on digitized mammograms

RATIONALE AND OBJECTIVES

To develop a method for differentiating malignant from benign masses in which a computer automatically extracts lesion features and merges them into an estimated likelihood of malignancy.

MATERIALS AND METHODS

Ninety-five mammograms depicting masses in 65 patients were digitized. Various features related to the margin and density of each mass were extracted automatically from the neighborhoods of the computer-identified mass regions. Selected features were merged into an estimated likelihood of malignancy by, using three different automated classifiers. The performance of the three classifiers in distinguishing between benign and malignant masses was evaluated by receiver operating characteristic analysis and compared with the performance of an experienced mammographer and that of five less experienced mammographers.

RESULTS

Our computer classification scheme yielded an area under the receiver operating characteristic curve (Az) value of 0.94, which was similar to that for an experienced mammographer (Az = 0.91) and was statistically significantly higher than the average performance of the radiologists with less mammographic experience (Az = 0.81) (P = .013). With the database used, the computer scheme achieved, at 100% sensitivity, a positive predictive value of 83%, which was 12% higher than that for the performance of the experienced mammographer and 21% higher than that for the average performance of the less experienced mammographers (P < .0001).

CONCLUSION

Automated computerized classification schemes may be useful in helping radiologists distinguish between benign and malignant masses and thus reducing the number of unnecessary biopsies.

Computer-Aided Diagnosis of Breast Cancer on Mammograms

Computer-aided diagnosis (CAD) is a diagnosis made by a physician who takes into account the computer output of quantitative analysis of mammograms. CAD schemes in mammography have been developed to detect lesions such as clustered microcalcifications and masses, and also to distinguish between benign and malignant lesions. Computerized schemes are composed of three major steps which are image processing, quantitation of image features, and data classification. The current performance level of detecting clustered microcalcifications by computer is approximately 85% at a false positive rate of 0.5 per mammogram, whereas the detection accuracy of masses is approximately 90% at a false positive rate of 2 per mammogram. Observer performance studies indicated that computer output can improve the performance of radiologists in detecting clustered microcalcifications by increasing the dtection accuracy to 90% from 80% at a specificity of 90%. The automated classification of clustered microcalcifications is based on quantitative analysis of image features of individual microcalcifications and cluster, followed by artificial neural networks (ANNs) for data classification. With our database, the computer scheme correctly identified 82% of patients with benign lesions, all of whom had biopsies (ie, the radiologist thought the microcalcifications were suspicious for malignancy), and 100% of patients with malignant lesions. On the same set of images, the average of five radiologists was only 27% correct in classifying lesions as benign at 100% sensitivity. The automated classification of masses is made by the quantitation of image features of masses together with a rule-based and ANNs method for data classification. The computer scheme achieved, at 100% sensitivity, a positive predictive value of 83%, which was 12% higher than that of the experienced mammographer and 21% higher than that of the average of less experienced mammographers. The first prototype intelligent workstation for mammography was developed at the University of Chicago, and applied to approximately 12000 screening cases for the detection of early breast cancers. Promising initial results were obtained with the workstation.

Classification of normal and abnormal lungs with interstitial diseases by rule-based method and artificial neural networks

We devised an automated classification scheme by using the rule-based method plus artificial neural networks (ANN) for distinction between normal and abnormal lungs with interstitial disease in digital chest radiographs. Four measures used in the classification scheme are determined from the texture and geometric-pattern feature analyses. The rms variation and the first moment of the power spectrum of lung patterns are determined as measures for the texture analysis. In addition, the total area of nodular opacities and the total length of linear opacities are determined as measures for the geometric-pattern feature analysis. In our classification scheme with these measures, we identify obviously normal and abnormal cases first by the rule-based method and then ANN is applied for the remaining difficult cases. The rule-based plus ANN method provided a sensitivity of 0.926 at the specificity of 0.900, which was considerably improved compared to performance of either the rule-based method alone or ANNs alone.

An improved computer-assisted diagnostic scheme using wavelet transform for detecting clustered microcalcifications in digital mammograms

RATIONALE AND OBJECTIVES

We evaluated the potential usefulness of a computer-assisted diagnostic (CAD) scheme incorporating the wavelet transform for detecting clustered microcalcifications in mammograms.

METHODS

A wavelet transform technique was applied to the detection of clustered microcalcifications. We examined several wavelets to study their effectiveness in detecting subtle microcalcifications. We used a database consisting of 39 mammograms containing 41 clusters of microcalcifications. The performance of the wavelet-based CAD scheme was evaluated using free-response receiver operating characteristic analysis.

RESULTS

The CAD scheme with the wavelet transform was useful in detecting some of the subtle microcalcifications that were not detected by our previous scheme, which was based on the difference-image technique. When the two schemes were combined, the overall performance was improved to a sensitivity of approximately 95%, with a false-positive rate of 1.5 clusters per image.

CONCLUSION

The wavelet transform approach can improve the detection of subtle clustered microcalcifications.

Teleradiology: state of the art in clinical environment

Teleradiology may even be a tautology since communication is an essential part of radiology. The value of a radiological report is strictly dependent on how fast and clearly it is transmitted to the referring physician. The radiologist has therefore learnt to combine imaging expertise with communicator's skills. However, the term teleradiology is usually reserved to define the instances in which computers and digital networks help to reduce the limitations imposed by the physical distance between radiologists and between radiologist and referring physician. In this paper the main clinical applications of teleradiology will be presented, distinguishing between intra-institutional teleradiology (that involves more units belonging to the same department or to the same hospital) and inter-institutional teleradiology (more classically aimed at primary interpretation and subspecialty consultation). Finally a brief mention will be made concerning the potential for Internet to provide radiologists with new professional and educational opportunities.