Evaluation of the accuracy of a computer-aided diagnosis (CAD) system in breast ultrasound according to the radiologist's experience

Artificial intelligence in breast ultrasound: application in clinical practice

Ultrasound (US) is a widely accessible and extensively used tool for breast imaging. It is commonly used as an additional screening tool, especially for women with dense breast tissue. Advances in artificial intelligence (AI) have led to the development of various AI systems that assist radiologists in identifying and diagnosing breast lesions using US. This article provides an overview of the background and supporting evidence for the use of AI in hand held breast US. It discusses the impact of AI on clinical workflow, covering breast cancer detection, diagnosis, prediction of molecular subtypes, evaluation of axillary lymph node status, and response to neoadjuvant chemotherapy. Additionally, the article highlights the potential significance of AI in breast US for low and middle income countries.

Knowledge Tensor-Aided Breast Ultrasound Image Assistant Inference Framework

Breast cancer is one of the most prevalent cancers in women nowadays, and medical intervention at an early stage of cancer can significantly improve the prognosis of patients. Breast ultrasound (BUS) is a widely used tool for the early screening of breast cancer in primary care hospitals but it relies heavily on the ability and experience of physicians. Accordingly, we propose a knowledge tensor-based Breast Imaging Reporting and Data System (BI-RADS)-score-assisted generalized inference model, which uses the BI-RADS score of senior physicians as the gold standard to construct a knowledge tensor model to infer the benignity and malignancy of breast tumors and axes the diagnostic results against those of junior physicians to provide an aid for breast ultrasound diagnosis. The experimental results showed that the diagnostic AUC of the knowledge tensor constructed using the BI-RADS characteristics labeled by senior radiologists achieved 0.983 (95% confidential interval (CI) = 0.975-0.992) for benign and malignant breast cancer, while the diagnostic performance of the knowledge tensor constructed using the BI-RADS characteristics labeled by junior radiologists was only 0.849 (95% CI = 0.823-0.876). With the knowledge tensor fusion, the AUC is improved to 0.887 (95% CI = 0.864-0.909). Therefore, our proposed knowledge tensor can effectively help reduce the misclassification of BI-RADS characteristics by senior radiologists and, thus, improve the diagnostic performance of breast-ultrasound-assisted diagnosis.

A review of the current state of the computer-aided diagnosis (CAD) systems for breast cancer diagnosis

Breast cancer is one of the most common cancers affecting females worldwide. Early detection and diagnosis of breast cancer may aid in timely treatment, reducing the mortality rate to a great extent. To diagnose breast cancer, computer-aided diagnosis (CAD) systems employ a variety of imaging modalities such as mammography, computerized tomography, magnetic resonance imaging, ultrasound, and histological imaging. CAD and breast-imaging specialists are in high demand for early detection and diagnosis. This system has the potential to enhance the partiality of traditional histopathological image analysis. This review aims to highlight the recent advancements and the current state of CAD systems for breast cancer detection using different modalities.

Dual-Intended Deep Learning Model for Breast Cancer Diagnosis in Ultrasound Imaging

Automated medical data analysis demonstrated a significant role in modern medicine, and cancer diagnosis/prognosis to achieve highly reliable and generalizable systems. In this study, an automated breast cancer screening method in ultrasound imaging is proposed. A convolutional deep autoencoder model is presented for simultaneous segmentation and radiomic extraction. The model segments the breast lesions while concurrently extracting radiomic features. With our deep model, we perform breast lesion segmentation, which is linked to low-dimensional deep-radiomic extraction (four features). Similarly, we used high dimensional conventional imaging throughputs and applied spectral embedding techniques to reduce its size from 354 to 12 radiomics. A total of 780 ultrasound images—437 benign, 210, malignant, and 133 normal—were used to train and validate the models in this study. To diagnose malignant lesions, we have performed training, hyperparameter tuning, cross-validation, and testing with a random forest model. This resulted in a binary classification accuracy of 78.5% (65.1–84.1%) for the maximal (full multivariate) cross-validated model for a combination of radiomic groups.

Association of Clinician Diagnostic Performance With Machine Learning-Based Decision Support Systems: A Systematic Review

IMPORTANCE

An increasing number of machine learning (ML)-based clinical decision support systems (CDSSs) are described in the medical literature, but this research focuses almost entirely on comparing CDSS directly with clinicians (human vs computer). Little is known about the outcomes of these systems when used as adjuncts to human decision-making (human vs human with computer).

OBJECTIVES

To conduct a systematic review to investigate the association between the interactive use of ML-based diagnostic CDSSs and clinician performance and to examine the extent of the CDSSs' human factors evaluation.

EVIDENCE REVIEW

A search of MEDLINE, Embase, PsycINFO, and grey literature was conducted for the period between January 1, 2010, and May 31, 2019. Peer-reviewed studies published in English comparing human clinician performance with and without interactive use of an ML-based diagnostic CDSSs were included. All metrics used to assess human performance were considered as outcomes. The risk of bias was assessed using Quality Assessment of Diagnostic Accuracy Studies (QUADAS-2) and Risk of Bias in Non-Randomised Studies-Intervention (ROBINS-I). Narrative summaries were produced for the main outcomes. Given the heterogeneity of medical conditions, outcomes of interest, and evaluation metrics, no meta-analysis was performed.

FINDINGS

A total of 8112 studies were initially retrieved and 5154 abstracts were screened; of these, 37 studies met the inclusion criteria. The median number of participating clinicians was 4 (interquartile range, 3-8). Of the 107 results that reported statistical significance, 54 (50%) were increased by the use of CDSSs, 4 (4%) were decreased, and 49 (46%) showed no change or an unclear change. In the subgroup of studies carried out in representative clinical settings, no association between the use of ML-based diagnostic CDSSs and improved clinician performance could be observed. Interobserver agreement was the commonly reported outcome whose change was the most strongly associated with CDSS use. Four studies (11%) reported on user feedback, and, in all but 1 case, clinicians decided to override at least some of the algorithms' recommendations. Twenty-eight studies (76%) were rated as having a high risk of bias in at least 1 of the 4 QUADAS-2 core domains, and 6 studies (16%) were considered to be at serious or critical risk of bias using ROBINS-I.

CONCLUSIONS AND RELEVANCE

This systematic review found only sparse evidence that the use of ML-based CDSSs is associated with improved clinician diagnostic performance. Most studies had a low number of participants, were at high or unclear risk of bias, and showed little or no consideration for human factors. Caution should be exercised when estimating the current potential of ML to improve human diagnostic performance, and more comprehensive evaluation should be conducted before deploying ML-based CDSSs in clinical settings. The results highlight the importance of considering supported human decisions as end points rather than merely the stand-alone CDSSs outputs.

Images Are Data: A Breast Imaging Perspective on a Contemporary Paradigm

BACKGROUND

 Considering radiological examinations not as mere images, but as a source of data, has become the key paradigm in the diagnostic imaging field. This change of perspective is particularly popular in breast imaging. It allows breast radiologists to apply algorithms derived from computer science, to realize innovative clinical applications, and to refine already established methods. In this context, the terminology "imaging biomarker", "radiomics", and "artificial intelligence" are of pivotal importance. These methods promise noninvasive, low-cost (e. g., in comparison to multigene arrays), and workflow-friendly (automated, only one examination, instantaneous results, etc.) delivery of clinically relevant information.

METHODS AND RESULTS

 This paper is designed as a narrative review on the previously mentioned paradigm. The focus is on key concepts in breast imaging and important buzzwords are explained. For all areas of breast imaging, exemplary studies and potential clinical use cases are discussed.

CONCLUSION

 Considering radiological examination as a source of data may optimize patient management by guiding individualized breast cancer diagnosis and oncologic treatment in the age of precision medicine.

KEY POINTS

  · In conventional breast imaging, examinations are interpreted based on patterns perceivable by visual inspection.. · The radiomics paradigm treats breast images as a source of data, containing information beyond what is visible to our eyes.. · This results in radiomic signatures that may be considered as imaging biomarkers, as they provide diagnostic, predictive, and prognostic information.. · Radiomics derived imaging biomarkers may be used to individualize breast cancer treatment in the era of precision medicine.. · The concept and key research of radiomics in the field of breast imaging will be discussed in this narrative review..

CITATION FORMAT

· Dietzel M, Clauser P, Kapetas P et al. Images Are Data: A Breast Imaging Perspective on a Contemporary Paradigm. Fortschr Röntgenstr 2021; 193: 898 - 908.

CR-Unet-Based Ultrasonic Follicle Monitoring to Reduce Diameter Variability and Generate Area Automatically as a Novel Biomarker for Follicular Maturity

Follicle size is closely related to ovarian function and is an important biomarker in transvaginal ultrasound examinations for assessing follicular maturity during an assisted reproduction cycle. However, manual measurement is time consuming and subject to high inter- and intra- observer variability. Based on the deep learning model CR-Unet described in our previous study, the aim of our present study was to investigate further the feasibility of using this model in clinical practice by validating its performance in reducing the inter- and intra-observer variability of follicle diameter measurement. This study also investigated whether follicular area is a better biomarker than diameter in assessing follicular maturity. Data on 106 ovaries and 230 follicles collected from 80 cases of single follicular cycles and 26 cases of multiple follicular cycles constituted the validation set. Intra-observer variability was 0.973 and 0.982 for the senior sonographer and junior sonographer in single follicular cycles and 0.979 (0.971, 0.985) and 0.920 (0.892, 0.943) in multiple follicular cycles, respectively, while CR-Unet had no intra-group variation. Bland-Altman plot analysis indicated that the 95% limits of agreement between senior sonographer and CR-Unet (-2.1 to 1.1 mm, -2.02 to 0.75 mm) were smaller than those between senior sonographer and junior sonographer (-1.51 to 1.15 mm, -2.1 to 1.56 mm) in single and multiple follicular cycles. The average operating times of diameter measurement taken by the junior sonographer, senior sonographer and CR-Unet were 7.54 ± 1.8, 4.87 ± 0.84 and 1.66 ± 0.76 s, respectively (p < 0.001). Correlation analysis indicated that both manual and automated follicular area correlated better with follicular volume than diameter. The deep learning algorithm and the new biomarker of follicular area hold potential for clinical application of ultrasonic follicular monitoring.

Impact of Data Presentation on Physician Performance Utilizing Artificial Intelligence-Based Computer-Aided Diagnosis and Decision Support Systems

Ultrasound (US) is a valuable imaging modality used to detect primary breast malignancy. However, radiologists have a limited ability to distinguish between benign and malignant lesions on US, leading to false-positive and false-negative results, which limit the positive predictive value of lesions sent for biopsy (PPV3) and specificity. A recent study demonstrated that incorporating an AI-based decision support (DS) system into US image analysis could help improve US diagnostic performance. While the DS system is promising, its efficacy in terms of its impact also needs to be measured when integrated into existing clinical workflows. The current study evaluates workflow schemas for DS integration and its impact on diagnostic accuracy. The impact on two different reading methodologies, sequential and independent, was assessed. This study demonstrates significant accuracy differences between the two workflow schemas as measured by area under the receiver operating curve (AUC), as well as inter-operator variability differences as measured by Kendall’s tau-b. This evaluation has practical implications on the utilization of such technologies in diagnostic environments as compared to previous studies.

Evaluation of a deep learning-based computer-aided diagnosis system for distinguishing benign from malignant thyroid nodules in ultrasound images

PURPOSE

Computer-aided diagnosis (CAD) systems assist in solving subjective diagnosis problems that typically rely on personal experience. A CAD system has been developed to differentiate malignant thyroid nodules from benign thyroid nodules in ultrasound images based on deep learning methods. The diagnostic performance was compared between the CAD system and the experienced attending radiologists.

METHODS

The ultrasound image dataset for training the CAD system included 651 malignant nodules and 386 benign nodules while the database for testing included 422 malignant nodules and 128 benign nodules. All the nodules were confirmed by pathology results. In the proposed CAD system, a support vector machine (SVM) is used for classification and fused features which combined the deep features extracted by a convolutional neural network (CNN) with the hand-crafted features such as the histogram of oriented gradient (HOG), local binary patterns (LBP), and scale invariant feature transform (SIFT) were obtained. The optimal feature subset was formed by selecting these fused features based on the maximum class separation distance and used as the training sample for the SVM.

RESULTS

The accuracy, sensitivity, and specificity of the CAD system were 92.5%, 96.4%, and 83.1%, respectively, which were higher than those of the experienced attending radiologists. The areas under the ROC curves of the CAD system and the attending radiologists were 0.881 and 0.819, respectively.

CONCLUSIONS

The CAD system for thyroid nodules exhibited a better diagnostic performance than experienced attending radiologists. The CAD system could be a reliable supplementary tool to diagnose thyroid nodules using ultrasonography. Macroscopic features in ultrasound images, such as the margins and shape of thyroid nodules, could influence the diagnostic efficiency of the CAD system.

Evaluation of the Quadri-Planes Method in Computer-Aided Diagnosis of Breast Lesions by Ultrasonography: Prospective Single-Center Study

Background

Computer-aided diagnosis (CAD) is a tool that can help radiologists diagnose breast lesions by ultrasonography. Previous studies have demonstrated that CAD can help reduce the incidence of missed diagnoses by radiologists. However, the optimal method to apply CAD to breast lesions using diagnostic planes has not been assessed.

Objective

The aim of this study was to compare the performance of radiologists with different levels of experience when using CAD with the quadri-planes method to detect breast tumors.

Methods

From November 2018 to October 2019, we enrolled patients in the study who had a breast mass as their most prominent symptom. We assigned 2 ultrasound radiologists (with 1 and 5 years of experience, respectively) to read breast ultrasonography images without CAD and then to perform a second reading while applying CAD with the quadri-planes method. We then compared the diagnostic performance of the readers for the 2 readings (without and with CAD). The McNemar test for paired data was used for statistical analysis.

Results

A total of 331 patients were included in this study (mean age 43.88 years, range 17-70, SD 12.10), including 512 lesions (mean diameter 1.85 centimeters, SD 1.19; range 0.26-9.5); 200/512 (39.1%) were malignant, and 312/512 (60.9%) were benign. For CAD, the area under the receiver operating characteristic curve (AUC) improved significantly from 0.76 (95% CI 0.71-0.79) with the cross-planes method to 0.84 (95% CI 0.80-0.88; P<.001) with the quadri-planes method. For the novice reader, the AUC significantly improved from 0.73 (95% CI 0.69-0.78) for the without-CAD mode to 0.83 (95% CI 0.80-0.87; P<.001) for the combined-CAD mode with the quadri-planes method. For the experienced reader, the AUC improved from 0.85 (95% CI 0.81-0.88) to 0.87 (95% CI 0.84-0.91; P=.15). The kappa indicating consistency between the experienced reader and the novice reader for the combined-CAD mode was 0.63. For the novice reader, the sensitivity significantly improved from 60.0% for the without-CAD mode to 79.0% for the combined-CAD mode (P=.004). The specificity, negative predictive value, positive predictive value, and accuracy improved from 84.9% to 87.8% (P=.53), 76.8% to 86.7% (P=.07), 71.9% to 80.6% (P=.13), and 75.2% to 84.4% (P=.12), respectively. For the experienced reader, the sensitivity improved significantly from 76.0% for the without-CAD mode to 87.0% for the combined-CAD mode (P=.045). The NPV and accuracy moderately improved from 85.8% and 86.3% to 91.0% (P=.27) and 87.0% (P=.84), respectively. The specificity and positive predictive value decreased from 87.4% to 81.3% (P=.25) and from 87.2% to 93.0% (P=.16), respectively.

Conclusions

S-Detect is a feasible diagnostic tool that can improve the sensitivity, accuracy, and AUC of the quadri-planes method for both novice and experienced readers while also improving the specificity for the novice reader. It demonstrates important application value in the clinical diagnosis of breast cancer.

Trial Registration

ChiCTR.org.cn 1800019649; http://www.chictr.org.cn/showproj.aspx?proj=33094

Should We Ignore, Follow, or Biopsy? Impact of Artificial Intelligence Decision Support on Breast Ultrasound Lesion Assessment

OBJECTIVE. The objective of this study was to assess the impact of artificial intelligence (AI)-based decision support (DS) on breast ultrasound (US) lesion assessment. MATERIALS AND METHODS. A multicenter retrospective review of 900 breast lesions (470/900 [52.2%] benign; 430/900 [47.8%] malignant) on US by 15 physicians (11 radiologists, two surgeons, two obstetrician/gynecologists). An AI system (Koios DS for Breast, Koios Medical) evaluated images and assigned them to one of four categories: benign, probably benign, suspicious, and probably malignant. Each reader reviewed cases twice: 750 cases with US only or with US plus DS; 4 weeks later, cases were reviewed in the opposite format. One hundred fifty additional cases were presented identically in each session. DS and reader sensitivity, specificity, and positive likelihood ratios (PLRs) were calculated as well as reader AUCs with and without DS. The Kendall τ-b correlation coefficient was used to assess intraand interreader variability. RESULTS. Mean reader AUC for cases reviewed with US only was 0.83 (95% CI, 0.78-0.89); for cases reviewed with US plus DS, mean AUC was 0.87 (95% CI, 0.84-0.90). PLR for the DS system was 1.98 (95% CI, 1.78-2.18) and was higher than the PLR for all readers but one. Fourteen readers had better AUC with US plus DS than with US only. Mean Kendall τ-b for US-only interreader variability was 0.54 (95% CI, 0.53-0.55); for US plus DS, it was 0.68 (95% CI, 0.67-0.69). Intrareader variability improved with DS; class switching (defined as crossing from BI-RADS category 3 to BI-RADS category 4A or above) occurred in 13.6% of cases with US only versus 10.8% of cases with US plus DS (p = 0.04). CONCLUSION. AI-based DS improves accuracy of sonographic breast lesion assessment while reducing inter- and intraobserver variability.

Classification of Breast Ultrasound Tomography by Using Textural Analysis

Background: Ultrasound imaging has become one of the most widely utilized adjunct tools in breast cancer screening due to its advantages. The computer-aided detection of breast ultrasound is rapid development via significant features extracted from images. Objectives: The main aim was to identify features of breast ultrasound image that can facilitate reasonable classification of ultrasound images between malignant and benign lesions. Patients and Methods: This research was a retrospective study in which 85 cases (35 malignant [positive group] and 50 benign [negative group] with diagnostic reports) with ultrasound images were collected. The B-mode ultrasound images have manually selected regions of interest (ROI) for estimated features of an image. Then, a fractal dimensional (FD) image was generated from the original ROI by using the box-counting method. Both FD and ROI images were extracted features, including mean, standard deviation, skewness, and kurtosis. These extracted features were tested as significant by t-test, receiver operating characteristic (ROC) analysis and Kappa coefficient. Results: The statistical analysis revealed that the mean texture of images performed the best in differentiating benign versus malignant tumors. As determined by the ROC analysis, the appropriate qualitative values for the mean and the LR model were 0.85 and 0.5, respectively. The sensitivity, specificity, accuracy, positive predicted value (PPV), negative predicted value (NPV), and Kappa for the mean was 0.77, 0.84, 0.81, 0.77, 0.84, and 0.61, respectively. Conclusion: The presented method was efficient in classifying malignant and benign tumors using image textures. Future studies on breast ultrasound texture analysis could focus on investigations of edge detection, texture estimation, classification models, and image features.

Performance of Computer-Aided Diagnosis in Ultrasonography for Detection of Breast Lesions Less and More Than 2 cm: Prospective Comparative Study

Background

Computer-aided diagnosis (CAD) is used as an aid tool by radiologists on breast lesion diagnosis in ultrasonography. Previous studies demonstrated that CAD can improve the diagnosis performance of radiologists. However, the optimal use of CAD on breast lesions according to size (below or above 2 cm) has not been assessed.

Objective

The aim of this study was to compare the performance of different radiologists using CAD to detect breast tumors less and more than 2 cm in size.

Methods

We prospectively enrolled 261 consecutive patients (mean age 43 years; age range 17-70 years), including 398 lesions (148 lesions>2 cm, 79 malignant and 69 benign; 250 lesions≤2 cm, 71 malignant and 179 benign) with breast mass as the prominent symptom. One novice radiologist with 1 year of ultrasonography experience and one experienced radiologist with 5 years of ultrasonography experience were each assigned to read the ultrasonography images without CAD, and then again at a second reading while applying the CAD S-Detect. We then compared the diagnostic performance of the readers in the two readings (without and combined with CAD) with breast imaging. The McNemar test for paired data was used for statistical analysis.

Results

For the novice reader, the area under the receiver operating characteristic curve (AUC) improved from 0.74 (95% CI 0.67-0.82) from the without-CAD mode to 0.88 (95% CI 0.83-0.93; P<.001) at the combined-CAD mode in lesions≤2 cm. For the experienced reader, the AUC improved from 0.84 (95% CI 0.77-0.90) to 0.90 (95% CI 0.86-0.94; P=.002). In lesions>2 cm, the AUC moderately decreased from 0.81 to 0.80 (novice reader) and from 0.90 to 0.82 (experienced reader). The sensitivity of the novice and experienced reader in lesions≤2 cm improved from 61.97% and 73.23% at the without-CAD mode to 90.14% and 97.18% (both P<.001) at the combined-CAD mode, respectively.

Conclusions

S-Detect is a feasible diagnostic tool that can improve the sensitivity for both novice and experienced readers, while also improving the negative predictive value and AUC for lesions≤2 cm, demonstrating important application value in the clinical diagnosis of breast cancer.

Trial Registration

Chinese Clinical Trial Registry ChiCTR1800019649; http://www.chictr.org.cn/showprojen.aspx?proj=33094

Computer-aided diagnosis system for breast ultrasound images using deep learning

The purpose of this study was to develop a computer-aided diagnosis (CAD) system for the classification of malignant and benign masses in the breast using ultrasonography based on a convolutional neural network (CNN), a state-of-the-art deep learning technique. We explored the regions for the correct classification by generating a heat map that presented the important regions used by the CNN for human malignancy/benign classification. Clinical data was obtained from a large-scale clinical trial previously conducted by the Japan Association of Breast and Thyroid Sonology. Images of 1536 breast masses (897 malignant and 639 benign) confirmed by pathological examinations were collected, with each breast mass captured from various angles using an ultrasound (US) imaging probe. We constructed an ensemble network by combining two CNN models (VGG19 and ResNet152) fine-tuned on balanced training data with augmentation and used the mass-level classification method to enable the CNN to classify a given mass using all views. For an independent test set consisting of 154 masses (77 malignant and 77 benign), our network showed outstanding classification performance with a sensitivity of 90.9% (95% confidence interval 84.5-97.3), a specificity of 87.0% (79.5-94.5), and area under the curve (AUC) of 0.951 (0.916-0.987) compared to that of the two CNN models. In addition, our study indicated that the breast masses themselves were not detected by the CNN as important regions for correct mass classification. Collectively, this CNN-based CAD system is expected to assist doctors by improving the diagnosis of breast cancer in clinical practice.

Performance and Reading Time of Automated Breast US with or without Computer-aided Detection

BackgroundComputer-aided detection (CAD) systems may be used to help radiologists interpret automated breast (AB) US images. However, the optimal use of CAD with AB US has, to the knowledge of the authors, not been determined.PurposeTo compare the performance and reading time of different readers by using AB US CAD system to detect breast cancer in different reading modes.Materials and MethodsIn this retrospective study, 1485 AB US images (282 with malignant lesions, 695 with benign lesions, and 508 healthy) in 1452 women (mean age, 43.7 years; age range, 19-82 years) including 529 (36.4%) women who were asymptomatic were collected between 2016 and 2017. A CAD system was used to interpret the images. Three novice readers with 1-3 years of US experience and three experienced readers with 5-10 years of US experience were assigned to read AB US images without CAD, at a second reading (after the reader completed a full unaided interpretation), and at concurrent reading (use of CAD at the start of the assessment). Diagnostic performances and reading times were compared by using analysis of variance.ResultsFor all readers, the mean area under the receiver operating characteristic curve improved from 0.88 (95% confidence interval [CI]: 0.85, 0.91) at without-CAD mode to 0.91 (95% CI: 0.89, 0.92; P < .001) at the second-reading mode and 0.90 (95% CI: 0.89, 0.92; P = .002) at the concurrent-reading mode. The mean sensitivity of novice readers in women who were asymptomatic improved from 67% (95% CI: 63%, 74%) at without-CAD mode to 88% (95% CI: 84%, 89%) at both the second-reading mode and the concurrent-reading mode (P = .003). Compared with the without-CAD and second-reading modes, the mean reading time per volume of concurrent reading was 16 seconds (95% CI: 11, 22; P < .001) and 27 seconds (95% CI: 21, 32; P < .001) shorter, respectively.ConclusionComputer-aided detection (CAD) was helpful for novice readers to improve cancer detection at automated breast US in women who were asymptomatic. CAD was more efficient when used concurrently for all readers.© RSNA, 2019Online supplemental material is available for this article.See also the editorial by Slanetz in this issue.

Feasibility of computer-assisted diagnosis for breast ultrasound: the results of the diagnostic performance of S-detect from a single center in China

Objective

To investigate the feasibility of a CAD system S-detect on a database from a single Chinese medical center.

Materials and methods

An experienced radiologist performed breast US examinations and made assessments of 266 consecutive breast lesions in 227 patients. S-detect classified the lesions automatically in a dichotomous form. An in-training resident who was blind to both the US diagnostic results and histological results reviewed the images afterward. The final histological results were considered as the diagnostic gold standard. The diagnostic performances and interrater agreements were analyzed.

Results

A total of 266 focal breast lesions (161 benign lesions and 105 malignant lesions) were assessed in this study. S-detect had a lower sensitivity (87.07%) and a higher specificity (72.27%) compared with the experienced radiologist (sensitivity 98.1% and specificity 65.43%). The sensitivity and specificity of S-detect were better than that of the resident (sensitivity 82.86% and specificity 68.94%). The AUC value of S-detect (0.807) showed no significant difference with the experienced radiologist (0.817) and was higher than that of the resident (0.758). S-detect had moderate agreement with the experienced radiologist.

Conclusion

In this single-center study, a high level of diagnostic performance of S-detect on 266 breast lesions of Chinese women was observed. S-detect had almost equal diagnostic capacity with an experienced radiologist and performed better than a novice reader. S-detect was also distinguished for its high specificity. These results supported the feasibility of S-detect in aiding the diagnosis of breast lesions on an independent database.

Development of a computer-aided tool for the pattern recognition of facial features in diagnosing Turner syndrome: comparison of diagnostic accuracy with clinical workers

Technologies applied for the recognition of facial features in diagnosing certain disorders seem to be promising in reducing the medical burden and improve the efficiency. This pilot study aimed to develop a computer-assisted tool for the pattern recognition of facial features for diagnosing Turner syndrome (TS). Photographs of 54 patients with TS and 158 female controls were collected from July 2016 to May 2017. Finally, photographs of 32 patients with TS and 96 age-matched controls were included in the study that were further divided equally into training and testing groups. The process of automatic classification consisted of image preprocessing, facial feature extraction, feature reduction and fusion, automatic classification, and result presentation. A total of 27 physicians and 21 medical students completed a web-based test including the same photographs used in computer testing. After training, the automatic facial classification system for diagnosing TS achieved a 68.8% sensitivity and 87.5% specificity (and a 67.6% average sensitivity and 87.9% average specificity after resampling), which was significantly higher than the average sensitivity (57.4%, P < 0.001) and specificity (75.4%, P < 0.001) of 48 participants, respectively. The accuracy of this system was satisfactory and better than the diagnosis by clinicians. However, the system necessitates further improvement for achieving a high diagnostic accuracy in clinical practice.

Application of computer-aided diagnosis in breast ultrasound interpretation: improvements in diagnostic performance according to reader experience

Purpose

The purpose of this study was to evaluate the usefulness of applying computer-aided diagnosis (CAD) to breast ultrasound (US), depending on the reader's experience with breast imaging.

Methods

Between October 2015 and January 2016, two experienced readers obtained and analyzed the grayscale US images of 200 cases according to the Breast Imaging Reporting and Data System (BI-RADS) lexicon and categories. They additionally applied CAD (S-Detect) to analyze the lesions and made a diagnostic decision subjectively, based on grayscale US with CAD. For the same cases, two inexperienced readers analyzed the grayscale US images using the BI-RADS lexicon and categories, added CAD, and came to a subjective diagnostic conclusion. We then compared the diagnostic performance depending on the reader's experience with breast imaging.

Results

The sensitivity values for the experienced readers, inexperienced readers, and CAD (for experienced and inexperienced readers) were 91.7%, 75.0%, 75.0%, and 66.7%, respectively. The specificity values for the experienced readers, inexperienced readers, and CAD (for experienced and inexperienced readers) were 76.6%, 71.8%, 78.2%, and 76.1%, respectively. When diagnoses were made subjectively in combination with CAD, the specificity significantly improved (76.6% to 80.3%) without a change in the sensitivity (91.7%) in the experienced readers. After subjective combination with CAD, both of the sensitivity and specificity improved in the inexperienced readers (75.0% to 83.3% and 71.8% to 77.1%). In addition, the area under the curve improved for both the experienced and inexperienced readers (0.84 to 0.86 and 0.73 to 0.80) after the addition of CAD.

Conclusion

CAD is more useful for less experienced readers. Combining CAD with breast US led to improved specificity for both experienced and inexperienced readers.

Computer-aided diagnosis program for classifying the risk of hepatocellular carcinoma on MR images following liver imaging reporting and data system (LI-RADS)

PURPOSE

To develop and evaluate a computer-aided diagnosis (CAD) program for liver lesions on magnetic resonance (MR) images for classification of the risk of hepatocellular carcinoma (HCC) following the liver imaging reporting and data system (LI-RADS).

MATERIALS AND METHODS

Liver MR images from 41 patients with hyperenhancing liver lesions categorized as LR 3, 4, and 5 were evaluated by two radiologists. The major LI-RADS features of each index liver lesion were recorded, including size (maximum transverse diameter), presence of hyperenhancement, washout appearance, and capsule appearance. A CAD program was implemented to register MR images at different contrast-enhancement phases, segment liver lesions, extract lesion features, and classify lesions according to LI-RADS. The LI-RADS features quantified by CAD were compared with those assessed by radiologists using the intraclass correlation coefficient (ICC) and receiver operator curve (ROC) analyses. The LI-RADS categorization between CAD and radiologists was evaluated using the weighted Cohen's kappa coefficient.

RESULTS

The mean and standard deviation of the lesion diameters were 21 ± 11 mm (range, 7-70 mm) by radiologists and 22 ± 11 mm (range, 8-72 mm) by CAD (ICC, 0.96-0.97). The area under the curve (AUC) for the washout assessment by CAD was 0.79-0.93 with sensitivity 0.69-0.82 and specificity 0.79-1. The AUC for the capsule assessment by CAD was 0.79-0.9 with sensitivity 0.75-0.9 and specificity 0.82-0.96. The classifications by the radiologists and CAD coincided in 76-83% lesions (k = 0.57-0.71), while the agreements between radiologists were in 78% lesions (k = 0.59).

CONCLUSION

We developed a CAD program for liver lesions on MR images and showed a substantial agreement in the LI-RADS-based classification of the risk of HCCs between the CAD and radiologists.

LEVEL OF EVIDENCE

1 Technical Efficacy: Stage 1 J. Magn. Reson. Imaging 2018;47:710-722.

A Computer-Aided Diagnosis System Using Artificial Intelligence for the Diagnosis and Characterization of Thyroid Nodules on Ultrasound: Initial Clinical Assessment

BACKGROUND

An initial clinical assessment is described of a new, commercially available, computer-aided diagnosis (CAD) system using artificial intelligence (AI) for thyroid ultrasound, and its performance is evaluated in the diagnosis of malignant thyroid nodules and categorization of nodule characteristics.

METHODS

Patients with thyroid nodules with decisive diagnosis, whether benign or malignant, were consecutively enrolled from November 2015 to February 2016. An experienced radiologist reviewed the ultrasound image characteristics of the thyroid nodules, while another radiologist assessed the same thyroid nodules using the CAD system, providing ultrasound characteristics and a diagnosis of whether nodules were benign or malignant. The diagnostic performance and agreement of US characteristics between the experienced radiologist and the CAD system were compared.

RESULTS

In total, 102 thyroid nodules from 89 patients were included; 59 (57.8%) were benign and 43 (42.2%) were malignant. The CAD system showed a similar sensitivity as the experienced radiologist (90.7% vs. 88.4%, p > 0.99), but a lower specificity and a lower area under the receiver operating characteristic (AUROC) curve (specificity: 74.6% vs. 94.9%, p = 0.002; AUROC: 0.83 vs. 0.92, p = 0.021). Classifications of the ultrasound characteristics (composition, orientation, echogenicity, and spongiform) between radiologist and CAD system were in substantial agreement (κ = 0.659, 0.740, 0.733, and 0.658, respectively), while the margin showed a fair agreement (κ = 0.239).

CONCLUSION

The sensitivity of the CAD system using AI for malignant thyroid nodules was as good as that of the experienced radiologist, while specificity and accuracy were lower than those of the experienced radiologist. The CAD system showed an acceptable agreement with the experienced radiologist for characterization of thyroid nodules.

Foundation and methodologies in computer-aided diagnosis systems for breast cancer detection

Breast cancer is the most prevalent cancer that affects women all over the world. Early detection and treatment of breast cancer could decline the mortality rate. Some issues such as technical reasons, which related to imaging quality and human error, increase misdiagnosis of breast cancer by radiologists. Computer-aided detection systems (CADs) are developed to overcome these restrictions and have been studied in many imaging modalities for breast cancer detection in recent years. The CAD systems improve radiologists' performance in finding and discriminating between the normal and abnormal tissues. These procedures are performed only as a double reader but the absolute decisions are still made by the radiologist. In this study, the recent CAD systems for breast cancer detection on different modalities such as mammography, ultrasound, MRI, and biopsy histopathological images are introduced. The foundation of CAD systems generally consist of four stages: Pre-processing, Segmentation, Feature extraction, and Classification. The approaches which applied to design different stages of CAD system are summarised. Advantages and disadvantages of different segmentation, feature extraction and classification techniques are listed. In addition, the impact of imbalanced datasets in classification outcomes and appropriate methods to solve these issues are discussed. As well as, performance evaluation metrics for various stages of breast cancer detection CAD systems are reviewed.

Computerized Diagnostic Assistant for the Automatic Detection of Pneumothorax on Ultrasound: A Pilot Study

INTRODUCTION

Bedside thoracic ultrasound (US) can rapidly diagnose pneumothorax (PTX) with improved accuracy over the physical examination and without the need for chest radiography (CXR); however, US is highly operator dependent. A computerized diagnostic assistant was developed by the United States Army Institute of Surgical Research to detect PTX on standard thoracic US images. This computer algorithm is designed to automatically detect sonographic signs of PTX by systematically analyzing B-mode US video clips for pleural sliding and M-mode still images for the seashore sign. This was a pilot study to estimate the diagnostic accuracy of the PTX detection computer algorithm when compared to an expert panel of US trained physicians.

METHODS

This was a retrospective study using archived thoracic US obtained on adult patients presenting to the emergency department (ED) between 5/23/2011 and 8/6/2014. Emergency medicine residents, fellows, attending physicians, physician assistants, and medical students performed the US examinations and stored the images in the picture archive and communications system (PACS). The PACS was queried for all ED bedside US examinations with reported positive PTX during the study period along with a random sample of negatives. The computer algorithm then interpreted the images, and we compared the results to an independent, blinded expert panel of three physicians, each with experience reviewing over 10,000 US examinations.

RESULTS

Query of the PACS system revealed 146 bedside thoracic US examinations for analysis. Thirteen examinations were indeterminate and were excluded. There were 79 true negatives, 33 true positives, 9 false negatives, and 12 false positives. The test characteristics of the algorithm when compared to the expert panel were sensitivity 79% (95 % CI [63-89]) and specificity 87% (95% CI [77-93]). For the 20 images scored as highest quality by the expert panel, the algorithm demonstrated 100% sensitivity (95% CI [56-100]) and 92% specificity (95% CI [62-100]).

CONCLUSION

This novel computer algorithm has potential to aid clinicians with the identification of the sonographic signs of PTX in the absence of expert physician sonographers. Further refinement and training of the algorithm is still needed, along with prospective validation, before it can be utilized in clinical practice.

Application of Computer-Aided Diagnosis to the Sonographic Evaluation of Cervical Lymph Nodes

We initiated an observer study to evaluate a computerized system developed in our previous study for automatic extraction of 10 features and estimation of the malignancy probability of cervical nodes in sonograms. In the present study, five expert radiologists and five resident radiologists interpreted the sonograms of 178 nodes. The malignancy rating and patient management recommendation (biopsy or follow-up) were made without and then with the computer aid. Under these two reading conditions, the performances of radiologists and agreement among a group of radiologists were evaluated by using the receiver operating characteristic (ROC) analysis and the κ statistic, respectively. With the computer aid, the performances of radiologists improved significantly, as indicated by the increase in the area under the ROC curve (Az) from 0.843 to 0.896 (p = 0.031) and from 0.705 to 0.822 (p < 0.001), for the expert and resident groups, respectively. Agreement among all 10 radiologists improved from slight to moderate as indicated by an increase in the κ value from 0.195 to 0.421 (p < 0.001). The average performance of residents with aid (Az = 0.822) was close to that of experts without aid (Az = 0.843). Results indicate that computer-aided diagnosis is useful to improve radiologist performance (especially that of inexperienced radiologists) in the ultrasonographic evaluation of cervical nodes and to reduce variability among radiologists.

Evaluating geodesic active contours in microcalcifications segmentation on mammograms

Breast cancer is the most commonly occurring type of cancer among women, and it is the major cause of female cancer-related deaths worldwide. Its incidence is increasing in developed as well as developing countries. Efficient strategies to reduce the high death rates due to breast cancer include early detection and tumor removal in the initial stages of the disease. Clinical and mammographic examinations are considered the best methods for detecting the early signs of breast cancer; however, these techniques are highly dependent on breast characteristics, equipment quality, and physician experience. Computer-aided diagnosis (CADx) systems have been developed to improve the accuracy of mammographic diagnosis; usually such systems may involve three steps: (i) segmentation; (ii) parameter extraction and selection of the segmented lesions and (iii) lesions classification. Literature considers the first step as the most important of them, as it has a direct impact on the lesions characteristics that will be used in the further steps. In this study, the original contribution is a microcalcification segmentation method based on the geodesic active contours (GAC) technique associated with anisotropic texture filtering as well as the radiologists' knowledge. Radiologists actively participate on the final step of the method, selecting the final segmentation that allows elaborating an adequate diagnosis hypothesis with the segmented microcalcifications presented in a region of interest (ROI). The proposed method was assessed by employing 1000 ROIs extracted from images of the Digital Database for Screening Mammography (DDSM). For the selected ROIs, the rate of adequately segmented microcalcifications to establish a diagnosis hypothesis was at least 86.9%, according to the radiologists. The quantitative test, based on the area overlap measure (AOM), yielded a mean of 0.52±0.20 for the segmented images, when all 2136 segmented microcalcifications were considered. Moreover, a statistical difference was observed between the AOM values for large and small microcalcifications. The proposed method had better or similar performance as compared to literature for microcalcifications with maximum diameters larger than 460μm. For smaller microcalcifications the performance was limited.

Computer-aided diagnosis improves detection of small intracranial aneurysms on MRA in a clinical setting

BACKGROUND AND PURPOSE

MRA is widely accepted as a noninvasive diagnostic tool for the detection of intracranial aneurysms, but detection is still a challenging task with rather low detection rates. Our aim was to examine the performance of a computer-aided diagnosis algorithm for detecting intracranial aneurysms on MRA in a clinical setting.

MATERIALS AND METHODS

Aneurysm detectability was evaluated retrospectively in 48 subjects with and without computer-aided diagnosis by 6 readers using a clinical 3D viewing system. Aneurysms ranged from 1.1 to 6.0 mm (mean = 3.12 mm, median = 2.50 mm). We conducted a multireader, multicase, double-crossover design, free-response, observer-performance study on sets of images from different MRA scanners by using DSA as the reference standard. Jackknife alternative free-response operating characteristic curve analysis with the figure of merit was used.

RESULTS

For all readers combined, the mean figure of merit improved from 0.655 to 0.759, indicating a change in the figure of merit attributable to computer-aided diagnosis of 0.10 (95% CI, 0.03-0.18), which was statistically significant (F(1,47) = 7.00, P = .011). Five of the 6 radiologists had improved performance with computer-aided diagnosis, primarily due to increased sensitivity.

CONCLUSIONS

In conditions similar to clinical practice, using computer-aided diagnosis significantly improved radiologists' detection of intracranial DSA-confirmed aneurysms of ≤6 mm.

Screening MR imaging versus screening ultrasound: pros and cons

Data support greater sensitivity of MR imaging compared with mammography and ultrasound in high-risk populations, in particular BRCA 1 and BRCA 2 carriers. Screening ultrasound improves cancer yield versus mammography alone in high-risk patients and in patients with dense breasts and is less expensive. Drawbacks include low positive predictive value, operator dependence, and significant physician time expenditure. Advances, such as refinement of automated whole-breast ultrasound, new outcomes data from ultrasound-detected masses in BI-RADS 3 and 4a categories, and development of new MR imaging sequences that allow rapid screening, potentially without use of contrast, will likely reveal the most appropriate tool over time.

Improving accuracy in reporting CT scans of oncology patients: assessing the effect of education and feedback interventions on the application of the Response Evaluation Criteria in Solid Tumors (RECIST) criteria

RATIONALE AND OBJECTIVES

In February 2010, our radiology department adopted the use of the Response Evaluation Criteria in Solid Tumors (RECIST) 1.1 criteria for newly diagnosed oncology patients. Prior to staff used RECIST 1.1, we hypothesized that education and feedback interventions could help clarify differences between RECIST 1.0 and the newly adopted RECIST 1.1 guidelines and result in appropriate and accurate utilization of both reporting systems. This study evaluates the effect of education and feedback interventions on the accuracy of computed tomography (CT) reporting using RECIST criteria.

MATERIALS AND METHODS

Consecutive CT scan reports and images were retrospectively reviewed during three different periods to assess for compliance and adherence to RECIST guidelines. Data collected included interpreting faculty, resident, type, and total number of errors per report. Significance testing of differences between cohorts was performed using an unequal variance t-test. Group 1 (baseline): RECIST 1.0 used; prior to adoption of RECIST 1.1 criteria. Group 2 (post distributed educational materials): Following adoption of RECIST 1.1 criteria and distribution of educational materials. Group 3 (post audit and feedback): Following the audit and feedback intervention.

RESULTS

The percentage of reports with errors decreased from 30% (baseline) to 28% (group 2) to 22% (group 3). Only the difference in error rate between the baseline and group 3 was significant (P = .03).

CONCLUSION

The combination of distributed educational materials and audit and feedback interventions improved the quality of radiology reports requiring RECIST criteria by reducing the number of studies with errors.