Single reading with computer-aided detection for screening mammography

Understanding the Landscape of Multi-Cancer Detection Tests: The Current Data and Clinical Considerations

Multi-cancer detection (MCD) tests are blood-based assays that screen for multiple cancers concurrently and offer a promising approach to improve early cancer detection and screening uptake. To date, there have been two prospective interventional studies evaluating MCD tests as a screening tool in human subjects. No MCD tests are currently approved by the FDA, but there is one commercially available MCD test. Ongoing trials continue to assess the efficacy, safety, and cost implications of MCD tests. In this review, we discuss the performance of CancerSEEK and Galleri, two leading MCD platforms, and discuss the clinical consideration for the broader application of this new technology.

Three scans are better than two for follow-up: An automatic method for finding missed and misidentified lesions in cross-sectional follow-up of oncology patients

PURPOSE

Missed and misidentified neoplastic lesions in longitudinal studies of oncology patients are pervasive and may affect the evaluation of the disease status. Two newly identified patterns of lesion changes, lone lesions and non-consecutive lesion changes, may help radiologists to detect these lesions. This study evaluated a new interpretation revision workflow of lesion annotations in three or more consecutive scans based on these suspicious patterns.

METHODS

The interpretation revision workflow was evaluated on manual and computed lesion annotations in longitudinal oncology patient studies. For the manual revision, a senior radiologist and a senior neurosurgeon (the readers) manually annotated the lesions in each scan and later revised their annotations to identify missed and misidentified lesions with the workflow using the automatically detected patterns. For the computerized revision, lesion annotations were first computed with a previously trained nnU-Net and were then automatically revised with an AI-based method that automates the workflow readers' decisions. The evaluation included 67 patient studies with 2295 metastatic lesions in lung (19 patients, 83 CT scans, 1178 lesions), liver (18 patients, 77 CECT scans, 800 lesions) and brain (30 patients, 102 T1W-Gad MRI scans, 317 lesions).

RESULTS

Revision of the manual lesion annotations revealed 120 missed lesions and 20 misidentified lesions in 31 out of 67 (46%) studies. The automatic revision reduced the number of computed missed lesions by 55 and computed misidentified lesions by 164 in 51 out of 67 (76%) studies.

CONCLUSION

Automatic analysis of three or more consecutive volumetric scans helps find missed and misidentified lesions and may improve the evaluation of temporal changes of oncological lesions.

Breast Cancer Detection with an Ensemble of Deep Learning Networks Using a Consensus-Adaptive Weighting Method

Breast cancer's high mortality rate is often linked to late diagnosis, with mammograms as key but sometimes limited tools in early detection. To enhance diagnostic accuracy and speed, this study introduces a novel computer-aided detection (CAD) ensemble system. This system incorporates advanced deep learning networks-EfficientNet, Xception, MobileNetV2, InceptionV3, and Resnet50-integrated via our innovative consensus-adaptive weighting (CAW) method. This method permits the dynamic adjustment of multiple deep networks, bolstering the system's detection capabilities. Our approach also addresses a major challenge in pixel-level data annotation of faster R-CNNs, highlighted in a prominent previous study. Evaluations on various datasets, including the cropped DDSM (Digital Database for Screening Mammography), DDSM, and INbreast, demonstrated the system's superior performance. In particular, our CAD system showed marked improvement on the cropped DDSM dataset, enhancing detection rates by approximately 1.59% and achieving an accuracy of 95.48%. This innovative system represents a significant advancement in early breast cancer detection, offering the potential for more precise and timely diagnosis, ultimately fostering improved patient outcomes.

Artificial Intelligence in Breast Cancer Diagnosis and Personalized Medicine

Breast cancer is a significant cause of cancer-related mortality in women worldwide. Early and precise diagnosis is crucial, and clinical outcomes can be markedly enhanced. The rise of artificial intelligence (AI) has ushered in a new era, notably in image analysis, paving the way for major advancements in breast cancer diagnosis and individualized treatment regimens. In the diagnostic workflow for patients with breast cancer, the role of AI encompasses screening, diagnosis, staging, biomarker evaluation, prognostication, and therapeutic response prediction. Although its potential is immense, its complete integration into clinical practice is challenging. Particularly, these challenges include the imperatives for extensive clinical validation, model generalizability, navigating the "black-box" conundrum, and pragmatic considerations of embedding AI into everyday clinical environments. In this review, we comprehensively explored the diverse applications of AI in breast cancer care, underlining its transformative promise and existing impediments. In radiology, we specifically address AI in mammography, tomosynthesis, risk prediction models, and supplementary imaging methods, including magnetic resonance imaging and ultrasound. In pathology, our focus is on AI applications for pathologic diagnosis, evaluation of biomarkers, and predictions related to genetic alterations, treatment response, and prognosis in the context of breast cancer diagnosis and treatment. Our discussion underscores the transformative potential of AI in breast cancer management and emphasizes the importance of focused research to realize the full spectrum of benefits of AI in patient care.

Mammography diagnosis of breast cancer screening through machine learning: a systematic review and meta-analysis

Breast cancer was the fourth leading cause of cancer-related death worldwide, and early mammography screening could decrease the breast cancer mortality. Artificial intelligence (AI)-assisted diagnose system based on machine learning (ML) methods can help improve the screening accuracy and efficacy. This study aimed to systematically review and make a meta-analysis on the diagnostic accuracy of mammography diagnosis of breast cancer through various ML methods. Springer Link, Science Direct (Elsevier), IEEE Xplore, PubMed and Web of Science were searched for relevant studies published from January 2000 to September 2021. The study was registered with the PROSPERO International Prospective Register of Systematic Reviews (protocol no. CRD42021284227). A Quality Assessment of Diagnostic Accuracy Studies-2 (QUADAS-2) was used to assess the included studies, and reporting was evaluated using the Preferred Reporting Items for Systematic reviews and Meta-Analyses (PRISMA). The pooled summary estimates for sensitivity, specificity, the area under the receiver operating characteristic curve (AUC) for three ML methods (convolutional neural network [CNN], artificial neural network [ANN], support vector machine [SVM]) were calculated. A total of 32 studies with 23,804 images were included in the meta-analysis. The overall pooled estimate for sensitivity, specificity and AUC was 0.914 [95% CI 0.868-0.945], 0.916 [95% CI 0.873-0.945] and 0.945 for mammography diagnosis of breast cancer through three ML methods. The pooled sensitivity, specificity and AUC of CNN were 0.961 [95% CI 0.886-0.988], 0.950 [95% CI 0.924-0.967] and 0.974. The pooled sensitivity, specificity and AUC of ANN were 0.837 [95% CI 0.772-0.886], 0.894 [95% CI 0.764-0.957] and 0.881. The pooled sensitivity, specificity and AUC of SVM were 0.889 [95% CI 0.807-0.939], 0.843 [95% CI 0.724-0.916] and 0.913. Machine learning methods (especially CNN) show excellent performance in mammography diagnosis of breast cancer screening based on retrospective studies. More rigorous prospective studies are needed to evaluate the longitudinal performance of AI.

Challenges of implementing computer-aided diagnostic models for neuroimages in a clinical setting

Advances in artificial intelligence have cultivated a strong interest in developing and validating the clinical utilities of computer-aided diagnostic models. Machine learning for diagnostic neuroimaging has often been applied to detect psychological and neurological disorders, typically on small-scale datasets or data collected in a research setting. With the collection and collation of an ever-growing number of public datasets that researchers can freely access, much work has been done in adapting machine learning models to classify these neuroimages by diseases such as Alzheimer's, ADHD, autism, bipolar disorder, and so on. These studies often come with the promise of being implemented clinically, but despite intense interest in this topic in the laboratory, limited progress has been made in clinical implementation. In this review, we analyze challenges specific to the clinical implementation of diagnostic AI models for neuroimaging data, looking at the differences between laboratory and clinical settings, the inherent limitations of diagnostic AI, and the different incentives and skill sets between research institutions, technology companies, and hospitals. These complexities need to be recognized in the translation of diagnostic AI for neuroimaging from the laboratory to the clinic.

Framing the fallibility of Computer-Aided Detection aids cancer detection

Computer-Aided Detection (CAD) has been proposed to help operators search for cancers in mammograms. Previous studies have found that although accurate CAD leads to an improvement in cancer detection, inaccurate CAD leads to an increase in both missed cancers and false alarms. This is known as the over-reliance effect. We investigated whether providing framing statements of CAD fallibility could keep the benefits of CAD while reducing over-reliance. In Experiment 1, participants were told about the benefits or costs of CAD, prior to the experiment. Experiment 2 was similar, except that participants were given a stronger warning and instruction set in relation to the costs of CAD. The results showed that although there was no effect of framing in Experiment 1, a stronger message in Experiment 2 led to a reduction in the over-reliance effect. A similar result was found in Experiment 3 where the target had a lower prevalence. The results show that although the presence of CAD can result in over-reliance on the technology, these effects can be mitigated by framing and instruction sets in relation to CAD fallibility.

Use of Artificial Intelligence for Digital Breast Tomosynthesis Screening: A Preliminary Real-world Experience

OBJECTIVE

The purpose of this study is to assess the "real-world" impact of an artificial intelligence (AI) tool designed to detect breast cancer in digital breast tomosynthesis (DBT) screening exams following 12 months of utilization in a subspecialized academic breast center.

METHODS

Following IRB approval, mammography audit reports, as specified in the BI-RADS atlas, were retrospectively generated for five radiologists reading at three locations during a 12-month time frame. One location had the AI tool (iCAD ProFound AI v2.0), and the other two locations did not. The co-primary endpoints were cancer detection rate (CDR) and abnormal interpretation rate (AIR). Secondary endpoints included positive predictive values (PPVs) for cancer among screenings with abnormal interpretations (PPV1) and for biopsies performed (PPV3). Odds ratios (OR) with two-sided 95% confidence intervals (CIs) summarized the impact of AI across radiologists using generalized estimating equations.

RESULTS

Nonsignificant differences were observed in CDR, AIR, and PPVs. The CDR was 7.3 with AI and 5.9 without AI (OR 1.3, 95% CI: 0.9-1.7). The AIR was 11.7% with AI and 11.8% without AI (OR 1.0, 95% CI: 0.8-1.3). The PPV1 was 6.2% with AI and 5.0% without AI (OR 1.3, 95% CI: 0.97-1.7). The PPV3 was 33.3% with AI and 32.0% without AI (OR 1.1, 95% CI: 0.8-1.5).

CONCLUSION

Although we are unable to show statistically significant changes in CDR and AIR outcomes in the two groups, the results are consistent with prior reader studies. There is a nonsignificant trend toward improvement in CDR with AI, without significant increases in AIR.

Artificial Intelligence as Supporting Reader in Breast Screening: A Novel Workflow to Preserve Quality and Reduce Workload

OBJECTIVE

To evaluate the effectiveness of a new strategy for using artificial intelligence (AI) as supporting reader for the detection of breast cancer in mammography-based double reading screening practice.

METHODS

Large-scale multi-site, multi-vendor data were used to retrospectively evaluate a new paradigm of AI-supported reading. Here, the AI served as the second reader only if it agrees with the recall/no-recall decision of the first human reader. Otherwise, a second human reader made an assessment followed by the standard clinical workflow. The data included 280 594 cases from 180 542 female participants screened for breast cancer at seven screening sites in two countries and using equipment from four hardware vendors. The statistical analysis included non-inferiority and superiority testing of cancer screening performance and evaluation of the reduction in workload, measured as arbitration rate and number of cases requiring second human reading.

RESULTS

Artificial intelligence as a supporting reader was found to be superior or noninferior on all screening metrics compared with human double reading while reducing the number of cases requiring second human reading by up to 87% (245 395/280 594). Compared with AI as an independent reader, the number of cases referred to arbitration was reduced from 13% (35 199/280 594) to 2% (5056/280 594).

CONCLUSION

The simulation indicates that the proposed workflow retains screening performance of human double reading while substantially reducing the workload. Further research should study the impact on the second human reader because they would only assess cases in which the AI prediction and first human reader disagree.

Multi-vendor evaluation of artificial intelligence as an independent reader for double reading in breast cancer screening on 275,900 mammograms

BACKGROUND

Double reading (DR) in screening mammography increases cancer detection and lowers recall rates, but has sustainability challenges due to workforce shortages. Artificial intelligence (AI) as an independent reader (IR) in DR may provide a cost-effective solution with the potential to improve screening performance. Evidence for AI to generalise across different patient populations, screening programmes and equipment vendors, however, is still lacking.

METHODS

This retrospective study simulated DR with AI as an IR, using data representative of real-world deployments (275,900 cases, 177,882 participants) from four mammography equipment vendors, seven screening sites, and two countries. Non-inferiority and superiority were assessed for relevant screening metrics.

RESULTS

DR with AI, compared with human DR, showed at least non-inferior recall rate, cancer detection rate, sensitivity, specificity and positive predictive value (PPV) for each mammography vendor and site, and superior recall rate, specificity, and PPV for some. The simulation indicates that using AI would have increased arbitration rate (3.3% to 12.3%), but could have reduced human workload by 30.0% to 44.8%.

CONCLUSIONS

AI has potential as an IR in the DR workflow across different screening programmes, mammography equipment and geographies, substantially reducing human reader workload while maintaining or improving standard of care.

TRIAL REGISTRATION

ISRCTN18056078 (20/03/2019; retrospectively registered).

Lessons learned from independent external validation of an AI tool to detect breast cancer using a representative UK data set

OBJECTIVE

To pilot a process for the independent external validation of an artificial intelligence (AI) tool to detect breast cancer using data from the NHS breast screening programme (NHSBSP).

METHODS

A representative data set of mammography images from 26,000 women attending 2 NHS screening centres, and an enriched data set of 2054 positive cases were used from the OPTIMAM image database. The use case of the AI tool was the replacement of the first or second human reader. The performance of the AI tool was compared to that of human readers in the NHSBSP.

RESULTS

Recommendations for future external validations of AI tools to detect breast cancer are provided. The tool recalled different breast cancers to the human readers. This study showed the importance of testing AI tools on all types of cases (including non-standard) and the clarity of any warning messages. The acceptable difference in sensitivity and specificity between the AI tool and human readers should be determined. Any information vital for the clinical application should be a required output for the AI tool. It is recommended that the interaction of radiologists with the AI tool, and the effect of the AI tool on arbitration be investigated prior to clinical use.

CONCLUSION

This pilot demonstrated several lessons for future independent external validation of AI tools for breast cancer detection.

ADVANCES IN KNOWLEDGE

Knowledge has been gained towards best practice procedures for performing independent external validations of AI tools for the detection of breast cancer using data from the NHS Breast Screening Programme.

Great debates in cardiac computed tomography: OPINION: "Artificial intelligence and the future of cardiovascular CT - Managing expectation and challenging hype"

This manuscript has been written as a follow-up to the "AI/ML great debate" featured at the 2021 Society of Cardiovascular Computed Tomography (SCCT) Annual Scientific Meeting. In debate style, we highlighti the need for expectation management of AI/ML, debunking the hype around current AI techniques, and countering the argument that in its current day format AI/ML is the "silver bullet" for the interpretation of daily clinical CCTA practice.

Performance of Radiologists and Radiographers in Double Reading Mammograms: The UK National Health Service Breast Screening Program

Background Double reading can be used in screening mammography, but it is labor intensive. There is limited evidence on whether trained radiographers (ie, technologists) may be used to provide double reading. Purpose To compare the performance of radiologists and radiographers double reading screening mammograms, considering reader experience level. Materials and Methods In this retrospective study, performance and experience data were obtained for radiologists and radiographer readers of all screening mammograms in England from April 2015 to March 2016. Cancer detection rate (CDR), recall rate (RR), and positive predictive value (PPV) of recall based on biopsy-proven findings were calculated for first readers. Performance metrics were analyzed according to reader professional group and years of reading experience using the analysis of variance test. P values less than .05 were considered to indicate statistically significant difference. Results During the study period, 401 readers (224 radiologists and 177 radiographers) double read 1 404 395 screening digital mammograms. There was no difference in CDR between radiologist and radiographer readers (mean, 7.84 vs 7.53 per 1000 examinations, respectively; P = .08) and no difference for readers with more than 10 years of experience compared with 5 years or fewer years of experience, regardless of professional group (mean, 7.75 vs 7.71 per 1000 examinations respectively, P = .87). No difference in the mean RR was observed between radiologists and radiographer readers (5.0% vs 5.2%, respectively, P = .63). A lower RR was seen for readers with more than 10 years of experience compared with 5 years or fewer, regardless of professional group (mean, 4.8% vs 5.8%, respectively; P = .001). No variation in PPV was observed between them (P = .42), with PPV values of 17.1% for radiologists versus 16.1% for radiographers. A higher PPV was seen for readers with more than 10 years of experience compared with 5 years or less, regardless of professional group (mean, 17.5% and 14.9%, respectively; P = .02). Conclusion No difference in performance was observed between radiographers and radiologists reading screening mammograms in a program that used double reading. Published under a CC BY 4.0 license Online supplemental material is available for this article. See also the editorial by Hooley and Durand in this issue.

Evolution of research trends in artificial intelligence for breast cancer diagnosis and prognosis over the past two decades: A bibliometric analysis

Objective

In recent years, among the available tools, the concurrent application of Artificial Intelligence (AI) has improved the diagnostic performance of breast cancer screening. In this context, the present study intends to provide a comprehensive overview of the evolution of AI for breast cancer diagnosis and prognosis research using bibliometric analysis.

Methodology

Therefore, in the present study, relevant peer-reviewed research articles published from 2000 to 2021 were downloaded from the Scopus and Web of Science (WOS) databases and later quantitatively analyzed and visualized using Bibliometrix (R package). Finally, open challenges areas were identified for future research work.

Results

The present study revealed that the number of literature studies published in AI for breast cancer detection and survival prediction has increased from 12 to 546 between the years 2000 to 2021. The United States of America (USA), the Republic of China, and India are the most productive publication-wise in this field. Furthermore, the USA leads in terms of the total citations; however, hungry and Holland take the lead positions in average citations per year. Wang J is the most productive author, and Zhan J is the most relevant author in this field. Stanford University in the USA is the most relevant affiliation by the number of published articles. The top 10 most relevant sources are Q1 journals with PLOS ONE and computer in Biology and Medicine are the leading journals in this field. The most trending topics related to our study, transfer learning and deep learning, were identified.

Conclusion

The present findings provide insight and research directions for policymakers and academic researchers for future collaboration and research in AI for breast cancer patients.

Artificial Intelligence (AI) for Screening Mammography, From the AJR Special Series on AI Applications

Artificial intelligence (AI) applications for screening mammography are being marketed for clinical use in the interpretative domains of lesion detection and diagnosis, triage, and breast density assessment and in the noninterpretive domains of breast cancer risk assessment, image quality control, image acquisition, and dose reduction. Evidence in support of these nascent applications, particularly for lesion detection and diagnosis, is largely based on multireader studies with cancer-enriched datasets rather than rigorous clinical evaluation aligned with the application's specific intended clinical use. This article reviews commercial AI algorithms for screening mammography that are currently available for clinical practice, their use, and evidence supporting their performance. Clinical implementation considerations, such as workflow integration, governance, and ethical issues, are also described. In addition, the future of AI for screening mammography is discussed, including the development of interpretive and noninterpretive AI applications and strategic priorities for research and development.

CoroNet: Deep Neural Network-Based End-to-End Training for Breast Cancer Diagnosis

In 2020, according to the publications of both the Global Cancer Observatory (GCO) and the World Health Organization (WHO), breast cancer (BC) represents one of the highest prevalent cancers in women worldwide. Almost 47% of the world’s 100,000 people are diagnosed with breast cancer, among females. Moreover, BC prevails among 38.8% of Egyptian women having cancer. Current deep learning developments have shown the common usage of deep convolutional neural networks (CNNs) for analyzing medical images. Unlike the randomly initialized ones, pre-trained natural image database (ImageNet)-based CNN models may become successfully fine-tuned to obtain improved findings. To conduct the automatic detection of BC by the CBIS-DDSM dataset, a CNN model, namely CoroNet, is proposed. It relies on the Xception architecture, which has been pre-trained on the ImageNet dataset and has been fully trained on whole-image BC according to mammograms. The convolutional design method is used in this paper, since it performs better than the other methods. On the prepared dataset, CoroNet was trained and tested. Experiments show that in a four-class classification, it may attain an overall accuracy of 94.92% (benign mass vs. malignant mass) and (benign calcification vs. malignant calcification). CoroNet has a classification accuracy of 88.67% for the two-class cases (calcifications and masses). The paper concluded that there are promising outcomes that could be improved because more training data are available.

Use of Diagnostic Imaging Modalities in Modern Screening, Diagnostics and Management of Breast Tumours 1st Central-Eastern European Professional Consensus Statement on Breast Cancer

Breast radiologists and nuclear medicine specialists updated their previous recommendation/guidance at the 4th Hungarian Breast Cancer Consensus Conference in Kecskemét. A recommendation is hereby made that breast tumours should be screened, diagnosed and treated according to these guidelines. These professional guidelines include the latest technical developments and research findings, including the role of imaging methods in therapy and follow-up. It includes details on domestic development proposals and also addresses related areas (forensic medicine, media, regulations, reimbursement). The entire material has been agreed with the related medical disciplines.

Comparing the number and relevance of false activations between 2 artificial intelligence computer-aided detection systems: the NOISE study

BACKGROUND AND AIMS

Artificial intelligence has been shown to be effective in polyp detection, and multiple computer-aided detection (CADe) systems have been developed. False-positive (FP) activation emerged as a possible way to benchmark CADe performance in clinical practice. The aim of this study was to validate a previously developed classification of FPs comparing the performances of different brands of approved CADe systems.

METHODS

We compared 2 different consecutive video libraries (40 video per arm) collected at Humanitas Research Hospital with 2 different CADe system brands (CADe A and CADe B). For each video, the number of CADe false activations, cause, and time spent by the endoscopist to examine the area erroneously highlighted were reported. The FP activations were classified according to the previously developed classification of FPs (the NOISE classification) according to their cause and relevance.

RESULTS

In CADe A 1021 FP activations were registered across the 40 videos (25.5 ± 12.2 FPs per colonoscopy), whereas in CADe B 1028 were identified (25.7 ± 13.2 FPs per colonoscopy; P = .53). Among them, 22.9 ± 9.9 (89.8% in CADe A) and 22.1 ± 10.0 (86.0% in CADe B) were because of artifacts from the bowel wall. Conversely, 2.6 ± 1.9 (10.2% in CADe A) and 3.5 ± 2.1 (14% in CADe B) were caused by bowel content (P = .45). Within CADe A each false activation required .2 ± .9 seconds, with 1.6 ± 1.0 FPs (6.3%) requiring additional time for endoscopic assessment. Comparable results were reported within CADe B with .2 ± .8 seconds spent per false activation and 1.8 ± 1.2 FPs per colonoscopy requiring additional inspection.

CONCLUSIONS

The use of a standardized nomenclature provided comparable results with either of the 2 recently approved CADe systems. (Clinical trial registration number: NCT04399590.).

Development and validation of a deep learning model for detection of breast cancers in mammography from multi-institutional datasets

Objectives

The objective of this study was to develop and validate a state-of-the-art, deep learning (DL)-based model for detecting breast cancers on mammography.

Methods

Mammograms in a hospital development dataset, a hospital test dataset, and a clinic test dataset were retrospectively collected from January 2006 through December 2017 in Osaka City University Hospital and Medcity21 Clinic. The hospital development dataset and a publicly available digital database for screening mammography (DDSM) dataset were used to train and to validate the RetinaNet, one type of DL-based model, with five-fold cross-validation. The model’s sensitivity and mean false positive indications per image (mFPI) and partial area under the curve (AUC) with 1.0 mFPI for both test datasets were externally assessed with the test datasets.

Results

The hospital development dataset, hospital test dataset, clinic test dataset, and DDSM development dataset included a total of 3179 images (1448 malignant images), 491 images (225 malignant images), 2821 images (37 malignant images), and 1457 malignant images, respectively. The proposed model detected all cancers with a 0.45–0.47 mFPI and had partial AUCs of 0.93 in both test datasets.

Conclusions

The DL-based model developed for this study was able to detect all breast cancers with a very low mFPI. Our DL-based model achieved the highest performance to date, which might lead to improved diagnosis for breast cancer.

Convolutional Neural Network-Based Computer-Assisted Diagnosis of Hashimoto's Thyroiditis on Ultrasound

PURPOSE

This study investigates the efficiency of deep learning models in the automated diagnosis of Hashimoto's thyroiditis (HT) using real-world ultrasound data from ultrasound examinations by computer-assisted diagnosis (CAD) with artificial intelligence.

METHODS

We retrospectively collected ultrasound images from patients with and without HT from 2 hospitals in China between September 2008 and February 2018. Images were divided into a training set (80%) and a validation set (20%). We ensembled 9 convolutional neural networks (CNNs) as the final model (CAD-HT) for HT classification. The model's diagnostic performance was validated and compared to 2 hospital validation sets. We also compared the accuracy of CAD-HT against seniors/junior radiologists. Subgroup analysis of CAD-HT performance for different thyroid hormone levels (hyperthyroidism, hypothyroidism, and euthyroidism) was also evaluated.

RESULTS

39 280 ultrasound images from 21 118 patients were included in this study. The accuracy, sensitivity, and specificity of the HT-CAD model were 0.892, 0.890, and 0.895, respectively. HT-CAD performance between 2 hospitals was not significantly different. The HT-CAD model achieved a higher performance (P < 0.001) when compared to senior radiologists, with a nearly 9% accuracy improvement. HT-CAD had almost similar accuracy (range 0.87-0.894) for the 3 subgroups based on thyroid hormone level.

CONCLUSION

The HT-CAD strategy based on CNN significantly improved the radiologists' diagnostic accuracy of HT. Our model demonstrates good performance and robustness in different hospitals and for different thyroid hormone levels.

The optimal use of computer aided detection to find low prevalence cancers

People miss a high proportion of targets that only appear rarely. This low prevalence (LP) effect has implications for applied search tasks such as the clinical reading of mammograms. Computer aided detection (CAD) has been used to help radiologists search mammograms by highlighting areas likely to contain a cancer. Previous research has found a benefit in search when CAD cues were correct but a cost to search when CAD cues were incorrect. The current research investigated whether there is an optimal way to present CAD to ensure low error rates when CAD is both correct and incorrect. Experiment 1 compared an automatic condition, where CAD appeared simultaneously with the display to an interactive condition, where participants could choose to use CAD. Experiment 2 compared the automatic condition to a confirm condition, where participants searched the display first before being shown the CAD cues. The results showed that miss errors were reduced overall in the confirm condition, with no cost to false alarms. Furthermore, having CAD be interactive, resulted in a low uptake where it was only used in 34% of trials. The results showed that the presentation mode of CAD can affect decision-making in LP search.

Machine Learning for Workflow Applications in Screening Mammography: Systematic Review and Meta-Analysis

Background Advances in computer processing and improvements in data availability have led to the development of machine learning (ML) techniques for mammographic imaging. Purpose To evaluate the reported performance of stand-alone ML applications for screening mammography workflow. Materials and Methods Ovid Embase, Ovid Medline, Cochrane Central Register of Controlled Trials, Scopus, and Web of Science literature databases were searched for relevant studies published from January 2012 to September 2020. The study was registered with the PROSPERO International Prospective Register of Systematic Reviews (protocol no. CRD42019156016). Stand-alone technology was defined as a ML algorithm that can be used independently of a human reader. Studies were quality assessed using the Quality Assessment of Diagnostic Accuracy Studies 2 and the Prediction Model Risk of Bias Assessment Tool, and reporting was evaluated using the Checklist for Artificial Intelligence in Medical Imaging. A primary meta-analysis included the top-performing algorithm and corresponding reader performance from which pooled summary estimates for the area under the receiver operating characteristic curve (AUC) were calculated using a bivariate model. Results Fourteen articles were included, which detailed 15 studies for stand-alone detection (n = 8) and triage (n = 7). Triage studies reported that 17%-91% of normal mammograms identified could be read by adapted screening, while "missing" an estimated 0%-7% of cancers. In total, an estimated 185 252 cases from three countries with more than 39 readers were included in the primary meta-analysis. The pooled sensitivity, specificity, and AUC was 75.4% (95% CI: 65.6, 83.2; P = .11), 90.6% (95% CI: 82.9, 95.0; P = .40), and 0.89 (95% CI: 0.84, 0.98), respectively, for algorithms, and 73.0% (95% CI: 60.7, 82.6), 88.6% (95% CI: 72.4, 95.8), and 0.85 (95% CI: 0.78, 0.97), respectively, for readers. Conclusion Machine learning (ML) algorithms that demonstrate a stand-alone application in mammographic screening workflows achieve or even exceed human reader detection performance and improve efficiency. However, this evidence is from a small number of retrospective studies. Therefore, further rigorous independent external prospective testing of ML algorithms to assess performance at preassigned thresholds is required to support these claims. ©RSNA, 2021 Online supplemental material is available for this article. See also the editorial by Whitman and Moseley in this issue.

Artificial intelligence for the real world of breast screening

Breast cancer screening with mammography reduces mortality in the women who attend by detecting high risk cancer early. It is far from perfect with variations in both sensitivity for the detection of cancer and very wide variations in specificity, leading to unnecessary recalls and biopsies. Over the last 12 months several papers have reported on AI algorithms that perform as well as human readers on large well curated population data sets. The nature of the test sets, the way the gold standard has been calculated, the definition of a positive call, and the statistics used all influence the results. Historically retrospective studies have not predicted the real-life performance of radiologist plus machine. So, it is important to perform prospective studies before introducing Artificial intelligence into real world breast screening.

Artificial Intelligence Algorithm Improves Radiologist Performance in Skeletal Age Assessment: A Prospective Multicenter Randomized Controlled Trial

Background Previous studies suggest that use of artificial intelligence (AI) algorithms as diagnostic aids may improve the quality of skeletal age assessment, though these studies lack evidence from clinical practice. Purpose To compare the accuracy and interpretation time of skeletal age assessment on hand radiograph examinations with and without the use of an AI algorithm as a diagnostic aid. Materials and Methods In this prospective randomized controlled trial, the accuracy of skeletal age assessment on hand radiograph examinations was performed with (n = 792) and without (n = 739) the AI algorithm as a diagnostic aid. For examinations with the AI algorithm, the radiologist was shown the AI interpretation as part of their routine clinical work and was permitted to accept or modify it. Hand radiographs were interpreted by 93 radiologists from six centers. The primary efficacy outcome was the mean absolute difference between the skeletal age dictated into the radiologists' signed report and the average interpretation of a panel of four radiologists not using a diagnostic aid. The secondary outcome was the interpretation time. A linear mixed-effects regression model with random center- and radiologist-level effects was used to compare the two experimental groups. Results Overall mean absolute difference was lower when radiologists used the AI algorithm compared with when they did not (5.36 months vs 5.95 months; P = .04). The proportions at which the absolute difference exceeded 12 months (9.3% vs 13.0%, P = .02) and 24 months (0.5% vs 1.8%, P = .02) were lower with the AI algorithm than without it. Median radiologist interpretation time was lower with the AI algorithm than without it (102 seconds vs 142 seconds, P = .001). Conclusion Use of an artificial intelligence algorithm improved skeletal age assessment accuracy and reduced interpretation times for radiologists, although differences were observed between centers. Clinical trial registration no. NCT03530098 © RSNA, 2021 Online supplemental material is available for this article. See also the editorial by Rubin in this issue.

Adoption of artificial intelligence in breast imaging: evaluation, ethical constraints and limitations

Retrospective studies have shown artificial intelligence (AI) algorithms can match as well as enhance radiologist's performance in breast screening. These tools can facilitate tasks not feasible by humans such as the automatic triage of patients and prediction of treatment outcomes. Breast imaging faces growing pressure with the exponential growth in imaging requests and a predicted reduced workforce to provide reports. Solutions to alleviate these pressures are being sought with an increasing interest in the adoption of AI to improve workflow efficiency as well as patient outcomes. Vast quantities of data are needed to test and monitor AI algorithms before and after their incorporation into healthcare systems. Availability of data is currently limited, although strategies are being devised to harness the data that already exists within healthcare institutions. Challenges that underpin the realisation of AI into everyday breast imaging cannot be underestimated and the provision of guidance from national agencies to tackle these challenges, taking into account views from a societal, industrial and healthcare prospective is essential. This review provides background on the evaluation and use of AI in breast imaging in addition to exploring key ethical, technical, legal and regulatory challenges that have been identified so far.

Deep Learning-Based Artificial Intelligence for Mammography

During the past decade, researchers have investigated the use of computer-aided mammography interpretation. With the application of deep learning technology, artificial intelligence (AI)-based algorithms for mammography have shown promising results in the quantitative assessment of parenchymal density, detection and diagnosis of breast cancer, and prediction of breast cancer risk, enabling more precise patient management. AI-based algorithms may also enhance the efficiency of the interpretation workflow by reducing both the workload and interpretation time. However, more in-depth investigation is required to conclusively prove the effectiveness of AI-based algorithms. This review article discusses how AI algorithms can be applied to mammography interpretation as well as the current challenges in its implementation in real-world practice.

Artificial Intelligence in the Healthcare System: An Overview

This chapter aims to present insights into the influence of artificial intelligence (AI) on medicine, public health, and the economy. PubMed and Google Scholar databases were used for the identification and collection of articles with search commands of "artificial intelligence" AND "public health" and "artificial intelligence" AND "medicine". A total of 273 articles specifically handling the issue of artificial intelligence, dating ten years back, in three major medical journals: Science, The Lancet, and The New England Journal of Medicine, were analyzed. Computational power gets stronger by the day, giving us new solutions and possibilities. Current medicine problems like personalized medicine, storage of data, and documentation overload will likely be replaced by AI shortly. The application of AI may also bring substantial benefits to other areas of medicine like the diagnostic and therapeutic processes. The development and spread of AI are inescapable as it lowers healthcare and administrative costs, improves medical efficiency, and predicts and prevents major disease complications. The use of AI in medicine seems destined to carry the day.

Computer-aided diagnosis in the era of deep learning

Computer-aided diagnosis (CAD) has been a major field of research for the past few decades. CAD uses machine learning methods to analyze imaging and/or nonimaging patient data and makes assessment of the patient's condition, which can then be used to assist clinicians in their decision-making process. The recent success of the deep learning technology in machine learning spurs new research and development efforts to improve CAD performance and to develop CAD for many other complex clinical tasks. In this paper, we discuss the potential and challenges in developing CAD tools using deep learning technology or artificial intelligence (AI) in general, the pitfalls and lessons learned from CAD in screening mammography and considerations needed for future implementation of CAD or AI in clinical use. It is hoped that the past experiences and the deep learning technology will lead to successful advancement and lasting growth in this new era of CAD, thereby enabling CAD to deliver intelligent aids to improve health care.

WHAT CAN WE ACTUALLY SEE USING COMPUTER AIDED DETECTION IN MAMMOGRAPHY?

The main goal of this study was to compare the results of computer aided detection (CAD) analysis in screening mammography with the results independently obtained by two radiologists for the same samples and to determine the sensitivity and specificity of CAD for breast lesions. A total of 436 mammograms were analyzed with CAD. For each screening mammogram, the changes in breast tissue recognized by CAD were compared to the interpretations of two radiologists. The sensitivity and specificity of CAD for breast lesions were calculated using contingency table. The sensitivity of CAD for all lesions was 54% and specificity 16%. CAD sensitivity for suspicious lesions only was 86%. CAD sensitivity for microcalcifications was 100% and specificity 45%. CAD mainly ‘mistook’ glandular parenchyma, connective tissue and blood vessels for breast lesions, and blood vessel calcifications and axillary folds for microcalcifications. In this study, we confirmed CAD as an excellent tool for recognizing microcalcifications with 100% sensitivity. However, it should not be used as a stand-alone tool in breast screening mammography due to the high rate of false-positive results.

Does Artificial Intelligence Outperform Natural Intelligence in Interpreting Musculoskeletal Radiological Studies? A Systematic Review

BACKGROUND

Machine learning (ML) is a subdomain of artificial intelligence that enables computers to abstract patterns from data without explicit programming. A myriad of impactful ML applications already exists in orthopaedics ranging from predicting infections after surgery to diagnostic imaging. However, no systematic reviews that we know of have compared, in particular, the performance of ML models with that of clinicians in musculoskeletal imaging to provide an up-to-date summary regarding the extent of applying ML to imaging diagnoses. By doing so, this review delves into where current ML developments stand in aiding orthopaedists in assessing musculoskeletal images.

QUESTIONS/PURPOSES

This systematic review aimed (1) to compare performance of ML models versus clinicians in detecting, differentiating, or classifying orthopaedic abnormalities on imaging by (A) accuracy, sensitivity, and specificity, (B) input features (for example, plain radiographs, MRI scans, ultrasound), (C) clinician specialties, and (2) to compare the performance of clinician-aided versus unaided ML models.

METHODS

A systematic review was performed in PubMed, Embase, and the Cochrane Library for studies published up to October 1, 2019, using synonyms for machine learning and all potential orthopaedic specialties. We included all studies that compared ML models head-to-head against clinicians in the binary detection of abnormalities in musculoskeletal images. After screening 6531 studies, we ultimately included 12 studies. We conducted quality assessment using the Methodological Index for Non-randomized Studies (MINORS) checklist. All 12 studies were of comparable quality, and they all clearly included six of the eight critical appraisal items (study aim, input feature, ground truth, ML versus human comparison, performance metric, and ML model description). This justified summarizing the findings in a quantitative form by calculating the median absolute improvement of the ML models compared with clinicians for the following metrics of performance: accuracy, sensitivity, and specificity.

RESULTS

ML models provided, in aggregate, only very slight improvements in diagnostic accuracy and sensitivity compared with clinicians working alone and were on par in specificity (3% (interquartile range [IQR] -2.0% to 7.5%), 0.06% (IQR -0.03 to 0.14), and 0.00 (IQR -0.048 to 0.048), respectively). Inputs used by the ML models were plain radiographs (n = 8), MRI scans (n = 3), and ultrasound examinations (n = 1). Overall, ML models outperformed clinicians more when interpreting plain radiographs than when interpreting MRIs (17 of 34 and 3 of 16 performance comparisons, respectively). Orthopaedists and radiologists performed similarly to ML models, while ML models mostly outperformed other clinicians (outperformance in 7 of 19, 7 of 23, and 6 of 10 performance comparisons, respectively). Two studies evaluated the performance of clinicians aided and unaided by ML models; both demonstrated considerable improvements in ML-aided clinician performance by reporting a 47% decrease of misinterpretation rate (95% confidence interval [CI] 37 to 54; p < 0.001) and a mean increase in specificity of 0.048 (95% CI 0.029 to 0.068; p < 0.001) in detecting abnormalities on musculoskeletal images.

CONCLUSIONS

At present, ML models have comparable performance to clinicians in assessing musculoskeletal images. ML models may enhance the performance of clinicians as a technical supplement rather than as a replacement for clinical intelligence. Future ML-related studies should emphasize how ML models can complement clinicians, instead of determining the overall superiority of one versus the other. This can be accomplished by improving transparent reporting, diminishing bias, determining the feasibility of implantation in the clinical setting, and appropriately tempering conclusions.

LEVEL OF EVIDENCE

Level III, diagnostic study.

Improving Breast Cancer Detection Accuracy of Mammography with the Concurrent Use of an Artificial Intelligence Tool

Purpose

To evaluate the benefits of an artificial intelligence (AI)-based tool for two-dimensional mammography in the breast cancer detection process.

Materials and Methods

In this multireader, multicase retrospective study, 14 radiologists assessed a dataset of 240 digital mammography images, acquired between 2013 and 2016, using a counterbalance design in which half of the dataset was read without AI and the other half with the help of AI during a first session and vice versa during a second session, which was separated from the first by a washout period. Area under the receiver operating characteristic curve (AUC), sensitivity, specificity, and reading time were assessed as endpoints.

Results

The average AUC across readers was 0.769 (95% CI: 0.724, 0.814) without AI and 0.797 (95% CI: 0.754, 0.840) with AI. The average difference in AUC was 0.028 (95% CI: 0.002, 0.055, P = .035). Average sensitivity was increased by 0.033 when using AI support (P = .021). Reading time changed dependently to the AI-tool score. For low likelihood of malignancy (< 2.5%), the time was about the same in the first reading session and slightly decreased in the second reading session. For higher likelihood of malignancy, the reading time was on average increased with the use of AI.

Conclusion

This clinical investigation demonstrated that the concurrent use of this AI tool improved the diagnostic performance of radiologists in the detection of breast cancer without prolonging their workflow.Supplemental material is available for this article.© RSNA, 2020.

[Investigation of Clinical Utility of Radiological Technologist's Reading Report as a Second Opinion for Medical Doctor Reading of Digital Mammogram]

PURPOSE

We investigated the clinical utility of a radiological technologist's (RT)'s reports (RRs) as a second opinion by the free-response receiver operating characteristic (FROC) observer study that compared the performance of medical doctors' (MDs') reading of digital mammogram with and without consulting the RR.

METHOD

One hundred women (39 malignant, 61 benign or normal) who underwent diagnostic mammography were selected from among 1674 routine clinical images classified by the degree of difficulty and categories for inclusion in the FROC study. The first FROC study performed by three RTs (RT 1-3) was conducted to collect the data for RR utilized in the second FROC study. The second FROC study was performed by five MDs, and the statistical significance of MDs' performances with and without reference to the RR was investigated by figure of merit (FOM).

RESULT

The FOM values of three RTs obtained in the first FROC study were 0.529, 0.576, and 0.539, respectively. In the second FROC study, RT 2 had the highest FOM, RT 1 the lowest false positives/case, and RT 3 the highest sensitivity. The average FOM values in the second FROC study for the five MDs with/without reference to the RR were as follows: RT 2's RR was 0.534/0.588 (p=0.003), RT 1's RR was 0.500/0.545 (p=0.099), and RT 3's RR was 0.569/0.592 (p=0.324).

CONCLUSION

We concluded that the MDs' performance of reading mammogram was statistically improved by consulting the RR when the RT's reading skill was high.

Clinical applications of machine learning in cardiovascular disease and its relevance to cardiac imaging

Artificial intelligence (AI) has transformed key aspects of human life. Machine learning (ML), which is a subset of AI wherein machines autonomously acquire information by extracting patterns from large databases, has been increasingly used within the medical community, and specifically within the domain of cardiovascular diseases. In this review, we present a brief overview of ML methodologies that are used for the construction of inferential and predictive data-driven models. We highlight several domains of ML application such as echocardiography, electrocardiography, and recently developed non-invasive imaging modalities such as coronary artery calcium scoring and coronary computed tomography angiography. We conclude by reviewing the limitations associated with contemporary application of ML algorithms within the cardiovascular disease field.

Development and Validation of a Deep Learning-based Automatic Detection Algorithm for Active Pulmonary Tuberculosis on Chest Radiographs

Background

Detection of active pulmonary tuberculosis on chest radiographs (CRs) is critical for the diagnosis and screening of tuberculosis. An automated system may help streamline the tuberculosis screening process and improve diagnostic performance.

Methods

We developed a deep learning–based automatic detection (DLAD) algorithm using 54c221 normal CRs and 6768 CRs with active pulmonary tuberculosis that were labeled and annotated by 13 board-certified radiologists. The performance of DLAD was validated using 6 external multicenter, multinational datasets. To compare the performances of DLAD with physicians, an observer performance test was conducted by 15 physicians including nonradiology physicians, board-certified radiologists, and thoracic radiologists. Image-wise classification and lesion-wise localization performances were measured using area under the receiver operating characteristic (ROC) curves and area under the alternative free-response ROC curves, respectively. Sensitivities and specificities of DLAD were calculated using 2 cutoffs (high sensitivity [98%] and high specificity [98%]) obtained through in-house validation.

Results

DLAD demonstrated classification performance of 0.977–1.000 and localization performance of 0.973–1.000. Sensitivities and specificities for classification were 94.3%–100% and 91.1%–100% using the high-sensitivity cutoff and 84.1%–99.0% and 99.1%–100% using the high-specificity cutoff. DLAD showed significantly higher performance in both classification (0.993 vs 0.746–0.971) and localization (0.993 vs 0.664–0.925) compared to all groups of physicians.

Conclusions

Our DLAD demonstrated excellent and consistent performance in the detection of active pulmonary tuberculosis on CR, outperforming physicians, including thoracic radiologists.

A review of the PERFORMS scheme in breast screening

This review details the aetiology of the PERFORMS self-assessment scheme in breast screening, together with its subsequent development, current implementation and future function. The purpose of the scheme is examined and the importance of its continuing role in a changing screening service described, together with current evolution.

Determinants of In-Hospital Mortality After Percutaneous Coronary Intervention: A Machine Learning Approach

Background

The ability to accurately predict the occurrence of in‐hospital death after percutaneous coronary intervention is important for clinical decision‐making. We sought to utilize the New York Percutaneous Coronary Intervention Reporting System in order to elucidate the determinants of in‐hospital mortality in patients undergoing percutaneous coronary intervention across New York State.

Methods and Results

We examined 479 804 patients undergoing percutaneous coronary intervention between 2004 and 2012, utilizing traditional and advanced machine learning algorithms to determine the most significant predictors of in‐hospital mortality. The entire data were randomly split into a training (80%) and a testing set (20%). Tuned hyperparameters were used to generate a trained model while the performance of the model was independently evaluated on the testing set after plotting a receiver‐operator characteristic curve and using the output measure of the area under the curve (AUC) and the associated 95% CIs. Mean age was 65.2±11.9 years and 68.5% were women. There were 2549 in‐hospital deaths within the patient population. A boosted ensemble algorithm (AdaBoost) had optimal discrimination with AUC of 0.927 (95% CI 0.923–0.929) compared with AUC of 0.913 for XGBoost (95% CI 0.906–0.919, P=0.02), AUC of 0.892 for Random Forest (95% CI 0.889–0.896, P<0.01), and AUC of 0.908 for logistic regression (95% CI 0.907–0.910, P<0.01). The 2 most significant predictors were age and ejection fraction.

Conclusions

A big data approach that utilizes advanced machine learning algorithms identifies new associations among risk factors and provides high accuracy for the prediction of in‐hospital mortality in patients undergoing percutaneous coronary intervention.

See Editorial by Garratt and Schneider

A deep learning-based automated diagnostic system for classifying mammographic lesions

BACKGROUND

Screening mammography has led to reduced breast cancer-specific mortality and is recommended worldwide. However, the resultant doctors' workload of reading mammographic scans needs to be addressed. Although computer-aided detection (CAD) systems have been developed to support readers, the findings are conflicting regarding whether traditional CAD systems improve reading performance. Rapid progress in the artificial intelligence (AI) field has led to the advent of newer CAD systems using deep learning-based algorithms which have the potential to reach human performance levels. Those systems, however, have been developed using mammography images mainly from women in western countries. Because Asian women characteristically have higher-density breasts, it is uncertain whether those AI systems can apply to Japanese women. In this study, we will construct a deep learning-based CAD system trained using mammography images from a large number of Japanese women with high quality reading.

METHODS

We will collect digital mammography images taken for screening or diagnostic purposes at multiple institutions in Japan. A total of 15,000 images, consisting of 5000 images with breast cancer and 10,000 images with benign lesions, will be collected. At least 1000 images of normal breasts will also be collected for use as reference data. With these data, we will construct a deep learning-based AI system to detect breast cancer on mammograms. The primary endpoint will be the sensitivity and specificity of the AI system with the test image set.

DISCUSSION

When the ability of AI reading is shown to be on a par with that of human reading, images of normal breasts or benign lesions that do not have to be read by a human can be selected by AI beforehand. Our AI might work well in Asian women who have similar breast density, size, and shape to those of Japanese women.

TRIAL REGISTRATION

UMIN, trial number UMIN000039009. Registered 26 December 2019, https://www.umin.ac.jp/ctr/.

Stand-Alone Artificial Intelligence for Breast Cancer Detection in Mammography: Comparison With 101 Radiologists

BACKGROUND

Artificial intelligence (AI) systems performing at radiologist-like levels in the evaluation of digital mammography (DM) would improve breast cancer screening accuracy and efficiency. We aimed to compare the stand-alone performance of an AI system to that of radiologists in detecting breast cancer in DM.

METHODS

Nine multi-reader, multi-case study datasets previously used for different research purposes in seven countries were collected. Each dataset consisted of DM exams acquired with systems from four different vendors, multiple radiologists' assessments per exam, and ground truth verified by histopathological analysis or follow-up, yielding a total of 2652 exams (653 malignant) and interpretations by 101 radiologists (28 296 independent interpretations). An AI system analyzed these exams yielding a level of suspicion of cancer present between 1 and 10. The detection performance between the radiologists and the AI system was compared using a noninferiority null hypothesis at a margin of 0.05.

RESULTS

The performance of the AI system was statistically noninferior to that of the average of the 101 radiologists. The AI system had a 0.840 (95% confidence interval [CI] = 0.820 to 0.860) area under the ROC curve and the average of the radiologists was 0.814 (95% CI = 0.787 to 0.841) (difference 95% CI = -0.003 to 0.055). The AI system had an AUC higher than 61.4% of the radiologists.

CONCLUSIONS

The evaluated AI system achieved a cancer detection accuracy comparable to an average breast radiologist in this retrospective setting. Although promising, the performance and impact of such a system in a screening setting needs further investigation.

CANDI: an R package and Shiny app for annotating radiographs and evaluating computer-aided diagnosis

Motivation

Radiologists have used algorithms for Computer-Aided Diagnosis (CAD) for decades. These algorithms use machine learning with engineered features, and there have been mixed findings on whether they improve radiologists’ interpretations. Deep learning offers superior performance but requires more training data and has not been evaluated in joint algorithm-radiologist decision systems.

Results

We developed the Computer-Aided Note and Diagnosis Interface (CANDI) for collaboratively annotating radiographs and evaluating how algorithms alter human interpretation. The annotation app collects classification, segmentation, and image captioning training data, and the evaluation app randomizes the availability of CAD tools to facilitate clinical trials on radiologist enhancement.

Availability and implementation

Demonstrations and source code are hosted at (https://candi.nextgenhealthcare.org), and (https://github.com/mbadge/candi), respectively, under GPL-3 license.

Supplementary information

Supplementary material is available at Bioinformatics online.

CAD and AI for breast cancer-recent development and challenges

Computer-aided diagnosis (CAD) has been a popular area of research and development in the past few decades. In CAD, machine learning methods and multidisciplinary knowledge and techniques are used to analyze the patient information and the results can be used to assist clinicians in their decision making process. CAD may analyze imaging information alone or in combination with other clinical data. It may provide the analyzed information directly to the clinician or correlate the analyzed results with the likelihood of certain diseases based on statistical modeling of the past cases in the population. CAD systems can be developed to provide decision support for many applications in the patient care processes, such as lesion detection, characterization, cancer staging, treatment planning and response assessment, recurrence and prognosis prediction. The new state-of-the-art machine learning technique, known as deep learning (DL), has revolutionized speech and text recognition as well as computer vision. The potential of major breakthrough by DL in medical image analysis and other CAD applications for patient care has brought about unprecedented excitement of applying CAD, or artificial intelligence (AI), to medicine in general and to radiology in particular. In this paper, we will provide an overview of the recent developments of CAD using DL in breast imaging and discuss some challenges and practical issues that may impact the advancement of artificial intelligence and its integration into clinical workflow.

Is there a safety-net effect with computer-aided detection?

Computer-aided detection (CAD) systems are used to aid readers interpreting screening mammograms. An expert reader searches the image initially unaided and then once again with the aid of CAD, which prompts automatically detected suspicious regions. This could lead to a "safety-net" effect, where the initial unaided search of the image is adversely affected by the fact that it is preliminary to an additional search with CAD and may, therefore, be less thorough. To investigate the existence of such an effect, we created a visual search experiment for nonexpert observers mirroring breast screening with CAD. Each observer searched 100 images for microcalcification clusters within synthetic images in both prompted (CAD) and unprompted (no-CAD) conditions. Fifty-two participants were recruited for the study, 48 of whom had their eye movements tracked in real-time; the other 4 participants could not be accurately calibrated, so only behavioral data were collected. In the CAD condition, before prompts were displayed, image coverage was significantly lower than coverage in the no-CAD condition (t 47 = 5.29 , p < 0.0001 ). Observer sensitivity was significantly greater for targets marked by CAD than the same targets in the no-CAD condition (t 51 = 6.56 , p < 0.001 ). For targets not marked by CAD, there was no significant difference in observer sensitivity in the CAD condition compared with the same targets in the no-CAD condition (t 51 = 0.54 , p = 0.59 ). These results suggest that the initial search may be influenced by the subsequent availability of CAD; if so, cross-sectional CAD efficacy studies should account for the effect when estimating benefit.

Epileptogenic zone localization using a new automatic quantitative analysis based on normal brain glucose metabolism database

OBJECTIVES

To assess the clinical value of voxel-based automatic quantitative analysis using a normal brain glucose metabolism database in the preoperative localization of focal intractable temporal lobe epilepsy patients.

METHODS

Patients with refractory temporal lobe epilepsy who underwent 18F-fluorodeoxyglucose positron emission tomography (¹⁸F-FDG PET) imaging were retrospectively enrolled from January to June 2017. Visual analysis was performed by two nuclear medicine radiologists, and the automatic quantitative analysis was carried out using MIMneuro software based the age- and gender-stratified normal brain glucose metabolism database. Setting postoperative outcomes as reference, the consistency between visual analysis and automatic quantitative analysis was tested by Cohen's kappa coefficient, and differences in localization of epileptic foci of the two methods were compared by Chi-square test.

RESULTS

A total of 32 patients intractable temporal lobe epilepsy were included in this study. There was a moderate agreement between the automatic quantitative analysis based on MIMneuro software and visual analysis (kappa coefficient = 0.472, p = 0.002). In terms of the efficiency of focus localization, the voxel-based automatic quantitative analysis was higher than that of visual analysis (Chi-square value = 6.969, p = 0.008).

CONCLUSIONS

The voxel-based automatic quantitative analysis combined with normal brain glucose metabolism database had a certain clinical application value for detection temporal lobe epilepsy.

Impact of Colorized Display of Mammograms on Lesion Detection

OBJECTIVE

To assess the effect of the colorized display of digital mammograms on observer detection of subtle breast lesions.

METHODS

Three separate observer studies compared detection performance using grayscale versus color display of 1) low-contrast mass-like objects in a standardized mammography phantom; 2) simulated microcalcifications in a background of normal breast parenchyma; and 3) standard-of-care clinical digital mammograms with subtle calcifications and masses. Colorization of the images was done by displaying each image pixel in blue, green, and red hues, or gray, maintaining DICOM-calibrated luminance scale and consistent luminance range. For the simulated calcifications and clinical mammogram studies, comparison of detection rates was computed using McNemar's test for paired differences.

RESULTS

For the phantom study, mass-like object detection was significantly better using a green colormap than grayscale (73.3% vs 70.8%, P = .009), with no significant improvement using blue or red colormaps (72.6% and 72.5%, respectively). For simulated microcalcifications, no significant difference was noted in detection using the green colormap, as compared with grayscale. For clinical digital screening mammograms, no significant difference was noted between gray and green colormaps for detection of microcalcifications. Green color display, however, resulted in decreased sensitivity for detection of subtle masses (63% vs 69%, P = .03).

CONCLUSION

Although modest improvement was demonstrated for a detection task using colorized display of a standard mammography phantom, no significant improvement was demonstrated using a color display for a simulated clinical detection task, and actual clinical performance was worse for colorized display of mammograms in comparison to standard grayscale display.