Impact of Data Presentation on Physician Performance Utilizing Artificial Intelligence-Based Computer-Aided Diagnosis and Decision Support Systems

Artificial Intelligence in BI-RADS Categorization of Breast Lesions on Ultrasound: Can We Omit Excessive Follow-ups and Biopsies?

RATIONALE AND OBJECTIVES

Artificial intelligence (AI) systems have been increasingly applied to breast ultrasonography. They are expected to decrease the workload of radiologists and to improve diagnostic accuracy. The aim of this study is to evaluate the performance of an AI system for the BI-RADS category assessment in breast masses detected on breast ultrasound. MATERIALS AND METHODS: A total of 715 masses detected in 530 patients were analyzed. Three breast imaging centers of the same institution and nine breast radiologists participated in this study. Ultrasound was performed by one radiologist who obtained two orthogonal views of each detected lesion. These images were retrospectively reviewed by a second radiologist blinded to the patient's clinical data. A commercial AI system evaluated images. The level of agreement between the AI system and the two radiologists and their diagnostic performance were calculated according to dichotomic BI-RADS category assessment.

RESULTS

This study included 715 breast masses. Of these, 134 (18.75%) were malignant, and 581 (81.25%) were benign. In discriminating benign and probably benign from suspicious lesions, the agreement between AI and the first and second radiologists was moderate statistically. The sensitivity and specificity of radiologist 1, radiologist 2, and AI were calculated as 98.51% and 80.72%, 97.76% and 75.56%, and 98.51% and 65.40%, respectively. For radiologist 1, the positive predictive value (PPV) was 54.10%, the negative predictive value (NPV) was 99.58%, and the accuracy was 84.06%. Radiologist 2 achieved a PPV of 47.99%, NPV of 99.32%, and accuracy of 79.72%. The AI system exhibited a PPV of 39.64%, NPV of 99.48%, and accuracy of 71.61%. Notably, none of the lesions categorized as BI-RADS 2 by AI were malignant, while 2 of the lesions classified as BI-RADS 3 by AI were subsequently confirmed as malignant. By considering AI-assigned BI-RADS 2 as safe, we could potentially avoid 11% (18 out of 163) of benign lesion biopsies and 46.2% (110 out of 238) of follow-ups.

CONCLUSION

AI proves effective in predicting malignancy. Integrating it into the clinical workflow has the potential to reduce unnecessary biopsies and short-term follow-ups, which, in turn, can contribute to sustainability in healthcare practices.

Machine learning and new insights for breast cancer diagnosis

Breast cancer (BC) is the most prominent form of cancer among females all over the world. The current methods of BC detection include X-ray mammography, ultrasound, computed tomography, magnetic resonance imaging, positron emission tomography and breast thermographic techniques. More recently, machine learning (ML) tools have been increasingly employed in diagnostic medicine for its high efficiency in detection and intervention. The subsequent imaging features and mathematical analyses can then be used to generate ML models, which stratify, differentiate and detect benign and malignant breast lesions. Given its marked advantages, radiomics is a frequently used tool in recent research and clinics. Artificial neural networks and deep learning (DL) are novel forms of ML that evaluate data using computer simulation of the human brain. DL directly processes unstructured information, such as images, sounds and language, and performs precise clinical image stratification, medical record analyses and tumour diagnosis. Herein, this review thoroughly summarizes prior investigations on the application of medical images for the detection and intervention of BC using radiomics, namely DL and ML. The aim was to provide guidance to scientists regarding the use of artificial intelligence and ML in research and the clinic.

Theory of radiologist interaction with instant messaging decision support tools: A sequential-explanatory study

Radiology specific clinical decision support systems (CDSS) and artificial intelligence are poorly integrated into the radiologist workflow. Current research and development efforts of radiology CDSS focus on 4 main interventions, based around exam centric time points-after image acquisition, intra-report support, post-report analysis, and radiology workflow adjacent. We review the literature surrounding CDSS tools in these time points, requirements for CDSS workflow augmentation, and technologies that support clinician to computer workflow augmentation. We develop a theory of radiologist-decision tool interaction using a sequential explanatory study design. The study consists of 2 phases, the first a quantitative survey and the second a qualitative interview study. The phase 1 survey identifies differences between average users and radiologist users in software interventions using the User Acceptance of Information Technology: Toward a Unified View (UTAUT) framework. Phase 2 semi-structured interviews provide narratives on why these differences are found. To build this theory, we propose a novel solution called Radibot-a conversational agent capable of engaging clinicians with CDSS as an assistant using existing instant messaging systems supporting hospital communications. This work contributes an understanding of how radiologist-users differ from the average user and can be utilized by software developers to increase satisfaction of CDSS tools within radiology.

Progress in the Application of Portable Ultrasound Combined with Artificial Intelligence in Pre-Hospital Emergency and Disaster Sites

With the miniaturization of ultrasound and the development of artificial intelligence, its application in disaster scenes and pre-hospital emergency care has become more and more common. This study summarizes the literature on portable ultrasound in pre-hospital emergency and disaster scene treatment in the past decade and reviews the development and application of portable ultrasound. Portable ultrasound diagnostic equipment can be used to diagnose abdominal bleeding, limb fracture, hemopneumothorax, pericardial effusion, etc., based on which trauma can be diagnosed pre-hospital and provide guiding suggestions for the next triage and rescue; in early rescue, portable ultrasound can guide emergency operations, such as tracheal intubation, pericardial cavity puncture, and thoracic and abdominal puncture as well as improve the accuracy and timeliness of operation techniques. In addition, with the development of artificial intelligence (AI), AI-assisted diagnosis can improve the diagnosis level of ultrasound at disaster sites. The portable ultrasound diagnosis system equipped with an AI robotic arm can maximize the pre-screening classification and fast and concise diagnosis and treatment of batch casualties, thus providing a reliable basis for batch casualty classification and evacuation at disaster accident sites.

ML-DSTnet: A Novel Hybrid Model for Breast Cancer Diagnosis Improvement Based on Image Processing Using Machine Learning and Dempster-Shafer Theory

Medical intelligence detection systems have changed with the help of artificial intelligence and have also faced challenges. Breast cancer diagnosis and classification are part of this medical intelligence system. Early detection can lead to an increase in treatment options. On the other hand, uncertainty is a case that has always been with the decision-maker. The system's parameters cannot be accurately estimated, and the wrong decision is made. To solve this problem, we have proposed a method in this article that reduces the ignorance of the problem with the help of Dempster-Shafer theory so that we can make a better decision. This research on the MIAS dataset, based on image processing machine learning and Dempster-Shafer mathematical theory, tries to improve the diagnosis and classification of benign, malignant masses. We first determine the results of the diagnosis of mass type with MLP by using the texture feature and CNN. We combine the results of the two classifications with Dempster-Shafer theory and improve its accuracy. The obtained results show that the proposed approach has better performance than others based on evaluation criteria such as accuracy of 99.10%, sensitivity of 98.4%, and specificity of 100%.

The feasibility of a novel computer-aided classification system for the characterisation and diagnosis of breast masses on ultrasound: a single-centre preliminary test study

AIM

To introduce a novel computer-aided classification (CAC) system and investigate the feasibility of characterising and diagnosing breast masses on ultrasound (US).

MATERIALS AND METHODS

A total of 246 breast masses were included. US features and the final assessment categories of the breast masses were analysed by a radiologist and the CAC system according to the Breast Imaging Reporting and Data System (BI-RADS) lexicon. The CAC system evaluated the BI-RADS assessment from the fusion of multi-view and colour Doppler US images without (SmartBreast) or with combining clinical variables (m-CAC system). The diagnostic performance and agreement of US characteristics between the radiologist and the CAC system were compared.

RESULTS

The agreement between the radiologist and the CAC system was substantial for mass shape (κ = 0.673), orientation (κ = 0.682), margin (κ = 0.622), posterior features (κ = 0.629), calcifications in a mass (κ = 0.709) and vascularity (κ = 0.745), fair for echo pattern (κ = 0.379), and moderate for BI-RADS assessment (κ = 0.575). With BI-RADS 4a as the cut-off value, the specificity (52.5% versus 25%, p<0.0001) and accuracy (73.98% versus 62.6%, p=0.0002) of the m-CAC system were improved without significant loss of sensitivity (94.44% versus 98.41%, p=0.1250) compared with the SmartBreast. The m-CAC system showed similar specificity (52.5% versus 45.83%, p=0.2430) and accuracy (73.98% versus 73.58%, p=1.0000) as the radiologist, but a lower sensitivity (94.44% versus 100%, p=0.0156).

CONCLUSION

The CAC system showed an acceptable agreement with the radiologist for characterisation of breast lesions. It has the potential to mimic the decision-making behaviour of radiologists for the classification of breast lesions.

AI: Can It Make a Difference to the Predictive Value of Ultrasound Breast Biopsy?

(1) Background: This study aims to compare the ground truth (pathology results) against the BI-RADS classification of images acquired while performing breast ultrasound diagnostic examinations that led to a biopsy and against the result of processing the same images through the AI algorithm KOIOS DS ^TM (KOIOS). (2) Methods: All results of biopsies performed with ultrasound guidance during 2019 were recovered from the pathology department. Readers selected the image which better represented the BI-RADS classification, confirmed correlation to the biopsied image, and submitted it to the KOIOS AI software. The results of the BI-RADS classification of the diagnostic study performed at our institution were set against the KOIOS classification and both were compared to the pathology reports. (3) Results: 403 cases were included in this study. Pathology rendered 197 malignant and 206 benign reports. Four biopsies on BI-RADS 0 and two images are included. Of fifty BI-RADS 3 cases biopsied, only seven rendered cancers. All but one had a positive or suspicious cytology; all were classified as suspicious by KOIOS. Using KOIOS, 17 B3 biopsies could have been avoided. Of 347 BI-RADS 4, 5, and 6 cases, 190 were malignant (54.7%). Because only KOIOS suspicious and probably malignant categories should be biopsied, 312 biopsies would have resulted in 187 malignant lesions (60%), but 10 cancers would have been missed. (4) Conclusions: KOIOS had a higher ratio of positive biopsies in this selected case study vis-à-vis the BI-RADS 4, 5 and 6 categories. A large number of biopsies in the BI-RADS 3 category could have been avoided.

Artificial Intelligence and Precision Medicine: A New Frontier for the Treatment of Brain Tumors

Brain tumors are a widespread and serious neurological phenomenon that can be life- threatening. The computing field has allowed for the development of artificial intelligence (AI), which can mimic the neural network of the human brain. One use of this technology has been to help researchers capture hidden, high-dimensional images of brain tumors. These images can provide new insights into the nature of brain tumors and help to improve treatment options. AI and precision medicine (PM) are converging to revolutionize healthcare. AI has the potential to improve cancer imaging interpretation in several ways, including more accurate tumor genotyping, more precise delineation of tumor volume, and better prediction of clinical outcomes. AI-assisted brain surgery can be an effective and safe option for treating brain tumors. This review discusses various AI and PM techniques that can be used in brain tumor treatment. These new techniques for the treatment of brain tumors, i.e., genomic profiling, microRNA panels, quantitative imaging, and radiomics, hold great promise for the future. However, there are challenges that must be overcome for these technologies to reach their full potential and improve healthcare.

Evaluation of physician performance using a concurrent-read artificial intelligence system to support breast ultrasound interpretation

Purpose

The purpose of this study was to compare the diagnostic performance and the interpretation time of breast ultrasound examination between reading without and with the artificial intelligence (AI) system as a concurrent reading aid.

Material and methods

A fully crossed multi-reader and multi-case (MRMC) reader study was conducted. Sixteen participating physicians were recruited and retrospectively interpreted 172 breast ultrasound cases in two reading scenarios, once without and once with the AI system (BU-CAD™, TaiHao Medical Inc.) assistance for concurrent reading. Interpretations of any given case set with and without the AI system were separated by at least 5 weeks. These reading results were compared to the reference standard and the area under the LROC curve (AUCLROC), sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV) were calculated for performance evaluations. The interpretation time was also compared between the unaided and aided scenarios.

Results

With the help of the AI system, the readers had higher diagnostic performance with an increase in the average AUCLROC from 0.7582 to 0.8294 with statistically significant. The sensitivity, specificity, PPV, and NPV were also improved from 95.77%, 24.07%, 44.18%, and 93.50%–98.17%, 30.67%, 46.91%, and 96.10%, respectively. Of these, the improvement in specificity reached statistical significance. The average interpretation time was significantly reduced by approximately 40% when the readers were assisted by the AI system.

Conclusion

The concurrent-read AI system improves the diagnostic performance in detecting and diagnosing breast lesions on breast ultrasound images. In addition, the interpretation time is effectively reduced for the interpreting physicians.

•

A reader study was conducted to compare the breast ultrasound interpreting performance without and with the aid of AI system.

•

The performance of breast ultrasound interpretation was improved by the AI system as a concurrent reading aid.

•

The breast ultrasound interpretation time is significantly reduced by the AI system as a concurrent reading aid.

•

The reproducibility experiments of the same lesion cropped by different rectangular proved the robustness of the AI system.

Differing benefits of artificial intelligence-based computer-aided diagnosis (AI-CAD) for breast US according to workflow and experience level

Purpose

This study evaluated how artificial intelligence-based computer-assisted diagnosis (AI-CAD) for breast ultrasonography (US) influences diagnostic performance and agreement between radiologists with varying experience levels in different workflows.

Methods

Images of 492 breast lesions (200 malignant and 292 benign masses) in 472 women taken from April 2017 to June 2018 were included. Six radiologists (three inexperienced [<1 year of experience] and three experienced [10-15 years of experience]) individually reviewed US images with and without the aid of AI-CAD, first sequentially and then simultaneously. Diagnostic performance and interobserver agreement were calculated and compared between radiologists and AI-CAD.

Results

After implementing AI-CAD, the specificity, positive predictive value (PPV), and accuracy significantly improved, regardless of experience and workflow (all P<0.001, respectively). The overall area under the receiver operating characteristic curve significantly increased in simultaneous reading, but only for inexperienced radiologists. The agreement for Breast Imaging Reporting and Database System (BI-RADS) descriptors generally increased when AI-CAD was used (κ=0.29-0.63 to 0.35-0.73). Inexperienced radiologists tended to concede to AI-CAD results more easily than experienced radiologists, especially in simultaneous reading (P<0.001). The conversion rates for final assessment changes from BI-RADS 2 or 3 to BI-RADS higher than 4a or vice versa were also significantly higher in simultaneous reading than sequential reading (overall, 15.8% and 6.2%, respectively; P<0.001) for both inexperienced and experienced radiologists.

Conclusion

Using AI-CAD to interpret breast US improved the specificity, PPV, and accuracy of radiologists regardless of experience level. AI-CAD may work better in simultaneous reading to improve diagnostic performance and agreement between radiologists, especially for inexperienced radiologists.

Doppler Ultrasound Imaging Combined with Fetal Heart Detection in Predicting Fetal Distress in Pregnancy-Induced Hypertension under the Guidance of Artificial Intelligence Algorithm

This study was to improve the feasibility and economic benefits of intelligent medical system Doppler ultrasound (DUS) imaging technology combined with fetal heart detection to predict the fetal distress in pregnancy-induced hypertension (PIH), so as to reduce the risk of deterioration of the patient's condition. The characteristics of DUS images were analyzed, and a diffusion filter reducing the specificity was adopted to improve the smooth speckle noise of DUS images. 120 pregnant women in hospital were the subjects of the study, all of whom received ultrasound cord blood flow testing and fetal heart monitoring. 88 PIH patients with fetal distress were diagnosed and included in the observation group, and 32 healthy pregnant women tested during the same period were identified as the control group. Clinical data were reviewed and analyzed. The diagnostic rates of fetal distress by simple fetal heart monitoring and DUS detection combined with fetal heart monitoring were compared. The results showed that 26.7% of fetal distress were diagnosed by fetal heart monitoring alone, and 73.3% of fetal distress were diagnosed by combined testing, so the diagnostic accuracy of the combined detection method was greatly higher than the single fetal heart detection (P < 0.05). The Pulsatility index (PI), resistance index (RI), and S/D values detected by the umbilical artery in the observation group were 1.48, 0.85, and 4.31, respectively. The PI, RI, and S/D values detected by the umbilical artery in the control group were 0.96, 0.64, and 3.59, respectively. The results of arterial detection were significantly higher than those of the control group, and the difference was of significant scientific significance (P < 0.05). In summary, the PI and RI values of the middle cerebral artery (MCA) detected by DUS diagnosis can effectively reflect the current status of the fetus in the uterus and reduce the mortality of the fetus. The images guided by DUS imaging technology can clearly show the current status of the fetus in the uterus, effectively improve the medical diagnostic efficiency, and have important reference value for the development of intelligent medical equipment.

Artificial intelligence system reduces false-positive findings in the interpretation of breast ultrasound exams

Though consistently shown to detect mammographically occult cancers, breast ultrasound has been noted to have high false-positive rates. In this work, we present an AI system that achieves radiologist-level accuracy in identifying breast cancer in ultrasound images. Developed on 288,767 exams, consisting of 5,442,907 B-mode and Color Doppler images, the AI achieves an area under the receiver operating characteristic curve (AUROC) of 0.976 on a test set consisting of 44,755 exams. In a retrospective reader study, the AI achieves a higher AUROC than the average of ten board-certified breast radiologists (AUROC: 0.962 AI, 0.924 ± 0.02 radiologists). With the help of the AI, radiologists decrease their false positive rates by 37.3% and reduce requested biopsies by 27.8%, while maintaining the same level of sensitivity. This highlights the potential of AI in improving the accuracy, consistency, and efficiency of breast ultrasound diagnosis.

Computer-Aided Diagnosis Improves the Detection of Clinically Significant Prostate Cancer on Multiparametric-MRI: A Multi-Observer Performance Study Involving Inexperienced Readers

Recently, Computer Aided Diagnosis (CAD) systems have been proposed to help radiologists in detecting and characterizing Prostate Cancer (PCa). However, few studies evaluated the performances of these systems in a clinical setting, especially when used by non-experienced readers. The main aim of this study is to assess the diagnostic performance of non-experienced readers when reporting assisted by the likelihood map generated by a CAD system, and to compare the results with the unassisted interpretation. Three resident radiologists were asked to review multiparametric-MRI of patients with and without PCa, both unassisted and assisted by a CAD system. In both reading sessions, residents recorded all positive cases, and sensitivity, specificity, negative and positive predictive values were computed and compared. The dataset comprised 90 patients (45 with at least one clinically significant biopsy-confirmed PCa). Sensitivity significantly increased in the CAD assisted mode for patients with at least one clinically significant lesion (GS > 6) (68.7% vs. 78.1%, p = 0.018). Overall specificity was not statistically different between unassisted and assisted sessions (94.8% vs. 89.6, p = 0.072). The use of the CAD system significantly increases the per-patient sensitivity of inexperienced readers in the detection of clinically significant PCa, without negatively affecting specificity, while significantly reducing overall reporting time.

Impact of Original and Artificially Improved Artificial Intelligence-based Computer-aided Diagnosis on Breast US Interpretation

OBJECTIVE

For breast US interpretation, to assess impact of computer-aided diagnosis (CADx) in original mode or with improved sensitivity or specificity.

METHODS

In this IRB approved protocol, orthogonal-paired US images of 319 lesions identified on screening, including 88 (27.6%) cancers (median 7 mm, range 1-34 mm), were reviewed by 9 breast imaging radiologists. Each observer provided BI-RADS assessments (2, 3, 4A, 4B, 4C, 5) before and after CADx in a mode-balanced design: mode 1, original CADx (outputs benign, probably benign, suspicious, or malignant); mode 2, artificially-high-sensitivity CADx (benign or malignant); and mode 3, artificially-high-specificity CADx (benign or malignant). Area under the receiver operating characteristic curve (AUC) was estimated under each modality and for standalone CADx outputs. Multi-reader analysis accounted for inter-reader variability and correlation between same-lesion assessments.

RESULTS

AUC of standalone CADx was 0.77 (95% CI: 0.72-0.83). For mode 1, average reader AUC was 0.82 (range 0.76-0.84) without CADx and not significantly changed with CADx. In high-sensitivity mode, all observers' AUCs increased: average AUC 0.83 (range 0.78-0.86) before CADx increased to 0.88 (range 0.84-0.90), P < 0.001. In high-specificity mode, all observers' AUCs increased: average AUC 0.82 (range 0.76-0.84) before CADx increased to 0.89 (range 0.87-0.92), P < 0.0001. Radiologists responded more frequently to malignant CADx cues in high-specificity mode (42.7% vs 23.2% mode 1, and 27.0% mode 2, P = 0.008).

CONCLUSION

Original CADx did not substantially impact radiologists' interpretations. Radiologists showed improved performance and were more responsive when CADx produced fewer false-positive malignant cues.

Role of Artificial Intelligence Applications in Real-Life Clinical Practice: Systematic Review

BACKGROUND

Artificial intelligence (AI) applications are growing at an unprecedented pace in health care, including disease diagnosis, triage or screening, risk analysis, surgical operations, and so forth. Despite a great deal of research in the development and validation of health care AI, only few applications have been actually implemented at the frontlines of clinical practice.

OBJECTIVE

The objective of this study was to systematically review AI applications that have been implemented in real-life clinical practice.

METHODS

We conducted a literature search in PubMed, Embase, Cochrane Central, and CINAHL to identify relevant articles published between January 2010 and May 2020. We also hand searched premier computer science journals and conferences as well as registered clinical trials. Studies were included if they reported AI applications that had been implemented in real-world clinical settings.

RESULTS

We identified 51 relevant studies that reported the implementation and evaluation of AI applications in clinical practice, of which 13 adopted a randomized controlled trial design and eight adopted an experimental design. The AI applications targeted various clinical tasks, such as screening or triage (n=16), disease diagnosis (n=16), risk analysis (n=14), and treatment (n=7). The most commonly addressed diseases and conditions were sepsis (n=6), breast cancer (n=5), diabetic retinopathy (n=4), and polyp and adenoma (n=4). Regarding the evaluation outcomes, we found that 26 studies examined the performance of AI applications in clinical settings, 33 studies examined the effect of AI applications on clinician outcomes, 14 studies examined the effect on patient outcomes, and one study examined the economic impact associated with AI implementation.

CONCLUSIONS

This review indicates that research on the clinical implementation of AI applications is still at an early stage despite the great potential. More research needs to assess the benefits and challenges associated with clinical AI applications through a more rigorous methodology.

Association of Clinician Diagnostic Performance With Machine Learning-Based Decision Support Systems: A Systematic Review

IMPORTANCE

An increasing number of machine learning (ML)-based clinical decision support systems (CDSSs) are described in the medical literature, but this research focuses almost entirely on comparing CDSS directly with clinicians (human vs computer). Little is known about the outcomes of these systems when used as adjuncts to human decision-making (human vs human with computer).

OBJECTIVES

To conduct a systematic review to investigate the association between the interactive use of ML-based diagnostic CDSSs and clinician performance and to examine the extent of the CDSSs' human factors evaluation.

EVIDENCE REVIEW

A search of MEDLINE, Embase, PsycINFO, and grey literature was conducted for the period between January 1, 2010, and May 31, 2019. Peer-reviewed studies published in English comparing human clinician performance with and without interactive use of an ML-based diagnostic CDSSs were included. All metrics used to assess human performance were considered as outcomes. The risk of bias was assessed using Quality Assessment of Diagnostic Accuracy Studies (QUADAS-2) and Risk of Bias in Non-Randomised Studies-Intervention (ROBINS-I). Narrative summaries were produced for the main outcomes. Given the heterogeneity of medical conditions, outcomes of interest, and evaluation metrics, no meta-analysis was performed.

FINDINGS

A total of 8112 studies were initially retrieved and 5154 abstracts were screened; of these, 37 studies met the inclusion criteria. The median number of participating clinicians was 4 (interquartile range, 3-8). Of the 107 results that reported statistical significance, 54 (50%) were increased by the use of CDSSs, 4 (4%) were decreased, and 49 (46%) showed no change or an unclear change. In the subgroup of studies carried out in representative clinical settings, no association between the use of ML-based diagnostic CDSSs and improved clinician performance could be observed. Interobserver agreement was the commonly reported outcome whose change was the most strongly associated with CDSS use. Four studies (11%) reported on user feedback, and, in all but 1 case, clinicians decided to override at least some of the algorithms' recommendations. Twenty-eight studies (76%) were rated as having a high risk of bias in at least 1 of the 4 QUADAS-2 core domains, and 6 studies (16%) were considered to be at serious or critical risk of bias using ROBINS-I.

CONCLUSIONS AND RELEVANCE

This systematic review found only sparse evidence that the use of ML-based CDSSs is associated with improved clinician diagnostic performance. Most studies had a low number of participants, were at high or unclear risk of bias, and showed little or no consideration for human factors. Caution should be exercised when estimating the current potential of ML to improve human diagnostic performance, and more comprehensive evaluation should be conducted before deploying ML-based CDSSs in clinical settings. The results highlight the importance of considering supported human decisions as end points rather than merely the stand-alone CDSSs outputs.

Detection of Cardiac Structural Abnormalities in Fetal Ultrasound Videos Using Deep Learning

Artificial Intelligence (AI) technologies have recently been applied to medical imaging for diagnostic support. With respect to fetal ultrasound screening of congenital heart disease (CHD), it is still challenging to achieve consistently accurate diagnoses owing to its manual operation and the technical differences among examiners. Hence, we proposed an architecture of Supervised Object detection with Normal data Only (SONO), based on a convolutional neural network (CNN), to detect cardiac substructures and structural abnormalities in fetal ultrasound videos. We used a barcode-like timeline to visualize the probability of detection and calculated an abnormality score of each video. Performance evaluations of detecting cardiac structural abnormalities utilized videos of sequential cross-sections around a four-chamber view (Heart) and three-vessel trachea view (Vessels). The mean value of abnormality scores in CHD cases was significantly higher than normal cases (p < 0.001). The areas under the receiver operating characteristic curve in Heart and Vessels produced by SONO were 0.787 and 0.891, respectively, higher than the other conventional algorithms. SONO achieves an automatic detection of each cardiac substructure in fetal ultrasound videos, and shows an applicability to detect cardiac structural abnormalities. The barcode-like timeline is informative for examiners to capture the clinical characteristic of each case, and it is also expected to acquire one of the important features in the field of medical AI: the development of “explainable AI.”

Can an Artificial Intelligence Decision Aid Decrease False-Positive Breast Biopsies?

ABSTRACT

This study aimed to evaluate the effect of an artificial intelligence (AI) support system on breast ultrasound diagnostic accuracy.In this Health Insurance Portability and Accountability Act-compliant, institutional review board-approved retrospective study, 200 lesions (155 benign, 45 malignant) were randomly selected from consecutive ultrasound-guided biopsies (June 2017-January 2019). Two readers, blinded to clinical history and pathology, evaluated lesions with and without an Food and Drug Administration-approved AI software. Lesion features, Breast Imaging Reporting and Data System (BI-RADS) rating (1-5), reader confidence level (1-5), and AI BI-RADS equivalent (1-5) were recorded. Statistical analysis was performed for diagnostic accuracy, negative predictive value, positive predictive value (PPV), sensitivity, and specificity of reader versus AI BI-RADS. Generalized estimating equation analysis was used for reader versus AI accuracy regarding lesion features and AI impact on low-confidence score lesions. Artificial intelligence effect on false-positive biopsy rate was determined. Statistical tests were conducted at a 2-sided 5% significance level.There was no significant difference in accuracy (73 vs 69.8%), negative predictive value (100% vs 98.5%), PPV (45.5 vs 42.4%), sensitivity (100% vs 96.7%), and specificity (65.2 vs 61.9; P = 0.118-0.409) for AI versus pooled reader assessment. Artificial intelligence was more accurate than readers for irregular shape (74.1% vs 57.4%, P = 0.002) and less accurate for round shape (26.5% vs 50.0%, P = 0.049). Artificial intelligence improved diagnostic accuracy for reader-rated low-confidence lesions with increased PPV (24.7% AI vs 19.3%, P = 0.004) and specificity (57.8% vs 44.6%, P = 0.008).Artificial intelligence decision support aid may help improve sonographic diagnostic accuracy, particularly in cases with low reader confidence, thereby decreasing false-positives.

Novel Approaches to Screening for Breast Cancer

Screening for breast cancer reduces breast cancer-related mortality and earlier detection facilitates less aggressive treatment. Unfortunately, current screening modalities are imperfect, suffering from limited sensitivity and high false-positive rates. Novel techniques in the field of breast imaging may soon play a role in breast cancer screening: digital breast tomosynthesis, contrast material-enhanced spectral mammography, US (automated three-dimensional breast US, transmission tomography, elastography, optoacoustic imaging), MRI (abbreviated and ultrafast, diffusion-weighted imaging), and molecular breast imaging. Artificial intelligence and radiomics have the potential to further improve screening strategies. Furthermore, nonimaging-based screening tests such as liquid biopsy and breathing tests may transform the screening landscape. © RSNA, 2020 Online supplemental material is available for this article.

Accuracy and efficiency of an artificial intelligence tool when counting breast mitoses

BACKGROUND

The mitotic count in breast carcinoma is an important prognostic marker. Unfortunately substantial inter- and intra-laboratory variation exists when pathologists manually count mitotic figures. Artificial intelligence (AI) coupled with whole slide imaging offers a potential solution to this problem. The aim of this study was to accordingly critique an AI tool developed to quantify mitotic figures in whole slide images of invasive breast ductal carcinoma.

METHODS

A representative H&E slide from 320 breast invasive ductal carcinoma cases was scanned at 40x magnification. Ten expert pathologists from two academic medical centers labeled mitotic figures in whole slide images to train and validate an AI algorithm to detect and count mitoses. Thereafter, 24 readers of varying expertise were asked to count mitotic figures with and without AI support in 140 high-power fields derived from a separate dataset. Their accuracy and efficiency of performing these tasks were calculated and statistical comparisons performed.

RESULTS

For each experience level the accuracy, precision and sensitivity of counting mitoses by users improved with AI support. There were 21 readers (87.5%) that identified more mitoses using AI support and 13 reviewers (54.2%) that decreased the quantity of falsely flagged mitoses with AI. More time was spent on this task for most participants when not provided with AI support. AI assistance resulted in an overall time savings of 27.8%.

CONCLUSIONS

This study demonstrates that pathology end-users were more accurate and efficient at quantifying mitotic figures in digital images of invasive breast carcinoma with the aid of AI. Higher inter-pathologist agreement with AI assistance suggests that such algorithms can also help standardize practice. Not surprisingly, there is much enthusiasm in pathology regarding the prospect of using AI in routine practice to perform mundane tasks such as counting mitoses.

Should We Ignore, Follow, or Biopsy? Impact of Artificial Intelligence Decision Support on Breast Ultrasound Lesion Assessment

OBJECTIVE. The objective of this study was to assess the impact of artificial intelligence (AI)-based decision support (DS) on breast ultrasound (US) lesion assessment. MATERIALS AND METHODS. A multicenter retrospective review of 900 breast lesions (470/900 [52.2%] benign; 430/900 [47.8%] malignant) on US by 15 physicians (11 radiologists, two surgeons, two obstetrician/gynecologists). An AI system (Koios DS for Breast, Koios Medical) evaluated images and assigned them to one of four categories: benign, probably benign, suspicious, and probably malignant. Each reader reviewed cases twice: 750 cases with US only or with US plus DS; 4 weeks later, cases were reviewed in the opposite format. One hundred fifty additional cases were presented identically in each session. DS and reader sensitivity, specificity, and positive likelihood ratios (PLRs) were calculated as well as reader AUCs with and without DS. The Kendall τ-b correlation coefficient was used to assess intraand interreader variability. RESULTS. Mean reader AUC for cases reviewed with US only was 0.83 (95% CI, 0.78-0.89); for cases reviewed with US plus DS, mean AUC was 0.87 (95% CI, 0.84-0.90). PLR for the DS system was 1.98 (95% CI, 1.78-2.18) and was higher than the PLR for all readers but one. Fourteen readers had better AUC with US plus DS than with US only. Mean Kendall τ-b for US-only interreader variability was 0.54 (95% CI, 0.53-0.55); for US plus DS, it was 0.68 (95% CI, 0.67-0.69). Intrareader variability improved with DS; class switching (defined as crossing from BI-RADS category 3 to BI-RADS category 4A or above) occurred in 13.6% of cases with US only versus 10.8% of cases with US plus DS (p = 0.04). CONCLUSION. AI-based DS improves accuracy of sonographic breast lesion assessment while reducing inter- and intraobserver variability.

Physician workforce in the United States of America: forecasting nationwide shortages

BACKGROUND

Physicians play a critical role in healthcare delivery. With an aging US population, population growth, and a greater insured population following the Affordable Care Act (ACA), healthcare demand is growing at an unprecedented pace. This study is to examine current and future physician job surplus/shortage trends across the United States of America from 2017 to 2030.

METHODS

Using projected changes in population size and age, the authors developed demand and supply models to forecast the physician shortage (difference between demand and supply) in each of the 50 states. Letter grades were then assigned based on projected physician shortage ratios (physician shortage per 100 000 people) to evaluate physician shortages and describe the changing physician workforce in each state.

RESULTS

On the basis of current trends, the number of states receiving a grade of "D" or "F" for their physician shortage ratio will increase from 4 in 2017 to 23 by 2030, with a total national deficit of 139 160 physician jobs. By 2030, the West is forecasted to have the greatest physician shortage ratio (69 physician jobs per 100 000 people), while the Northeast will have a surplus of 50 jobs per 100 000 people.

CONCLUSION

There will be physician workforce shortages throughout the country in 2030. Outcomes of this study provide a foundation to discuss effective and efficient ways to curb the worsening shortage over the coming decades and meet current and future population demands. Increased efforts to understand shortage dynamics are warranted.

[Artificial intelligence in the diagnosis of breast cancer : Yesterday, today and tomorrow]

BACKGROUND

Artificial intelligence (AI) is increasingly applied in the field of breast imaging.

OBJECTIVES

What are the main areas where AI is applied in breast imaging and what AI and computer-aided diagnosis (CAD) systems are already available?

MATERIALS AND METHODS

Basic literature and vendor-supplied information are screened for relevant information, which is then pooled, structured and discussed from the perspective of breast imaging.

RESULTS

Original CAD systems in mammography date almost 25 years back. They are much more widely applied in the United States than in Europe. The initial CAD systems exhibited limited diagnostic abilities and disproportionally high rates of false positive results. Since 2012, deep learning mechanisms have been applied and expand the application possibilities of AI.

CONCLUSION

To date there is no algorithm that has beyond doubt been proven to outperform double reporting by two certified breast radiologists. AI could, however, in the foreseeable future, take over the following tasks: preselection of abnormal examinations to substantially reduce workload of the radiologists by either excluding normal findings from human review or by replacing the double reader in screening. Furthermore, the establishment of radio-patho-genomic correlations and their translation into clinical practice is hardly conceivable without AI.