1
|
Fritz BA, King CR, Abdelhack M, Chen Y, Kronzer A, Abraham J, Tripathi S, Ben Abdallah A, Kannampallil T, Budelier TP, Helsten D, Montes de Oca A, Mehta D, Sontha P, Higo O, Kerby P, Gregory SH, Wildes TS, Avidan MS. Effect of machine learning models on clinician prediction of postoperative complications: the Perioperative ORACLE randomised clinical trial. Br J Anaesth 2024:S0007-0912(24)00468-9. [PMID: 39261226 DOI: 10.1016/j.bja.2024.08.004] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/23/2024] [Revised: 07/19/2024] [Accepted: 08/07/2024] [Indexed: 09/13/2024] Open
Abstract
BACKGROUND Anaesthesiologists might be able to mitigate risk if they know which patients are at greatest risk for postoperative complications. This trial examined the impact of machine learning models on clinician risk assessment. METHODS This single-centre, prospective, randomised clinical trial enrolled surgical patients aged ≥18 yr. Anaesthesiologists and nurse anaesthetists providing remote telemedicine support reviewed electronic health records with (assisted group) or without (unassisted group) reviewing machine learning predictions. Clinicians predicted the likelihood of postoperative 30-day all-cause mortality and postoperative acute kidney injury (AKI) within 7 days. The primary outcome was area under the receiver operating characteristic curve (AUROC) for clinician predictions of mortality and AKI, comparing AUROCs between assisted and unassisted assessments. RESULTS We analysed 5071 patients (mean [range] age: 58 [18-100] yr; 52% female) assessed by 89 clinicians. Of these, 98 (2.2%) patients died within 30 days of surgery and 450 (11.1%) patients sustained AKI. Clinician predictions agreed with the models more strongly in the assisted vs unassisted group (weighted kappa 0.75 vs 0.62 for death, mean difference: 0.13 [95% CI 0.10-0.17]; and 0.79 vs 0.54 for AKI, mean difference: 0.25 [95% CI 0.21-0.29]). Clinical prediction of death was similar between the assisted (AUROC 0.793) and unassisted (AUROC 0.780) groups (mean difference: 0.013 [95% CI -0.070 to 0.097]; P=0.76). Prediction of AKI had an AUROC of 0.734 in the assisted group vs 0.688 in the unassisted group (difference 0.046 [95% CI -0.003 to 0.091]; P=0.06). CONCLUSIONS Clinician performance was not improved by machine learning assistance. Further work is needed to clarify the role of machine learning in real-time perioperative risk stratification. CLINICAL TRIAL REGISTRATION NCT05042804.
Collapse
Affiliation(s)
- Bradley A Fritz
- Department of Anesthesiology, Washington University School of Medicine, Saint Louis, MO, USA.
| | - Christopher R King
- Department of Anesthesiology, Washington University School of Medicine, Saint Louis, MO, USA
| | - Mohamed Abdelhack
- Department of Computer Science and Engineering, Washington University McKelvey School of Engineering, Saint Louis, MO, USA; Krembil Centre for Neuroinformatics, Centre for Addiction and Mental Health, Toronto, ON, Canada
| | - Yixin Chen
- Department of Computer Science and Engineering, Washington University McKelvey School of Engineering, Saint Louis, MO, USA
| | - Alex Kronzer
- Department of Anesthesiology, Washington University School of Medicine, Saint Louis, MO, USA
| | - Joanna Abraham
- Department of Anesthesiology, Washington University School of Medicine, Saint Louis, MO, USA; Institute for Informatics, Data Science, and Biostatistics, Washington University School of Medicine, Saint Louis, MO, USA
| | - Sandhya Tripathi
- Department of Anesthesiology, Washington University School of Medicine, Saint Louis, MO, USA
| | - Arbi Ben Abdallah
- Department of Anesthesiology, Washington University School of Medicine, Saint Louis, MO, USA
| | - Thomas Kannampallil
- Department of Anesthesiology, Washington University School of Medicine, Saint Louis, MO, USA; Institute for Informatics, Data Science, and Biostatistics, Washington University School of Medicine, Saint Louis, MO, USA
| | - Thaddeus P Budelier
- Department of Anesthesiology, Washington University School of Medicine, Saint Louis, MO, USA
| | - Daniel Helsten
- Department of Anesthesiology, Washington University School of Medicine, Saint Louis, MO, USA
| | - Arianna Montes de Oca
- Department of Anesthesiology, Washington University School of Medicine, Saint Louis, MO, USA
| | - Divya Mehta
- Department of Anesthesiology, Washington University School of Medicine, Saint Louis, MO, USA
| | - Pratyush Sontha
- Department of Anesthesiology, Washington University School of Medicine, Saint Louis, MO, USA
| | - Omokhaye Higo
- Department of Anesthesiology, Washington University School of Medicine, Saint Louis, MO, USA
| | - Paul Kerby
- Department of Anesthesiology, Washington University School of Medicine, Saint Louis, MO, USA
| | - Stephen H Gregory
- Department of Anesthesiology, Washington University School of Medicine, Saint Louis, MO, USA
| | - Troy S Wildes
- Department of Anesthesiology, University of Nebraska Medical Center, Omaha, NE, USA
| | - Michael S Avidan
- Department of Anesthesiology, Washington University School of Medicine, Saint Louis, MO, USA
| |
Collapse
|
2
|
Wong CR, Zhu A, Baltzer HL. The Accuracy of Artificial Intelligence Models in Hand/Wrist Fracture and Dislocation Diagnosis: A Systematic Review and Meta-Analysis. JBJS Rev 2024; 12:01874474-202409000-00006. [PMID: 39236148 DOI: 10.2106/jbjs.rvw.24.00106] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 09/07/2024]
Abstract
BACKGROUND Early and accurate diagnosis is critical to preserve function and reduce healthcare costs in patients with hand and wrist injury. As such, artificial intelligence (AI) models have been developed for the purpose of diagnosing fractures through imaging. The purpose of this systematic review and meta-analysis was to determine the accuracy of AI models in identifying hand and wrist fractures and dislocations. METHODS Adhering to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses Diagnostic Test Accuracy guidelines, Ovid MEDLINE, Embase, and Cochrane Central Register of Controlled Trials were searched from their inception to October 10, 2023. Studies were included if they utilized an AI model (index test) for detecting hand and wrist fractures and dislocations in pediatric (<18 years) or adult (>18 years) patients through any radiologic imaging, with the reference standard established through image review by a medical expert. Results were synthesized through bivariate analysis. Risk of bias was assessed using the QUADAS-2 tool. This study was registered with PROSPERO (CRD42023486475). Certainty of evidence was assessed using Grading of Recommendations Assessment, Development, and Evaluation. RESULTS A systematic review identified 36 studies. Most studies assessed wrist fractures (27.90%) through radiograph imaging (94.44%), with radiologists serving as the reference standard (66.67%). AI models demonstrated area under the curve (0.946), positive likelihood ratio (7.690; 95% confidence interval, 6.400-9.190), and negative likelihood ratio (0.112; 0.0848-0.145) in diagnosing hand and wrist fractures and dislocations. Examining only studies characterized by a low risk of bias, sensitivity analysis did not reveal any difference from the overall results. Overall certainty of evidence was moderate. CONCLUSION In demonstrating the accuracy of AI models in hand and wrist fracture and dislocation diagnosis, we have demonstrated that the potential use of AI in diagnosing hand and wrist fractures is promising. LEVEL OF EVIDENCE Level III. See Instructions for Authors for a complete description of levels of evidence.
Collapse
Affiliation(s)
- Chloe R Wong
- Division of Plastic, Reconstructive & Aesthetic Surgery, Department of Surgery, University of Toronto, Toronto, Ontario, Canada
| | - Alice Zhu
- Division of General Surgery, Department of Surgery, University of Toronto, Toronto, Ontario, Canada
| | - Heather L Baltzer
- Division of Plastic, Reconstructive & Aesthetic Surgery, Department of Surgery, University of Toronto, Toronto, Ontario, Canada
| |
Collapse
|
3
|
Janssen SM, Bouzembrak Y, Tekinerdogan B. Artificial Intelligence in Malnutrition: A Systematic Literature Review. Adv Nutr 2024; 15:100264. [PMID: 38971229 PMCID: PMC11403436 DOI: 10.1016/j.advnut.2024.100264] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/12/2024] [Revised: 06/03/2024] [Accepted: 06/26/2024] [Indexed: 07/08/2024] Open
Abstract
Malnutrition among the population of the world is a frequent yet underdiagnosed problem in both children and adults. Development of malnutrition screening and diagnostic tools for early detection of malnutrition is necessary to prevent long-term complications to patients' health and well-being. Most of these tools are based on predefined questionnaires and consensus guidelines. The use of artificial intelligence (AI) allows for automated tools to detect malnutrition in an earlier stage to prevent long-term consequences. In this study, a systematic literature review was carried out with the goal of providing detailed information on what patient groups, screening tools, machine learning algorithms, data types, and variables are being used, as well as the current limitations and implementation stage of these AI-based tools. The results showed that a staggering majority exceeding 90% of all AI models go unused in day-to-day clinical practice. Furthermore, supervised learning models seemed to be the most popular type of learning. Alongside this, disease-related malnutrition was the most common category of malnutrition found in the analysis of all primary studies. This research provides a resource for researchers to identify directions for their research on the use of AI in malnutrition.
Collapse
Affiliation(s)
- Sander Mw Janssen
- Information Technology Group, Wageningen University and Research, Wageningen, The Netherlands
| | - Yamine Bouzembrak
- Information Technology Group, Wageningen University and Research, Wageningen, The Netherlands.
| | - Bedir Tekinerdogan
- Information Technology Group, Wageningen University and Research, Wageningen, The Netherlands
| |
Collapse
|
4
|
Brauner LE, Yao Y, Grigull L, Klawonn F. Patient-Oriented Questionnaires and Machine Learning for Rare Disease Diagnosis: A Systematic Review. J Clin Med 2024; 13:5132. [PMID: 39274347 PMCID: PMC11396573 DOI: 10.3390/jcm13175132] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/17/2024] [Revised: 08/16/2024] [Accepted: 08/26/2024] [Indexed: 09/16/2024] Open
Abstract
Background: A major challenge faced by patients with rare diseases (RDs) often stems from delays in diagnosis, typically due to nonspecific clinical symptoms or doctors' limited experience in connecting symptoms to the underlying RD. Using patient-oriented questionnaires (POQs) as a data source for machine learning (ML) techniques can serve as a potential solution. These questionnaires enable patients to portray their day-to-day experiences living with their condition, irrespective of clinical symptoms. This systematic review-registered at PROSPERO with the Registration-ID: CRD42023490838-aims to present the current state of research in this domain by conducting a systematic literature search and identifying the potentials and limitations of this methodology. Methods: The review adheres to Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines and was primarily funded by the German Federal Ministry of Education and Research under grant no. 16DHBKI056 (ki4all). The methodology involved a systematic search across the databases PubMed, Semantic Scholar and Google Scholar, covering articles published until June 2023. The inclusion criteria encompass examining the use of POQs in diagnosing rare and common diseases. Additionally, studies that focused on applying ML techniques to the resulting datasets were considered for inclusion. The primary objective was to include English as well as German research that involved the generation of predictions regarding the underlying disease based on the information gathered from POQs. Furthermore, studies exploring identifying predictive indicators associated with the underlying disease were also included in the literature review. The following data were extracted from the selected studies: year of publication, number of questions in the POQs, answer scale in the questionnaires, the ML algorithms used, the input data for the ML algorithms, the performance of these algorithms and how the performance was measured. In addition, information on the development of the questionnaires was recorded. Results: This search retrieved 421 results in total. After one superficial and two comprehensive screening runs performed by two authors independently, we ended up with 26 studies for further consideration. Sixteen of these studies deal with diseases and ML algorithms to analyse data; the other ten studies provide contributing research in this field. We discuss several potentials and limitations of the evaluated approach. Conclusions: Overall, the results show that the full potential has not yet been exploited and that further research in this direction is worthwhile, because the study results show that ML algorithms can achieve promising results on POQ data; however, their use in everyday medical practice has not yet been investigated.
Collapse
Affiliation(s)
- Lea Eileen Brauner
- Department of Computer Science, Ostfalia University of Applied Sciences, 38302 Wolfenbuettel, Germany
| | - Yao Yao
- Department of Computer Science, Ostfalia University of Applied Sciences, 38302 Wolfenbuettel, Germany
| | - Lorenz Grigull
- Center for Rare Diseases Bonn (ZSEB), University Hospital of Bonn, 53127 Bonn, Germany
| | - Frank Klawonn
- Department of Computer Science, Ostfalia University of Applied Sciences, 38302 Wolfenbuettel, Germany
- Helmholtz Centre for Infection Research, 38124 Braunschweig, Germany
| |
Collapse
|
5
|
Dingel J, Kleine AK, Cecil J, Sigl AL, Lermer E, Gaube S. Predictors of Health Care Practitioners' Intention to Use AI-Enabled Clinical Decision Support Systems: Meta-Analysis Based on the Unified Theory of Acceptance and Use of Technology. J Med Internet Res 2024; 26:e57224. [PMID: 39102675 PMCID: PMC11333871 DOI: 10.2196/57224] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/15/2024] [Revised: 05/03/2024] [Accepted: 05/13/2024] [Indexed: 08/07/2024] Open
Abstract
BACKGROUND Artificial intelligence-enabled clinical decision support systems (AI-CDSSs) offer potential for improving health care outcomes, but their adoption among health care practitioners remains limited. OBJECTIVE This meta-analysis identified predictors influencing health care practitioners' intention to use AI-CDSSs based on the Unified Theory of Acceptance and Use of Technology (UTAUT). Additional predictors were examined based on existing empirical evidence. METHODS The literature search using electronic databases, forward searches, conference programs, and personal correspondence yielded 7731 results, of which 17 (0.22%) studies met the inclusion criteria. Random-effects meta-analysis, relative weight analyses, and meta-analytic moderation and mediation analyses were used to examine the relationships between relevant predictor variables and the intention to use AI-CDSSs. RESULTS The meta-analysis results supported the application of the UTAUT to the context of the intention to use AI-CDSSs. The results showed that performance expectancy (r=0.66), effort expectancy (r=0.55), social influence (r=0.66), and facilitating conditions (r=0.66) were positively associated with the intention to use AI-CDSSs, in line with the predictions of the UTAUT. The meta-analysis further identified positive attitude (r=0.63), trust (r=0.73), anxiety (r=-0.41), perceived risk (r=-0.21), and innovativeness (r=0.54) as additional relevant predictors. Trust emerged as the most influential predictor overall. The results of the moderation analyses show that the relationship between social influence and use intention becomes weaker with increasing age. In addition, the relationship between effort expectancy and use intention was stronger for diagnostic AI-CDSSs than for devices that combined diagnostic and treatment recommendations. Finally, the relationship between facilitating conditions and use intention was mediated through performance and effort expectancy. CONCLUSIONS This meta-analysis contributes to the understanding of the predictors of intention to use AI-CDSSs based on an extended UTAUT model. More research is needed to substantiate the identified relationships and explain the observed variations in effect sizes by identifying relevant moderating factors. The research findings bear important implications for the design and implementation of training programs for health care practitioners to ease the adoption of AI-CDSSs into their practice.
Collapse
Affiliation(s)
- Julius Dingel
- Human-AI-Interaction Group, Center for Leadership and People Management, Ludwig Maximilian University of Munich, Munich, Germany
| | - Anne-Kathrin Kleine
- Human-AI-Interaction Group, Center for Leadership and People Management, Ludwig Maximilian University of Munich, Munich, Germany
| | - Julia Cecil
- Human-AI-Interaction Group, Center for Leadership and People Management, Ludwig Maximilian University of Munich, Munich, Germany
| | - Anna Leonie Sigl
- Department of Liberal Arts and Sciences, Technical University of Applied Sciences Augsburg, Augsburg, Germany
| | - Eva Lermer
- Human-AI-Interaction Group, Center for Leadership and People Management, Ludwig Maximilian University of Munich, Munich, Germany
- Department of Liberal Arts and Sciences, Technical University of Applied Sciences Augsburg, Augsburg, Germany
| | - Susanne Gaube
- Human Factors in Healthcare, Global Business School for Health, University College London, London, United Kingdom
| |
Collapse
|
6
|
Ortiz A, Mulsant BH. Beyond Step Count: Are We Ready to Use Digital Phenotyping to Make Actionable Individual Predictions in Psychiatry? J Med Internet Res 2024; 26:e59826. [PMID: 39102686 PMCID: PMC11333868 DOI: 10.2196/59826] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/23/2024] [Revised: 07/09/2024] [Accepted: 07/16/2024] [Indexed: 08/07/2024] Open
Abstract
Some models for mental disorders or behaviors (eg, suicide) have been successfully developed, allowing predictions at the population level. However, current demographic and clinical variables are neither sensitive nor specific enough for making individual actionable clinical predictions. A major hope of the "Decade of the Brain" was that biological measures (biomarkers) would solve these issues and lead to precision psychiatry. However, as models are based on sociodemographic and clinical data, even when these biomarkers differ significantly between groups of patients and control participants, they are still neither sensitive nor specific enough to be applied to individual patients. Technological advances over the past decade offer a promising approach based on new measures that may be essential for understanding mental disorders and predicting their trajectories. Several new tools allow us to continuously monitor objective behavioral measures (eg, hours of sleep) and densely sample subjective measures (eg, mood). The promise of this approach, referred to as digital phenotyping, was recognized almost a decade ago, with its potential impact on psychiatry being compared to the impact of the microscope on biological sciences. However, despite the intuitive belief that collecting densely sampled data (big data) improves clinical outcomes, recent clinical trials have not shown that incorporating digital phenotyping improves clinical outcomes. This viewpoint provides a stepwise development and implementation approach, similar to the one that has been successful in the prediction and prevention of cardiovascular disease, to achieve clinically actionable predictions in psychiatry.
Collapse
Affiliation(s)
- Abigail Ortiz
- Department of Psychiatry, Temerty Faculty of Medicine, University of Toronto, Toronto, ON, Canada
| | - Benoit H Mulsant
- Department of Psychiatry, Temerty Faculty of Medicine, University of Toronto, Toronto, ON, Canada
| |
Collapse
|
7
|
Parry R, Wright K, Bellinge JW, Ebert MA, Rowshanfarzad P, Francis RJ, Schultz CJ. Training and assessing convolutional neural network performance in automatic vascular segmentation using Ga-68 DOTATATE PET/CT. THE INTERNATIONAL JOURNAL OF CARDIOVASCULAR IMAGING 2024:10.1007/s10554-024-03171-2. [PMID: 38967895 DOI: 10.1007/s10554-024-03171-2] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Received: 10/21/2023] [Accepted: 06/22/2024] [Indexed: 07/06/2024]
Abstract
To evaluate a convolutional neural network's performance (nnU-Net) in the assessment of vascular contours, calcification and PET tracer activity using Ga-68 DOTATATE PET/CT. Patients who underwent Ga-68 DOTATATE PET/CT imaging over a 12-month period for neuroendocrine investigation were included. Manual cardiac and aortic segmentations were performed by an experienced observer. Scans were randomly allocated in ratio 64:16:20 for training, validation and testing of the nnU-Net model. PET tracer uptake and calcium scoring were compared between segmentation methods and different observers. 116 patients (53.5% female) with a median age of 64.5 years (range 23-79) were included. There were strong, positive correlations between all segmentations (mostly r > 0.98). There were no significant differences between manual and AI segmentation of SUVmean for global cardiac (mean ± SD 0.71 ± 0.22 vs. 0.71 ± 0.22; mean diff 0.001 ± 0.008, p > 0.05), ascending aorta (mean ± SD 0.44 ± 0.14 vs. 0.44 ± 0.14; mean diff 0.002 ± 0.01, p > 0.05), aortic arch (mean ± SD 0.44 ± 0.10 vs. 0.43 ± 0.10; mean diff 0.008 ± 0.16, p > 0.05) and descending aorta (mean ± SD < 0.001; 0.58 ± 0.12 vs. 0.57 ± 0.12; mean diff 0.01 ± 0.03, p > 0.05) contours. There was excellent agreement between the majority of manual and AI segmentation measures (r ≥ 0.80) and in all vascular contour calcium scores. Compared with the manual segmentation approach, the CNN required a significantly lower workflow time. AI segmentation of vascular contours using nnU-Net resulted in very similar measures of PET tracer uptake and vascular calcification when compared to an experienced observer and significantly reduced workflow time.
Collapse
Affiliation(s)
- R Parry
- School of Medicine, The University of Western Australia, Perth, Australia.
- Department of Cardiology, Royal Perth Hospital, Perth, Australia.
| | - K Wright
- School of Physics, Mathematics and Computing, The University of Western Australia, Crawley, WA, Australia
| | - J W Bellinge
- School of Medicine, The University of Western Australia, Perth, Australia
- Department of Cardiology, Royal Perth Hospital, Perth, Australia
| | - M A Ebert
- School of Physics, Mathematics and Computing, The University of Western Australia, Crawley, WA, Australia
- Department of Radiation Oncology, Sir Charles Gairdner Hospital, Perth, Australia
- School of Medicine and Population Health, University of Wisconsin, Madison, WI, USA
| | - P Rowshanfarzad
- School of Physics, Mathematics and Computing, The University of Western Australia, Crawley, WA, Australia
| | - R J Francis
- School of Medicine, The University of Western Australia, Perth, Australia
- Department of Nuclear Medicine, Sir Charles Gairdner Hospital, Perth, Australia
| | - C J Schultz
- School of Medicine, The University of Western Australia, Perth, Australia
- Department of Cardiology, Royal Perth Hospital, Perth, Australia
| |
Collapse
|
8
|
Butler JM, Taft T, Taber P, Rutter E, Fix M, Baker A, Weir C, Nevers M, Classen D, Cosby K, Jones M, Chapman A, Jones BE. Pneumonia diagnosis performance in the emergency department: a mixed-methods study about clinicians' experiences and exploration of individual differences and response to diagnostic performance feedback. J Am Med Inform Assoc 2024; 31:1503-1513. [PMID: 38796835 PMCID: PMC11187426 DOI: 10.1093/jamia/ocae112] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/14/2023] [Revised: 03/25/2024] [Accepted: 05/22/2024] [Indexed: 05/29/2024] Open
Abstract
OBJECTIVES We sought to (1) characterize the process of diagnosing pneumonia in an emergency department (ED) and (2) examine clinician reactions to a clinician-facing diagnostic discordance feedback tool. MATERIALS AND METHODS We designed a diagnostic feedback tool, using electronic health record data from ED clinicians' patients to establish concordance or discordance between ED diagnosis, radiology reports, and hospital discharge diagnosis for pneumonia. We conducted semistructured interviews with 11 ED clinicians about pneumonia diagnosis and reactions to the feedback tool. We administered surveys measuring individual differences in mindset beliefs, comfort with feedback, and feedback tool usability. We qualitatively analyzed interview transcripts and descriptively analyzed survey data. RESULTS Thematic results revealed: (1) the diagnostic process for pneumonia in the ED is characterized by diagnostic uncertainty and may be secondary to goals to treat and dispose the patient; (2) clinician diagnostic self-evaluation is a fragmented, inconsistent process of case review and follow-up that a feedback tool could fill; (3) the feedback tool was described favorably, with task and normative feedback harnessing clinician values of high-quality patient care and personal excellence; and (4) strong reactions to diagnostic feedback varied from implicit trust to profound skepticism about the validity of the concordance metric. Survey results suggested a relationship between clinicians' individual differences in learning and failure beliefs, feedback experience, and usability ratings. DISCUSSION AND CONCLUSION Clinicians value feedback on pneumonia diagnoses. Our results highlight the importance of feedback about diagnostic performance and suggest directions for considering individual differences in feedback tool design and implementation.
Collapse
Affiliation(s)
- Jorie M Butler
- Department of Biomedical Informatics, University of Utah Spencer Fox Eccles School of Medicine, Salt Lake City, UT 84108, United States
- Department of Internal Medicine, Division of Geriatrics, University of Utah Spencer Fox Eccles School of Medicine, Salt Lake City, UT 84132, United States
- Salt Lake City VA Informatics Decision-Enhancement and Analytic Sciences (IDEAS) Center for Innovation, Salt Lake City, UT 84148, United States
- Geriatrics Research, Education, and Clinical Center (GRECC), VA Salt Lake City Health Care System, Salt Lake City, UT 84148, United States
| | - Teresa Taft
- Department of Biomedical Informatics, University of Utah Spencer Fox Eccles School of Medicine, Salt Lake City, UT 84108, United States
| | - Peter Taber
- Department of Biomedical Informatics, University of Utah Spencer Fox Eccles School of Medicine, Salt Lake City, UT 84108, United States
- Salt Lake City VA Informatics Decision-Enhancement and Analytic Sciences (IDEAS) Center for Innovation, Salt Lake City, UT 84148, United States
| | - Elizabeth Rutter
- Department of Emergency Medicine, University of Utah Spencer Fox Eccles School of Medicine, Salt Lake City, UT 84108, United States
| | - Megan Fix
- Department of Emergency Medicine, University of Utah Spencer Fox Eccles School of Medicine, Salt Lake City, UT 84108, United States
| | - Alden Baker
- Department of Family and Preventive Medicine, Division of Physician Assistant Studies, University of Utah Spencer Fox Eccles School of Medicine, Salt Lake City, UT 84108, United States
| | - Charlene Weir
- Department of Biomedical Informatics, University of Utah Spencer Fox Eccles School of Medicine, Salt Lake City, UT 84108, United States
| | - McKenna Nevers
- Department of Internal Medicine, Division of Epidemiology, University of Utah Spencer Fox Eccles School of Medicine, Salt Lake City, UT 84108, United States
| | - David Classen
- Department of Internal Medicine, Division of Epidemiology, University of Utah Spencer Fox Eccles School of Medicine, Salt Lake City, UT 84108, United States
| | - Karen Cosby
- Department of Emergency Medicine, Cook County Hospital, Rush Medical College, Chicago, IL 60612, United States
| | - Makoto Jones
- Salt Lake City VA Informatics Decision-Enhancement and Analytic Sciences (IDEAS) Center for Innovation, Salt Lake City, UT 84148, United States
- Department of Internal Medicine, Division of Epidemiology, University of Utah Spencer Fox Eccles School of Medicine, Salt Lake City, UT 84108, United States
| | - Alec Chapman
- Department of Population Health Sciences, University of Utah Spencer Fox Eccles School of Medicine, Salt Lake City, UT 84108, United States
| | - Barbara E Jones
- Salt Lake City VA Informatics Decision-Enhancement and Analytic Sciences (IDEAS) Center for Innovation, Salt Lake City, UT 84148, United States
- Department of Internal Medicine, Division of Pulmonology, University of Utah Spencer Fox Eccles School of Medicine, Salt Lake City, UT 84108, United States
| |
Collapse
|
9
|
Wu H, Chen Z, Gu J, Jiang Y, Gao S, Chen W, Miao C. Predicting Chronic Pain and Treatment Outcomes Using Machine Learning Models Based on High-dimensional Clinical Data From a Large Retrospective Cohort. Clin Ther 2024; 46:490-498. [PMID: 38824080 DOI: 10.1016/j.clinthera.2024.04.012] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/29/2023] [Revised: 04/13/2024] [Accepted: 04/27/2024] [Indexed: 06/03/2024]
Abstract
PURPOSE To identify factors and indicators that affect chronic pain and pain relief, and to develop predictive models using machine learning. METHODS We analyzed the data of 67,028 outpatient cases and 11,310 valid samples with pain from a large retrospective cohort. We used decision tree, random forest, AdaBoost, neural network, and logistic regression to discover significant indicators and to predict pain and treatment relief. FINDINGS The random forest model had the highest accuracy, F1 value, precision, and recall rates for predicting pain relief. The main factors affecting pain and treatment relief included body mass index, blood pressure, age, body temperature, heart rate, pulse, and neutrophil/lymphocyte × platelet ratio. The logistic regression model had high sensitivity and specificity for predicting pain occurrence. IMPLICATIONS Machine learning models can be used to analyze the risk factors and predictors of chronic pain and pain relief, and to provide personalized and evidence-based pain management.
Collapse
Affiliation(s)
- Han Wu
- Department of Anesthesiology, Zhongshan Hospital, Fudan University, Shanghai, China; Shanghai Key laboratory of Perioperative Stress and Protection, Shanghai, China
| | - Zhaoyuan Chen
- Department of Anesthesiology, Zhongshan Hospital, Fudan University, Shanghai, China; Shanghai Key laboratory of Perioperative Stress and Protection, Shanghai, China
| | - Jiahui Gu
- Department of Anesthesiology, Zhongshan Hospital, Fudan University, Shanghai, China; Shanghai Key laboratory of Perioperative Stress and Protection, Shanghai, China
| | - Yi Jiang
- Department of Anesthesiology, Zhongshan Hospital, Fudan University, Shanghai, China; Shanghai Key laboratory of Perioperative Stress and Protection, Shanghai, China
| | - Shenjia Gao
- Department of Anesthesiology, Zhongshan Hospital, Fudan University, Shanghai, China; Shanghai Key laboratory of Perioperative Stress and Protection, Shanghai, China
| | - Wankun Chen
- Department of Anesthesiology, Zhongshan Hospital, Fudan University, Shanghai, China; Shanghai Key laboratory of Perioperative Stress and Protection, Shanghai, China.
| | - Changhong Miao
- Department of Anesthesiology, Zhongshan Hospital, Fudan University, Shanghai, China; Shanghai Key laboratory of Perioperative Stress and Protection, Shanghai, China.
| |
Collapse
|
10
|
Fritz BA, King CR, Abdelhack M, Chen Y, Kronzer A, Abraham J, Tripathi S, Abdallah AB, Kannampallil T, Budelier TP, Helsten D, Montes de Oca A, Mehta D, Sontha P, Higo O, Kerby P, Gregory SH, Wildes TS, Avidan MS. Effect of Machine Learning on Anaesthesiology Clinician Prediction of Postoperative Complications: The Perioperative ORACLE Randomised Clinical Trial. MEDRXIV : THE PREPRINT SERVER FOR HEALTH SCIENCES 2024:2024.05.22.24307754. [PMID: 38826471 PMCID: PMC11142290 DOI: 10.1101/2024.05.22.24307754] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/04/2024]
Abstract
Background Anaesthesiology clinicians can implement risk mitigation strategies if they know which patients are at greatest risk for postoperative complications. Although machine learning models predicting complications exist, their impact on clinician risk assessment is unknown. Methods This single-centre randomised clinical trial enrolled patients age ≥18 undergoing surgery with anaesthesiology services. Anaesthesiology clinicians providing remote intraoperative telemedicine support reviewed electronic health records with (assisted group) or without (unassisted group) also reviewing machine learning predictions. Clinicians predicted the likelihood of postoperative 30-day all-cause mortality and postoperative acute kidney injury within 7 days. Area under the receiver operating characteristic curve (AUROC) for the clinician predictions was determined. Results Among 5,071 patient cases reviewed by 89 clinicians, the observed incidence was 2% for postoperative death and 11% for acute kidney injury. Clinician predictions agreed with the models more strongly in the assisted versus unassisted group (weighted kappa 0.75 versus 0.62 for death [difference 0.13, 95%CI 0.10-0.17] and 0.79 versus 0.54 for kidney injury [difference 0.25, 95%CI 0.21-0.29]). Clinicians predicted death with AUROC of 0.793 in the assisted group and 0.780 in the unassisted group (difference 0.013, 95%CI -0.070 to 0.097). Clinicians predicted kidney injury with AUROC of 0.734 in the assisted group and 0.688 in the unassisted group (difference 0.046, 95%CI -0.003 to 0.091). Conclusions Although there was evidence that the models influenced clinician predictions, clinician performance was not statistically significantly different with and without machine learning assistance. Further work is needed to clarify the role of machine learning in real-time perioperative risk stratification. Trial Registration ClinicalTrials.gov NCT05042804.
Collapse
Affiliation(s)
- Bradley A Fritz
- Department of Anesthesiology, Washington University School of Medicine, Saint Louis, USA
| | - Christopher R King
- Department of Anesthesiology, Washington University School of Medicine, Saint Louis, USA
| | - Mohamed Abdelhack
- Department of Computer Science and Engineering, Washington University McKelvey School of Engineering, Saint Louis, USA
- Krembil Centre for Neuroinformatics, Centre for Addiction and Mental Health, Toronto, Canada
| | - Yixin Chen
- Department of Computer Science and Engineering, Washington University McKelvey School of Engineering, Saint Louis, USA
| | - Alex Kronzer
- Department of Anesthesiology, Washington University School of Medicine, Saint Louis, USA
| | - Joanna Abraham
- Department of Anesthesiology, Washington University School of Medicine, Saint Louis, USA
- Institute for Informatics, Data Science and Biostatistics, Washington University School of Medicine, Saint Louis, USA
| | - Sandhya Tripathi
- Department of Anesthesiology, Washington University School of Medicine, Saint Louis, USA
| | - Arbi Ben Abdallah
- Department of Anesthesiology, Washington University School of Medicine, Saint Louis, USA
| | - Thomas Kannampallil
- Department of Anesthesiology, Washington University School of Medicine, Saint Louis, USA
- Institute for Informatics, Data Science and Biostatistics, Washington University School of Medicine, Saint Louis, USA
| | - Thaddeus P Budelier
- Department of Anesthesiology, Washington University School of Medicine, Saint Louis, USA
| | - Daniel Helsten
- Department of Anesthesiology, Washington University School of Medicine, Saint Louis, USA
| | - Arianna Montes de Oca
- Department of Anesthesiology, Washington University School of Medicine, Saint Louis, USA
| | - Divya Mehta
- Department of Anesthesiology, Washington University School of Medicine, Saint Louis, USA
| | - Pratyush Sontha
- Department of Anesthesiology, Washington University School of Medicine, Saint Louis, USA
| | - Omokhaye Higo
- Department of Anesthesiology, Washington University School of Medicine, Saint Louis, USA
| | - Paul Kerby
- Department of Anesthesiology, Washington University School of Medicine, Saint Louis, USA
| | - Stephen H. Gregory
- Department of Anesthesiology, Washington University School of Medicine, Saint Louis, USA
| | - Troy S. Wildes
- Department of Anesthesiology, University of Nebraska Medical Center, Omaha, USA
| | - Michael S Avidan
- Department of Anesthesiology, Washington University School of Medicine, Saint Louis, USA
| |
Collapse
|
11
|
Çubukçu HC, Topcu Dİ, Yenice S. Machine learning-based clinical decision support using laboratory data. Clin Chem Lab Med 2024; 62:793-823. [PMID: 38015744 DOI: 10.1515/cclm-2023-1037] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/15/2023] [Accepted: 11/17/2023] [Indexed: 11/30/2023]
Abstract
Artificial intelligence (AI) and machine learning (ML) are becoming vital in laboratory medicine and the broader context of healthcare. In this review article, we summarized the development of ML models and how they contribute to clinical laboratory workflow and improve patient outcomes. The process of ML model development involves data collection, data cleansing, feature engineering, model development, and optimization. These models, once finalized, are subjected to thorough performance assessments and validations. Recently, due to the complexity inherent in model development, automated ML tools were also introduced to streamline the process, enabling non-experts to create models. Clinical Decision Support Systems (CDSS) use ML techniques on large datasets to aid healthcare professionals in test result interpretation. They are revolutionizing laboratory medicine, enabling labs to work more efficiently with less human supervision across pre-analytical, analytical, and post-analytical phases. Despite contributions of the ML tools at all analytical phases, their integration presents challenges like potential model uncertainties, black-box algorithms, and deskilling of professionals. Additionally, acquiring diverse datasets is hard, and models' complexity can limit clinical use. In conclusion, ML-based CDSS in healthcare can greatly enhance clinical decision-making. However, successful adoption demands collaboration among professionals and stakeholders, utilizing hybrid intelligence, external validation, and performance assessments.
Collapse
Affiliation(s)
- Hikmet Can Çubukçu
- General Directorate of Health Services, Rare Diseases Department, Turkish Ministry of Health, Ankara, Türkiye
- Hacettepe University Institute of Informatics, Ankara, Türkiye
| | - Deniz İlhan Topcu
- Health Sciences University İzmir Tepecik Education and Research Hospital, Medical Biochemistry, İzmir, Türkiye
| | - Sedef Yenice
- Florence Nightingale Hospital, Istanbul, Türkiye
| |
Collapse
|
12
|
Spielvogel CP, Haberl D, Mascherbauer K, Ning J, Kluge K, Traub-Weidinger T, Davies RH, Pierce I, Patel K, Nakuz T, Göllner A, Amereller D, Starace M, Monaci A, Weber M, Li X, Haug AR, Calabretta R, Ma X, Zhao M, Mascherbauer J, Kammerlander A, Hengstenberg C, Menezes LJ, Sciagra R, Treibel TA, Hacker M, Nitsche C. Diagnosis and prognosis of abnormal cardiac scintigraphy uptake suggestive of cardiac amyloidosis using artificial intelligence: a retrospective, international, multicentre, cross-tracer development and validation study. Lancet Digit Health 2024; 6:e251-e260. [PMID: 38519153 DOI: 10.1016/s2589-7500(23)00265-0] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/15/2023] [Revised: 11/21/2023] [Accepted: 12/11/2023] [Indexed: 03/24/2024]
Abstract
BACKGROUND The diagnosis of cardiac amyloidosis can be established non-invasively by scintigraphy using bone-avid tracers, but visual assessment is subjective and can lead to misdiagnosis. We aimed to develop and validate an artificial intelligence (AI) system for standardised and reliable screening of cardiac amyloidosis-suggestive uptake and assess its prognostic value, using a multinational database of 99mTc-scintigraphy data across multiple tracers and scanners. METHODS In this retrospective, international, multicentre, cross-tracer development and validation study, 16 241 patients with 19 401 scans were included from nine centres: one hospital in Austria (consecutive recruitment Jan 4, 2010, to Aug 19, 2020), five hospital sites in London, UK (consecutive recruitment Oct 1, 2014, to Sept 29, 2022), two centres in China (selected scans from Jan 1, 2021, to Oct 31, 2022), and one centre in Italy (selected scans from Jan 1, 2011, to May 23, 2023). The dataset included all patients referred to whole-body 99mTc-scintigraphy with an anterior view and all 99mTc-labelled tracers currently used to identify cardiac amyloidosis-suggestive uptake. Exclusion criteria were image acquisition at less than 2 h (99mTc-3,3-diphosphono-1,2-propanodicarboxylic acid, 99mTc-hydroxymethylene diphosphonate, and 99mTc-methylene diphosphonate) or less than 1 h (99mTc-pyrophosphate) after tracer injection and if patients' imaging and clinical data could not be linked. Ground truth annotation was derived from centralised core-lab consensus reading of at least three independent experts (CN, TT-W, and JN). An AI system for detection of cardiac amyloidosis-associated high-grade cardiac tracer uptake was developed using data from one centre (Austria) and independently validated in the remaining centres. A multicase, multireader study and a medical algorithmic audit were conducted to assess clinician performance compared with AI and to evaluate and correct failure modes. The system's prognostic value in predicting mortality was tested in the consecutively recruited cohorts using cox proportional hazards models for each cohort individually and for the combined cohorts. FINDINGS The prevalence of cases positive for cardiac amyloidosis-suggestive uptake was 142 (2%) of 9176 patients in the Austrian, 125 (2%) of 6763 patients in the UK, 63 (62%) of 102 patients in the Chinese, and 103 (52%) of 200 patients in the Italian cohorts. In the Austrian cohort, cross-validation performance showed an area under the curve (AUC) of 1·000 (95% CI 1·000-1·000). Independent validation yielded AUCs of 0·997 (0·993-0·999) for the UK, 0·925 (0·871-0·971) for the Chinese, and 1·000 (0·999-1·000) for the Italian cohorts. In the multicase multireader study, five physicians disagreed in 22 (11%) of 200 cases (Fleiss' kappa 0·89), with a mean AUC of 0·946 (95% CI 0·924-0·967), which was inferior to AI (AUC 0·997 [0·991-1·000], p=0·0040). The medical algorithmic audit demonstrated the system's robustness across demographic factors, tracers, scanners, and centres. The AI's predictions were independently prognostic for overall mortality (adjusted hazard ratio 1·44 [95% CI 1·19-1·74], p<0·0001). INTERPRETATION AI-based screening of cardiac amyloidosis-suggestive uptake in patients undergoing scintigraphy was reliable, eliminated inter-rater variability, and portended prognostic value, with potential implications for identification, referral, and management pathways. FUNDING Pfizer.
Collapse
Affiliation(s)
- Clemens P Spielvogel
- Department of Biomedical Imaging and Image-Guided Therapy, Division of Nuclear Medicine, Medical University of Vienna, Vienna, Austria
| | - David Haberl
- Department of Biomedical Imaging and Image-Guided Therapy, Division of Nuclear Medicine, Medical University of Vienna, Vienna, Austria
| | - Katharina Mascherbauer
- Department of Medicine II, Division of Cardiology, Medical University of Vienna, Vienna, Austria
| | - Jing Ning
- Christian Doppler Laboratory for Applied Metabolomics, Medical University of Vienna, Vienna, Austria
| | - Kilian Kluge
- Department of Biomedical Imaging and Image-Guided Therapy, Division of Nuclear Medicine, Medical University of Vienna, Vienna, Austria
| | - Tatjana Traub-Weidinger
- Department of Biomedical Imaging and Image-Guided Therapy, Division of Nuclear Medicine, Medical University of Vienna, Vienna, Austria
| | - Rhodri H Davies
- Institute of Cardiovascular Science, University College London, London, UK; Bart's Heart Centre, St Bartholomew's Hospital, West Smithfield, London, London, UK
| | - Iain Pierce
- Institute of Cardiovascular Science, University College London, London, UK; Bart's Heart Centre, St Bartholomew's Hospital, West Smithfield, London, London, UK
| | - Kush Patel
- Bart's Heart Centre, St Bartholomew's Hospital, West Smithfield, London, London, UK
| | - Thomas Nakuz
- Department of Biomedical Imaging and Image-Guided Therapy, Division of Nuclear Medicine, Medical University of Vienna, Vienna, Austria
| | - Adelina Göllner
- Department of Biomedical Imaging and Image-Guided Therapy, Division of Nuclear Medicine, Medical University of Vienna, Vienna, Austria
| | - Dominik Amereller
- Department of Biomedical Imaging and Image-Guided Therapy, Division of Nuclear Medicine, Medical University of Vienna, Vienna, Austria
| | - Maria Starace
- Department of Experimental and Clinical Biomedical Sciences, Nuclear Medicine Unit, University of Florence, Florence, Italy
| | - Alice Monaci
- Department of Experimental and Clinical Biomedical Sciences, Nuclear Medicine Unit, University of Florence, Florence, Italy
| | - Michael Weber
- Department of Biomedical Imaging and Image-Guided Therapy, Division of Nuclear Medicine, Medical University of Vienna, Vienna, Austria
| | - Xiang Li
- Department of Biomedical Imaging and Image-Guided Therapy, Division of Nuclear Medicine, Medical University of Vienna, Vienna, Austria
| | - Alexander R Haug
- Department of Biomedical Imaging and Image-Guided Therapy, Division of Nuclear Medicine, Medical University of Vienna, Vienna, Austria; Christian Doppler Laboratory for Applied Metabolomics, Medical University of Vienna, Vienna, Austria
| | - Raffaella Calabretta
- Department of Biomedical Imaging and Image-Guided Therapy, Division of Nuclear Medicine, Medical University of Vienna, Vienna, Austria
| | - Xiaowei Ma
- Department of Nuclear Medicine, Second Xiangya Hospital, Central South University, Changsha, China
| | - Min Zhao
- Department of Nuclear Medicine, Third Xiangya Hospital, Central South University, Changsha, China
| | - Julia Mascherbauer
- Department of Medicine II, Division of Cardiology, Medical University of Vienna, Vienna, Austria; Karl Landsteiner University of Health Sciences, Department of Internal Medicine 3, University Hospital St Pölten, Krems, Austria
| | - Andreas Kammerlander
- Department of Medicine II, Division of Cardiology, Medical University of Vienna, Vienna, Austria
| | - Christian Hengstenberg
- Department of Medicine II, Division of Cardiology, Medical University of Vienna, Vienna, Austria
| | - Leon J Menezes
- Bart's Heart Centre, St Bartholomew's Hospital, West Smithfield, London, London, UK
| | - Roberto Sciagra
- Department of Experimental and Clinical Biomedical Sciences, Nuclear Medicine Unit, University of Florence, Florence, Italy
| | - Thomas A Treibel
- Institute of Cardiovascular Science, University College London, London, UK; Bart's Heart Centre, St Bartholomew's Hospital, West Smithfield, London, London, UK
| | - Marcus Hacker
- Department of Biomedical Imaging and Image-Guided Therapy, Division of Nuclear Medicine, Medical University of Vienna, Vienna, Austria
| | - Christian Nitsche
- Institute of Cardiovascular Science, University College London, London, UK; Department of Medicine II, Division of Cardiology, Medical University of Vienna, Vienna, Austria; Bart's Heart Centre, St Bartholomew's Hospital, West Smithfield, London, London, UK.
| |
Collapse
|
13
|
Giddings R, Joseph A, Callender T, Janes SM, van der Schaar M, Sheringham J, Navani N. Factors influencing clinician and patient interaction with machine learning-based risk prediction models: a systematic review. Lancet Digit Health 2024; 6:e131-e144. [PMID: 38278615 DOI: 10.1016/s2589-7500(23)00241-8] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/23/2023] [Revised: 10/20/2023] [Accepted: 11/14/2023] [Indexed: 01/28/2024]
Abstract
Machine learning (ML)-based risk prediction models hold the potential to support the health-care setting in several ways; however, use of such models is scarce. We aimed to review health-care professional (HCP) and patient perceptions of ML risk prediction models in published literature, to inform future risk prediction model development. Following database and citation searches, we identified 41 articles suitable for inclusion. Article quality varied with qualitative studies performing strongest. Overall, perceptions of ML risk prediction models were positive. HCPs and patients considered that models have the potential to add benefit in the health-care setting. However, reservations remain; for example, concerns regarding data quality for model development and fears of unintended consequences following ML model use. We identified that public views regarding these models might be more negative than HCPs and that concerns (eg, extra demands on workload) were not always borne out in practice. Conclusions are tempered by the low number of patient and public studies, the absence of participant ethnic diversity, and variation in article quality. We identified gaps in knowledge (particularly views from under-represented groups) and optimum methods for model explanation and alerts, which require future research.
Collapse
Affiliation(s)
- Rebecca Giddings
- Lungs for Living Research Centre, UCL Respiratory, University College London, London, UK.
| | - Anabel Joseph
- Lungs for Living Research Centre, UCL Respiratory, University College London, London, UK
| | - Thomas Callender
- Lungs for Living Research Centre, UCL Respiratory, University College London, London, UK
| | - Sam M Janes
- Lungs for Living Research Centre, UCL Respiratory, University College London, London, UK
| | - Mihaela van der Schaar
- Department of Applied Mathematics and Theoretical Physics, University of Cambridge, Cambridge, UK; The Alan Turing Institute, London, UK
| | - Jessica Sheringham
- Department of Applied Health Research, University College London, London, UK
| | - Neal Navani
- Lungs for Living Research Centre, UCL Respiratory, University College London, London, UK
| |
Collapse
|
14
|
Welzel C, Cotte F, Wekenborg M, Vasey B, McCulloch P, Gilbert S. Holistic Human-Serving Digitization of Health Care Needs Integrated Automated System-Level Assessment Tools. J Med Internet Res 2023; 25:e50158. [PMID: 38117545 PMCID: PMC10765286 DOI: 10.2196/50158] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/21/2023] [Revised: 09/01/2023] [Accepted: 10/26/2023] [Indexed: 12/21/2023] Open
Abstract
Digital health tools, platforms, and artificial intelligence- or machine learning-based clinical decision support systems are increasingly part of health delivery approaches, with an ever-greater degree of system interaction. Critical to the successful deployment of these tools is their functional integration into existing clinical routines and workflows. This depends on system interoperability and on intuitive and safe user interface design. The importance of minimizing emergent workflow stress through human factors research and purposeful design for integration cannot be overstated. Usability of tools in practice is as important as algorithm quality. Regulatory and health technology assessment frameworks recognize the importance of these factors to a certain extent, but their focus remains mainly on the individual product rather than on emergent system and workflow effects. The measurement of performance and user experience has so far been performed in ad hoc, nonstandardized ways by individual actors using their own evaluation approaches. We propose that a standard framework for system-level and holistic evaluation could be built into interacting digital systems to enable systematic and standardized system-wide, multiproduct, postmarket surveillance and technology assessment. Such a system could be made available to developers through regulatory or assessment bodies as an application programming interface and could be a requirement for digital tool certification, just as interoperability is. This would enable health systems and tool developers to collect system-level data directly from real device use cases, enabling the controlled and safe delivery of systematic quality assessment or improvement studies suitable for the complexity and interconnectedness of clinical workflows using developing digital health technologies.
Collapse
Affiliation(s)
- Cindy Welzel
- Else Kröner Fresenius Center for Digital Health, TUD Dresden University of Technology, Dresden, Germany
| | | | - Magdalena Wekenborg
- Else Kröner Fresenius Center for Digital Health, TUD Dresden University of Technology, Dresden, Germany
| | - Baptiste Vasey
- Nuffield Department of Surgical Sciences, University of Oxford, Oxford, United Kingdom
| | - Peter McCulloch
- Nuffield Department of Surgical Sciences, University of Oxford, Oxford, United Kingdom
| | - Stephen Gilbert
- Else Kröner Fresenius Center for Digital Health, TUD Dresden University of Technology, Dresden, Germany
| |
Collapse
|
15
|
Habib AR, Gross CP. FDA Regulations of AI-Driven Clinical Decision Support Devices Fall Short. JAMA Intern Med 2023; 183:1401-1402. [PMID: 37812411 DOI: 10.1001/jamainternmed.2023.5006] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 10/10/2023]
Affiliation(s)
- Anand R Habib
- Department of Medicine, Yale University, New Haven, Connecticut
- National Clinician Scholars Program, Yale University, New Haven, Connecticut
| | - Cary P Gross
- Department of Medicine, Yale University, New Haven, Connecticut
- National Clinician Scholars Program, Yale University, New Haven, Connecticut
- Associate Editor, JAMA Internal Medicine
| |
Collapse
|
16
|
Haydon HM, Snoswell CL, Jones C, Carey M, Taylor M, Horstmanshof L, Hicks R, Lotfaliany M, Banbury A. Digital health literacy to enhance workforce skills and clinical effectiveness: A response to 'Digital health literacy: Helpful today, dependency tomorrow? Contingency planning in a digital age'. Australas J Ageing 2023; 42:803-804. [PMID: 37986677 DOI: 10.1111/ajag.13257] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/17/2023] [Revised: 10/26/2023] [Accepted: 10/30/2023] [Indexed: 11/22/2023]
Affiliation(s)
- Helen M Haydon
- Centre for Online Health, University of Queensland, Brisbane, Queensland, Australia
- Centre for Health Services Research, University of Queensland, Brisbane, Queensland, Australia
| | - Centaine L Snoswell
- Centre for Online Health, University of Queensland, Brisbane, Queensland, Australia
- Centre for Health Services Research, University of Queensland, Brisbane, Queensland, Australia
| | - Cindy Jones
- Faculty of Health Sciences & Medicine, Bond University, Gold Coast, Queensland, Australia
- Menzies Health Institute Queensland, Griffith University, Brisbane, Queensland, Australia
| | - Melissa Carey
- Centre for Health Research, The University of Southern Queensland, Ipswich, Queensland, Australia
- University of Auckland, Auckland, New Zealand
| | - Melissa Taylor
- School of Nursing and Midwifery, Centre for Health Research, The University of Southern Queensland, Ipswich, Queensland, Australia
| | - Louise Horstmanshof
- Faculty of Health, Southern Cross University, Lismore, New South Wales, Australia
| | - Richard Hicks
- School of Psychology, Faculty of Society and Design, Bond University, Gold Coast, Queensland, Australia
| | - Mojtaba Lotfaliany
- Centre for Online Health, University of Queensland, Brisbane, Queensland, Australia
- Centre for Health Services Research, University of Queensland, Brisbane, Queensland, Australia
- The Institute for Mental and Physical Health and Clinical Translation (IMPACT), School of Medicine, Barwon Health, Deakin University, Geelong, Victoria, Australia
| | - Annie Banbury
- Centre for Online Health, University of Queensland, Brisbane, Queensland, Australia
- Centre for Health Services Research, University of Queensland, Brisbane, Queensland, Australia
| |
Collapse
|
17
|
Chen Z, Liang N, Zhang H, Li H, Yang Y, Zong X, Chen Y, Wang Y, Shi N. Harnessing the power of clinical decision support systems: challenges and opportunities. Open Heart 2023; 10:e002432. [PMID: 38016787 PMCID: PMC10685930 DOI: 10.1136/openhrt-2023-002432] [Citation(s) in RCA: 8] [Impact Index Per Article: 8.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 07/25/2023] [Accepted: 10/31/2023] [Indexed: 11/30/2023] Open
Abstract
Clinical decision support systems (CDSSs) are increasingly integrated into healthcare settings to improve patient outcomes, reduce medical errors and enhance clinical efficiency by providing clinicians with evidence-based recommendations at the point of care. However, the adoption and optimisation of these systems remain a challenge. This review aims to provide an overview of the current state of CDSS, discussing their development, implementation, benefits, limitations and future directions. We also explore the potential for enhancing their effectiveness and provide an outlook for future developments in this field. There are several challenges in CDSS implementation, including data privacy concerns, system integration and clinician acceptance. While CDSS have demonstrated significant potential, their adoption and optimisation remain a challenge.
Collapse
Affiliation(s)
- Zhao Chen
- Institute of Basic Research in Clinical Medicine, China Academy of Chinese Medical Sciences, Beijing, China
| | - Ning Liang
- Institute of Basic Research in Clinical Medicine, China Academy of Chinese Medical Sciences, Beijing, China
| | - Haili Zhang
- Institute of Basic Research in Clinical Medicine, China Academy of Chinese Medical Sciences, Beijing, China
| | - Huizhen Li
- Institute of Basic Research in Clinical Medicine, China Academy of Chinese Medical Sciences, Beijing, China
| | - Yijiu Yang
- Institute of Basic Research in Clinical Medicine, China Academy of Chinese Medical Sciences, Beijing, China
| | - Xingyu Zong
- Institute of Basic Research in Clinical Medicine, China Academy of Chinese Medical Sciences, Beijing, China
| | - Yaxin Chen
- Institute of Basic Research in Clinical Medicine, China Academy of Chinese Medical Sciences, Beijing, China
| | - Yanping Wang
- Institute of Basic Research in Clinical Medicine, China Academy of Chinese Medical Sciences, Beijing, China
| | - Nannan Shi
- Institute of Basic Research in Clinical Medicine, China Academy of Chinese Medical Sciences, Beijing, China
| |
Collapse
|
18
|
Ito N, Kadomatsu S, Fujisawa M, Fukaguchi K, Ishizawa R, Kanda N, Kasugai D, Nakajima M, Goto T, Tsugawa Y. The Accuracy and Potential Racial and Ethnic Biases of GPT-4 in the Diagnosis and Triage of Health Conditions: Evaluation Study. JMIR MEDICAL EDUCATION 2023; 9:e47532. [PMID: 37917120 PMCID: PMC10654908 DOI: 10.2196/47532] [Citation(s) in RCA: 2] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 03/23/2023] [Revised: 07/07/2023] [Accepted: 09/05/2023] [Indexed: 11/03/2023]
Abstract
BACKGROUND Whether GPT-4, the conversational artificial intelligence, can accurately diagnose and triage health conditions and whether it presents racial and ethnic biases in its decisions remain unclear. OBJECTIVE We aim to assess the accuracy of GPT-4 in the diagnosis and triage of health conditions and whether its performance varies by patient race and ethnicity. METHODS We compared the performance of GPT-4 and physicians, using 45 typical clinical vignettes, each with a correct diagnosis and triage level, in February and March 2023. For each of the 45 clinical vignettes, GPT-4 and 3 board-certified physicians provided the most likely primary diagnosis and triage level (emergency, nonemergency, or self-care). Independent reviewers evaluated the diagnoses as "correct" or "incorrect." Physician diagnosis was defined as the consensus of the 3 physicians. We evaluated whether the performance of GPT-4 varies by patient race and ethnicity, by adding the information on patient race and ethnicity to the clinical vignettes. RESULTS The accuracy of diagnosis was comparable between GPT-4 and physicians (the percentage of correct diagnosis was 97.8% (44/45; 95% CI 88.2%-99.9%) for GPT-4 and 91.1% (41/45; 95% CI 78.8%-97.5%) for physicians; P=.38). GPT-4 provided appropriate reasoning for 97.8% (44/45) of the vignettes. The appropriateness of triage was comparable between GPT-4 and physicians (GPT-4: 30/45, 66.7%; 95% CI 51.0%-80.0%; physicians: 30/45, 66.7%; 95% CI 51.0%-80.0%; P=.99). The performance of GPT-4 in diagnosing health conditions did not vary among different races and ethnicities (Black, White, Asian, and Hispanic), with an accuracy of 100% (95% CI 78.2%-100%). P values, compared to the GPT-4 output without incorporating race and ethnicity information, were all .99. The accuracy of triage was not significantly different even if patients' race and ethnicity information was added. The accuracy of triage was 62.2% (95% CI 46.5%-76.2%; P=.50) for Black patients; 66.7% (95% CI 51.0%-80.0%; P=.99) for White patients; 66.7% (95% CI 51.0%-80.0%; P=.99) for Asian patients, and 62.2% (95% CI 46.5%-76.2%; P=.69) for Hispanic patients. P values were calculated by comparing the outputs with and without conditioning on race and ethnicity. CONCLUSIONS GPT-4's ability to diagnose and triage typical clinical vignettes was comparable to that of board-certified physicians. The performance of GPT-4 did not vary by patient race and ethnicity. These findings should be informative for health systems looking to introduce conversational artificial intelligence to improve the efficiency of patient diagnosis and triage.
Collapse
Affiliation(s)
- Naoki Ito
- TXP Medical Co Ltd, Tokyo, Japan
- Faculty of Medicine, The University of Tokyo, Tokyo, Japan
| | - Sakina Kadomatsu
- TXP Medical Co Ltd, Tokyo, Japan
- Faculty of Medicine, International University of Health and Welfare, Chiba, Japan
| | - Mineto Fujisawa
- TXP Medical Co Ltd, Tokyo, Japan
- Faculty of Medicine, The University of Tokyo, Tokyo, Japan
| | - Kiyomitsu Fukaguchi
- TXP Medical Co Ltd, Tokyo, Japan
- Department of Emergency Medicine, Shonan Kamakura General Hospital, Kanagawa, Japan
| | - Ryo Ishizawa
- TXP Medical Co Ltd, Tokyo, Japan
- Department of Emergency and Critical Care Medicine, Tokyo Medical Center National Hospital Organization, Tokyo, Japan
| | - Naoki Kanda
- TXP Medical Co Ltd, Tokyo, Japan
- Division of General Internal Medicine, Jichi Medical University Hospital, Tochigi, Japan
| | - Daisuke Kasugai
- TXP Medical Co Ltd, Tokyo, Japan
- Department of Emergency and Critical Care Medicine, Nagoya University Graduate School of Medicine, Aichi, Japan
| | - Mikio Nakajima
- TXP Medical Co Ltd, Tokyo, Japan
- Emergency Life-Saving Technique Academy of Tokyo Foundation for Ambulance Service Development, Tokyo, Japan
| | | | - Yusuke Tsugawa
- Division of General Internal Medicine and Health Services Research, David Geffen School of Medicine, The University of California, Los Angeles, Los Angeles, CA, United States
- Department of Health Policy and Management, UCLA Fielding School of Public Health, Los Angeles, CA, United States
| |
Collapse
|
19
|
Joo H, Mathis MR, Tam M, James C, Han P, Mangrulkar RS, Friedman CP, Vydiswaran VGV. Applying AI and Guidelines to Assist Medical Students in Recognizing Patients With Heart Failure: Protocol for a Randomized Trial. JMIR Res Protoc 2023; 12:e49842. [PMID: 37874618 PMCID: PMC10630872 DOI: 10.2196/49842] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/19/2023] [Revised: 09/16/2023] [Accepted: 09/20/2023] [Indexed: 10/25/2023] Open
Abstract
BACKGROUND The integration of artificial intelligence (AI) into clinical practice is transforming both clinical practice and medical education. AI-based systems aim to improve the efficacy of clinical tasks, enhancing diagnostic accuracy and tailoring treatment delivery. As it becomes increasingly prevalent in health care for high-quality patient care, it is critical for health care providers to use the systems responsibly to mitigate bias, ensure effective outcomes, and provide safe clinical practices. In this study, the clinical task is the identification of heart failure (HF) prior to surgery with the intention of enhancing clinical decision-making skills. HF is a common and severe disease, but detection remains challenging due to its subtle manifestation, often concurrent with other medical conditions, and the absence of a simple and effective diagnostic test. While advanced HF algorithms have been developed, the use of these AI-based systems to enhance clinical decision-making in medical education remains understudied. OBJECTIVE This research protocol is to demonstrate our study design, systematic procedures for selecting surgical cases from electronic health records, and interventions. The primary objective of this study is to measure the effectiveness of interventions aimed at improving HF recognition before surgery, the second objective is to evaluate the impact of inaccurate AI recommendations, and the third objective is to explore the relationship between the inclination to accept AI recommendations and their accuracy. METHODS Our study used a 3 × 2 factorial design (intervention type × order of prepost sets) for this randomized trial with medical students. The student participants are asked to complete a 30-minute e-learning module that includes key information about the intervention and a 5-question quiz, and a 60-minute review of 20 surgical cases to determine the presence of HF. To mitigate selection bias in the pre- and posttests, we adopted a feature-based systematic sampling procedure. From a pool of 703 expert-reviewed surgical cases, 20 were selected based on features such as case complexity, model performance, and positive and negative labels. This study comprises three interventions: (1) a direct AI-based recommendation with a predicted HF score, (2) an indirect AI-based recommendation gauged through the area under the curve metric, and (3) an HF guideline-based intervention. RESULTS As of July 2023, 62 of the enrolled medical students have fulfilled this study's participation, including the completion of a short quiz and the review of 20 surgical cases. The subject enrollment commenced in August 2022 and will end in December 2023, with the goal of recruiting 75 medical students in years 3 and 4 with clinical experience. CONCLUSIONS We demonstrated a study protocol for the randomized trial, measuring the effectiveness of interventions using AI and HF guidelines among medical students to enhance HF recognition in preoperative care with electronic health record data. INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID) DERR1-10.2196/49842.
Collapse
Affiliation(s)
- Hyeon Joo
- Department of Learning Health Sciences, University of Michigan, Ann Arbor, MI, United States
| | - Michael R Mathis
- Department of Anesthesiology, University of Michigan, Ann Arbor, MI, United States
| | - Marty Tam
- Department of Internal Medicine, Cardiology, University of Michigan, Ann Arbor, MI, United States
| | - Cornelius James
- Department of Learning Health Sciences, University of Michigan, Ann Arbor, MI, United States
- Department of Pediatrics, University of Michigan, Ann Arbor, MI, United States
- Department of Internal Medicine, University of Michigan, Ann Arbor, MI, United States
| | - Peijin Han
- Department of Computational Medicine and Bioinformatics, University of Michigan, Ann Arbor, MI, United States
| | - Rajesh S Mangrulkar
- Department of Learning Health Sciences, University of Michigan, Ann Arbor, MI, United States
- Department of Internal Medicine, University of Michigan, Ann Arbor, MI, United States
| | - Charles P Friedman
- Department of Learning Health Sciences, University of Michigan, Ann Arbor, MI, United States
- School of Information, University of Michigan, Ann Arbor, MI, United States
| | - V G Vinod Vydiswaran
- Department of Learning Health Sciences, University of Michigan, Ann Arbor, MI, United States
- School of Information, University of Michigan, Ann Arbor, MI, United States
| |
Collapse
|
20
|
Wohlgemut JM, Pisirir E, Kyrimi E, Stoner RS, Marsh W, Perkins ZB, Tai NRM. Methods used to evaluate usability of mobile clinical decision support systems for healthcare emergencies: a systematic review and qualitative synthesis. JAMIA Open 2023; 6:ooad051. [PMID: 37449057 PMCID: PMC10336299 DOI: 10.1093/jamiaopen/ooad051] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/28/2023] [Revised: 06/15/2023] [Accepted: 06/30/2023] [Indexed: 07/18/2023] Open
Abstract
Objective The aim of this study was to determine the methods and metrics used to evaluate the usability of mobile application Clinical Decision Support Systems (CDSSs) used in healthcare emergencies. Secondary aims were to describe the characteristics and usability of evaluated CDSSs. Materials and Methods A systematic literature review was conducted using Pubmed/Medline, Embase, Scopus, and IEEE Xplore databases. Quantitative data were descriptively analyzed, and qualitative data were described and synthesized using inductive thematic analysis. Results Twenty-three studies were included in the analysis. The usability metrics most frequently evaluated were efficiency and usefulness, followed by user errors, satisfaction, learnability, effectiveness, and memorability. Methods used to assess usability included questionnaires in 20 (87%) studies, user trials in 17 (74%), interviews in 6 (26%), and heuristic evaluations in 3 (13%). Most CDSS inputs consisted of manual input (18, 78%) rather than automatic input (2, 9%). Most CDSS outputs comprised a recommendation (18, 78%), with a minority advising a specific treatment (6, 26%), or a score, risk level or likelihood of diagnosis (6, 26%). Interviews and heuristic evaluations identified more usability-related barriers and facilitators to adoption than did questionnaires and user testing studies. Discussion A wide range of metrics and methods are used to evaluate the usability of mobile CDSS in medical emergencies. Input of information into CDSS was predominantly manual, impeding usability. Studies employing both qualitative and quantitative methods to evaluate usability yielded more thorough results. Conclusion When planning CDSS projects, developers should consider multiple methods to comprehensively evaluate usability.
Collapse
Affiliation(s)
- Jared M Wohlgemut
- Corresponding Author: Jared M. Wohlgemut, MSc, Centre for Trauma Sciences, Blizard Institute, Queen Mary University of London, 4 Newark St, London E1 2AT, UK;
| | - Erhan Pisirir
- Department of Electrical Engineering and Computer Science, Queen Mary University of London, London, UK
| | - Evangelia Kyrimi
- Department of Electrical Engineering and Computer Science, Queen Mary University of London, London, UK
| | - Rebecca S Stoner
- Centre for Trauma Sciences, Blizard Institute, Queen Mary University of London, London, UK
- Trauma Service, Royal London Hospital, Barts NHS Health Trust, London, UK
| | - William Marsh
- Department of Electrical Engineering and Computer Science, Queen Mary University of London, London, UK
| | - Zane B Perkins
- Centre for Trauma Sciences, Blizard Institute, Queen Mary University of London, London, UK
- Trauma Service, Royal London Hospital, Barts NHS Health Trust, London, UK
| | - Nigel R M Tai
- Centre for Trauma Sciences, Blizard Institute, Queen Mary University of London, London, UK
- Trauma Service, Royal London Hospital, Barts NHS Health Trust, London, UK
- Academic Department of Military Surgery and Trauma, Royal Centre of Defence Medicine, Birmingham, UK
| |
Collapse
|
21
|
El Naqa I, Karolak A, Luo Y, Folio L, Tarhini AA, Rollison D, Parodi K. Translation of AI into oncology clinical practice. Oncogene 2023; 42:3089-3097. [PMID: 37684407 DOI: 10.1038/s41388-023-02826-z] [Citation(s) in RCA: 6] [Impact Index Per Article: 6.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/07/2023] [Revised: 08/23/2023] [Accepted: 08/25/2023] [Indexed: 09/10/2023]
Abstract
Artificial intelligence (AI) is a transformative technology that is capturing popular imagination and can revolutionize biomedicine. AI and machine learning (ML) algorithms have the potential to break through existing barriers in oncology research and practice such as automating workflow processes, personalizing care, and reducing healthcare disparities. Emerging applications of AI/ML in the literature include screening and early detection of cancer, disease diagnosis, response prediction, prognosis, and accelerated drug discovery. Despite this excitement, only few AI/ML models have been properly validated and fewer have become regulated products for routine clinical use. In this review, we highlight the main challenges impeding AI/ML clinical translation. We present different clinical use cases from the domains of radiology, radiation oncology, immunotherapy, and drug discovery in oncology. We dissect the unique challenges and opportunities associated with each of these cases. Finally, we summarize the general requirements for successful AI/ML implementation in the clinic, highlighting specific examples and points of emphasis including the importance of multidisciplinary collaboration of stakeholders, role of domain experts in AI augmentation, transparency of AI/ML models, and the establishment of a comprehensive quality assurance program to mitigate risks of training bias and data drifts, all culminating toward safer and more beneficial AI/ML applications in oncology labs and clinics.
Collapse
Affiliation(s)
- Issam El Naqa
- Department of Machine Learning, Moffitt Cancer Center, Tampa, FL, 33612, USA.
| | - Aleksandra Karolak
- Department of Machine Learning, Moffitt Cancer Center, Tampa, FL, 33612, USA
| | - Yi Luo
- Department of Machine Learning, Moffitt Cancer Center, Tampa, FL, 33612, USA
| | - Les Folio
- Diagnostic Imaging & Interventional Radiology, Moffitt Cancer Center, Tampa, FL, 33612, USA
| | - Ahmad A Tarhini
- Cutaneous Oncology and Immunology, Moffitt Cancer Center, Tampa, FL, 33612, USA
| | - Dana Rollison
- Department of Cancer Epidemiology, Moffitt Cancer Center, Tampa, FL, 33612, USA
| | - Katia Parodi
- Department of Medical Physics, Ludwig-Maximilians-Universität München, Munich, Germany
| |
Collapse
|
22
|
Mello-Thoms C, Mello CAB. Clinical applications of artificial intelligence in radiology. Br J Radiol 2023; 96:20221031. [PMID: 37099398 PMCID: PMC10546456 DOI: 10.1259/bjr.20221031] [Citation(s) in RCA: 6] [Impact Index Per Article: 6.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/03/2022] [Revised: 03/28/2023] [Accepted: 03/28/2023] [Indexed: 04/27/2023] Open
Abstract
The rapid growth of medical imaging has placed increasing demands on radiologists. In this scenario, artificial intelligence (AI) has become an attractive partner, one that may complement case interpretation and may aid in various non-interpretive aspects of the work in the radiological clinic. In this review, we discuss interpretative and non-interpretative uses of AI in the clinical practice, as well as report on the barriers to AI's adoption in the clinic. We show that AI currently has a modest to moderate penetration in the clinical practice, with many radiologists still being unconvinced of its value and the return on its investment. Moreover, we discuss the radiologists' liabilities regarding the AI decisions, and explain how we currently do not have regulation to guide the implementation of explainable AI or of self-learning algorithms.
Collapse
Affiliation(s)
| | - Carlos A B Mello
- Centro de Informática, Universidade Federal de Pernambuco, Recife, Brazil
| |
Collapse
|
23
|
Davis MA, Lim N, Jordan J, Yee J, Gichoya JW, Lee R. Imaging Artificial Intelligence: A Framework for Radiologists to Address Health Equity, From the AJR Special Series on DEI. AJR Am J Roentgenol 2023; 221:302-308. [PMID: 37095660 DOI: 10.2214/ajr.22.28802] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 02/24/2023]
Abstract
Artificial intelligence (AI) holds promise for helping patients access new and individualized health care pathways while increasing efficiencies for health care practitioners. Radiology has been at the forefront of this technology in medicine; many radiology practices are implementing and trialing AI-focused products. AI also holds great promise for reducing health disparities and promoting health equity. Radiology is ideally positioned to help reduce disparities given its central and critical role in patient care. The purposes of this article are to discuss the potential benefits and pitfalls of deploying AI algorithms in radiology, specifically highlighting the impact of AI on health equity; to explore ways to mitigate drivers of inequity; and to enhance pathways for creating better health care for all individuals, centering on a practical framework that helps radiologists address health equity during deployment of new tools.
Collapse
Affiliation(s)
- Melissa A Davis
- Department of Diagnostic Radiology, Yale University School of Medicine, 789 Howard Ave, PO Box 20842, New Haven, CT 06520
| | | | - John Jordan
- Stanford University School of Medicine, Stanford, CA
| | - Judy Yee
- Montefiore Medical Center, Albert Einstein College of Medicine, New York, NY
| | | | - Ryan Lee
- Jefferson Health, Philadelphia, PA
| |
Collapse
|
24
|
Choi DH, Lim MH, Kim KH, Shin SD, Hong KJ, Kim S. Development of an artificial intelligence bacteremia prediction model and evaluation of its impact on physician predictions focusing on uncertainty. Sci Rep 2023; 13:13518. [PMID: 37598221 PMCID: PMC10439897 DOI: 10.1038/s41598-023-40708-2] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/14/2023] [Accepted: 08/16/2023] [Indexed: 08/21/2023] Open
Abstract
Prediction of bacteremia is a clinically important but challenging task. An artificial intelligence (AI) model has the potential to facilitate early bacteremia prediction, aiding emergency department (ED) physicians in making timely decisions and reducing unnecessary medical costs. In this study, we developed and externally validated a Bayesian neural network-based AI bacteremia prediction model (AI-BPM). We also evaluated its impact on physician predictive performance considering both AI and physician uncertainties using historical patient data. A retrospective cohort of 15,362 adult patients with blood cultures performed in the ED was used to develop the AI-BPM. The AI-BPM used structured and unstructured text data acquired during the early stage of ED visit, and provided both the point estimate and 95% confidence interval (CI) of its predictions. High AI-BPM uncertainty was defined as when the predetermined bacteremia risk threshold (5%) was included in the 95% CI of the AI-BPM prediction, and low AI-BPM uncertainty was when it was not included. In the temporal validation dataset (N = 8,188), the AI-BPM achieved area under the receiver operating characteristic curve (AUC) of 0.754 (95% CI 0.737-0.771), sensitivity of 0.917 (95% CI 0.897-0.934), and specificity of 0.340 (95% CI 0.330-0.351). In the external validation dataset (N = 7,029), the AI-BPM's AUC was 0.738 (95% CI 0.722-0.755), sensitivity was 0.927 (95% CI 0.909-0.942), and specificity was 0.319 (95% CI 0.307-0.330). The AUC of the post-AI physicians predictions (0.703, 95% CI 0.654-0.753) was significantly improved compared with that of the pre-AI predictions (0.639, 95% CI 0.585-0.693; p-value < 0.001) in the sampled dataset (N = 1,000). The AI-BPM especially improved the predictive performance of physicians in cases with high physician uncertainty (low subjective confidence) and low AI-BPM uncertainty. Our results suggest that the uncertainty of both the AI model and physicians should be considered for successful AI model implementation.
Collapse
Affiliation(s)
- Dong Hyun Choi
- Department of Biomedical Engineering, Seoul National University College of Medicine, Seoul, South Korea
| | - Min Hyuk Lim
- Transdisciplinary Department of Medicine and Advanced Technology, Seoul National University Hospital, Seoul, South Korea
- Innovative Medical Technology Research Institute, Seoul National University Hospital, Seoul, South Korea
- Institute of Medical and Biological Engineering, Seoul National University, Seoul, South Korea
| | - Ki Hong Kim
- Department of Emergency Medicine, Seoul National University Hospital, Seoul, South Korea
- Department of Emergency Medicine, Seoul National University College of Medicine, Seoul, South Korea
- Laboratory of Emergency Medical Services, Seoul National University Hospital Biomedical Research Institute, Seoul, South Korea
| | - Sang Do Shin
- Department of Emergency Medicine, Seoul National University Hospital, Seoul, South Korea
- Department of Emergency Medicine, Seoul National University College of Medicine, Seoul, South Korea
- Laboratory of Emergency Medical Services, Seoul National University Hospital Biomedical Research Institute, Seoul, South Korea
| | - Ki Jeong Hong
- Department of Emergency Medicine, Seoul National University Hospital, Seoul, South Korea.
- Department of Emergency Medicine, Seoul National University College of Medicine, Seoul, South Korea.
- Laboratory of Emergency Medical Services, Seoul National University Hospital Biomedical Research Institute, Seoul, South Korea.
| | - Sungwan Kim
- Department of Biomedical Engineering, Seoul National University College of Medicine, Seoul, South Korea.
- Institute of Bioengineering, Seoul National University, Seoul, South Korea.
| |
Collapse
|
25
|
Ng CKC. Generative Adversarial Network (Generative Artificial Intelligence) in Pediatric Radiology: A Systematic Review. CHILDREN (BASEL, SWITZERLAND) 2023; 10:1372. [PMID: 37628371 PMCID: PMC10453402 DOI: 10.3390/children10081372] [Citation(s) in RCA: 2] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Received: 07/21/2023] [Revised: 08/07/2023] [Accepted: 08/09/2023] [Indexed: 08/27/2023]
Abstract
Generative artificial intelligence, especially with regard to the generative adversarial network (GAN), is an important research area in radiology as evidenced by a number of literature reviews on the role of GAN in radiology published in the last few years. However, no review article about GAN in pediatric radiology has been published yet. The purpose of this paper is to systematically review applications of GAN in pediatric radiology, their performances, and methods for their performance evaluation. Electronic databases were used for a literature search on 6 April 2023. Thirty-seven papers met the selection criteria and were included. This review reveals that the GAN can be applied to magnetic resonance imaging, X-ray, computed tomography, ultrasound and positron emission tomography for image translation, segmentation, reconstruction, quality assessment, synthesis and data augmentation, and disease diagnosis. About 80% of the included studies compared their GAN model performances with those of other approaches and indicated that their GAN models outperformed the others by 0.1-158.6%. However, these study findings should be used with caution because of a number of methodological weaknesses. For future GAN studies, more robust methods will be essential for addressing these issues. Otherwise, this would affect the clinical adoption of the GAN-based applications in pediatric radiology and the potential advantages of GAN could not be realized widely.
Collapse
Affiliation(s)
- Curtise K. C. Ng
- Curtin Medical School, Curtin University, GPO Box U1987, Perth, WA 6845, Australia; or ; Tel.: +61-8-9266-7314; Fax: +61-8-9266-2377
- Curtin Health Innovation Research Institute (CHIRI), Faculty of Health Sciences, Curtin University, GPO Box U1987, Perth, WA 6845, Australia
| |
Collapse
|
26
|
Trottet C, Vogels T, Keitel K, Kulinkina AV, Tan R, Cobuccio L, Jaggi M, Hartley MA. Modular Clinical Decision Support Networks (MoDN)-Updatable, interpretable, and portable predictions for evolving clinical environments. PLOS DIGITAL HEALTH 2023; 2:e0000108. [PMID: 37459285 PMCID: PMC10351690 DOI: 10.1371/journal.pdig.0000108] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 08/18/2022] [Accepted: 06/12/2023] [Indexed: 07/20/2023]
Abstract
Clinical Decision Support Systems (CDSS) have the potential to improve and standardise care with probabilistic guidance. However, many CDSS deploy static, generic rule-based logic, resulting in inequitably distributed accuracy and inconsistent performance in evolving clinical environments. Data-driven models could resolve this issue by updating predictions according to the data collected. However, the size of data required necessitates collaborative learning from analogous CDSS's, which are often imperfectly interoperable (IIO) or unshareable. We propose Modular Clinical Decision Support Networks (MoDN) which allow flexible, privacy-preserving learning across IIO datasets, as well as being robust to the systematic missingness common to CDSS-derived data, while providing interpretable, continuous predictive feedback to the clinician. MoDN is a novel decision tree composed of feature-specific neural network modules that can be combined in any number or combination to make any number or combination of diagnostic predictions, updatable at each step of a consultation. The model is validated on a real-world CDSS-derived dataset, comprising 3,192 paediatric outpatients in Tanzania. MoDN significantly outperforms 'monolithic' baseline models (which take all features at once at the end of a consultation) with a mean macro F1 score across all diagnoses of 0.749 vs 0.651 for logistic regression and 0.620 for multilayer perceptron (p < 0.001). To test collaborative learning between IIO datasets, we create subsets with various percentages of feature overlap and port a MoDN model trained on one subset to another. Even with only 60% common features, fine-tuning a MoDN model on the new dataset or just making a composite model with MoDN modules matched the ideal scenario of sharing data in a perfectly interoperable setting. MoDN integrates into consultation logic by providing interpretable continuous feedback on the predictive potential of each question in a CDSS questionnaire. The modular design allows it to compartmentalise training updates to specific features and collaboratively learn between IIO datasets without sharing any data.
Collapse
Affiliation(s)
- Cécile Trottet
- Intelligent Global Health Research Group, Machine Learning and Optimization Laboratory, Swiss Federal Institute of Technology (EPFL), Lausanne, Switzerland
| | - Thijs Vogels
- Intelligent Global Health Research Group, Machine Learning and Optimization Laboratory, Swiss Federal Institute of Technology (EPFL), Lausanne, Switzerland
| | - Kristina Keitel
- Division of Pediatric Emergency Medicine, Department of Pediatrics, Inselspital, Bern University Hospital, University of Bern, Switzerland
| | - Alexandra V. Kulinkina
- Digital Health Unit, Swiss Center for International Health, Swiss Tropical and Public Health Institute, Allschwil, Switzerland
- University of Basel, Basel, Switzerland
| | - Rainer Tan
- Clinical Research Unit, Swiss Tropical and Public Health Institute, Allschwil, Switzerland
- Ifakara Health Institute, Ifakara, Tanzania
- Center for Primary Care and Public Health (Unisanté), Lausanne, Switzerland
| | - Ludovico Cobuccio
- Clinical Research Unit, Swiss Tropical and Public Health Institute, Allschwil, Switzerland
| | - Martin Jaggi
- Machine Learning and Optimization Laboratory, Swiss Federal Institute of Technology (EPFL), Lausanne, Switzerland
| | - Mary-Anne Hartley
- Intelligent Global Health Research Group, Machine Learning and Optimization Laboratory, Swiss Federal Institute of Technology (EPFL), Lausanne, Switzerland
- Laboratory of Intelligent Global Health Technologies, Biomedical Informatics and Data Science, Yale School of Medicine, New Haven, CT, USA
| |
Collapse
|
27
|
Fraser AG, Biasin E, Bijnens B, Bruining N, Caiani EG, Cobbaert K, Davies RH, Gilbert SH, Hovestadt L, Kamenjasevic E, Kwade Z, McGauran G, O'Connor G, Vasey B, Rademakers FE. Artificial intelligence in medical device software and high-risk medical devices - a review of definitions, expert recommendations and regulatory initiatives. Expert Rev Med Devices 2023; 20:467-491. [PMID: 37157833 DOI: 10.1080/17434440.2023.2184685] [Citation(s) in RCA: 4] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 05/10/2023]
Abstract
INTRODUCTION Artificial intelligence (AI) encompasses a wide range of algorithms with risks when used to support decisions about diagnosis or treatment, so professional and regulatory bodies are recommending how they should be managed. AREAS COVERED AI systems may qualify as standalone medical device software (MDSW) or be embedded within a medical device. Within the European Union (EU) AI software must undergo a conformity assessment procedure to be approved as a medical device. The draft EU Regulation on AI proposes rules that will apply across industry sectors, while for devices the Medical Device Regulation also applies. In the CORE-MD project (Coordinating Research and Evidence for Medical Devices), we have surveyed definitions and summarize initiatives made by professional consensus groups, regulators, and standardization bodies. EXPERT OPINION The level of clinical evidence required should be determined according to each application and to legal and methodological factors that contribute to risk, including accountability, transparency, and interpretability. EU guidance for MDSW based on international recommendations does not yet describe the clinical evidence needed for medical AI software. Regulators, notified bodies, manufacturers, clinicians and patients would all benefit from common standards for the clinical evaluation of high-risk AI applications and transparency of their evidence and performance.
Collapse
Affiliation(s)
- Alan G Fraser
- University Hospital of Wales, School of Medicine, Cardiff University, Heath Park, Cardiff, U.K
- KU Leuven, Leuven, Belgium
| | | | - Bart Bijnens
- Engineering Sciences, Institut d'Investigacions Biomèdiques August Pi i Sunyer (IDIBAPS), Barcelona, Spain
| | - Nico Bruining
- Department of Clinical and Experimental Information processing (Digital Cardiology), Erasmus Medical Center, Thoraxcenter, Rotterdam, the Netherlands
| | - Enrico G Caiani
- Department of Electronics, Information and Biomedical Engineering, Politecnico di Milano, Milan, Italy
| | | | - Rhodri H Davies
- Institute of Cardiovascular Science, University College London, London, U.K
| | - Stephen H Gilbert
- Technische Universität Dresden, Else Kröner Fresenius Center for Digital Health, Dresden, Germany
| | | | | | | | | | | | - Baptiste Vasey
- Nuffield Department of Surgical Sciences, University of Oxford, Oxford, UK
| | | |
Collapse
|
28
|
Tong WJ, Wu SH, Cheng MQ, Huang H, Liang JY, Li CQ, Guo HL, He DN, Liu YH, Xiao H, Hu HT, Ruan SM, Li MD, Lu MD, Wang W. Integration of Artificial Intelligence Decision Aids to Reduce Workload and Enhance Efficiency in Thyroid Nodule Management. JAMA Netw Open 2023; 6:e2313674. [PMID: 37191957 PMCID: PMC10189570 DOI: 10.1001/jamanetworkopen.2023.13674] [Citation(s) in RCA: 3] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 12/26/2022] [Accepted: 04/01/2023] [Indexed: 05/17/2023] Open
Abstract
Importance To optimize the integration of artificial intelligence (AI) decision aids and reduce workload in thyroid nodule management, it is critical to incorporate personalized AI into the decision-making processes of radiologists with varying levels of expertise. Objective To develop an optimized integration of AI decision aids for reducing radiologists' workload while maintaining diagnostic performance compared with traditional AI-assisted strategy. Design, Setting, and Participants In this diagnostic study, a retrospective set of 1754 ultrasonographic images of 1048 patients with 1754 thyroid nodules from July 1, 2018, to July 31, 2019, was used to build an optimized strategy based on how 16 junior and senior radiologists incorporated AI-assisted diagnosis results with different image features. In the prospective set of this diagnostic study, 300 ultrasonographic images of 268 patients with 300 thyroid nodules from May 1 to December 31, 2021, were used to compare the optimized strategy with the traditional all-AI strategy in terms of diagnostic performance and workload reduction. Data analyses were completed in September 2022. Main Outcomes and Measures The retrospective set of images was used to develop an optimized integration of AI decision aids for junior and senior radiologists based on the selection of AI-assisted significant or nonsignificant features. In the prospective set of images, the diagnostic performance, time-based cost, and assisted diagnosis were compared between the optimized strategy and the traditional all-AI strategy. Results The retrospective set included 1754 ultrasonographic images from 1048 patients (mean [SD] age, 42.1 [13.2] years; 749 women [71.5%]) with 1754 thyroid nodules (mean [SD] size, 16.4 [10.6] mm); 748 nodules (42.6%) were benign, and 1006 (57.4%) were malignant. The prospective set included 300 ultrasonographic images from 268 patients (mean [SD] age, 41.7 [14.1] years; 194 women [72.4%]) with 300 thyroid nodules (mean [SD] size, 17.2 [6.8] mm); 125 nodules (41.7%) were benign, and 175 (58.3%) were malignant. For junior radiologists, the ultrasonographic features that were not improved by AI assistance included cystic or almost completely cystic nodules, anechoic nodules, spongiform nodules, and nodules smaller than 5 mm, whereas for senior radiologists the features that were not improved by AI assistance were cystic or almost completely cystic nodules, anechoic nodules, spongiform nodules, very hypoechoic nodules, nodules taller than wide, lobulated or irregular nodules, and extrathyroidal extension. Compared with the traditional all-AI strategy, the optimized strategy was associated with increased mean task completion times for junior radiologists (reader 11, from 15.2 seconds [95% CI, 13.2-17.2 seconds] to 19.4 seconds [95% CI, 15.6-23.3 seconds]; reader 12, from 12.7 seconds [95% CI, 11.4-13.9 seconds] to 15.6 seconds [95% CI, 13.6-17.7 seconds]), but shorter times for senior radiologists (reader 14, from 19.4 seconds [95% CI, 18.1-20.7 seconds] to 16.8 seconds [95% CI, 15.3-18.3 seconds]; reader 16, from 12.5 seconds [95% CI, 12.1-12.9 seconds] to 10.0 seconds [95% CI, 9.5-10.5 seconds]). There was no significant difference in sensitivity (range, 91%-100%) or specificity (range, 94%-98%) between the 2 strategies for readers 11 to 16. Conclusions and Relevance This diagnostic study suggests that an optimized AI strategy in thyroid nodule management may reduce diagnostic time-based costs without sacrificing diagnostic accuracy for senior radiologists, while the traditional all-AI strategy may still be more beneficial for junior radiologists.
Collapse
Affiliation(s)
- Wen-Juan Tong
- Department of Medical Ultrasonics, Institute of Diagnostic and Interventional Ultrasound, Ultrasomics Artificial Intelligence X-Lab, The First Affiliated Hospital, Sun Yat-sen University, Guangzhou, China
| | - Shao-Hong Wu
- Department of Medical Ultrasonics, Institute of Diagnostic and Interventional Ultrasound, Ultrasomics Artificial Intelligence X-Lab, The First Affiliated Hospital, Sun Yat-sen University, Guangzhou, China
| | - Mei-Qing Cheng
- Department of Medical Ultrasonics, Institute of Diagnostic and Interventional Ultrasound, Ultrasomics Artificial Intelligence X-Lab, The First Affiliated Hospital, Sun Yat-sen University, Guangzhou, China
| | - Hui Huang
- Department of Medical Ultrasonics, Institute of Diagnostic and Interventional Ultrasound, Ultrasomics Artificial Intelligence X-Lab, The First Affiliated Hospital, Sun Yat-sen University, Guangzhou, China
| | - Jin-Yu Liang
- Department of Medical Ultrasonics, Institute of Diagnostic and Interventional Ultrasound, Ultrasomics Artificial Intelligence X-Lab, The First Affiliated Hospital, Sun Yat-sen University, Guangzhou, China
| | - Chao-Qun Li
- Department of Medical Ultrasonics, Institute of Diagnostic and Interventional Ultrasound, Ultrasomics Artificial Intelligence X-Lab, The First Affiliated Hospital, Sun Yat-sen University, Guangzhou, China
| | - Huan-Ling Guo
- Department of Medical Ultrasonics, Institute of Diagnostic and Interventional Ultrasound, Ultrasomics Artificial Intelligence X-Lab, The First Affiliated Hospital, Sun Yat-sen University, Guangzhou, China
| | - Dan-Ni He
- Department of Medical Ultrasonics, The Seventh Affiliated Hospital, Sun Yat-sen University, Shenzhen, China
| | - Yi-Hao Liu
- Clinical Trials Unit, The First Affiliated Hospital, Sun Yat-sen University, Guangzhou, China
| | - Han Xiao
- Department of Medical Ultrasonics, Institute of Diagnostic and Interventional Ultrasound, Ultrasomics Artificial Intelligence X-Lab, The First Affiliated Hospital, Sun Yat-sen University, Guangzhou, China
| | - Hang-Tong Hu
- Department of Medical Ultrasonics, Institute of Diagnostic and Interventional Ultrasound, Ultrasomics Artificial Intelligence X-Lab, The First Affiliated Hospital, Sun Yat-sen University, Guangzhou, China
| | - Si-Min Ruan
- Department of Medical Ultrasonics, Institute of Diagnostic and Interventional Ultrasound, Ultrasomics Artificial Intelligence X-Lab, The First Affiliated Hospital, Sun Yat-sen University, Guangzhou, China
| | - Ming-De Li
- Department of Medical Ultrasonics, Institute of Diagnostic and Interventional Ultrasound, Ultrasomics Artificial Intelligence X-Lab, The First Affiliated Hospital, Sun Yat-sen University, Guangzhou, China
| | - Ming-De Lu
- Department of Medical Ultrasonics, Institute of Diagnostic and Interventional Ultrasound, Ultrasomics Artificial Intelligence X-Lab, The First Affiliated Hospital, Sun Yat-sen University, Guangzhou, China
- Department of Hepatobiliary Surgery, The First Affiliated Hospital, Sun Yat-sen University, Guangzhou, China
| | - Wei Wang
- Department of Medical Ultrasonics, Institute of Diagnostic and Interventional Ultrasound, Ultrasomics Artificial Intelligence X-Lab, The First Affiliated Hospital, Sun Yat-sen University, Guangzhou, China
| |
Collapse
|
29
|
Ng CKC. Diagnostic Performance of Artificial Intelligence-Based Computer-Aided Detection and Diagnosis in Pediatric Radiology: A Systematic Review. CHILDREN (BASEL, SWITZERLAND) 2023; 10:children10030525. [PMID: 36980083 PMCID: PMC10047006 DOI: 10.3390/children10030525] [Citation(s) in RCA: 3] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Received: 01/31/2023] [Revised: 02/13/2023] [Accepted: 03/07/2023] [Indexed: 03/30/2023]
Abstract
Artificial intelligence (AI)-based computer-aided detection and diagnosis (CAD) is an important research area in radiology. However, only two narrative reviews about general uses of AI in pediatric radiology and AI-based CAD in pediatric chest imaging have been published yet. The purpose of this systematic review is to investigate the AI-based CAD applications in pediatric radiology, their diagnostic performances and methods for their performance evaluation. A literature search with the use of electronic databases was conducted on 11 January 2023. Twenty-three articles that met the selection criteria were included. This review shows that the AI-based CAD could be applied in pediatric brain, respiratory, musculoskeletal, urologic and cardiac imaging, and especially for pneumonia detection. Most of the studies (93.3%, 14/15; 77.8%, 14/18; 73.3%, 11/15; 80.0%, 8/10; 66.6%, 2/3; 84.2%, 16/19; 80.0%, 8/10) reported model performances of at least 0.83 (area under receiver operating characteristic curve), 0.84 (sensitivity), 0.80 (specificity), 0.89 (positive predictive value), 0.63 (negative predictive value), 0.87 (accuracy), and 0.82 (F1 score), respectively. However, a range of methodological weaknesses (especially a lack of model external validation) are found in the included studies. In the future, more AI-based CAD studies in pediatric radiology with robust methodology should be conducted for convincing clinical centers to adopt CAD and realizing its benefits in a wider context.
Collapse
Affiliation(s)
- Curtise K C Ng
- Curtin Medical School, Curtin University, GPO Box U1987, Perth, WA 6845, Australia
- Curtin Health Innovation Research Institute (CHIRI), Faculty of Health Sciences, Curtin University, GPO Box U1987, Perth, WA 6845, Australia
| |
Collapse
|
30
|
Lex JR, Di Michele J, Koucheki R, Pincus D, Whyne C, Ravi B. Artificial Intelligence for Hip Fracture Detection and Outcome Prediction: A Systematic Review and Meta-analysis. JAMA Netw Open 2023; 6:e233391. [PMID: 36930153 PMCID: PMC10024206 DOI: 10.1001/jamanetworkopen.2023.3391] [Citation(s) in RCA: 4] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 03/18/2023] Open
Abstract
IMPORTANCE Artificial intelligence (AI) enables powerful models for establishment of clinical diagnostic and prognostic tools for hip fractures; however the performance and potential impact of these newly developed algorithms are currently unknown. OBJECTIVE To evaluate the performance of AI algorithms designed to diagnose hip fractures on radiographs and predict postoperative clinical outcomes following hip fracture surgery relative to current practices. DATA SOURCES A systematic review of the literature was performed using the MEDLINE, Embase, and Cochrane Library databases for all articles published from database inception to January 23, 2023. A manual reference search of included articles was also undertaken to identify any additional relevant articles. STUDY SELECTION Studies developing machine learning (ML) models for the diagnosis of hip fractures from hip or pelvic radiographs or to predict any postoperative patient outcome following hip fracture surgery were included. DATA EXTRACTION AND SYNTHESIS This study followed the Preferred Reporting Items for Systematic Reviews and Meta-analyses and was registered with PROSPERO. Eligible full-text articles were evaluated and relevant data extracted independently using a template data extraction form. For studies that predicted postoperative outcomes, the performance of traditional predictive statistical models, either multivariable logistic or linear regression, was recorded and compared with the performance of the best ML model on the same out-of-sample data set. MAIN OUTCOMES AND MEASURES Diagnostic accuracy of AI models was compared with the diagnostic accuracy of expert clinicians using odds ratios (ORs) with 95% CIs. Areas under the curve for postoperative outcome prediction between traditional statistical models (multivariable linear or logistic regression) and ML models were compared. RESULTS Of 39 studies that met all criteria and were included in this analysis, 18 (46.2%) used AI models to diagnose hip fractures on plain radiographs and 21 (53.8%) used AI models to predict patient outcomes following hip fracture surgery. A total of 39 598 plain radiographs and 714 939 hip fractures were used for training, validating, and testing ML models specific to diagnosis and postoperative outcome prediction, respectively. Mortality and length of hospital stay were the most predicted outcomes. On pooled data analysis, compared with clinicians, the OR for diagnostic error of ML models was 0.79 (95% CI, 0.48-1.31; P = .36; I2 = 60%) for hip fracture radiographs. For the ML models, the mean (SD) sensitivity was 89.3% (8.5%), specificity was 87.5% (9.9%), and F1 score was 0.90 (0.06). The mean area under the curve for mortality prediction was 0.84 with ML models compared with 0.79 for alternative controls (P = .09). CONCLUSIONS AND RELEVANCE The findings of this systematic review and meta-analysis suggest that the potential applications of AI to aid with diagnosis from hip radiographs are promising. The performance of AI in diagnosing hip fractures was comparable with that of expert radiologists and surgeons. However, current implementations of AI for outcome prediction do not seem to provide substantial benefit over traditional multivariable predictive statistics.
Collapse
Affiliation(s)
- Johnathan R. Lex
- Division of Orthopaedic Surgery, Department of Surgery, University of Toronto, Toronto, Ontario, Canada
- Institute of Biomedical Engineering, University of Toronto, Toronto, Ontario, Canada
- Orthopaedics Biomechanics Laboratory, Sunnybrook Research Institute, Toronto, Ontario, Canada
| | - Joseph Di Michele
- Division of Orthopaedic Surgery, Department of Surgery, University of Toronto, Toronto, Ontario, Canada
| | - Robert Koucheki
- Institute of Biomedical Engineering, University of Toronto, Toronto, Ontario, Canada
- Temerty Faculty of Medicine, University of Toronto, Toronto, Ontario, Canada
| | - Daniel Pincus
- Division of Orthopaedic Surgery, Department of Surgery, University of Toronto, Toronto, Ontario, Canada
- Division of Orthopaedic Surgery, Sunnybrook Health Sciences Centre, Toronto, Ontario, Canada
| | - Cari Whyne
- Orthopaedics Biomechanics Laboratory, Sunnybrook Research Institute, Toronto, Ontario, Canada
| | - Bheeshma Ravi
- Division of Orthopaedic Surgery, Department of Surgery, University of Toronto, Toronto, Ontario, Canada
- Division of Orthopaedic Surgery, Sunnybrook Health Sciences Centre, Toronto, Ontario, Canada
| |
Collapse
|
31
|
Ursprung S, Woitek R. The Steep Road to Artificial Intelligence–mediated Radiology. Radiol Artif Intell 2023; 5:e230017. [PMID: 37035434 PMCID: PMC10077073 DOI: 10.1148/ryai.230017] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/18/2023] [Revised: 02/01/2023] [Accepted: 02/06/2023] [Indexed: 03/06/2023]
Affiliation(s)
- Stephan Ursprung
- From the Department of Radiology, University Hospital Tuebingen, Tuebingen, Germany (S.U.); Department of Radiology, University of Cambridge School of Clinical Medicine, Box 218, Cambridge Biomedical Campus, Hills Road, Cambridge CB2 0QQ, England (S.U., R.W.); and Research Center Medical Image Analysis and AI (MIAAI), Danube Private University, Krems, Austria (R.W.)
| | - Ramona Woitek
- From the Department of Radiology, University Hospital Tuebingen, Tuebingen, Germany (S.U.); Department of Radiology, University of Cambridge School of Clinical Medicine, Box 218, Cambridge Biomedical Campus, Hills Road, Cambridge CB2 0QQ, England (S.U., R.W.); and Research Center Medical Image Analysis and AI (MIAAI), Danube Private University, Krems, Austria (R.W.)
| |
Collapse
|
32
|
Villa-Camacho JC, Baikpour M, Chou SHS. Artificial Intelligence for Breast US. JOURNAL OF BREAST IMAGING 2023; 5:11-20. [PMID: 38416959 DOI: 10.1093/jbi/wbac077] [Citation(s) in RCA: 3] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/30/2022] [Indexed: 03/01/2024]
Abstract
US is a widely available, commonly used, and indispensable imaging modality for breast evaluation. It is often the primary imaging modality for the detection and diagnosis of breast cancer in low-resource settings. In addition, it is frequently employed as a supplemental screening tool via either whole breast handheld US or automated breast US among women with dense breasts. In recent years, a variety of artificial intelligence systems have been developed to assist radiologists with the detection and diagnosis of breast lesions on US. This article reviews the background and evidence supporting the use of artificial intelligence tools for breast US, describes implementation strategies and impact on clinical workflow, and discusses potential emerging roles and future directions.
Collapse
Affiliation(s)
| | - Masoud Baikpour
- Massachusetts General Hospital, Department of Radiology, Boston, MA, USA
| | - Shinn-Huey S Chou
- Massachusetts General Hospital, Department of Radiology, Boston, MA, USA
| |
Collapse
|
33
|
Vasey B, Novak A, Ather S, Ibrahim M, McCulloch P. DECIDE-AI: a new reporting guideline and its relevance to artificial intelligence studies in radiology. Clin Radiol 2023; 78:130-136. [PMID: 36639172 DOI: 10.1016/j.crad.2022.09.131] [Citation(s) in RCA: 5] [Impact Index Per Article: 5.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/06/2022] [Revised: 09/18/2022] [Accepted: 09/29/2022] [Indexed: 01/12/2023]
Abstract
DECIDE-AI is a new, stage-specific reporting guideline for the early and live clinical evaluation of decision-support systems based on artificial intelligence (AI). It answers a need for more attention to the human factors influencing clinical AI performance and more transparent reporting of clinical studies investigating AI systems. Given the rapid expansion of AI systems and the concentration of related studies in radiology, these new standards are likely to find a place in radiological literature in the near future. This review highlights some of the specificities of AI as complex intervention, why a new reporting guideline was needed for early stage, live evaluation of this technology, and how DECIDE-AI and other AI reporting guidelines can be useful to radiologists and researchers.
Collapse
Affiliation(s)
- B Vasey
- Nuffield Department of Surgical Sciences, University of Oxford, Oxford, UK; Department of Surgery, Geneva University Hospital, Geneva, Switzerland.
| | - A Novak
- Emergency Medicine Research Oxford (EMROx), Oxford University Hospitals NHS Foundation Trust, Oxford, UK
| | - S Ather
- National Consortium for Intelligent Medical Imaging, University of Oxford, Oxford, UK; Oxford University Hospitals NHS Foundation Trust, Oxford, UK
| | - M Ibrahim
- Nuffield Department of Surgical Sciences, University of Oxford, Oxford, UK; Department of Surgery, Maimonides Medical Center, Brooklyn, NY, USA
| | - P McCulloch
- Nuffield Department of Surgical Sciences, University of Oxford, Oxford, UK
| |
Collapse
|
34
|
Lemoine É, Neves Briard J, Rioux B, Podbielski R, Nauche B, Toffa D, Keezer M, Lesage F, Nguyen DK, Bou Assi E. Computer-assisted analysis of routine electroencephalogram to identify hidden biomarkers of epilepsy: protocol for a systematic review. BMJ Open 2023; 13:e066932. [PMID: 36693684 PMCID: PMC9884857 DOI: 10.1136/bmjopen-2022-066932] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 01/26/2023] Open
Abstract
INTRODUCTION The diagnosis of epilepsy frequently relies on the visual interpretation of the electroencephalogram (EEG) by a neurologist. The hallmark of epilepsy on EEG is the interictal epileptiform discharge (IED). This marker lacks sensitivity: it is only captured in a small percentage of 30 min routine EEGs in patients with epilepsy. In the past three decades, there has been growing interest in the use of computational methods to analyse the EEG without relying on the detection of IEDs, but none have made it to the clinical practice. We aim to review the diagnostic accuracy of quantitative methods applied to ambulatory EEG analysis to guide the diagnosis and management of epilepsy. METHODS AND ANALYSIS The protocol complies with the recommendations for systematic reviews of diagnostic test accuracy by Cochrane. We will search MEDLINE, EMBASE, EBM reviews, IEEE Explore along with grey literature for articles, conference papers and conference abstracts published after 1961. We will include observational studies that present a computational method to analyse the EEG for the diagnosis of epilepsy in adults or children without relying on the identification of IEDs or seizures. The reference standard is the diagnosis of epilepsy by a physician. We will report the estimated pooled sensitivity and specificity, and receiver operating characteristic area under the curve (ROC AUC) for each marker. If possible, we will perform a meta-analysis of the sensitivity and specificity and ROC AUC for each individual marker. We will assess the risk of bias using an adapted QUADAS-2 tool. We will also describe the algorithms used for signal processing, feature extraction and predictive modelling, and comment on the reproducibility of the different studies. ETHICS AND DISSEMINATION Ethical approval was not required. Findings will be disseminated through peer-reviewed publication and presented at conferences related to this field. PROSPERO REGISTRATION NUMBER CRD42022292261.
Collapse
Affiliation(s)
- Émile Lemoine
- Department of Neurosciences, University of Montreal, Montreal, Québec, Canada
- Institute of Biomedical Engineering, Ecole Polytechnique de Montreal, Montreal, Québec, Canada
| | - Joel Neves Briard
- Department of Neurosciences, University of Montreal, Montreal, Québec, Canada
- University of Montreal Hospital Centre Research Centre, Montreal, Québec, Canada
| | - Bastien Rioux
- Department of Neurosciences, University of Montreal, Montreal, Québec, Canada
- University of Montreal Hospital Centre Research Centre, Montreal, Québec, Canada
| | - Renata Podbielski
- University of Montreal Hospital Centre Research Centre, Montreal, Québec, Canada
| | - Bénédicte Nauche
- University of Montreal Hospital Centre Research Centre, Montreal, Québec, Canada
| | - Denahin Toffa
- Department of Neurosciences, University of Montreal, Montreal, Québec, Canada
- University of Montreal Hospital Centre Research Centre, Montreal, Québec, Canada
| | - Mark Keezer
- Department of Neurosciences, University of Montreal, Montreal, Québec, Canada
- Stichting Epilepsie Instellingen Nederland (SEIN), Heemstede, The Netherlands
| | - Frédéric Lesage
- Institute of Biomedical Engineering, Ecole Polytechnique de Montreal, Montreal, Québec, Canada
| | - Dang K Nguyen
- Department of Neurosciences, University of Montreal, Montreal, Québec, Canada
- University of Montreal Hospital Centre Research Centre, Montreal, Québec, Canada
| | - Elie Bou Assi
- Department of Neurosciences, University of Montreal, Montreal, Québec, Canada
- University of Montreal Hospital Centre Research Centre, Montreal, Québec, Canada
| |
Collapse
|
35
|
Castagna F, Garton A, McBurney P, Parsons S, Sassoon I, Sklar EI. EQRbot: A chatbot delivering EQR argument-based explanations. Front Artif Intell 2023; 6:1045614. [PMID: 37035536 PMCID: PMC10076765 DOI: 10.3389/frai.2023.1045614] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/15/2022] [Accepted: 02/20/2023] [Indexed: 04/11/2023] Open
Abstract
Recent years have witnessed the rise of several new argumentation-based support systems, especially in the healthcare industry. In the medical sector, it is imperative that the exchange of information occurs in a clear and accurate way, and this has to be reflected in any employed virtual systems. Argument Schemes and their critical questions represent well-suited formal tools for modeling such information and exchanges since they provide detailed templates for explanations to be delivered. This paper details the EQR argument scheme and deploys it to generate explanations for patients' treatment advice using a chatbot (EQRbot). The EQR scheme (devised as a pattern of Explanation-Question-Response interactions between agents) comprises multiple premises that can be interrogated to disclose additional data. The resulting explanations, obtained as instances of the employed argumentation reasoning engine and the EQR template, will then feed the conversational agent that will exhaustively convey the requested information and answers to follow-on users' queries as personalized Telegram messages. Comparisons with a previous baseline and existing argumentation-based chatbots illustrate the improvements yielded by EQRbot against similar conversational agents.
Collapse
Affiliation(s)
- Federico Castagna
- School of Computer Science, University of Lincoln, Lincoln, United Kingdom
- *Correspondence: Federico Castagna
| | - Alexandra Garton
- School of Computer Science, University of Lincoln, Lincoln, United Kingdom
| | - Peter McBurney
- Department of Informatics, King's College London, London, United Kingdom
| | - Simon Parsons
- School of Computer Science, University of Lincoln, Lincoln, United Kingdom
| | - Isabel Sassoon
- Department of Computer Science, Brunel University London, London, United Kingdom
| | - Elizabeth I. Sklar
- Lincoln Institute for Agri-Food Technology, University of Lincoln, Lincoln, United Kingdom
| |
Collapse
|
36
|
Monteith S, Glenn T, Geddes J, Whybrow PC, Achtyes E, Bauer M. Expectations for Artificial Intelligence (AI) in Psychiatry. Curr Psychiatry Rep 2022; 24:709-721. [PMID: 36214931 PMCID: PMC9549456 DOI: 10.1007/s11920-022-01378-5] [Citation(s) in RCA: 5] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Accepted: 09/15/2022] [Indexed: 01/29/2023]
Abstract
PURPOSE OF REVIEW Artificial intelligence (AI) is often presented as a transformative technology for clinical medicine even though the current technology maturity of AI is low. The purpose of this narrative review is to describe the complex reasons for the low technology maturity and set realistic expectations for the safe, routine use of AI in clinical medicine. RECENT FINDINGS For AI to be productive in clinical medicine, many diverse factors that contribute to the low maturity level need to be addressed. These include technical problems such as data quality, dataset shift, black-box opacity, validation and regulatory challenges, and human factors such as a lack of education in AI, workflow changes, automation bias, and deskilling. There will also be new and unanticipated safety risks with the introduction of AI. The solutions to these issues are complex and will take time to discover, develop, validate, and implement. However, addressing the many problems in a methodical manner will expedite the safe and beneficial use of AI to augment medical decision making in psychiatry.
Collapse
Affiliation(s)
- Scott Monteith
- Michigan State University College of Human Medicine, Traverse City Campus, Traverse City, MI, 49684, USA.
| | - Tasha Glenn
- ChronoRecord Association, Fullerton, CA, USA
| | - John Geddes
- Department of Psychiatry, University of Oxford, Warneford Hospital, Oxford, UK
| | - Peter C Whybrow
- Department of Psychiatry and Biobehavioral Sciences, Semel Institute for Neuroscience and Human Behavior, University of California Los Angeles (UCLA), Los Angeles, CA, USA
| | - Eric Achtyes
- Michigan State University College of Human Medicine, Grand Rapids, MI, 49684, USA
- Network180, Grand Rapids, MI, USA
| | - Michael Bauer
- Department of Psychiatry and Psychotherapy, University Hospital Carl Gustav Carus Medical Faculty, Technische Universität Dresden, Dresden, Germany
| |
Collapse
|
37
|
Böhnke J, Varghese J, Karch A, Rübsamen N. Systematic review identifies deficiencies in reporting of diagnostic test accuracy among clinical decision support systems. J Clin Epidemiol 2022; 151:171-184. [PMID: 35987404 DOI: 10.1016/j.jclinepi.2022.08.003] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/20/2022] [Revised: 07/21/2022] [Accepted: 08/10/2022] [Indexed: 12/25/2022]
Abstract
OBJECTIVES This systematic review assesses the reporting quality and risk of bias in studies evaluating the diagnostic test accuracy (DTA) of clinical decision support systems (CDSS). STUDY DESIGN AND SETTING The Cochrane Library, PubMed/MEDLINE, Scopus, and Web of Science were searched for studies, published between January 1, 2016 and May 31, 2021, evaluating the DTA of CDSS for human patients. Articles using a patient's self-diagnosis, assessing disease severity, focusing on treatment/follow-up, or comparing pre-post CDSS implementation periods were excluded. All eligible studies were assessed for reporting quality using STARD 2015 and for risk of bias using QUADAS-2. Item ratings were presented using heat maps. This study was reported as per PRISMA-DTA. RESULTS In total, 158 of 2,820 screened articles were included in the analysis. The studies were heterogeneous in terms of study characteristics, reporting quality, risk of biases, and applicability concerns with few highly rated studies. Mostly the overall quality was deficient for items addressing the domains 'methodology,' 'results,' and 'other information'. CONCLUSION Our analysis revealed shortcomings in critical domains of reporting quality and risk of bias, indicating the need for additional guidance and training in an interdisciplinary scientific field with mixed biostatistical expertise.
Collapse
Affiliation(s)
- Julia Böhnke
- University of Münster, Institute of Epidemiology and Social Medicine, Münster, Germany.
| | - Julian Varghese
- University of Münster, Institute of Medical Informatics, Münster, Germany
| | - André Karch
- University of Münster, Institute of Epidemiology and Social Medicine, Münster, Germany
| | - Nicole Rübsamen
- University of Münster, Institute of Epidemiology and Social Medicine, Münster, Germany
| |
Collapse
|
38
|
Besculides M, Mazumdar M, Phlegar S, Freeman R, Wilson S, Joshi H, Kia A, Gorbenko K. Implementing a Machine Learning Screening Tool for Malnutrition: Insights from Qualitative Research Applicable to Other ML-Based CDSS (Preprint). JMIR Form Res 2022. [PMID: 37440303 PMCID: PMC10375393 DOI: 10.2196/42262] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 02/01/2023] Open
Abstract
BACKGROUND Machine learning (ML)-based clinical decision support systems (CDSS) are popular in clinical practice settings but are often criticized for being limited in usability, interpretability, and effectiveness. Evaluating the implementation of ML-based CDSS is critical to ensure CDSS is acceptable and useful to clinicians and helps them deliver high-quality health care. Malnutrition is a common and underdiagnosed condition among hospital patients, which can have serious adverse impacts. Early identification and treatment of malnutrition are important. OBJECTIVE This study aims to evaluate the implementation of an ML tool, Malnutrition Universal Screening Tool (MUST)-Plus, that predicts hospital patients at high risk for malnutrition and identify best implementation practices applicable to this and other ML-based CDSS. METHODS We conducted a qualitative postimplementation evaluation using in-depth interviews with registered dietitians (RDs) who use MUST-Plus output in their everyday work. After coding the data, we mapped emergent themes onto select domains of the nonadoption, abandonment, scale-up, spread, and sustainability (NASSS) framework. RESULTS We interviewed 17 of the 24 RDs approached (71%), representing 37% of those who use MUST-Plus output. Several themes emerged: (1) enhancements to the tool were made to improve accuracy and usability; (2) MUST-Plus helped identify patients that would not otherwise be seen; perceived usefulness was highest in the original site; (3) perceived accuracy varied by respondent and site; (4) RDs valued autonomy in prioritizing patients; (5) depth of tool understanding varied by hospital and level; (6) MUST-Plus was integrated into workflows and electronic health records; and (7) RDs expressed a desire to eventually have 1 automated screener. CONCLUSIONS Our findings suggest that continuous involvement of stakeholders at new sites given staff turnover is vital to ensure buy-in. Qualitative research can help identify the potential bias of ML tools and should be widely used to ensure health equity. Ongoing collaboration among CDSS developers, data scientists, and clinical providers may help refine CDSS for optimal use and improve the acceptability of CDSS in the clinical context.
Collapse
|
39
|
Chalmer R, Ayers E, Weiss EF, Malik R, Ehrlich A, Wang C, Zwerling J, Ansari A, Possin KL, Verghese J. The 5-Cog paradigm to improve detection of cognitive impairment and dementia: clinical trial protocol. Neurodegener Dis Manag 2022; 12:171-184. [PMID: 35603666 PMCID: PMC9245592 DOI: 10.2217/nmt-2021-0043] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/27/2022] [Accepted: 05/05/2022] [Indexed: 11/21/2022] Open
Abstract
Cognitive impairment related to dementia is under-diagnosed in primary care despite availability of numerous cognitive assessment tools; under-diagnosis is more prevalent for members of racial and ethnic minority groups. Clinical decision-support systems may improve rates of primary care providers responding to positive cognitive assessments with appropriate follow-up. The 5-Cog study is a randomized controlled trial in 1200 predominantly Black and Hispanic older adults from an urban underserved community who are presenting to primary care with cognitive concerns. The study will validate a novel 5-minute cognitive assessment coupled with an electronic medical record-embedded decision tree to overcome the barriers of current cognitive assessment paradigms in primary care and facilitate improved dementia care.
Collapse
Affiliation(s)
- Rachel Chalmer
- Department of Medicine, Division of Geriatrics, Montefiore Medical Center & Albert Einstein College of Medicine, Bronx, NY 10467, USA
| | - Emmeline Ayers
- Department of Neurology, Montefiore Medical Center & Albert Einstein College of Medicine, Bronx, NY 10467, USA
| | - Erica F Weiss
- Department of Neurology, Montefiore Medical Center & Albert Einstein College of Medicine, Bronx, NY 10467, USA
| | - Rubina Malik
- Department of Medicine, Division of Geriatrics, Montefiore Medical Center & Albert Einstein College of Medicine, Bronx, NY 10467, USA
| | - Amy Ehrlich
- Department of Medicine, Division of Geriatrics, Montefiore Medical Center & Albert Einstein College of Medicine, Bronx, NY 10467, USA
| | - Cuiling Wang
- Department of Epidemiology & Population Health, Montefiore Medical Center & Albert Einstein College of Medicine, Bronx, NY 10461, USA
| | - Jessica Zwerling
- Department of Neurology, Montefiore Medical Center & Albert Einstein College of Medicine, Bronx, NY 10467, USA
| | - Asif Ansari
- Department of Medicine, Division of Geriatrics, Montefiore Medical Center & Albert Einstein College of Medicine, Bronx, NY 10467, USA
| | - Katherine L Possin
- Department of Neurology, Memory & Aging Center, University of California San Francisco, San Francisco, CA 94158, USA
| | - Joe Verghese
- Department of Medicine, Division of Geriatrics, Montefiore Medical Center & Albert Einstein College of Medicine, Bronx, NY 10467, USA
- Department of Neurology, Montefiore Medical Center & Albert Einstein College of Medicine, Bronx, NY 10467, USA
| |
Collapse
|
40
|
Ray J, Wijesekera L, Cirstea S. Machine learning and clinical neurophysiology. J Neurol 2022; 269:6678-6684. [PMID: 35907045 DOI: 10.1007/s00415-022-11283-9] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/13/2022] [Revised: 07/05/2022] [Accepted: 07/09/2022] [Indexed: 11/29/2022]
Abstract
Clinical neurophysiology constructs a wealth of dynamic information pertaining to the integrity and function of both central and peripheral nervous systems. As with many technological fields, there has been an explosion of data in neurophysiology over recent years, and this requires considerable analysis by experts. Computational algorithms and especially advances in machine learning (ML) have the ability to assist with this task and potentially reveal hidden insights. In this update article, we will provide a brief overview where such technology is being applied in clinical neurophysiology and possible future directions.
Collapse
Affiliation(s)
- Julian Ray
- Department of Clinical Neurophysiology, Addenbrooke's Hospital, Cambridge University Hospitals Neurosciences, Cambridge, UK.
| | - Lokesh Wijesekera
- Department of Clinical Neurophysiology, Addenbrooke's Hospital, Cambridge University Hospitals Neurosciences, Cambridge, UK
| | - Silvia Cirstea
- Department of Clinical Neurophysiology, Addenbrooke's Hospital, Cambridge University Hospitals Neurosciences, Cambridge, UK
| |
Collapse
|
41
|
Al-Zaiti SS, Alghwiri AA, Hu X, Clermont G, Peace A, Macfarlane P, Bond R. A clinician's guide to understanding and critically appraising machine learning studies: a checklist for Ruling Out Bias Using Standard Tools in Machine Learning (ROBUST-ML). EUROPEAN HEART JOURNAL. DIGITAL HEALTH 2022; 3:125-140. [PMID: 36713011 PMCID: PMC9708024 DOI: 10.1093/ehjdh/ztac016] [Citation(s) in RCA: 12] [Impact Index Per Article: 6.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Received: 12/20/2021] [Revised: 02/11/2022] [Indexed: 05/06/2023]
Abstract
Developing functional machine learning (ML)-based models to address unmet clinical needs requires unique considerations for optimal clinical utility. Recent debates about the rigours, transparency, explainability, and reproducibility of ML models, terms which are defined in this article, have raised concerns about their clinical utility and suitability for integration in current evidence-based practice paradigms. This featured article focuses on increasing the literacy of ML among clinicians by providing them with the knowledge and tools needed to understand and critically appraise clinical studies focused on ML. A checklist is provided for evaluating the rigour and reproducibility of the four ML building blocks: data curation, feature engineering, model development, and clinical deployment. Checklists like this are important for quality assurance and to ensure that ML studies are rigourously and confidently reviewed by clinicians and are guided by domain knowledge of the setting in which the findings will be applied. Bridging the gap between clinicians, healthcare scientists, and ML engineers can address many shortcomings and pitfalls of ML-based solutions and their potential deployment at the bedside.
Collapse
Affiliation(s)
| | - Alaa A Alghwiri
- Data Science Core, The Provost Office, University of Pittsburgh, Pittsburgh PA, USA
| | - Xiao Hu
- Center for Data Science, Emory University, Atlanta, GA, USA
| | - Gilles Clermont
- Departments of Critical Care Medicine, Mathematics, Clinical and Translational Science, and Industrial Engineering, University of Pittsburgh, Pittsburgh, PA, USA
| | - Aaron Peace
- The Clinical Translational Research and Innovation Centre, Northern Ireland, UK
| | - Peter Macfarlane
- Institute of Health and Wellbeing, Electrocardiology Section, University of Glasgow, Glasgow, UK
| | - Raymond Bond
- School of Computing, Ulster University, Ulster, UK
| |
Collapse
|
42
|
Abstract
Clinical informatics can support quality improvement and patient safety in the pediatric intensive care unit (PICU) in several ways including data extraction, analysis, and decision support enabled by electronic health records (EHRs), and databases and registries. Clinical decision support (CDS), embedded in EHRs, now an integral part of the workflow in the PICU, includes several tools and is increasingly leveraging artificial intelligence (AI). Understanding the opportunities and challenges can improve the engagement of clinicians with the design, validation, and implementation of CDS, improve satisfaction with CDS, and improve patient safety, care quality, and value.
Collapse
|
43
|
Scott IA. Using information technology to reduce diagnostic error: still a bridge too far? Intern Med J 2022; 52:908-911. [PMID: 35718736 DOI: 10.1111/imj.15804] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/21/2022] [Accepted: 04/28/2022] [Indexed: 11/28/2022]
Affiliation(s)
- Ian A Scott
- Internal Medicine and Clinical Epidemiology, Princess Alexandra Hospital, Brisbane, Queensland, Australia.,School of Clinical Medicine, University of Queensland, Brisbane, Queensland, Australia
| |
Collapse
|
44
|
Vasey B, Nagendran M, Campbell B, Clifton DA, Collins GS, Denaxas S, Denniston AK, Faes L, Geerts B, Ibrahim M, Liu X, Mateen BA, Mathur P, McCradden MD, Morgan L, Ordish J, Rogers C, Saria S, Ting DSW, Watkinson P, Weber W, Wheatstone P, McCulloch P. Reporting guideline for the early stage clinical evaluation of decision support systems driven by artificial intelligence: DECIDE-AI. BMJ 2022; 377:e070904. [PMID: 35584845 PMCID: PMC9116198 DOI: 10.1136/bmj-2022-070904] [Citation(s) in RCA: 60] [Impact Index Per Article: 30.0] [Reference Citation Analysis] [MESH Headings] [Grants] [Track Full Text] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Accepted: 04/26/2022] [Indexed: 01/04/2023]
Affiliation(s)
- Baptiste Vasey
- Nuffield Department of Surgical Sciences, University of Oxford, Oxford, UK
- Institute of Biomedical Engineering, Department of Engineering Science, University of Oxford, Oxford, UK
- Critical Care Research Group, Nuffield Department of Clinical Neurosciences, University of Oxford, Oxford, UK
| | - Myura Nagendran
- UKRI Centre for Doctoral Training in AI for Healthcare, Imperial College London, London, UK
| | - Bruce Campbell
- University of Exeter Medical School, Exeter, UK
- Royal Devon and Exeter Hospital, Exeter, UK
| | - David A Clifton
- Institute of Biomedical Engineering, Department of Engineering Science, University of Oxford, Oxford, UK
| | - Gary S Collins
- Centre for Statistics in Medicine, Nuffield Department of Orthopaedics, Rheumatology and Musculoskeletal Sciences, University of Oxford, Oxford, UK
| | - Spiros Denaxas
- Institute of Health Informatics, University College London, London, UK
- British Heart Foundation Data Science Centre, London, UK
- Health Data Research UK, London, UK
- UCL Hospitals Biomedical Research Centre, London, UK
| | - Alastair K Denniston
- University Hospitals Birmingham NHS Foundation Trust, Birmingham, UK
- Academic Unit of Ophthalmology, Institute of Inflammation and Ageing, College of Medical and Dental Sciences, University of Birmingham, Birmingham, UK
- Moorfields Eye Hospital NHS Foundation Trust, London, UK
| | - Livia Faes
- Moorfields Eye Hospital NHS Foundation Trust, London, UK
| | | | - Mudathir Ibrahim
- Nuffield Department of Surgical Sciences, University of Oxford, Oxford, UK
- Department of Surgery, Maimonides Medical Center, New York, NY, USA
| | - Xiaoxuan Liu
- University Hospitals Birmingham NHS Foundation Trust, Birmingham, UK
- Academic Unit of Ophthalmology, Institute of Inflammation and Ageing, College of Medical and Dental Sciences, University of Birmingham, Birmingham, UK
| | - Bilal A Mateen
- Institute of Health Informatics, University College London, London, UK
- Wellcome Trust, London, UK
- Alan Turing Institute, London, UK
| | - Piyush Mathur
- Department of General Anesthesiology, Anesthesiology Institute, Cleveland Clinic, Cleveland, OH, USA
| | - Melissa D McCradden
- Hospital for Sick Children, Toronto, ON, Canada
- Dalla Lana School of Public Health, University of Toronto, Toronto, ON, Canada
| | | | - Johan Ordish
- The Medicines and Healthcare products Regulatory Agency, London, UK
| | | | - Suchi Saria
- Departments of Computer Science, Statistics, and Health Policy, and Division of Informatics, Johns Hopkins University, Baltimore, MD, USA
- Bayesian Health, New York, NY, USA
| | - Daniel S W Ting
- Singapore National Eye Center, Singapore Eye Research Institute, Singapore
- Duke-NUS Medical School, National University of Singapore, Singapore
| | - Peter Watkinson
- Critical Care Research Group, Nuffield Department of Clinical Neurosciences, University of Oxford, Oxford, UK
- NIHR Biomedical Research Centre Oxford, Oxford University Hospitals NHS Trust, Oxford, UK
| | | | | | - Peter McCulloch
- Nuffield Department of Surgical Sciences, University of Oxford, Oxford, UK
| |
Collapse
|
45
|
Sibbald M, Abdulla B, Keuhl A, Norman G, Monteiro S, Sherbino J. Electronic diagnostic support in emergency physician triage: a qualitative study (Preprint). JMIR Hum Factors 2022; 9:e39234. [PMID: 36178728 PMCID: PMC9568817 DOI: 10.2196/39234] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/04/2022] [Revised: 08/05/2022] [Accepted: 08/29/2022] [Indexed: 12/05/2022] Open
Abstract
Background Not thinking of a diagnosis is a leading cause of diagnostic error in the emergency department, resulting in delayed treatment, morbidity, and excess mortality. Electronic differential diagnostic support (EDS) results in small but significant reductions in diagnostic error. However, the uptake of EDS by clinicians is limited. Objective We sought to understand physician perceptions and barriers to the uptake of EDS within the emergency department triage process. Methods We conducted a qualitative study using a research associate to rapidly prototype an embedded EDS into the emergency department triage process. Physicians involved in the triage assessment of a busy emergency department were provided the output of an EDS based on the triage complaint by an embedded researcher to simulate an automated system that would draw from the electronic medical record. Physicians were interviewed immediately after their experience. Verbatim transcripts were analyzed by a team using open and axial coding, informed by direct content analysis. Results In all, 4 themes emerged from 14 interviews: (1) the quality of the EDS was inferred from the scope and prioritization of the diagnoses present in the EDS differential; (2) the trust of the EDS was linked to varied beliefs around the diagnostic process and potential for bias; (3) clinicians foresaw more benefit to EDS use for colleagues and trainees rather than themselves; and (4) clinicians felt strongly that EDS output should not be included in the patient record. Conclusions The adoption of an EDS into an emergency department triage process will require a system that provides diagnostic suggestions appropriate for the scope and context of the emergency department triage process, transparency of system design, and affordances for clinician beliefs about the diagnostic process and addresses clinician concern around including EDS output in the patient record.
Collapse
Affiliation(s)
- Matthew Sibbald
- McMaster Education Research, Innovation & Theory (MERIT) Program, Department of Medicine, McMaster University, Hamilton, ON, Canada
| | - Bashayer Abdulla
- McMaster Education Research, Innovation & Theory (MERIT) Program, Department of Medicine, McMaster University, Hamilton, ON, Canada
| | - Amy Keuhl
- McMaster Education Research, Innovation & Theory (MERIT) Program, Department of Medicine, McMaster University, Hamilton, ON, Canada
| | - Geoffrey Norman
- McMaster Education Research, Innovation & Theory (MERIT) Program, Department of Health Research Methods, Evidence & Impact, McMaster University, Hamilton, ON, Canada
| | - Sandra Monteiro
- McMaster Education Research, Innovation & Theory (MERIT) Program, Department of Medicine, McMaster University, Hamilton, ON, Canada
| | - Jonathan Sherbino
- McMaster Education Research, Innovation & Theory (MERIT) Program, Department of Medicine, McMaster University, Hamilton, ON, Canada
| |
Collapse
|
46
|
Vasey B, Nagendran M, Campbell B, Clifton DA, Collins GS, Denaxas S, Denniston AK, Faes L, Geerts B, Ibrahim M, Liu X, Mateen BA, Mathur P, McCradden MD, Morgan L, Ordish J, Rogers C, Saria S, Ting DSW, Watkinson P, Weber W, Wheatstone P, McCulloch P. Reporting guideline for the early-stage clinical evaluation of decision support systems driven by artificial intelligence: DECIDE-AI. Nat Med 2022; 28:924-933. [PMID: 35585198 DOI: 10.1038/s41591-022-01772-9] [Citation(s) in RCA: 126] [Impact Index Per Article: 63.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/20/2021] [Accepted: 03/03/2022] [Indexed: 12/31/2022]
Abstract
A growing number of artificial intelligence (AI)-based clinical decision support systems are showing promising performance in preclinical, in silico evaluation, but few have yet demonstrated real benefit to patient care. Early-stage clinical evaluation is important to assess an AI system's actual clinical performance at small scale, ensure its safety, evaluate the human factors surrounding its use and pave the way to further large-scale trials. However, the reporting of these early studies remains inadequate. The present statement provides a multi-stakeholder, consensus-based reporting guideline for the Developmental and Exploratory Clinical Investigations of DEcision support systems driven by Artificial Intelligence (DECIDE-AI). We conducted a two-round, modified Delphi process to collect and analyze expert opinion on the reporting of early clinical evaluation of AI systems. Experts were recruited from 20 pre-defined stakeholder categories. The final composition and wording of the guideline was determined at a virtual consensus meeting. The checklist and the Explanation & Elaboration (E&E) sections were refined based on feedback from a qualitative evaluation process. In total, 123 experts participated in the first round of Delphi, 138 in the second round, 16 in the consensus meeting and 16 in the qualitative evaluation. The DECIDE-AI reporting guideline comprises 17 AI-specific reporting items (made of 28 subitems) and ten generic reporting items, with an E&E paragraph provided for each. Through consultation and consensus with a range of stakeholders, we developed a guideline comprising key items that should be reported in early-stage clinical studies of AI-based decision support systems in healthcare. By providing an actionable checklist of minimal reporting items, the DECIDE-AI guideline will facilitate the appraisal of these studies and replicability of their findings.
Collapse
Affiliation(s)
- Baptiste Vasey
- Nuffield Department of Surgical Sciences, University of Oxford, Oxford, UK.
- Institute of Biomedical Engineering, Department of Engineering Science, University of Oxford, Oxford, UK.
- Critical Care Research Group, Nuffield Department of Clinical Neurosciences, University of Oxford, Oxford, UK.
| | - Myura Nagendran
- UKRI Centre for Doctoral Training in AI for Healthcare, Imperial College London, London, UK
| | - Bruce Campbell
- University of Exeter Medical School, Exeter, UK
- Royal Devon and Exeter Hospital, Exeter, UK
| | - David A Clifton
- Institute of Biomedical Engineering, Department of Engineering Science, University of Oxford, Oxford, UK
| | - Gary S Collins
- Centre for Statistics in Medicine, Nuffield Department of Orthopaedics, Rheumatology & Musculoskeletal Sciences, University of Oxford, Oxford, UK
| | - Spiros Denaxas
- Institute of Health Informatics, University College London, London, UK
- British Heart Foundation Data Science Centre, London, UK
- Health Data Research UK, London, UK
- UCL Hospitals Biomedical Research Centre, London, UK
| | - Alastair K Denniston
- University Hospitals Birmingham NHS Foundation Trust, Birmingham, UK
- Academic Unit of Ophthalmology, Institute of Inflammation and Ageing, College of Medical and Dental Sciences, University of Birmingham, Birmingham, UK
- Moorfields Eye Hospital NHS Foundation Trust, London, UK
| | - Livia Faes
- Moorfields Eye Hospital NHS Foundation Trust, London, UK
| | - Bart Geerts
- Healthplus.ai-R&D BV, Amsterdam, The Netherlands
| | - Mudathir Ibrahim
- Nuffield Department of Surgical Sciences, University of Oxford, Oxford, UK
- Department of Surgery, Maimonides Medical Center, Brooklyn, NY, USA
| | - Xiaoxuan Liu
- University Hospitals Birmingham NHS Foundation Trust, Birmingham, UK
- Academic Unit of Ophthalmology, Institute of Inflammation and Ageing, College of Medical and Dental Sciences, University of Birmingham, Birmingham, UK
| | - Bilal A Mateen
- Institute of Health Informatics, University College London, London, UK
- The Wellcome Trust, London, UK
- The Alan Turing Institute, London, UK
| | - Piyush Mathur
- Department of General Anesthesiology, Anesthesiology Institute, Cleveland Clinic, Cleveland, OH, USA
| | - Melissa D McCradden
- The Hospital for Sick Children, Toronto ON, Canada
- Dalla Lana School of Public Health, University of Toronto, Toronto ON, Canada
| | | | - Johan Ordish
- Medicines and Healthcare products Regulatory Agency, London, UK
| | | | - Suchi Saria
- Departments of Computer Science, Statistics, and Health Policy, and Division of Informatics, Johns Hopkins University, Baltimore, MD, USA
- Bayesian Health, New York, NY, USA
| | - Daniel S W Ting
- Singapore National Eye Center, Singapore Eye Research Institute, Singapore, Singapore
- Duke-NUS Medical School, National University of Singapore, Singapore, Singapore
| | - Peter Watkinson
- Critical Care Research Group, Nuffield Department of Clinical Neurosciences, University of Oxford, Oxford, UK
- NIHR Biomedical Research Centre Oxford, Oxford University Hospitals NHS Trust, Oxford, UK
| | | | | | - Peter McCulloch
- Nuffield Department of Surgical Sciences, University of Oxford, Oxford, UK
| |
Collapse
|
47
|
Cifra CL, Custer JW, Fackler JC. A Research Agenda for Diagnostic Excellence in Critical Care Medicine. Crit Care Clin 2022; 38:141-157. [PMID: 34794628 PMCID: PMC8963385 DOI: 10.1016/j.ccc.2021.07.003] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/03/2023]
Abstract
Diagnosing critically ill patients in the intensive care unit is difficult. As a result, diagnostic errors in the intensive care unit are common and have been shown to cause harm. Research to improve diagnosis in critical care medicine has accelerated in past years. However, much work remains to fully elucidate the diagnostic process in critical care. To achieve diagnostic excellence, interdisciplinary research is needed, adopting a balanced strategy of continued biomedical discovery while addressing the complex care delivery systems underpinning the diagnosis of critical illness.
Collapse
|
48
|
Niemiec E. Will the EU Medical Device Regulation help to improve the safety and performance of medical AI devices? Digit Health 2022; 8:20552076221089079. [PMID: 35386955 PMCID: PMC8977702 DOI: 10.1177/20552076221089079] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/19/2021] [Accepted: 03/06/2022] [Indexed: 12/23/2022] Open
Abstract
Concerns have been raised over the quality of evidence on the performance of medical
artificial intelligence devices, including devices that are already on the market in the
USA and Europe. Recently, the Medical Device Regulation, which aims to set high standards
of safety and quality, has become applicable in the European Union. The aim of this
article is to discuss whether, and how, the Medical Device Regulation will help improve
the safety and performance of medical artificial intelligence devices entering the market.
The Medical Device Regulation introduces new rules for risk classification of the devices,
which will result in more devices subjected to a higher degree of scrutiny before entering
the market; more stringent requirements on clinical evaluation, including the requirement
for appraisal of clinical data; new requirements for post-market surveillance, which may
help spot early on any new, unexpected side effects and risks of the devices; and
requirements for notified bodies, including for expertise of the personnel and
consideration of relevant best practice documents. The guidance of the Medical Device
Coordination Group on clinical evaluation of medical device software and the MEDDEV2.7
guideline on clinical evaluation also attend to some of the problems identified in studies
on medical artificial intelligence devices. The Medical Device Regulation will likely help
improve the safety and performance of the medical artificial intelligence devices on the
European market. The impact of the Regulation, however, is also dependent on its adequate
enforcement by the European Union member states.
Collapse
Affiliation(s)
- Emilia Niemiec
- Medical Ethics Division, Department of Clinical Sciences, Lund University, Sweden
| |
Collapse
|
49
|
Sarafidis M, Manta O, Kouris I, Schlee W, Kikidis D, Vellidou E, Koutsouris D. Why a Clinical Decision Support System is needed for Tinnitus? ANNUAL INTERNATIONAL CONFERENCE OF THE IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. ANNUAL INTERNATIONAL CONFERENCE 2021; 2021:2075-2078. [PMID: 34891697 DOI: 10.1109/embc46164.2021.9630137] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/14/2023]
Abstract
Tinnitus is the perception of a phantom sound and the individual's reaction to it. Although much progress has been made, tinnitus remains an unresolved scientific and clinical issue, affecting more than 10% of the general population and having a high prevalence and socioeconomic burden. Clinical decision support systems (CDSS) are used to assist clinicians in their complex decision-making processes, having been proved that they improve healthcare delivery. In this paper, we present a CDSS for tinnitus, attempting to address the question which treatment approach is optimal for a particular patient based on specific parameters. The CDSS will be developed in the context of the EU-funded "UNITI" project and, after the project completion, it will be able to determine the suitability and expected attachment of a particular patient to a list of available clinical interventions, utilizing predictive and classification machine learning models.Clinical Relevance - The proposed clinically utilizable CDSS will be able to suggest the optimal treatment strategy for the tinnitus patient based on a set of heterogeneous data.
Collapse
|
50
|
Asselbergs FW, Fraser AG. Artificial intelligence in cardiology: the debate continues. EUROPEAN HEART JOURNAL. DIGITAL HEALTH 2021; 2:721-726. [PMID: 36713089 PMCID: PMC9708032 DOI: 10.1093/ehjdh/ztab090] [Citation(s) in RCA: 6] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 09/28/2021] [Accepted: 10/12/2021] [Indexed: 02/01/2023]
Abstract
In 1955, when John McCarthy and his colleagues proposed their first study of artificial intelligence, they suggested that 'every aspect of learning or any other feature of intelligence can in principle be so precisely described that a machine can be made to simulate it'. Whether that might ever be possible would depend on how we define intelligence, but what is indisputable is that new methods are needed to analyse and interpret the copious information provided by digital medical images, genomic databases, and biobanks. Technological advances have enabled applications of artificial intelligence (AI) including machine learning (ML) to be implemented into clinical practice, and their related scientific literature is exploding. Advocates argue enthusiastically that AI will transform many aspects of clinical cardiovascular medicine, while sceptics stress the importance of caution and the need for more evidence. This report summarizes the main opposing arguments that were presented in a debate at the 2021 Congress of the European Society of Cardiology. Artificial intelligence is an advanced analytical technique that should be considered when conventional statistical methods are insufficient, but testing a hypothesis or solving a clinical problem-not finding another application for AI-remains the most important objective. Artificial intelligence and ML methods should be transparent and interpretable, if they are to be approved by regulators and trusted to provide support for clinical decisions. Physicians need to understand AI methods and collaborate with engineers. Few applications have yet been shown to have a positive impact on clinical outcomes, so investment in research is essential.
Collapse
Affiliation(s)
- Folkert W Asselbergs
- Division Heart and Lungs, Department of Cardiology, University Medical Center Utrecht, Heidelberglaan 100, 3584 CX Utrecht, Netherlands,Institute of Health Informatics and Institute of Cardiovascular Science, University College London, 222 Euston Rd, London NW1 2DA, UK,NIHR BRC Clinical Research Informatics Unit, University College London Hospital, London, UK
| | - Alan G Fraser
- School of Medicine, Cardiff University, University Hospital of Wales, Heath Park, Cardiff CF14 4XW, UK,Cardiovascular Imaging and Dynamics, Katholieke Universiteit Leuven, UZ Gasthuisberg, Herestraat 49, 3000 Leuven, Belgium,Corresponding author. Tel: +44 (0)29 2184 5366, Fax: +44 (0)29 2184 4473,
| |
Collapse
|