1
|
Martindale APL, Ng B, Ngai V, Kale AU, Ferrante di Ruffano L, Golub RM, Collins GS, Moher D, McCradden MD, Oakden-Rayner L, Rivera SC, Calvert M, Kelly CJ, Lee CS, Yau C, Chan AW, Keane PA, Beam AL, Denniston AK, Liu X. Concordance of randomised controlled trials for artificial intelligence interventions with the CONSORT-AI reporting guidelines. Nat Commun 2024; 15:1619. [PMID: 38388497 PMCID: PMC10883966 DOI: 10.1038/s41467-024-45355-3] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/27/2023] [Accepted: 01/22/2024] [Indexed: 02/24/2024] Open
Abstract
The Consolidated Standards of Reporting Trials extension for Artificial Intelligence interventions (CONSORT-AI) was published in September 2020. Since its publication, several randomised controlled trials (RCTs) of AI interventions have been published but their completeness and transparency of reporting is unknown. This systematic review assesses the completeness of reporting of AI RCTs following publication of CONSORT-AI and provides a comprehensive summary of RCTs published in recent years. 65 RCTs were identified, mostly conducted in China (37%) and USA (18%). Median concordance with CONSORT-AI reporting was 90% (IQR 77-94%), although only 10 RCTs explicitly reported its use. Several items were consistently under-reported, including algorithm version, accessibility of the AI intervention or code, and references to a study protocol. Only 3 of 52 included journals explicitly endorsed or mandated CONSORT-AI. Despite a generally high concordance amongst recent AI RCTs, some AI-specific considerations remain systematically poorly reported. Further encouragement of CONSORT-AI adoption by journals and funders may enable more complete adoption of the full CONSORT-AI guidelines.
Collapse
Affiliation(s)
| | - Benjamin Ng
- Birmingham and Midland Eye Centre, Sandwell and West Birmingham NHS Trust, Birmingham, UK
- Christ Church, University of Oxford, Oxford, UK
| | - Victoria Ngai
- University College London Medical School, London, UK
| | - Aditya U Kale
- Institute of Inflammation and Ageing, University of Birmingham, Birmingham, UK
- University Hospitals Birmingham NHS Foundation Trust, Birmingham, UK
- National Institute for Health and Care Research (NIHR) Birmingham Biomedical Research Centre, University of Birmingham, Birmingham, UK
| | | | - Robert M Golub
- Northwestern University Feinberg School of Medicine, Chicago, IL, USA
| | - Gary S Collins
- Centre for Statistics in Medicine//UK EQUATOR Centre, Nuffield Department of Orthopaedics, Rheumatology and Musculoskeletal Sciences, University of Oxford, Oxford, UK
| | - David Moher
- Centre for Journalology, Clinical Epidemiology Program, Ottawa Hospital Research Institute, Ottowa, ON, Canada
| | - Melissa D McCradden
- Department of Bioethics, The Hospital for Sick Children, Toronto, ON, Canada
- Genetics & Genome Biology Research Program, Peter Gilgan Centre for Research & Learning, Toronto, ON, Canada
- Division of Clinical and Public Health, Dalla Lana School of Public Health, Toronto, ON, Canada
| | - Lauren Oakden-Rayner
- Australian Institute for Machine Learning, University of Adelaide, Adelaide, SA, Australia
| | - Samantha Cruz Rivera
- Birmingham Health Partners Centre for Regulatory Science and Innovation, University of Birmingham, Birmingham, UK
- Centre for Patient Reported Outcomes Research (CPROR), Institute of Applied Health Research, College of Medical and Dental Sciences, University of Birmingham, Birmingham, UK
| | - Melanie Calvert
- National Institute for Health and Care Research (NIHR) Birmingham Biomedical Research Centre, University of Birmingham, Birmingham, UK
- Birmingham Health Partners Centre for Regulatory Science and Innovation, University of Birmingham, Birmingham, UK
- Centre for Patient Reported Outcomes Research (CPROR), Institute of Applied Health Research, College of Medical and Dental Sciences, University of Birmingham, Birmingham, UK
- NIHR Applied Research Collaboration (ARC) West Midlands, University of Birmingham, Birmingham, UK
- NIHR Blood and Transplant Research Unit (BTRU) in Precision Transplant and Cellular Therapeutics, University of Birmingham, Birmingham, UK
| | | | | | - Christopher Yau
- Nuffield Department of Women's and Reproductive Health, University of Oxford, Oxford, UK
- Health Data Research UK, London, UK
| | - An-Wen Chan
- Department of Medicine, Women's College Hospital. University of Toronto, Toronto, ON, Canada
| | - Pearse A Keane
- NIHR Biomedical Research Centre at Moorfields, Moorfields Eye Hospital NHS Foundation Trust and UCL Institute of Ophthalmology, London, UK
| | - Andrew L Beam
- Department of Epidemiology, Harvard. T.H. Chan School of Public Health, Boston, MA, USA
- Department of Biomedical Informatics, Harvard Medical School, Boston, MA, USA
| | - Alastair K Denniston
- Institute of Inflammation and Ageing, University of Birmingham, Birmingham, UK
- University Hospitals Birmingham NHS Foundation Trust, Birmingham, UK
- National Institute for Health and Care Research (NIHR) Birmingham Biomedical Research Centre, University of Birmingham, Birmingham, UK
- Birmingham Health Partners Centre for Regulatory Science and Innovation, University of Birmingham, Birmingham, UK
- NIHR Biomedical Research Centre at Moorfields, Moorfields Eye Hospital NHS Foundation Trust and UCL Institute of Ophthalmology, London, UK
| | - Xiaoxuan Liu
- Institute of Inflammation and Ageing, University of Birmingham, Birmingham, UK.
- University Hospitals Birmingham NHS Foundation Trust, Birmingham, UK.
- Birmingham Health Partners Centre for Regulatory Science and Innovation, University of Birmingham, Birmingham, UK.
| |
Collapse
|
2
|
Panebianco N, Baston C. Training, Competency, and Interdisciplinary Collaboration in Point-of-Care Ultrasound. Semin Ultrasound CT MR 2024; 45:91-97. [PMID: 38056786 DOI: 10.1053/j.sult.2023.12.010] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/08/2023]
Abstract
Point-of-care ultrasound can provide critical diagnostic information and add to procedural safety. As with any clinical skill, however, it must be applied by an adequately trained provider, with systems to ensure safety. Training can include a mix of hands-on training with traditional didactics, online coursework, and simulation, but each carries its own costs and benefits worth of review. Following training it is essential to think about assessment of competency in point-of-care ultrasound to reflect the combination of cognitive and procedural skills that makes up the practice. Within the frameworks described, expansion can be expected across specialty and professional boundaries.
Collapse
Affiliation(s)
- Nova Panebianco
- Perelman School of Medicine at the University of Pennsylvania, Department of Emergency Medicine, Philadelphia, PA.
| | - Cameron Baston
- Perelman School of Medicine at the University of Pennsylvania, Department of Medicine, Philadelphia, PA
| |
Collapse
|
3
|
Dadon Z, Orlev A, Butnaru A, Rosenmann D, Glikson M, Gottlieb S, Alpert EA. Empowering Medical Students: Harnessing Artificial Intelligence for Precision Point-of-Care Echocardiography Assessment of Left Ventricular Ejection Fraction. Int J Clin Pract 2023; 2023:5225872. [PMID: 38078051 PMCID: PMC10699938 DOI: 10.1155/2023/5225872] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 05/12/2023] [Revised: 10/14/2023] [Accepted: 11/14/2023] [Indexed: 12/18/2023] Open
Abstract
Introduction Point-of-care ultrasound (POCUS) use is now universal among nonexperts. Artificial intelligence (AI) is currently employed by nonexperts in various imaging modalities to assist in diagnosis and decision making. Aim To evaluate the diagnostic accuracy of POCUS, operated by medical students with the assistance of an AI-based tool for assessing the left ventricular ejection fraction (LVEF) of patients admitted to a cardiology department. Methods Eight students underwent a 6-hour didactic and hands-on training session. Participants used a hand-held ultrasound device (HUD) equipped with an AI-based tool for the automatic evaluation of LVEF. The clips were assessed for LVEF by three methods: visually by the students, by students + the AI-based tool, and by the cardiologists. All LVEF measurements were compared to formal echocardiography completed within 24 hours and were evaluated for LVEF using the Simpson method and eyeballing assessment by expert echocardiographers. Results The study included 88 patients (aged 58.3 ± 16.3 years). The AI-based tool measurement was unsuccessful in 6 cases. Comparing LVEF reported by students' visual evaluation and students + AI vs. cardiologists revealed a correlation of 0.51 and 0.83, respectively. Comparing these three evaluation methods with the echocardiographers revealed a moderate/substantial agreement for the students + AI and cardiologists but only a fair agreement for the students' visual evaluation. Conclusion Medical students' utilization of an AI-based tool with a HUD for LVEF assessment achieved a level of accuracy similar to that of cardiologists. Furthermore, the use of AI by the students achieved moderate to substantial inter-rater reliability with expert echocardiographers' evaluation.
Collapse
Affiliation(s)
- Ziv Dadon
- Jesselson Integrated Heart Center, Shaare Zedek Medical Center, Jerusalem, Israel
- Faculty of Medicine, Hebrew University of Jerusalem, Jerusalem, Israel
| | - Amir Orlev
- Jesselson Integrated Heart Center, Shaare Zedek Medical Center, Jerusalem, Israel
- Faculty of Medicine, Hebrew University of Jerusalem, Jerusalem, Israel
| | - Adi Butnaru
- Jesselson Integrated Heart Center, Shaare Zedek Medical Center, Jerusalem, Israel
| | - David Rosenmann
- Jesselson Integrated Heart Center, Shaare Zedek Medical Center, Jerusalem, Israel
| | - Michael Glikson
- Jesselson Integrated Heart Center, Shaare Zedek Medical Center, Jerusalem, Israel
- Faculty of Medicine, Hebrew University of Jerusalem, Jerusalem, Israel
| | - Shmuel Gottlieb
- Jesselson Integrated Heart Center, Shaare Zedek Medical Center, Jerusalem, Israel
- Sackler Faculty of Medicine, Tel Aviv University, Tel Aviv, Israel
| | - Evan Avraham Alpert
- Faculty of Medicine, Hebrew University of Jerusalem, Jerusalem, Israel
- Department of Emergency Medicine, Shaare Zedek Medical Center, Jerusalem, Israel
| |
Collapse
|
4
|
Dadon Z, Carasso S, Gottlieb S. The Role of Hand-Held Cardiac Ultrasound in Patients with COVID-19. Biomedicines 2023; 11:biomedicines11020239. [PMID: 36830776 PMCID: PMC9953352 DOI: 10.3390/biomedicines11020239] [Citation(s) in RCA: 3] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/24/2022] [Revised: 01/04/2023] [Accepted: 01/05/2023] [Indexed: 01/19/2023] Open
Abstract
The role of point-of-care ultrasound (POCUS) in patient management has been established in recent years as an important tool. It is increasingly used by multiple medical disciplines in numerous clinical settings, for different applications and diagnostic purposes and in the guidance of procedures. The introduction of small-sized and inexpensive hand-held ultrasound devices (HUDs) has addressed some of the POCUS-related challenges and has thus extended POCUS' applicability. HUD utilization is even more relevant in the COVID-19 setting given the operators' infection risk, excessive workload concerns and general equipment contamination. This review focuses on the available technology, usefulness, feasibility and clinical applications of HUD for echocardiogram assessment in patients with COVID-19.
Collapse
Affiliation(s)
- Ziv Dadon
- Jesselson Integrated Heart Center, Shaare Zedek Medical Center, Jerusalem 9103102, Israel
| | - Shemy Carasso
- Jesselson Integrated Heart Center, Shaare Zedek Medical Center, Jerusalem 9103102, Israel
- The Azrieli Faculty of Medicine, Bar-Ilan University, Zefat 1311502, Israel
| | - Shmuel Gottlieb
- Jesselson Integrated Heart Center, Shaare Zedek Medical Center, Jerusalem 9103102, Israel
- The Sackler Faculty of Medicine, Tel Aviv University, Tel Aviv 6997801, Israel
- Correspondence:
| |
Collapse
|