1
|
Moterani VC, Abbade JF, Borges VTM, Fonseca CGF, Desiderio N, Moterani Junior NJW, Gonçalves Moterani LBB. [Guidelines for clinical trial protocols for interventions involving artificial intelligence: the SPIRIT-AI extensionDiretrizes para protocolos de ensaios clínicos com intervenções que utilizam inteligência artificial: a extensão SPIRIT-AI]. Rev Panam Salud Publica 2023; 47:e149. [PMID: 38361499 PMCID: PMC10868409 DOI: 10.26633/rpsp.2023.149] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/24/2020] [Accepted: 07/23/2020] [Indexed: 01/10/2024] Open
Abstract
The SPIRIT 2013 statement aims to improve the completeness of clinical trial protocol reporting by providing evidence-based recommendations for the minimum set of items to be addressed. This guidance has been instrumental in promoting transparent evaluation of new interventions. More recently, there has been a growing recognition that interventions involving artificial intelligence (AI) need to undergo rigorous, prospective evaluation to demonstrate their impact on health outcomes. The SPIRIT-AI (Standard Protocol Items: Recommendations for Interventional Trials-Artificial Intelligence) extension is a new reporting guideline for clinical trial protocols evaluating interventions with an AI component. It was developed in parallel with its companion statement for trial reports: CONSORT-AI (Consolidated Standards of Reporting Trials-Artificial Intelligence). Both guidelines were developed through a staged consensus process involving literature review and expert consultation to generate 26 candidate items, which were consulted upon by an international multi-stakeholder group in a two-stage Delphi survey (103 stakeholders), agreed upon in a consensus meeting (31 stakeholders) and refined through a checklist pilot (34 participants). The SPIRIT-AI extension includes 15 new items that were considered sufficiently important for clinical trial protocols of AI interventions. These new items should be routinely reported in addition to the core SPIRIT 2013 items. SPIRIT-AI recommends that investigators provide clear descriptions of the AI intervention, including instructions and skills required for use, the setting in which the AI intervention will be integrated, considerations for the handling of input and output data, the human-AI interaction and analysis of error cases. SPIRIT-AI will help promote transparency and completeness for clinical trial protocols for AI interventions. Its use will assist editors and peer reviewers, as well as the general readership, to understand, interpret and critically appraise the design and risk of bias for a planned clinical trial.
Collapse
Affiliation(s)
- Vinicius Cesar Moterani
- Universidade Estadual Paulista “Júlio de Mesquita Filho,”BotucatuBrazilUniversidade Estadual Paulista “Júlio de Mesquita Filho,” Botucatu, Brazil
| | - Joelcio Francisco Abbade
- Universidade Estadual Paulista “Júlio de Mesquita Filho,”BotucatuBrazilUniversidade Estadual Paulista “Júlio de Mesquita Filho,” Botucatu, Brazil
| | - Vera Therezinha Medeiros Borges
- Universidade Estadual Paulista “Júlio de Mesquita Filho,”BotucatuBrazilUniversidade Estadual Paulista “Júlio de Mesquita Filho,” Botucatu, Brazil
| | - Cecilia Guimarães Ferreira Fonseca
- Universidade Estadual Paulista “Júlio de Mesquita Filho,”BotucatuBrazilUniversidade Estadual Paulista “Júlio de Mesquita Filho,” Botucatu, Brazil
| | - Nathalia Desiderio
- Marilia Medical SchoolMariliaBrazilMarilia Medical School, Marilia, Brazil
| | | | | |
Collapse
|
2
|
Toonders SAJ, van der Meer HA, van Bruxvoort T, Veenhof C, Speksnijder CM. Effectiveness of remote physiotherapeutic e-Health interventions on pain in patients with musculoskeletal disorders: a systematic review. Disabil Rehabil 2023; 45:3620-3638. [PMID: 36369923 DOI: 10.1080/09638288.2022.2135775] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/25/2022] [Revised: 10/04/2022] [Accepted: 10/09/2022] [Indexed: 11/15/2022]
Abstract
PURPOSE To systematically review the literature on effectiveness of remote physiotherapeutic e-Health interventions on pain in patients with musculoskeletal disorders. MATERIALS AND METHODS Using online data sources PubMed, Embase, and Cochrane in adults with musculoskeletal disorders with a pain-related complaint. Remote physiotherapeutic e-Health interventions were analysed. Control interventions were not specified. Outcomes on effect of remote e-Health interventions in terms of pain intensity. RESULTS From 11,811 studies identified, 27 studies were included. There is limited evidence for the effectiveness for remote e-Health for patients with back pain based on five articles. Twelve articles studied chronic pain and the effectiveness was dependent on the control group and involvement of healthcare providers. In patients with osteoarthritis (five articles), total knee surgery (two articles), and knee pain (three articles) no significant effects were found for remote e-Health compared to control groups. CONCLUSIONS There is limited evidence for the effectiveness of remote physiotherapeutic e-Health interventions to decrease pain intensity in patients with back pain. There is some evidence for effectiveness of remote e-Health in patients with chronic pain. For patients with osteoarthritis, after total knee surgery and knee pain, there appears to be no effect of e-Health when solely looking at reduction of pain. Implications for rehabilitationThis review shows that e-Health can be an effective way of reducing pain in some populations.Remote physiotherapeutic e-Health interventions may decrease pain intensity in patients with back pain.Autonomous e-Health is more effective than no treatment in patients with chronic pain.There is no effect of e-Health in reduction of pain for patients with osteoarthritis, after total knee surgery and knee pain.
Collapse
Affiliation(s)
- Suze A J Toonders
- Department of Health Innovation and Technology, Fontys University of Applied Sciences, Eindhoven, Netherlands
- Center for Physical Therapy Research and Innovation in Primary Care, Leidsche Rijn Julius Health Care Centers, Utrecht, The Netherlands
- Department of Rehabilitation, Physical Therapy Science and Sport, Physical Therapy Research Group, Brain Center Rudolf Magnus, University Medical Center Utrecht, Utrecht, The Netherlands
| | - Hedwig A van der Meer
- Department of Oral-Maxillofacial Surgery and Special Dental Care, University Medical Center Utrecht, Utrecht University, Utrecht, The Netherlands
- Department of Orofacial Pain and Disfunction, Academic Centre for Dentistry Amsterdam (ACTA), University of Amsterdam and Vrije Universiteit (VU) University Amsterdam, Amsterdam, The Netherlands
- Faculty of Health, Amsterdam University of Applied Sciences, Amsterdam, The Netherlands
| | - Thijs van Bruxvoort
- Product Management, Thijs van Bruxvoort, Founda B.V, Amsterdam, The Netherlands
| | - Cindy Veenhof
- Center for Physical Therapy Research and Innovation in Primary Care, Leidsche Rijn Julius Health Care Centers, Utrecht, The Netherlands
- Department of Rehabilitation, Physical Therapy Science and Sport, Physical Therapy Research Group, Brain Center Rudolf Magnus, University Medical Center Utrecht, Utrecht, The Netherlands
- Research Group Innovation of Human Movement Care, HU University of Applied Sciences Utrecht, Utrecht, The Netherlands
| | - Caroline M Speksnijder
- Department of Oral-Maxillofacial Surgery and Special Dental Care, University Medical Center Utrecht, Utrecht University, Utrecht, The Netherlands
| |
Collapse
|
3
|
Liu X, Cruz Rivera S, Moher D, Calvert MJ, Denniston AK. [Reporting guidelines for clinical trial reports for interventions involving artificial intelligence: the CONSORT-AI extensionDiretrizes para relatórios de ensaios clínicos com intervenções que utilizam inteligência artificial: a extensão CONSORT-AI]. Rev Panam Salud Publica 2023; 48:e13. [PMID: 38352035 PMCID: PMC10863743 DOI: 10.26633/rpsp.2024.13] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/24/2020] [Accepted: 07/23/2020] [Indexed: 02/16/2024] Open
Abstract
The CONSORT 2010 statement provides minimum guidelines for reporting randomized trials. Its widespread use has been instrumental in ensuring transparency in the evaluation of new interventions. More recently, there has been a growing recognition that interventions involving artificial intelligence (AI) need to undergo rigorous, prospective evaluation to demonstrate impact on health outcomes. The CONSORT-AI (Consolidated Standards of Reporting Trials-Artificial Intelligence) extension is a new reporting guideline for clinical trials evaluating interventions with an AI component. It was developed in parallel with its companion statement for clinical trial protocols: SPIRIT-AI (Standard Protocol Items: Recommendations for Interventional Trials-Artificial Intelligence). Both guidelines were developed through a staged consensus process involving literature review and expert consultation to generate 29 candidate items, which were assessed by an international multi-stakeholder group in a two-stage Delphi survey (103 stakeholders), agreed upon in a two-day consensus meeting (31 stakeholders) and refined through a checklist pilot (34 participants). The CONSORT-AI extension includes 14 new items that were considered sufficiently important for AI interventions that they should be routinely reported in addition to the core CONSORT 2010 items. CONSORT-AI recommends that investigators provide clear descriptions of the AI intervention, including instructions and skills required for use, the setting in which the AI intervention is integrated, the handling of inputs and outputs of the AI intervention, the human-AI interaction and provision of an analysis of error cases. CONSORT-AI will help promote transparency and completeness in reporting clinical trials for AI interventions. It will assist editors and peer reviewers, as well as the general readership, to understand, interpret and critically appraise the quality of clinical trial design and risk of bias in the reported outcomes.
Collapse
Affiliation(s)
- Xiaoxuan Liu
- Moorfields Eye Hospital NHS Foundation TrustLondresReino UnidoMoorfields Eye Hospital NHS Foundation Trust, Londres, Reino Unido.
- Academic Unit of OphthalmologyInstitute of Inflammation and AgeingUniversity of BirminghamBirminghamReino UnidoAcademic Unit of Ophthalmology, Institute of Inflammation and Ageing, University of Birmingham, Birmingham, Reino Unido.
- University Hospitals Birmingham NHS Foundation TrustBirminghamReino UnidoUniversity Hospitals Birmingham NHS Foundation Trust, Birmingham, Reino Unido.
- Health Data Research Reino UnidoLondresReino UnidoHealth Data Research Reino Unido, Londres, Reino Unido.
- Birmingham Health Partners Centre for Regulatory Science and InnovationUniversity of BirminghamBirminghamReino UnidoBirmingham Health Partners Centre for Regulatory Science and Innovation, University of Birmingham, Birmingham, Reino Unido.
| | - Samantha Cruz Rivera
- Birmingham Health Partners Centre for Regulatory Science and InnovationUniversity of BirminghamBirminghamReino UnidoBirmingham Health Partners Centre for Regulatory Science and Innovation, University of Birmingham, Birmingham, Reino Unido.
- Centre for Patient Reported Outcomes ResearchInstitute of Applied Health ResearchUniversity of BirminghamBirmingham Centre for Patient Reported Outcomes Research, Institute of Applied Health Research, University of Birmingham, Birmingham.
- Institute of Applied Health ResearchUniversity of BirminghamBirminghamReino UnidoInstitute of Applied Health Research, University of Birmingham, Birmingham, Reino Unido.
| | - David Moher
- Centre for JournalologyClinical Epidemiology ProgramOttawa Hospital Research InstituteOttawaCanadáCentre for Journalology, Clinical Epidemiology Program, Ottawa Hospital Research Institute, Ottawa, Canadá.
- School of Epidemiology and Public HealthFaculty of MedicineUniversity of OttawaOttawaCanadaSchool of Epidemiology and Public Health, Faculty of Medicine, University of Ottawa, Ottawa, Canada.
| | - Melanie J. Calvert
- Health Data Research Reino UnidoLondresReino UnidoHealth Data Research Reino Unido, Londres, Reino Unido.
- Birmingham Health Partners Centre for Regulatory Science and InnovationUniversity of BirminghamBirminghamReino UnidoBirmingham Health Partners Centre for Regulatory Science and Innovation, University of Birmingham, Birmingham, Reino Unido.
- Centre for Patient Reported Outcomes ResearchInstitute of Applied Health ResearchUniversity of BirminghamBirmingham Centre for Patient Reported Outcomes Research, Institute of Applied Health Research, University of Birmingham, Birmingham.
- Institute of Applied Health ResearchUniversity of BirminghamBirminghamReino UnidoInstitute of Applied Health Research, University of Birmingham, Birmingham, Reino Unido.
- National Institute of Health Research Birmingham Biomedical Research CentreUniversity of Birmingham and University Hospitals Birmingham NHS Foundation TrustBirminghamReino UnidoNational Institute of Health Research Birmingham Biomedical Research Centre, University of Birmingham and University Hospitals Birmingham NHS Foundation Trust, Birmingham, Reino Unido.
- National Institute of Health Research Applied Research Collaborative West MidlandsCoventryReino Unido.National Institute of Health Research Applied Research Collaborative West Midlands, Coventry, Reino Unido.
- National Institute of Health Research Surgical Reconstruction and Microbiology CentreUniversity of Birmingham and University Hospitals Birmingham NHS Foundation TrustBirminghamReino UnidoNational Institute of Health Research Surgical Reconstruction and Microbiology Centre, University of Birmingham and University Hospitals Birmingham NHS Foundation Trust, Birmingham, Reino Unido.
| | - Alastair K. Denniston
- Academic Unit of OphthalmologyInstitute of Inflammation and AgeingUniversity of BirminghamBirminghamReino UnidoAcademic Unit of Ophthalmology, Institute of Inflammation and Ageing, University of Birmingham, Birmingham, Reino Unido.
- University Hospitals Birmingham NHS Foundation TrustBirminghamReino UnidoUniversity Hospitals Birmingham NHS Foundation Trust, Birmingham, Reino Unido.
- Health Data Research Reino UnidoLondresReino UnidoHealth Data Research Reino Unido, Londres, Reino Unido.
- Birmingham Health Partners Centre for Regulatory Science and InnovationUniversity of BirminghamBirminghamReino UnidoBirmingham Health Partners Centre for Regulatory Science and Innovation, University of Birmingham, Birmingham, Reino Unido.
- Centre for Patient Reported Outcomes ResearchInstitute of Applied Health ResearchUniversity of BirminghamBirmingham Centre for Patient Reported Outcomes Research, Institute of Applied Health Research, University of Birmingham, Birmingham.
- NIHR Biomedical Research Center at Moorfields Eye Hospital NHS Foundation Trust and UCL Institute of OphthalmologyLondresReino UnidoNIHR Biomedical Research Center at Moorfields Eye Hospital NHS Foundation Trust and UCL Institute of Ophthalmology, Londres, Reino Unido.
| | | |
Collapse
|
4
|
Cruz Rivera S, Liu X, Chan AW, Denniston AK, Calvert MJ. [Guidelines for clinical trial protocols for interventions involving artificial intelligence: the SPIRIT-AI extensionDiretrizes para protocolos de ensaios clínicos com intervenções que utilizam inteligência artificial: a extensão SPIRIT-AI]. Rev Panam Salud Publica 2023; 48:e12. [PMID: 38304411 PMCID: PMC10832304 DOI: 10.26633/rpsp.2024.12] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/24/2020] [Accepted: 07/23/2020] [Indexed: 02/03/2024] Open
Abstract
The SPIRIT 2013 statement aims to improve the completeness of clinical trial protocol reporting by providing evidence-based recommendations for the minimum set of items to be addressed. This guidance has been instrumental in promoting transparent evaluation of new interventions. More recently, there has been a growing recognition that interventions involving artificial intelligence (AI) need to undergo rigorous, prospective evaluation to demonstrate their impact on health outcomes. The SPIRIT-AI (Standard Protocol Items: Recommendations for Interventional Trials-Artificial Intelligence) extension is a new reporting guideline for clinical trial protocols evaluating interventions with an AI component. It was developed in parallel with its companion statement for trial reports: CONSORT-AI (Consolidated Standards of Reporting Trials-Artificial Intelligence). Both guidelines were developed through a staged consensus process involving literature review and expert consultation to generate 26 candidate items, which were consulted upon by an international multi-stakeholder group in a two-stage Delphi survey (103 stakeholders), agreed upon in a consensus meeting (31 stakeholders) and refined through a checklist pilot (34 participants). The SPIRIT-AI extension includes 15 new items that were considered sufficiently important for clinical trial protocols of AI interventions. These new items should be routinely reported in addition to the core SPIRIT 2013 items. SPIRIT-AI recommends that investigators provide clear descriptions of the AI intervention, including instructions and skills required for use, the setting in which the AI intervention will be integrated, considerations for the handling of input and output data, the human-AI interaction and analysis of error cases. SPIRIT-AI will help promote transparency and completeness for clinical trial protocols for AI interventions. Its use will assist editors and peer reviewers, as well as the general readership, to understand, interpret and critically appraise the design and risk of bias for a planned clinical trial.
Collapse
Affiliation(s)
- Samantha Cruz Rivera
- Centre for Patient Reported Outcomes ResearchInstitute of Applied Health ResearchUniversity of BirminghamBirminghamReino UnidoCentre for Patient Reported Outcomes Research, Institute of Applied Health Research, University of Birmingham, Birmingham, Reino Unido.
- Institute of Applied Health ResearchUniversity of BirminghamBirminghamReino UnidoInstitute of Applied Health Research, University of Birmingham, Birmingham, Reino Unido.
- Birmingham Health Partners Centre for Regulatory Science and InnovationUniversity of BirminghamBirminghamReino UnidoBirmingham Health Partners Centre for Regulatory Science and Innovation, University of Birmingham, Birmingham, Reino Unido.
| | - Xiaoxuan Liu
- Birmingham Health Partners Centre for Regulatory Science and InnovationUniversity of BirminghamBirminghamReino UnidoBirmingham Health Partners Centre for Regulatory Science and Innovation, University of Birmingham, Birmingham, Reino Unido.
- Academic Unit of OphthalmologyInstitute of Inflammation and AgeingUniversity of BirminghamBirminghamReino UnidoAcademic Unit of Ophthalmology, Institute of Inflammation and Ageing, University of Birmingham, Birmingham, Reino Unido.
- University Hospitals Birmingham NHS Foundation TrustBirminghamReino UnidoUniversity Hospitals Birmingham NHS Foundation Trust, Birmingham, Reino Unido.
- Health Data Research UKLondresReino UnidoHealth Data Research UK, Londres, Reino Unido.
- Moorfields Eye Hospital NHS Foundation TrustLondresReino UnidoMoorfields Eye Hospital NHS Foundation Trust, Londres, Reino Unido.
| | - An-Wen Chan
- Department of Medicine, Women’s College Research InstituteWomen’s College HospitalUniversity of TorontoOntarioCanadáDepartment of Medicine, Women’s College Research Institute, Women’s College Hospital, University of Toronto, Ontario, Canadá.
| | - Alastair K. Denniston
- Centre for Patient Reported Outcomes ResearchInstitute of Applied Health ResearchUniversity of BirminghamBirminghamReino UnidoCentre for Patient Reported Outcomes Research, Institute of Applied Health Research, University of Birmingham, Birmingham, Reino Unido.
- Birmingham Health Partners Centre for Regulatory Science and InnovationUniversity of BirminghamBirminghamReino UnidoBirmingham Health Partners Centre for Regulatory Science and Innovation, University of Birmingham, Birmingham, Reino Unido.
- Academic Unit of OphthalmologyInstitute of Inflammation and AgeingUniversity of BirminghamBirminghamReino UnidoAcademic Unit of Ophthalmology, Institute of Inflammation and Ageing, University of Birmingham, Birmingham, Reino Unido.
- University Hospitals Birmingham NHS Foundation TrustBirminghamReino UnidoUniversity Hospitals Birmingham NHS Foundation Trust, Birmingham, Reino Unido.
- Health Data Research UKLondresReino UnidoHealth Data Research UK, Londres, Reino Unido.
- National Institute of Health Research Biomedical Research Centre for OphthalmologyMoorfields Hospital London NHS Foundation Trust and University College LondonInstitute of OphthalmologyLondresReino UnidoNational Institute of Health Research Biomedical Research Centre for Ophthalmology, Moorfields Hospital London NHS Foundation Trust and University College London, Institute of Ophthalmology, Londres, Reino Unido.
| | - Melanie J. Calvert
- Centre for Patient Reported Outcomes ResearchInstitute of Applied Health ResearchUniversity of BirminghamBirminghamReino UnidoCentre for Patient Reported Outcomes Research, Institute of Applied Health Research, University of Birmingham, Birmingham, Reino Unido.
- Institute of Applied Health ResearchUniversity of BirminghamBirminghamReino UnidoInstitute of Applied Health Research, University of Birmingham, Birmingham, Reino Unido.
- Birmingham Health Partners Centre for Regulatory Science and InnovationUniversity of BirminghamBirminghamReino UnidoBirmingham Health Partners Centre for Regulatory Science and Innovation, University of Birmingham, Birmingham, Reino Unido.
- Health Data Research UKLondresReino UnidoHealth Data Research UK, Londres, Reino Unido.
- National Institute of Health Research Birmingham Biomedical Research CentreUniversity of Birmingham and University Hospitals Birmingham NHS Foundation TrustBirminghamReino UnidoNational Institute of Health Research Birmingham Biomedical Research Centre, University of Birmingham and University Hospitals Birmingham NHS Foundation Trust, Birmingham, Reino Unido.
- National Institute of Health Research Applied Research Collaborative West MidlandsCoventryReino UnidoNational Institute of Health Research Applied Research Collaborative West Midlands, Coventry, Reino Unido.
- National Institute of Health Research Surgical Reconstruction and Microbiology CentreUniversity of Birmingham and University Hospitals Birmingham NHS Foundation TrustBirminghamReino UnidoNational Institute of Health Research Surgical Reconstruction and Microbiology Centre, University of Birmingham and University Hospitals Birmingham NHS Foundation Trust, Birmingham, Reino Unido.
| | | |
Collapse
|
5
|
Rentz C, Far MS, Boltes M, Schnitzler A, Amunts K, Dukart J, Minnerop M. System Comparison for Gait and Balance Monitoring Used for the Evaluation of a Home-Based Training. SENSORS (BASEL, SWITZERLAND) 2022; 22:4975. [PMID: 35808470 PMCID: PMC9269735 DOI: 10.3390/s22134975] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 06/10/2022] [Revised: 06/24/2022] [Accepted: 06/28/2022] [Indexed: 06/15/2023]
Abstract
There are currently no standard methods for evaluating gait and balance performance at home. Smartphones include acceleration sensors and may represent a promising and easily accessible tool for this purpose. We performed an interventional feasibility study and compared a smartphone-based approach with two standard gait analysis systems (force plate and motion capturing systems). Healthy adults (n = 25, 44.1 ± 18.4 years) completed two laboratory evaluations before and after a three-week gait and balance training at home. There was an excellent agreement between all systems for stride time and cadence during normal, tandem and backward gait, whereas correlations for gait velocity were lower. Balance variables of both standard systems were moderately intercorrelated across all stance tasks, but only few correlated with the corresponding smartphone measures. Significant differences over time were found for several force plate and mocap system-obtained gait variables of normal, backward and tandem gait. Changes in balance variables over time were more heterogeneous and not significant for any system. The smartphone seems to be a suitable method to measure cadence and stride time of different gait, but not balance, tasks in healthy adults. Additional optimizations in data evaluation and processing may further improve the agreement between the analysis systems.
Collapse
Affiliation(s)
- Clara Rentz
- Institute of Neuroscience and Medicine (INM-1), Research Centre Juelich, 52428 Juelich, Germany; (K.A.); (M.M.)
| | - Mehran Sahandi Far
- Institute of Neuroscience and Medicine, Brain & Behaviour (INM-7), Research Centre Juelich, 52428 Juelich, Germany; (M.S.F.); (J.D.)
- Institute of Systems Neuroscience, Medical Faculty, Heinrich-Heine University Duesseldorf, 40225 Duesseldorf, Germany
| | - Maik Boltes
- Institute for Advanced Simulation (IAS-7), Research Centre Juelich, 52428 Juelich, Germany;
| | - Alfons Schnitzler
- Department of Neurology, Center for Movement Disorders and Neuromodulation, Medical Faculty, Heinrich-Heine University Duesseldorf, 40225 Duesseldorf, Germany;
- Institute of Clinical Neuroscience and Medical Psychology, Medical Faculty, Heinrich-Heine University Duesseldorf, 40225 Duesseldorf, Germany
| | - Katrin Amunts
- Institute of Neuroscience and Medicine (INM-1), Research Centre Juelich, 52428 Juelich, Germany; (K.A.); (M.M.)
- C. and O. Vogt Institute for Brain Research, Medical Faculty, University Hospital Duesseldorf, Heinrich-Heine University Duesseldorf, 40225 Duesseldorf, Germany
| | - Juergen Dukart
- Institute of Neuroscience and Medicine, Brain & Behaviour (INM-7), Research Centre Juelich, 52428 Juelich, Germany; (M.S.F.); (J.D.)
- Institute of Systems Neuroscience, Medical Faculty, Heinrich-Heine University Duesseldorf, 40225 Duesseldorf, Germany
| | - Martina Minnerop
- Institute of Neuroscience and Medicine (INM-1), Research Centre Juelich, 52428 Juelich, Germany; (K.A.); (M.M.)
- Department of Neurology, Center for Movement Disorders and Neuromodulation, Medical Faculty, Heinrich-Heine University Duesseldorf, 40225 Duesseldorf, Germany;
- Institute of Clinical Neuroscience and Medical Psychology, Medical Faculty, Heinrich-Heine University Duesseldorf, 40225 Duesseldorf, Germany
| |
Collapse
|
6
|
Liu X, Cruz Rivera S, Moher D, Calvert MJ, Denniston AK. Reporting guidelines for clinical trial reports for interventions involving artificial intelligence: the CONSORT-AI extension. Lancet Digit Health 2020; 2:e537-e548. [PMID: 33328048 PMCID: PMC8183333 DOI: 10.1016/s2589-7500(20)30218-1] [Citation(s) in RCA: 103] [Impact Index Per Article: 25.8] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/04/2020] [Accepted: 08/20/2020] [Indexed: 02/06/2023]
Abstract
The CONSORT 2010 statement provides minimum guidelines for reporting randomised trials. Its widespread use has been instrumental in ensuring transparency in the evaluation of new interventions. More recently, there has been a growing recognition that interventions involving artificial intelligence (AI) need to undergo rigorous, prospective evaluation to demonstrate impact on health outcomes. The CONSORT-AI (Consolidated Standards of Reporting Trials-Artificial Intelligence) extension is a new reporting guideline for clinical trials evaluating interventions with an AI component. It was developed in parallel with its companion statement for clinical trial protocols: SPIRIT-AI (Standard Protocol Items: Recommendations for Interventional Trials-Artificial Intelligence). Both guidelines were developed through a staged consensus process involving literature review and expert consultation to generate 29 candidate items, which were assessed by an international multi-stakeholder group in a two-stage Delphi survey (103 stakeholders), agreed upon in a two-day consensus meeting (31 stakeholders), and refined through a checklist pilot (34 participants). The CONSORT-AI extension includes 14 new items that were considered sufficiently important for AI interventions that they should be routinely reported in addition to the core CONSORT 2010 items. CONSORT-AI recommends that investigators provide clear descriptions of the AI intervention, including instructions and skills required for use, the setting in which the AI intervention is integrated, the handling of inputs and outputs of the AI intervention, the human-AI interaction and provision of an analysis of error cases. CONSORT-AI will help promote transparency and completeness in reporting clinical trials for AI interventions. It will assist editors and peer reviewers, as well as the general readership, to understand, interpret, and critically appraise the quality of clinical trial design and risk of bias in the reported outcomes.
Collapse
Affiliation(s)
- Xiaoxuan Liu
- Academic Unit of Ophthalmology, Institute of Inflammation and Ageing, College of Medical and Dental Sciences, University of Birmingham, Birmingham, UK; Birmingham Health Partners Centre for Regulatory Science and Innovation, University of Birmingham, Birmingham, UK; Department of Ophthalmology, University Hospitals Birmingham NHS Foundation Trust, Birmingham, UK; Moorfields Eye Hospital NHS Foundation Trust, London, UK; Health Data Research UK, London, UK
| | - Samantha Cruz Rivera
- Centre for Patient Reported Outcome Research, Institute of Applied Health Research, University of Birmingham, Birmingham, UK; Birmingham Health Partners Centre for Regulatory Science and Innovation, University of Birmingham, Birmingham, UK
| | - David Moher
- Centre for Journalology, Clinical Epidemiology Program, Ottawa Hospital Research Institute, Ottawa, ON, Canada; School of Epidemiology and Public Health, University of Ottawa, Ottawa, ON, Canada
| | - Melanie J Calvert
- Centre for Patient Reported Outcome Research, Institute of Applied Health Research, University of Birmingham, Birmingham, UK; Birmingham Health Partners Centre for Regulatory Science and Innovation, University of Birmingham, Birmingham, UK; Health Data Research UK, London, UK; National National Institute of Health Research Surgical Reconstruction and Microbiology Centre, and National Institute of Health Research Birmingham Biomedical Research Centre, University of Birmingham and University Hospitals Birmingham NHS Foundation Trust, Birmingham, UK; National Institute of Health Research Applied Research Collaborative West Midlands, Birmingham, UK
| | - Alastair K Denniston
- Academic Unit of Ophthalmology, Institute of Inflammation and Ageing, College of Medical and Dental Sciences, University of Birmingham, Birmingham, UK; Centre for Patient Reported Outcome Research, Institute of Applied Health Research, University of Birmingham, Birmingham, UK; Birmingham Health Partners Centre for Regulatory Science and Innovation, University of Birmingham, Birmingham, UK; Department of Ophthalmology, University Hospitals Birmingham NHS Foundation Trust, Birmingham, UK; National Institute of Health Research Biomedical Research Centre for Ophthalmology, Moorfields Eye Hospital London NHS Foundation Trust and University College London, Institute of Ophthalmology, London, UK; Health Data Research UK, London, UK.
| |
Collapse
|
7
|
Rivera SC, Liu X, Chan AW, Denniston AK, Calvert MJ. Guidelines for clinical trial protocols for interventions involving artificial intelligence: the SPIRIT-AI Extension. BMJ 2020; 370:m3210. [PMID: 32907797 PMCID: PMC7490785 DOI: 10.1136/bmj.m3210] [Citation(s) in RCA: 124] [Impact Index Per Article: 31.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Accepted: 08/04/2020] [Indexed: 02/06/2023]
Abstract
The SPIRIT 2013 (The Standard Protocol Items: Recommendations for Interventional Trials) statement aims to improve the completeness of clinical trial protocol reporting, by providing evidence-based recommendations for the minimum set of items to be addressed. This guidance has been instrumental in promoting transparent evaluation of new interventions. More recently, there is a growing recognition that interventions involving artificial intelligence need to undergo rigorous, prospective evaluation to demonstrate their impact on health outcomes.The SPIRIT-AI extension is a new reporting guideline for clinical trials protocols evaluating interventions with an AI component. It was developed in parallel with its companion statement for trial reports: CONSORT-AI. Both guidelines were developed using a staged consensus process, involving a literature review and expert consultation to generate 26 candidate items, which were consulted on by an international multi-stakeholder group in a 2-stage Delphi survey (103 stakeholders), agreed on in a consensus meeting (31 stakeholders) and refined through a checklist pilot (34 participants).The SPIRIT-AI extension includes 15 new items, which were considered sufficiently important for clinical trial protocols of AI interventions. These new items should be routinely reported in addition to the core SPIRIT 2013 items. SPIRIT-AI recommends that investigators provide clear descriptions of the AI intervention, including instructions and skills required for use, the setting in which the AI intervention will be integrated, considerations around the handling of input and output data, the human-AI interaction and analysis of error cases.SPIRIT-AI will help promote transparency and completeness for clinical trial protocols for AI interventions. Its use will assist editors and peer-reviewers, as well as the general readership, to understand, interpret and critically appraise the design and risk of bias for a planned clinical trial.
Collapse
Affiliation(s)
- Samantha Cruz Rivera
- Centre for Patient Reported Outcome Research, Institute of Applied Health Research, University of Birmingham, Birmingham, UK
- Birmingham Health Partners Centre for Regulatory Science and Innovation, University of Birmingham, Birmingham, UK
| | - Xiaoxuan Liu
- Birmingham Health Partners Centre for Regulatory Science and Innovation, University of Birmingham, Birmingham, UK
- Academic Unit of Ophthalmology, Institute of Inflammation and Ageing, College of Medical and Dental Sciences, University of Birmingham, UK
- Department of Ophthalmology, University Hospitals Birmingham NHS Foundation Trust, Birmingham, UK
- Moorfields Eye Hospital NHS Foundation Trust, London, UK
- Health Data Research UK, London, UK
| | - An-Wen Chan
- Department of Medicine, Women's College Research Institute, Women's College Hospital, University of Toronto, Ontario, Canada
| | - Alastair K Denniston
- Centre for Patient Reported Outcome Research, Institute of Applied Health Research, University of Birmingham, Birmingham, UK.
- Birmingham Health Partners Centre for Regulatory Science and Innovation, University of Birmingham, Birmingham, UK
- Academic Unit of Ophthalmology, Institute of Inflammation and Ageing, College of Medical and Dental Sciences, University of Birmingham, UK
- Department of Ophthalmology, University Hospitals Birmingham NHS Foundation Trust, Birmingham, UK
- Moorfields Eye Hospital NHS Foundation Trust, London, UK
- National Institute of Health Research Biomedical Research Centre for Ophthalmology, Moorfields Hospital London NHS Foundation Trust and University College London, Institute of Ophthalmology, London, UK
| | - Melanie J Calvert
- Centre for Patient Reported Outcome Research, Institute of Applied Health Research, University of Birmingham, Birmingham, UK
- Birmingham Health Partners Centre for Regulatory Science and Innovation, University of Birmingham, Birmingham, UK
- Health Data Research UK, London, UK
- National Institute of Health Research Surgical Reconstruction and Microbiology Centre, University of Birmingham and University Hospitals Birmingham NHS Foundation Trust, Birmingham, UK
- National Institute of Health Research Birmingham Biomedical Research Centre, University of Birmingham and University Hospitals Birmingham NHS Foundation Trust, Birmingham, UK
- National Institute of Health Research Applied Research Collaborative West Midlands, Birmingham, UK
| |
Collapse
|
8
|
Cruz Rivera S, Liu X, Chan AW, Denniston AK, Calvert MJ. Guidelines for clinical trial protocols for interventions involving artificial intelligence: the SPIRIT-AI extension. LANCET DIGITAL HEALTH 2020; 2:e549-e560. [PMID: 33328049 PMCID: PMC8212701 DOI: 10.1016/s2589-7500(20)30219-3] [Citation(s) in RCA: 120] [Impact Index Per Article: 30.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 08/04/2020] [Accepted: 08/20/2020] [Indexed: 12/13/2022]
Abstract
The SPIRIT 2013 statement aims to improve the completeness of clinical trial protocol reporting by providing evidence-based recommendations for the minimum set of items to be addressed. This guidance has been instrumental in promoting transparent evaluation of new interventions. More recently, there has been a growing recognition that interventions involving artificial intelligence (AI) need to undergo rigorous, prospective evaluation to demonstrate their impact on health outcomes. The SPIRIT-AI (Standard Protocol Items: Recommendations for Interventional Trials-Artificial Intelligence) extension is a new reporting guideline for clinical trial protocols evaluating interventions with an AI component. It was developed in parallel with its companion statement for trial reports: CONSORT-AI (Consolidated Standards of Reporting Trials-Artificial Intelligence). Both guidelines were developed through a staged consensus process involving literature review and expert consultation to generate 26 candidate items, which were consulted upon by an international multi-stakeholder group in a two-stage Delphi survey (103 stakeholders), agreed upon in a consensus meeting (31 stakeholders) and refined through a checklist pilot (34 participants). The SPIRIT-AI extension includes 15 new items that were considered sufficiently important for clinical trial protocols of AI interventions. These new items should be routinely reported in addition to the core SPIRIT 2013 items. SPIRIT-AI recommends that investigators provide clear descriptions of the AI intervention, including instructions and skills required for use, the setting in which the AI intervention will be integrated, considerations for the handling of input and output data, the human–AI interaction and analysis of error cases. SPIRIT-AI will help promote transparency and completeness for clinical trial protocols for AI interventions. Its use will assist editors and peer reviewers, as well as the general readership, to understand, interpret, and critically appraise the design and risk of bias for a planned clinical trial.
Collapse
Affiliation(s)
- Samantha Cruz Rivera
- Centre for Patient Reported Outcome Research, Institute of Applied Health Research, University of Birmingham, Birmingham, UK; Birmingham Health Partners Centre for Regulatory Science and Innovation, University of Birmingham, Birmingham, UK
| | - Xiaoxuan Liu
- Academic Unit of Ophthalmology, Institute of Inflammation and Ageing, College of Medical and Dental Sciences, University of Birmingham, Birmingham, UK; Birmingham Health Partners Centre for Regulatory Science and Innovation, University of Birmingham, Birmingham, UK; Department of Ophthalmology, University Hospitals Birmingham NHS Foundation Trust, Birmingham, UK; Moorfields Eye Hospital NHS Foundation Trust, London, UK; Health Data Research UK, London, UK
| | - An-Wen Chan
- Department of Medicine, Women's College Research Institute, Women's College Hospital, University of Toronto, Toronto, ON, Canada
| | - Alastair K Denniston
- Centre for Patient Reported Outcome Research, Institute of Applied Health Research, University of Birmingham, Birmingham, UK; Academic Unit of Ophthalmology, Institute of Inflammation and Ageing, College of Medical and Dental Sciences, University of Birmingham, Birmingham, UK; Birmingham Health Partners Centre for Regulatory Science and Innovation, University of Birmingham, Birmingham, UK; Department of Ophthalmology, University Hospitals Birmingham NHS Foundation Trust, Birmingham, UK; National Institute of Health Research Biomedical Research Centre for Ophthalmology, Moorfields Hospital London NHS Foundation Trust and University College London, Institute of Ophthalmology, London, UK; Health Data Research UK, London, UK.
| | - Melanie J Calvert
- Centre for Patient Reported Outcome Research, Institute of Applied Health Research, University of Birmingham, Birmingham, UK; Birmingham Health Partners Centre for Regulatory Science and Innovation, University of Birmingham, Birmingham, UK; Health Data Research UK, London, UK; National Institute of Health Research Surgical Reconstruction and Microbiology Centre, and National Institute of Health Research Birmingham Biomedical Research Centre, University of Birmingham and University Hospitals Birmingham NHS Foundation Trust, Birmingham, UK; National Institute of Health Research Applied Research Collaborative West Midlands, Birmingham, UK
| |
Collapse
|
9
|
Liu X, Rivera SC, Moher D, Calvert MJ, Denniston AK. Reporting guidelines for clinical trial reports for interventions involving artificial intelligence: the CONSORT-AI Extension. BMJ 2020; 370:m3164. [PMID: 32909959 PMCID: PMC7490784 DOI: 10.1136/bmj.m3164] [Citation(s) in RCA: 160] [Impact Index Per Article: 40.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Accepted: 08/04/2020] [Indexed: 02/07/2023]
Abstract
The CONSORT 2010 (Consolidated Standards of Reporting Trials) statement provides minimum guidelines for reporting randomised trials. Its widespread use has been instrumental in ensuring transparency when evaluating new interventions. More recently, there has been a growing recognition that interventions involving artificial intelligence (AI) need to undergo rigorous, prospective evaluation to demonstrate impact on health outcomes.The CONSORT-AI extension is a new reporting guideline for clinical trials evaluating interventions with an AI component. It was developed in parallel with its companion statement for clinical trial protocols: SPIRIT-AI. Both guidelines were developed through a staged consensus process, involving a literature review and expert consultation to generate 29 candidate items, which were assessed by an international multi-stakeholder group in a two-stage Delphi survey (103 stakeholders), agreed on in a two-day consensus meeting (31 stakeholders) and refined through a checklist pilot (34 participants).The CONSORT-AI extension includes 14 new items, which were considered sufficiently important for AI interventions, that they should be routinely reported in addition to the core CONSORT 2010 items. CONSORT-AI recommends that investigators provide clear descriptions of the AI intervention, including instructions and skills required for use, the setting in which the AI intervention is integrated, the handling of inputs and outputs of the AI intervention, the human-AI interaction and providing analysis of error cases.CONSORT-AI will help promote transparency and completeness in reporting clinical trials for AI interventions. It will assist editors and peer-reviewers, as well as the general readership, to understand, interpret and critically appraise the quality of clinical trial design and risk of bias in the reported outcomes.
Collapse
Affiliation(s)
- Xiaoxuan Liu
- Academic Unit of Ophthalmology, Institute of Inflammation and Ageing, College of Medical and Dental Sciences, University of Birmingham, UK
- Department of Ophthalmology, University Hospitals Birmingham NHS Foundation Trust, Birmingham, UK
- Moorfields Eye Hospital NHS Foundation Trust, London, UK
- Health Data Research UK, London, UK
- Birmingham Health Partners Centre for Regulatory Science and Innovation, University of Birmingham, Birmingham, UK
| | - Samantha Cruz Rivera
- Birmingham Health Partners Centre for Regulatory Science and Innovation, University of Birmingham, Birmingham, UK
- Centre for Patient Reported Outcome Research, Institute of Applied Health Research, University of Birmingham, Birmingham, UK
| | - David Moher
- Centre for Journalology, Clinical Epidemiology Program, Ottawa Hospital Research Institute, Ottawa, Canada
- School of Epidemiology and Public Health, University of Ottawa, Ottawa, Canada
| | - Melanie J Calvert
- Health Data Research UK, London, UK
- Birmingham Health Partners Centre for Regulatory Science and Innovation, University of Birmingham, Birmingham, UK
- Centre for Patient Reported Outcome Research, Institute of Applied Health Research, University of Birmingham, Birmingham, UK
- National Institute of Health Research Surgical Reconstruction and Microbiology Centre, University of Birmingham and University Hospitals Birmingham NHS Foundation Trust, Birmingham, UK
- National Institute of Health Research Birmingham Biomedical Research Centre, University of Birmingham and University Hospitals Birmingham NHS Foundation Trust, Birmingham, UK
- National Institute of Health Research Applied Research Collaborative West Midlands, Birmingham, UK
| | - Alastair K Denniston
- Academic Unit of Ophthalmology, Institute of Inflammation and Ageing, College of Medical and Dental Sciences, University of Birmingham, UK
- Department of Ophthalmology, University Hospitals Birmingham NHS Foundation Trust, Birmingham, UK
- Health Data Research UK, London, UK
- Birmingham Health Partners Centre for Regulatory Science and Innovation, University of Birmingham, Birmingham, UK
- Centre for Patient Reported Outcome Research, Institute of Applied Health Research, University of Birmingham, Birmingham, UK
- National Institute of Health Research Biomedical Research Centre for Ophthalmology, Moorfields Hospital London NHS Foundation Trust and University College London, Institute of Ophthalmology, London, UK
| |
Collapse
|
10
|
Cruz Rivera S, Liu X, Chan AW, Denniston AK, Calvert MJ. Guidelines for clinical trial protocols for interventions involving artificial intelligence: the SPIRIT-AI extension. Nat Med 2020; 26:1351-1363. [PMID: 32908284 PMCID: PMC7598944 DOI: 10.1038/s41591-020-1037-7] [Citation(s) in RCA: 207] [Impact Index Per Article: 51.8] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/24/2020] [Accepted: 07/23/2020] [Indexed: 02/06/2023]
Abstract
The SPIRIT 2013 statement aims to improve the completeness of clinical trial protocol reporting by providing evidence-based recommendations for the minimum set of items to be addressed. This guidance has been instrumental in promoting transparent evaluation of new interventions. More recently, there has been a growing recognition that interventions involving artificial intelligence (AI) need to undergo rigorous, prospective evaluation to demonstrate their impact on health outcomes. The SPIRIT-AI (Standard Protocol Items: Recommendations for Interventional Trials-Artificial Intelligence) extension is a new reporting guideline for clinical trial protocols evaluating interventions with an AI component. It was developed in parallel with its companion statement for trial reports: CONSORT-AI (Consolidated Standards of Reporting Trials-Artificial Intelligence). Both guidelines were developed through a staged consensus process involving literature review and expert consultation to generate 26 candidate items, which were consulted upon by an international multi-stakeholder group in a two-stage Delphi survey (103 stakeholders), agreed upon in a consensus meeting (31 stakeholders) and refined through a checklist pilot (34 participants). The SPIRIT-AI extension includes 15 new items that were considered sufficiently important for clinical trial protocols of AI interventions. These new items should be routinely reported in addition to the core SPIRIT 2013 items. SPIRIT-AI recommends that investigators provide clear descriptions of the AI intervention, including instructions and skills required for use, the setting in which the AI intervention will be integrated, considerations for the handling of input and output data, the human-AI interaction and analysis of error cases. SPIRIT-AI will help promote transparency and completeness for clinical trial protocols for AI interventions. Its use will assist editors and peer reviewers, as well as the general readership, to understand, interpret and critically appraise the design and risk of bias for a planned clinical trial.
Collapse
Affiliation(s)
- Samantha Cruz Rivera
- Centre for Patient Reported Outcomes Research, Institute of Applied Health Research, University of Birmingham, Birmingham, UK
- Institute of Applied Health Research, University of Birmingham, Birmingham, UK
- Birmingham Health Partners Centre for Regulatory Science and Innovation, University of Birmingham, Birmingham, UK
| | - Xiaoxuan Liu
- Birmingham Health Partners Centre for Regulatory Science and Innovation, University of Birmingham, Birmingham, UK
- Academic Unit of Ophthalmology, Institute of Inflammation and Ageing, University of Birmingham, Birmingham, UK
- University Hospitals Birmingham NHS Foundation Trust, Birmingham, UK
- Health Data Research UK, London, UK
- Moorfields Eye Hospital NHS Foundation Trust, London, UK
| | - An-Wen Chan
- Department of Medicine, Women's College Research Institute, Women's College Hospital, University of Toronto, Ontario, Canada
| | - Alastair K Denniston
- Centre for Patient Reported Outcomes Research, Institute of Applied Health Research, University of Birmingham, Birmingham, UK.
- Birmingham Health Partners Centre for Regulatory Science and Innovation, University of Birmingham, Birmingham, UK.
- Academic Unit of Ophthalmology, Institute of Inflammation and Ageing, University of Birmingham, Birmingham, UK.
- University Hospitals Birmingham NHS Foundation Trust, Birmingham, UK.
- Health Data Research UK, London, UK.
- National Institute of Health Research Biomedical Research Centre for Ophthalmology, Moorfields Hospital London NHS Foundation Trust and University College London, Institute of Ophthalmology, London, UK.
| | - Melanie J Calvert
- Centre for Patient Reported Outcomes Research, Institute of Applied Health Research, University of Birmingham, Birmingham, UK
- Institute of Applied Health Research, University of Birmingham, Birmingham, UK
- Birmingham Health Partners Centre for Regulatory Science and Innovation, University of Birmingham, Birmingham, UK
- Health Data Research UK, London, UK
- National Institute of Health Research Birmingham Biomedical Research Centre, University of Birmingham and University Hospitals Birmingham NHS Foundation Trust, Birmingham, UK
- National Institute of Health Research Applied Research Collaborative West Midlands, Coventry, UK
- National Institute of Health Research Surgical Reconstruction and Microbiology Centre, University of Birmingham and University Hospitals Birmingham NHS Foundation Trust, Birmingham, UK
| |
Collapse
|
11
|
Liu X, Cruz Rivera S, Moher D, Calvert MJ, Denniston AK. Reporting guidelines for clinical trial reports for interventions involving artificial intelligence: the CONSORT-AI extension. Nat Med 2020; 26:1364-1374. [PMID: 32908283 PMCID: PMC7598943 DOI: 10.1038/s41591-020-1034-x] [Citation(s) in RCA: 312] [Impact Index Per Article: 78.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/24/2020] [Accepted: 07/23/2020] [Indexed: 02/07/2023]
Abstract
The CONSORT 2010 statement provides minimum guidelines for reporting randomized trials. Its widespread use has been instrumental in ensuring transparency in the evaluation of new interventions. More recently, there has been a growing recognition that interventions involving artificial intelligence (AI) need to undergo rigorous, prospective evaluation to demonstrate impact on health outcomes. The CONSORT-AI (Consolidated Standards of Reporting Trials-Artificial Intelligence) extension is a new reporting guideline for clinical trials evaluating interventions with an AI component. It was developed in parallel with its companion statement for clinical trial protocols: SPIRIT-AI (Standard Protocol Items: Recommendations for Interventional Trials-Artificial Intelligence). Both guidelines were developed through a staged consensus process involving literature review and expert consultation to generate 29 candidate items, which were assessed by an international multi-stakeholder group in a two-stage Delphi survey (103 stakeholders), agreed upon in a two-day consensus meeting (31 stakeholders) and refined through a checklist pilot (34 participants). The CONSORT-AI extension includes 14 new items that were considered sufficiently important for AI interventions that they should be routinely reported in addition to the core CONSORT 2010 items. CONSORT-AI recommends that investigators provide clear descriptions of the AI intervention, including instructions and skills required for use, the setting in which the AI intervention is integrated, the handling of inputs and outputs of the AI intervention, the human-AI interaction and provision of an analysis of error cases. CONSORT-AI will help promote transparency and completeness in reporting clinical trials for AI interventions. It will assist editors and peer reviewers, as well as the general readership, to understand, interpret and critically appraise the quality of clinical trial design and risk of bias in the reported outcomes.
Collapse
Affiliation(s)
- Xiaoxuan Liu
- Moorfields Eye Hospital NHS Foundation Trust, London, UK
- Academic Unit of Ophthalmology, Institute of Inflammation and Ageing, University of Birmingham, Birmingham, UK
- University Hospitals Birmingham NHS Foundation Trust, Birmingham, UK
- Health Data Research UK, London, UK
- Birmingham Health Partners Centre for Regulatory Science and Innovation, University of Birmingham, Birmingham, UK
| | - Samantha Cruz Rivera
- Birmingham Health Partners Centre for Regulatory Science and Innovation, University of Birmingham, Birmingham, UK
- Centre for Patient Reported Outcomes Research, Institute of Applied Health Research, University of Birmingham, Birmingham, UK
- Institute of Applied Health Research, University of Birmingham, Birmingham, UK
| | - David Moher
- Centre for Journalology, Clinical Epidemiology Program, Ottawa Hospital Research Institute, Ottawa, Canada
- School of Epidemiology and Public Health, Faculty of Medicine, University of Ottawa, Ottawa, Canada
| | - Melanie J Calvert
- Health Data Research UK, London, UK
- Birmingham Health Partners Centre for Regulatory Science and Innovation, University of Birmingham, Birmingham, UK
- Centre for Patient Reported Outcomes Research, Institute of Applied Health Research, University of Birmingham, Birmingham, UK
- Institute of Applied Health Research, University of Birmingham, Birmingham, UK
- National Institute of Health Research Birmingham Biomedical Research Centre, University of Birmingham and University Hospitals Birmingham NHS Foundation Trust, Birmingham, UK
- National Institute of Health Research Applied Research Collaborative West Midlands, Coventry, UK
- National Institute of Health Research Surgical Reconstruction and Microbiology Centre, University of Birmingham and University Hospitals Birmingham NHS Foundation Trust, Birmingham, UK
| | - Alastair K Denniston
- Academic Unit of Ophthalmology, Institute of Inflammation and Ageing, University of Birmingham, Birmingham, UK.
- University Hospitals Birmingham NHS Foundation Trust, Birmingham, UK.
- Health Data Research UK, London, UK.
- Birmingham Health Partners Centre for Regulatory Science and Innovation, University of Birmingham, Birmingham, UK.
- Centre for Patient Reported Outcomes Research, Institute of Applied Health Research, University of Birmingham, Birmingham, UK.
- NIHR Biomedical Research Center at Moorfields Eye Hospital NHS Foundation Trust and UCL Institute of Ophthalmology, London, UK.
| |
Collapse
|
12
|
Naeemabadi MR, Fazlali H, Najafi S, Dinesen B, Hansen J. Telerehabilitation for Patients With Knee Osteoarthritis: A Focused Review of Technologies and Teleservices. JMIR BIOMEDICAL ENGINEERING 2020. [DOI: 10.2196/16991] [Citation(s) in RCA: 8] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/17/2023] Open
Abstract
Background
Telerehabilitation programs are designed with the aim of improving the quality of services as well as overcoming existing limitations in terms of resource management and accessibility of services. This review will collect recent studies investigating telerehabilitation programs for patients with knee osteoarthritis while focusing on the technologies and services provided in the programs.
Objective
The main objective of this review is to identify and discuss the modes of service delivery and technologies in telerehabilitation programs for patients with knee osteoarthritis. The gaps, strengths, and weaknesses of programs will be discussed individually.
Methods
Studies published in English since 2000 were retrieved from the EMBASE, Scopus, Web of Science, Cumulative Index to Nursing and Allied Health Literature (CINAHL), PubMed, Physiotherapy Evidence Database (PEDro), and PsycINFO databases. The search words “telerehabilitation,” “telehealth,” “telemedicine,” “teletherapy,” and “ehealth” were combined with “knee” and “rehabilitation” to generate a data set of studies for screening and review. The final group of studies reviewed here includes those that implemented teletreatment for patients for at least 2 weeks of rehabilitation.
Results
In total, 1198 studies were screened, and the full text of 154 studies was reviewed. Of these, 38 studies were included, and data were extracted accordingly. Four modes of telerehabilitation service delivery were identified: phone-based, video-based, sensor-based, and expert system–based telerehabilitation. The intervention services provided in the studies included information, training, communication, monitoring, and tracking. Video-based telerehabilitation programs were frequently used. Among the identified services, information and educational material were introduced in only one-quarter of the studies.
Conclusions
Video-based telerehabilitation programs can be considered the best alternative solution to conventional treatment. This study shows that, in recent years, sensor-based solutions have also become more popular due to rapid developments in sensor technology. Nevertheless, communication and human-generated feedback remain as important as monitoring and intervention services.
Collapse
|
13
|
Abstract
OBJECTIVE To evaluate the effect of education interventions compared with any type of comparator on managing patellofemoral pain (PFP). DESIGN Intervention systematic review. PROSPERO identifier: CRD42018088671. LITERATURE SEARCH MEDLINE, Embase, CINAHL, and Web of Science were searched for studies evaluating the effect of education on clinical and functional outcomes in people with PFP. STUDY SELECTION CRITERIA Two reviewers independently assessed studies for inclusion and quality. We included randomized controlled trials on PFP where at least 1 group received an education intervention (in isolation or in combination with other interventions). DATA SYNTHESIS Available data were synthesized via meta-analysis where possible; data that were not appropriate for pooling were synthesized qualitatively. Interpretation was guided by the Grading of Recommendations Assessment, Development and Evaluation approach. RESULTS Nine trials were identified. Low-credibility evidence indicated that health education material alone was inferior to exercise therapy for pain and function outcomes. Low- and very low-credibility evidence indicated that health professional-delivered education alone produced outcomes similar to those of exercise therapy combined with health professional-delivered education for pain and function, respectively. CONCLUSION Health professional-delivered education may produce similar outcomes in pain and function compared to exercise therapy plus health professional-delivered education in people with PFP. J Orthop Sports Phys Ther 2020;50(7):388-396. Epub 29 Apr 2020. doi:10.2519/jospt.2020.9400.
Collapse
|
14
|
Demographics and rates of surgical arthroscopy and postoperative rehabilitative preferences of arthroscopists from the Arthroscopy Association of North America (AANA). J Orthop 2018; 15:591-595. [PMID: 29881200 DOI: 10.1016/j.jor.2018.05.033] [Citation(s) in RCA: 16] [Impact Index Per Article: 2.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 03/30/2018] [Accepted: 05/06/2018] [Indexed: 11/21/2022] Open
Abstract
Survey of 869 arthroscopists regarding joint-specific arthroscopic procedures and postoperative rehabilitative preferences revealed comparable support for use of supervised physical therapy (SPT) and home exercise programs (HEPs) but stronger preference for joint-specific HEP applications (wrist, knee). Among respondents utilizing HEPs, modality of delivery (verbal/handout/web-based) didn't differ by joint, yet only 2.9% utilized web-based HEPs. This is the first known study to identify postoperative rehabilitation preferences. With 1.77 million estimated arthroscopic procedures annually (mean: 325.4 procedures/respondent), this study highlights under-utilization of web-based HEPs. Reliable, web-based HEPs can improve post-arthroscopic outcomes for patients, arthroscopic surgeons, and rehabilitative specialists while being cost efficient.
Collapse
|
15
|
Bright P, Hambly K. Patients Using an Online Forum for Reporting Progress When Engaging With a Six-Week Exercise Program for Knee Conditioning: Feasibility Study. JMIR Rehabil Assist Technol 2018; 5:e9. [PMID: 29699967 PMCID: PMC5945989 DOI: 10.2196/rehab.8567] [Citation(s) in RCA: 4] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/30/2017] [Revised: 02/26/2018] [Accepted: 03/16/2018] [Indexed: 12/21/2022] Open
Abstract
BACKGROUND The use of electronic health (eHealth) and Web-based resources for patients with knee pain is expanding. Padlet is an online noticeboard that can facilitate patient interaction by posting virtual “sticky notes.” OBJECTIVE The primary aim of this study was to determine feasibility of patients in a 6-week knee exercise program using Padlet as an online forum for self-reporting on outcome progression. METHODS Undergraduate manual therapy students were recruited as part of a 6-week study into knee conditioning. Participants were encouraged to post maximum effort readings from quadriceps and gluteal home exercises captured from standard bathroom scales on a bespoke Padlet. Experience and progression reporting were encouraged. Posted data were analyzed for association between engagement, entry frequency, and participant characteristics. Individual data facilitated single-subject, multiple-baseline analysis using statistical process control. Experiential narrative was analyzed thematically. RESULTS Nineteen participants were recruited (47%, 9/19 female); ages ranged from 19 to 53 years. Twelve individuals (63%) opted to engage with the forum (range 4-40 entries), with five (42%) reporting across all 6 weeks. Gender did not influence reporting (odds ratio [OR] 0.76, 95% CI 0.06-6.93). No significant difference manifested between body mass index and engagement P=.46); age and entry frequency did not correlate (R2=.054, 95% CI –0.42 to 0.51, P=.83). Statistically significant conditioning profiles arose in single participants. Themes of pain, mitigation, and response were inducted from the experiences posted. CONCLUSIONS Patients will engage with an online forum for reporting progress when undertaking exercise programs. In contrast to related literature, no significant association was found with reporting and gender, age, or body mass index. Individual posted data allowed multiple-baseline analysis and experiential induction from participants. Conditioning responses were evident on visual inspection. The importance of individualized visual data to patients and the role of forums in monitoring patients’ progress in symptomatic knee pain populations need further consideration.
Collapse
Affiliation(s)
- Philip Bright
- School of Sport and Exercise Sciences, Medway Campus, University of Kent, Chatham, United Kingdom.,Research Department, European School of Osteopathy, Maidstone, United Kingdom
| | - Karen Hambly
- School of Sport and Exercise Sciences, Medway Campus, University of Kent, Maidstone, United Kingdom
| |
Collapse
|
16
|
Bright P, Hambly K. What Is the Proportion of Studies Reporting Patient and Practitioner Satisfaction with Software Support Tools Used in the Management of Knee Pain and Is This Related to Sample Size, Effect Size, and Journal Impact Factor? Telemed J E Health 2017; 24:562-576. [PMID: 29265954 DOI: 10.1089/tmj.2017.0207] [Citation(s) in RCA: 5] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/27/2023] Open
Abstract
INTRODUCTION E-health software tools have been deployed in managing knee conditions. Reporting of patient and practitioner satisfaction in studies regarding e-health usage is not widely explored. The objective of this review was to identify studies describing patient and practitioner satisfaction with software use concerning knee pain. MATERIALS AND METHODS A computerized search was undertaken: four electronic databases were searched from January 2007 until January 2017. Keywords were decision dashboard, clinical decision, Web-based resource, evidence support, and knee. Full texts were scanned for effect of size reporting and satisfaction scales from participants and practitioners. Binary regression was run; impact factor and sample size were predictors with indicators for satisfaction and effect size reporting as dependent variables. RESULTS Seventy-seven articles were retrieved; 37 studies were included in final analysis. Ten studies reported patient satisfaction ratings (27.8%): a single study reported both patient and practitioner satisfaction (2.8%). Randomized control trials were the most common design (35%) and knee osteoarthritis the most prevalent condition (38%). Electronic patient-reported outcome measures and Web-based training were the most common interventions. No significant dependency was found within the regression models (p > 0.05). DISCUSSION AND CONCLUSIONS The proportion of reporting of patient satisfaction was low; practitioner satisfaction was poorly represented. There may be implications for the suitability of administering e-health, a medium for capturing further meta-evidence needs to be established and used as best practice for implicated studies in future. This is the first review of its kind to address patient and practitioner satisfaction with knee e-health.
Collapse
Affiliation(s)
- Philip Bright
- 1 Research Department, European School of Osteopathy , Kent, United Kingdom
- 2 School of Sports and Exercise Sciences, University of Kent at Medway , Kent, United Kingdom
| | - Karen Hambly
- 2 School of Sports and Exercise Sciences, University of Kent at Medway , Kent, United Kingdom
| |
Collapse
|