1
|
Low GK, Subedi S, Omosumwen OF, Jiee SF, Devkota S, Shanmuganathan S, Doyle Z. Development and validation of observational and qualitative study protocol reporting checklists for novice researchers (ObsQual checklist). EVALUATION AND PROGRAM PLANNING 2024; 106:102468. [PMID: 39029287 DOI: 10.1016/j.evalprogplan.2024.102468] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 05/09/2024] [Revised: 06/20/2024] [Accepted: 07/15/2024] [Indexed: 07/21/2024]
Abstract
INTRODUCTION Currently, no reporting guidelines exist for observational and qualitative study protocols. In an effort to enhance the quality of research protocols, we introduce two study protocol reporting checklists that we have developed. MAIN RECOMMENDATIONS These checklists include educational components and examples intended to assist novice researchers. Through the analysis of 333 study protocols submitted for ethical review, our checklists have been developed and validated, demonstrating their applicability across various observational and qualitative study designs. CHANGES IN MANAGEMENT We provide insights into the systematic implementation of these checklists alongside complementary elements that support their effectiveness. We recommend longitudinal monitoring and evaluation of checklist utilization.
Collapse
Affiliation(s)
- Gary Kk Low
- Research Directorate, Nepean Hospital, Nepean Blue Mountain Local Health District, Derby St, Kingswood, NSW, 2750, Australia; Sydney Medical School, Faculty of Medicine and Health, University of Sydney, Australia.
| | - Sudarshan Subedi
- Department of Community Services, Torrens University of Australia, Adelaide, SA, Australia
| | | | - Sam Froze Jiee
- Department of Community Medicine and Public Health, Faculty of Medicine and Health Sciences, Universiti Malaysia Sarawak, Malaysia
| | | | - Selvanaayagam Shanmuganathan
- Menzies Centre for Health Policy and Economics, Faculty of Medicine and Health, The University of Sydney, Australia; Ministry of Health, Malaysia
| | - Zelda Doyle
- Rural Clinical School, School of Medicine, Faculty of Medicine, Nursing, Midwifery and Health Sciences, The University of Notre Dame, NSW, Australia
| |
Collapse
|
2
|
Matvienko-Sikar K, O'Shea J, Kennedy S, Thomas SD, Avery K, Byrne M, McHugh S, O' Connor DB, Saldanha IJ, Smith V, Toomey E, Dwan K, Kirkham JJ. Selective outcome reporting in trials of behavioural health interventions in health psychology and behavioural medicine journals: a review. Health Psychol Rev 2024:1-15. [PMID: 38923431 DOI: 10.1080/17437199.2024.2367613] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 11/27/2023] [Accepted: 06/09/2024] [Indexed: 06/28/2024]
Abstract
Selective outcome reporting can result in overestimation of treatment effects, research waste, and reduced openness and transparency. This review aimed to examine selective outcome reporting in trials of behavioural health interventions and determine potential outcome reporting bias. A review of nine health psychology and behavioural medicine journals was conducted to identify randomised controlled trials of behavioural health interventions published since 2019. Discrepancies in outcome reporting were observed in 90% of the 29 trials with corresponding registrations/protocols. Discrepancies included 72% of trials omitting prespecified outcomes; 55% of trials introduced new outcomes. Thirty-eight percent of trials omitted prespecified and introduced new outcomes. Three trials (10%) downgraded primary outcomes in registrations/protocols to secondary outcomes in final reports; downgraded outcomes were not statistically significant in two trials. Five trials (17%) upgraded secondary outcomes to primary outcomes; upgraded outcomes were statistically significant in all trials. In final reports, three trials (7%) omitted outcomes from the methods section; three trials (7%) introduced new outcomes in results that were not in the methods. These findings indicate that selective outcome reporting is a problem in behavioural health intervention trials. Journal- and trialist-level approaches are needed to minimise selective outcome reporting in health psychology and behavioural medicine.
Collapse
Affiliation(s)
| | - Jen O'Shea
- School of Applied Psychology, University College Cork, Cork, Ireland
| | | | - Siobhan D Thomas
- School of Applied Psychology, University College Cork, Cork, Ireland
| | - Kerry Avery
- Bristol Medical School: Population Health Sciences, University of Bristol, Bristol, UK
| | - Molly Byrne
- School of Psychology, University of Galway, Galway, Ireland
| | - Sheena McHugh
- School of Public Health, University College Cork, Cork, Ireland
| | | | - Ian J Saldanha
- Center for Clinical Trials and Evidence Synthesis, Johns Hopkins Bloomberg School of Public Health, Baltimore, MD, USA
| | - Valerie Smith
- School of Nursing, Midwifery, and Health Systems, University College Dublin, Dublin, Ireland
| | - Elaine Toomey
- School of Nursing & Midwifery, University College Galway, Galway, Ireland
| | - Kerry Dwan
- Liverpool School of Tropical Medicine, Liverpool, UK
| | - Jamie J Kirkham
- Centre for Biostatistics, The University of Manchester, Manchester Academic Health Science Centre, Manchester, UK
| |
Collapse
|
3
|
Rosengaard LO, Andersen MZ, Rosenberg J, Fonnes S. Five aspects of research waste in biomedicine: A scoping review. J Evid Based Med 2024; 17:351-359. [PMID: 38798014 DOI: 10.1111/jebm.12616] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 12/27/2023] [Accepted: 05/12/2024] [Indexed: 05/29/2024]
Abstract
BACKGROUND The number of published journal articles has grown exponentially during the last 30 years, which may have led to some wasteful research. However, the terminology associated with research waste remains unclear. To address this, we aimed to identify, define, and categorize the aspects of research waste in published biomedical reports. METHODS In this scoping review, we systematically searched for biomedical literature reports from 1993 to 2023 in two databases, focusing on those addressing and defining research waste. Through data charting, we analyzed and categorized the aspects of research waste. RESULTS Based on 4285 initial records in the searches, a total of 832 reports were included in the analysis. The included reports were primarily narrative reviews (26%) and original reports (21%). We categorized research waste into five aspects: methodological, invisible, negligible, underreported, and structural (MINUS) research waste. More than half of the reports (56%) covered methodological research waste concerning flaws in study design, study conduct, or analysis. Invisible research waste covered nonpublication, discontinuation, and lack of data-sharing. Negligible research waste primarily concerned unnecessary repetition, for example, stemming from the absence of preceding a trial with a systematic review of the literature. Underreported research waste mainly included poor reporting, resulting in a lack of transparency. Structural research waste comprised inadequate management, collaboration, prioritization, implementation, and dissemination. CONCLUSION MINUS encapsulates the five main aspects of research waste. Recognizing these aspects of research waste is important for addressing and preventing further research waste and thereby ensuring efficient resource allocation and scientific integrity.
Collapse
Affiliation(s)
- Louise Olsbro Rosengaard
- Center for Perioperative Optimization, Department of Surgery, Copenhagen University Hospital-Herlev and Gentofte, Herlev, Denmark
| | - Mikkel Zola Andersen
- Center for Perioperative Optimization, Department of Surgery, Copenhagen University Hospital-Herlev and Gentofte, Herlev, Denmark
| | - Jacob Rosenberg
- Center for Perioperative Optimization, Department of Surgery, Copenhagen University Hospital-Herlev and Gentofte, Herlev, Denmark
| | - Siv Fonnes
- Center for Perioperative Optimization, Department of Surgery, Copenhagen University Hospital-Herlev and Gentofte, Herlev, Denmark
| |
Collapse
|
4
|
Sammy A, Baba A, Klassen TP, Moher D, Offringa M. A Decade of Efforts to Add Value to Child Health Research Practices. J Pediatr 2024; 265:113840. [PMID: 38000771 DOI: 10.1016/j.jpeds.2023.113840] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 08/09/2023] [Revised: 10/25/2023] [Accepted: 11/15/2023] [Indexed: 11/26/2023]
Abstract
OBJECTIVE To identify practices that add value to improve the design, conduct, and reporting of child health research and reduce research waste. STUDY DESIGN In order to categorize the contributions of members of Standards for Research (StaR) in Child Health network, we developed a novel Child Health Improving Research Practices (CHIRP) framework comprised of 5 domains meant to counteract avoidable child health research waste and improve quality: 1) address research questions relevant to children, their families, clinicians, and researchers; 2) apply appropriate research design, conduct and analysis; 3) ensure efficient research oversight and regulation; 4) Provide accessible research protocols and reports; and 5) develop unbiased and usable research reports, including 17 responsible research practice recommendations. All child health research relevant publications by the 48 original StaR standards' authors over the last decade were identified, and main topic areas were categorized using this framework. RESULTS A total of 247 publications were included in the final sample: 100 publications (41%) in domain 1 (3 recommendations), 77 publications (31%) in domain 2 (3), 35 publications (14%) in domain 3 (4), 20 publications (8%) in domain 4 (4), and 15 publications (6%) in domain 5 (3). We identified readily implementable "responsible" research practices to counter child health research waste and improve quality, especially in the areas of patients and families' engagement throughout the research process, developing Core Outcome Sets, and addressing ethics and regulatory oversight issues. CONCLUSION While most of the practices are readily implementable, increased awareness of methodological issues and wider guideline uptake is needed to improve child health research. The CHIRP Framework can be used to guide responsible research practices that add value to child health research.
Collapse
Affiliation(s)
- Adrian Sammy
- Child Health Evaluative Sciences, The Hospital for Sick Children Research Institute, Toronto, Ontario, Canada
| | - Ami Baba
- Child Health Evaluative Sciences, The Hospital for Sick Children Research Institute, Toronto, Ontario, Canada
| | - Terry P Klassen
- Department of Pediatrics and Child Health, Children's Hospital Research Institute of Manitoba, Rady Faculty of Health Sciences, University of Manitoba, Winnipeg, Manitoba, Canada
| | - David Moher
- Centre for Journalology, Clinical Epidemiology Program, Ottawa Hospital Research Institute, Ottawa, Ontario, Canada; School of Epidemiology and Public Health, Faculty of Medicine, University of Ottawa, Ottawa, Ontario, Canada
| | - Martin Offringa
- Child Health Evaluative Sciences, The Hospital for Sick Children Research Institute, Toronto, Ontario, Canada; Division of Neonatology, The Hospital for Sick Children, Toronto, Ontario, Canada; Institute of Health Policy, Management and Evaluation, University of Toronto, Toronto, Ontario, Canada.
| |
Collapse
|
5
|
Stocking K, Watson A, Kirkham JJ, Wilkinson J, Vail A. Gynecological trials frequently exclude people based on their symptoms rather than their condition: a systematic review of Cochrane reviews and their component trials. J Clin Epidemiol 2023; 163:29-36. [PMID: 37778735 DOI: 10.1016/j.jclinepi.2023.09.012] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/27/2023] [Revised: 09/15/2023] [Accepted: 09/25/2023] [Indexed: 10/03/2023]
Abstract
OBJECTIVES To identify strategies used in recent randomized controlled trials (RCTs) and their associated Cochrane Reviews where patients with the same gynecological condition present with different symptoms but would plausibly benefit from a common intervention. STUDY DESIGN AND SETTING We searched the Cochrane library (February 2022) for reviews in polycystic ovarian syndrome (PCOS) and endometriosis. Reviews were included if the intervention was intended to treat all condition-specific symptoms. For each trial we recorded the strategy used and the number of potentially eligible participants excluded as a direct result of the chosen strategy. For each review we recorded the numbers of RCTs and participants excluded on the basis of symptoms experienced. RESULTS There were 89 distinct PCOS trials in 13 reviews, and 13 Endometriosis trials in 11 reviews. Most trials restricted their eligibility to participants with specific symptoms (55% PCOS, 46% endometriosis). The second most common strategy was to measure and analyze clinical outcomes that were not relevant to all participants (38% PCOS, 31% endometriosis). Reviews excluded 27% of trials in participants evaluating the same intervention in participants experiencing the same condition based on the outcomes measured in the trials. CONCLUSION Most gynecological trials exclude patients who could benefit from treatment or measure outcomes not relevant to all participants. We introduce a taxonomy to describe trial design strategies for conditions with heterogeneous symptoms.
Collapse
Affiliation(s)
- Katie Stocking
- Centre for Biostatistics, Faculty of Biology and Health Sciences, University of Manchester, Manchester, UK.
| | - Andrew Watson
- Department of Obstetrics and Gynaecology, Tameside & Glossop Acute Services NHS Trust, Ashton-Under-Lyne, UK
| | - Jamie J Kirkham
- Centre for Biostatistics, Faculty of Biology and Health Sciences, University of Manchester, Manchester, UK
| | - Jack Wilkinson
- Centre for Biostatistics, Faculty of Biology and Health Sciences, University of Manchester, Manchester, UK
| | - Andy Vail
- Centre for Biostatistics, Faculty of Biology and Health Sciences, University of Manchester, Manchester, UK
| |
Collapse
|
6
|
Lazzarini SG, Stella Yousif M, Bargeri S, Castellini G, Gianola S. Reasons for missing evidence in rehabilitation meta-analyses: a cross-sectional meta-research study. BMC Med Res Methodol 2023; 23:245. [PMID: 37865743 PMCID: PMC10590516 DOI: 10.1186/s12874-023-02064-7] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/20/2023] [Accepted: 10/10/2023] [Indexed: 10/23/2023] Open
Abstract
BACKGROUND Systematic reviews of randomized controlled trials are the best evidence for informing on intervention effectiveness. Their results, however, can be biased due to omitted evidence in the quantitative analyses. We aimed to assess the proportion of randomized controlled trials omitted from meta-analyses in the rehabilitation field and explore related reasons. METHODS This is a cross-sectional meta-research study. For each systematic review included in a published selected sample in the rehabilitation field, we identified an index meta-analysis on the primary outcome and the main comparison. We then looked at all the studies considered eligible for the chosen comparison in the systematic review and identified those trials that have been omitted (i.e., not included) from each index meta-analysis. Reasons for omission were collected based on an eight-reason classification. We used descriptive statistics to describe the proportion of omitted trials overall and according to each reason. RESULTS Starting from a cohort of 827 systematic reviews, 131 index meta-analyses comprising a total of 1761 eligible trials were selected. Only 16 index meta-analyses included all eligible studies while 15 omitted studies without providing references. From the remaining 100 index meta-analyses, 717 trials (40,7%) were omitted overall. Specific reasons for omission were: "unable to distinguish between selective reporting and inadequate planning" (39,3%, N = 282), "inadequate planning" (17%, N = 122), "justified to be not included" (15,1%, N = 108), "incomplete reporting" (8,4%, N = 60), "selective reporting" (3,3%, N = 24) and other situations (e.g., outcome present but no motivation for omission) (5,2%, N = 37). The 11,7% (N = 84) of omitted trials were not assessed due to non-English language or full text not available. CONCLUSIONS Almost half of the eligible trials were omitted from their index meta-analyses. Better reporting, protocol registration, definition and adoption of core outcome sets are needed to prevent omission of evidence in systematic reviews.
Collapse
Affiliation(s)
| | - Marzia Stella Yousif
- Department of Clinical Science and Translational Medicine, Faculty of Medicine and Surgery, University of Rome Tor Vergata, Rome, Italy
| | - Silvia Bargeri
- Unit of Clinical Epidemiology, IRCCS Istituto Ortopedico Galeazzi, Milan, Italy
| | - Greta Castellini
- Unit of Clinical Epidemiology, IRCCS Istituto Ortopedico Galeazzi, Milan, Italy
| | - Silvia Gianola
- Unit of Clinical Epidemiology, IRCCS Istituto Ortopedico Galeazzi, Milan, Italy
| |
Collapse
|
7
|
Green HR, Dawson A, Elfeky A, Pickles D, Treweek S, Gillies K. Exploring the perspectives of selectors and collecters of trial outcome data: an international qualitative study. BMC Med Res Methodol 2023; 23:229. [PMID: 37821867 PMCID: PMC10568821 DOI: 10.1186/s12874-023-02054-9] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/12/2022] [Accepted: 10/01/2023] [Indexed: 10/13/2023] Open
Abstract
INTRODUCTION Selecting and collecting data to support appropriate primary and secondary outcomes is a critical step in designing trials that can change clinical practice. In this study, we aimed to investigate who contributes to the process of selecting and collecting trial outcomes, and how these people are involved. This work serves two main purposes: (1) it provides the trials community with evidence to demonstrate how outcomes are currently selected and collected, and (2) it allows people involved in trial design and conduct to pick apart these processes to consider how efficiencies and improvements can be made. METHODS One-with-one semi-structured interviews, supported by a topic guide to ensure coverage of key content. The Framework approach was used for thematic analysis of data, and themes were linked through constant comparison of data both within and across participant groups. Interviews took place between July 2020 and January 2021. Participants were twenty-nine international trialists from various contributor groups, working primarily on designing and/or delivering phase III pragmatic effectiveness trials. Their experience spanned various funders, trial settings, clinical specialties, intervention types, and participant populations. RESULTS We identified three descriptive themes encompassing the process of primary and secondary outcome selection, collection, and the publication of outcome data. Within these themes, participants raised issues around the following: 1) Outcome selection: clarity of the research question; confidence in selecting trial outcomes and how confidence decreases with increased experience; interplay between different interested parties; how patients and the public are involved in outcome selection; perceived impact of poor outcome selection including poor recruitment and/or retention; and use of core outcome sets. 2) Outcome collection: disconnect between decisions made by outcome selectors and the practical work done by outcome collectors; potential impact of outcome measures on trial participants; potential impact on trial staff workload; and use of routinely collected data. 3) Publication of outcome data: difficulties in finding time to write and revise manuscripts for publication due to time and funding constraints. Participants overwhelmingly focused on the process of outcome selection, a topic they talked about unprompted. When prompted, participants do discuss outcome collection, but poor communication between selectors and collectors at the trial design stage means that outcome selection is rarely linked with the data collection workload it generates. DISCUSSION People involved in the design and conduct of trials fail to connect decisions around outcome selection with data collection workload. Publication of outcome data and effective dissemination of trial results are hindered due to the project-based culture of some academic clinical trial research.
Collapse
Affiliation(s)
- Heidi R Green
- Health Services Research Unit, University of Aberdeen, Aberdeen, UK.
- COUCH Health, Manchester, UK.
| | | | - Adel Elfeky
- Health Services Research Unit, University of Aberdeen, Aberdeen, UK
- Division of Health Sciences, Warwick Medical School, University of Warwick, Warwick, UK
| | - David Pickles
- Rheumatology department, Leeds Teaching Hospitals NHS Trust, Leeds, UK
| | - Shaun Treweek
- Health Services Research Unit, University of Aberdeen, Aberdeen, UK
| | - Katie Gillies
- Health Services Research Unit, University of Aberdeen, Aberdeen, UK
| |
Collapse
|
8
|
Mathews JJ, Chow R, Wennberg E, Lau J, Hannon B, Zimmermann C. Telehealth palliative care interventions for patients with advanced cancer: a scoping review. Support Care Cancer 2023; 31:451. [PMID: 37421447 DOI: 10.1007/s00520-023-07907-z] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/24/2023] [Accepted: 06/26/2023] [Indexed: 07/10/2023]
Abstract
PURPOSE Telehealth allows patients to maintain contact with healthcare providers without necessitating travel, and is becoming increasingly utilized. The purpose of this study is to describe the components of telehealth palliative care interventions for patients with advanced cancer before the COVID-19 pandemic; identify any intervention components associated with improvements in outcomes; and evaluate reporting of interventions. METHODS This scoping review was registered on the Open Science Framework. We searched 5 medical databases from inception to June 19, 2020. Inclusion criteria were: age ≥ 18, advanced cancer, asynchronous or synchronous telehealth intervention, and specialized palliative care interventions in any setting. We assessed the quality of intervention reporting using the Template for Intervention Description and Replication (TIDieR) checklist. RESULTS Twenty-three studies met the inclusion criteria: 15 (65%) quantitative (7 randomized controlled trials, 5 feasibility trials, 3 retrospective chart reviews); 4 (17%) mixed methods, and 4 (17%) qualitative. Most quantitative and mixed methods studies were conducted in North America (12/19, 63%), reported on hybrid (in-person and telehealth) interventions (9/19, 47%), and were delivered by nurses (12/19, 63%) in the home setting (14/19, 74%). In most studies that reported improvements in patient- or caregiver-reported outcomes, the content was psychoeducational and resulted in improvements for psychological symptoms. No study provided complete reporting on all 12 TIDieR checklist items. CONCLUSION Telehealth studies are needed that reflect palliative care's mission to provide multidisciplinary team-based care that improves quality of life in diverse settings, and that provide detailed reporting of interventions.
Collapse
Affiliation(s)
- Jean Jacob Mathews
- Division of Palliative Medicine, Department of Medicine and Department of Oncology, Queen's University, Kingston, Ontario, Canada
| | - Ronald Chow
- Department of Supportive Care, Princess Margaret Cancer Centre, University Health Network, 620 University Ave, 12-300, Toronto, Ontario, M5G 2C1, Canada
- Temerty Faculty of Medicine, University of Toronto, Toronto, Ontario, Canada
| | - Erica Wennberg
- Temerty Faculty of Medicine, University of Toronto, Toronto, Ontario, Canada
- Institute of Health Policy, Management and Evaluation, Dalla Lana School of Public Health, University of Toronto, Toronto, Ontario, Canada
| | - Jenny Lau
- Department of Supportive Care, Princess Margaret Cancer Centre, University Health Network, 620 University Ave, 12-300, Toronto, Ontario, M5G 2C1, Canada
- Temerty Faculty of Medicine, University of Toronto, Toronto, Ontario, Canada
- Division of Palliative Care, Department of Medicine, University Health Network, Toronto, Ontario, Canada
| | - Breffni Hannon
- Department of Supportive Care, Princess Margaret Cancer Centre, University Health Network, 620 University Ave, 12-300, Toronto, Ontario, M5G 2C1, Canada
- Temerty Faculty of Medicine, University of Toronto, Toronto, Ontario, Canada
- Division of Palliative Care, Department of Medicine, University Health Network, Toronto, Ontario, Canada
| | - Camilla Zimmermann
- Department of Supportive Care, Princess Margaret Cancer Centre, University Health Network, 620 University Ave, 12-300, Toronto, Ontario, M5G 2C1, Canada.
- Temerty Faculty of Medicine, University of Toronto, Toronto, Ontario, Canada.
- Institute of Health Policy, Management and Evaluation, Dalla Lana School of Public Health, University of Toronto, Toronto, Ontario, Canada.
- Division of Palliative Care, Department of Medicine, University Health Network, Toronto, Ontario, Canada.
| |
Collapse
|
9
|
Collet L, Lanore A, Alaterre C, Constantin JM, Martin GL, Caille A, James A, Dechartres A. Heterogeneity in the definition of delirium in ICUs and association with the intervention effect in randomized controlled trials: a meta-epidemiological study. Crit Care 2023; 27:170. [PMID: 37143091 PMCID: PMC10161412 DOI: 10.1186/s13054-023-04411-y] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/06/2023] [Accepted: 03/20/2023] [Indexed: 05/06/2023] Open
Abstract
PURPOSE To evaluate the heterogeneity in the definition of delirium in randomized controlled trials (RCTs) included in meta-analyses of delirium in intensive care units (ICUs) and to explore whether intervention effect depends on the definition used. METHODS We searched PubMed for meta-analyses including RCTs evaluating prevention or treatment strategies of delirium in ICU. The definition of delirium was collected from RCTs and classified as validated (DSM criteria, CAM-ICU, ICDSC, NEECHAM, DRS-R98) or non-validated (non-validated scales, set of symptoms, physician appreciation or not reported). We conducted a meta-epidemiological analysis to compare intervention effects between trials using or not a validated definition by a two-step method as primary analysis and a multilevel model as secondary analysis. A ratio of odds ratios (ROR) < 1 indicated larger intervention effects in trials using a non-validated definition. RESULTS Of 149 RCTs (41 meta-analyses), 109 (73.1%) used a validated definition and 40 (26.8%) did not (including 31 [20.8%] not reporting the definition). The primary analysis of 7 meta-analyses (30 RCTs) found no significant difference in intervention effects between trials using a validated definition and the others (ROR = 0.54, 95% CI 0.27-1.08), whereas the secondary multilevel analysis including 12 meta-analyses (67 RCTs) found significantly larger effects for trials using a non-validated versus a validated definition (ROR = 0.36, 95% CI 0.21-0.62). CONCLUSION The definition of delirium was heterogeneous across RCTs, with one-fifth not reporting how they evaluated delirium. We did not find a significant association with intervention effect in the primary analysis. The secondary analysis including more studies revealed significantly larger intervention effects in trials using a non-validated versus a validated definition.
Collapse
Affiliation(s)
- Lucie Collet
- Sorbonne Université, INSERM, Institut Pierre Louis d'Épidémiologie et de Santé Publique, AP-HP, Hôpital Pitié-Salpetrière, Département de Santé Publique, 47-83 Boulevard de l'Hôpital, 75013, Paris, France
- Sorbonne Université, GRC 29, DMU DREAM, AP-HP, Hôpital Pitié-Salpetrière, Département d'Anesthésie-Réanimation, Paris, France
| | - Aymeric Lanore
- Sorbonne Université, INSERM, Institut Pierre Louis d'Épidémiologie et de Santé Publique, AP-HP, Hôpital Pitié-Salpetrière, Département de Santé Publique, 47-83 Boulevard de l'Hôpital, 75013, Paris, France
- CIC Neurosciences, Department of Neurology, Assistance Publique Hôpitaux de Paris, Hôpital Pitié-Salpêtrière, Paris, France
| | - Camille Alaterre
- Sorbonne Université, GRC 29, DMU DREAM, AP-HP, Hôpital Pitié-Salpetrière, Département d'Anesthésie-Réanimation, Paris, France
| | - Jean-Michel Constantin
- Sorbonne Université, GRC 29, DMU DREAM, AP-HP, Hôpital Pitié-Salpetrière, Département d'Anesthésie-Réanimation, Paris, France
| | - Guillaume L Martin
- Sorbonne Université, INSERM, Institut Pierre Louis d'Épidémiologie et de Santé Publique, AP-HP, Hôpital Pitié-Salpetrière, Département de Santé Publique, 47-83 Boulevard de l'Hôpital, 75013, Paris, France
| | - Agnès Caille
- Université́ de Tours, Université́ de Nantes, INSERM, SPHERE U1246, Tours, France; INSERM CIC 1415, CHRU de Tours, Tours, France
| | - Arthur James
- Sorbonne Université, GRC 29, DMU DREAM, AP-HP, Hôpital Pitié-Salpetrière, Département d'Anesthésie-Réanimation, Paris, France
| | - Agnès Dechartres
- Sorbonne Université, INSERM, Institut Pierre Louis d'Épidémiologie et de Santé Publique, AP-HP, Hôpital Pitié-Salpetrière, Département de Santé Publique, 47-83 Boulevard de l'Hôpital, 75013, Paris, France.
| |
Collapse
|
10
|
Liu Y, Wang S, Chen W, Tan Y, Dun W, Zhang Y, Lu T, Hou X, Liu J. The Consistency between Registered Acupuncture-Moxibustion Clinical Studies and Their Published Studies and Update Status of Registered Information. Complement Med Res 2023; 30:307-316. [PMID: 36944314 DOI: 10.1159/000530245] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/25/2021] [Accepted: 03/14/2023] [Indexed: 03/23/2023]
Abstract
BACKGROUND Few studies have analyzed the consistency between registered acupuncture-moxibustion clinical studies and their published research results as well as their update status of registered information. METHODS We searched for acupuncture-moxibustion clinical studies that were registered at the World Health Organization International Clinical Trials Registry Platform between 2013 and 2015 and collected data regarding their characteristics and update status. Published results of these registered studies were identified and compared with registered information. RESULTS A total of 425 registered acupuncture-moxibustion clinical studies were included; 379 (89.2%) of them were interventional studies, and the remaining 46 (10.8%) were observational studies. Forty-six studies (10.8%) were found to have published results, and 51 published articles were identified. Overall, 73.2% (311) of registered studies did not update the research status in time; 46.6% (198) stopped updating before recruiting; 21.6% (92) stopped updating after recruiting; and 4.9% (21) stopped updating after completion. Regarding the 46 studies with published results, 29 (63.0%) were considered to be affected by reporting bias. These reporting biases predominantly involved the omission of some predefined outcomes or endpoints (16 studies), contradictions regarding descriptions of sample sizes (9 studies), discrepancies in treatment measurements or group distribution (7 studies), and inconsistent treatment durations (4 studies). When compared with other studies, significant and various reporting biases could also be commonly found in fields other than acupuncture-moxibustion. CONCLUSIONS There were many discrepancies between registered information and published reports on acupuncture-moxibustion, which could also be commonly observed in other fields. Moreover, a large proportion of registered studies did not update their research status in time. Efforts should be made to improve the reporting quality and timely updates. Hintergrund Es gibt nur wenige Studien, in denen die Übereinstimmung zwischen den registrierten klinischen Studien zur Akupunktur und Moxibustion mit den veröffentlichten Studienergebnissen und dem Aktualisierungsstand der Informationen im Register untersucht wurde. Methoden Wir suchten nach klinischen Studien zur Akupunktur und Moxibustion, die zwischen 2013 und 2015 auf der International Clinical Trials Registry Platform der Weltgesundheitsorganisation registriert wurden, und erhoben Daten zu ihren Merkmalen und ihrem Aktualisierungsstand. Die veröffentlichten Ergebnisse der registrierten Studien wurden identifiziert und mit den Informationen im Register verglichen. Ergebnisse Insgesamt wurden 425 registrierte klinische Studien zur Akupunktur und Moxibustion eingeschlossen, davon waren 379 (89,2 %) Interventionsstudien und die restlichen 46 (10,8 %) waren Beobachtungsstudien. Es wurden 46 Studien (10,8 %) mit veröffentlichten Ergebnissen gefunden und 51 veröffentlichte Artikel identifiziert. Insgesamt wurde bei 73,2 % (311) der registrierten Studien der Forschungsstand nicht zeitnah aktualisiert; bei 46,6 % (198) wurde die Aktualisierung vor der Rekrutierung eingestellt; bei 21,6 % (92) wurde die Aktualisierung nach der Rekrutierung eingestellt und bei 4,9 % (21) wurde die Aktualisierung nach Abschluss der Studie eingestellt. Von den 46 Studien mit veröffentlichten Ergebnissen wurden 29 (63,0 %) als von Publikationsverzerrung betroffen angesehen. Diese Publikationsverzerrung betraf vor allem die Auslassung einiger vordefinierter Zielkriterien oder Endpunkte (16 Studien), Widersprüche bei der Beschreibung des Stichprobenumfangs (9 Studien), Diskrepanzen bei den Behandlungsmessungen oder der Gruppenverteilung (7 Studien) und Inkonsistenzen bei der Behandlungsdauer (4 Studien). Beim Vergleich mit anderen Studien wurden auch in anderen Bereichen als Akupunktur und Moxibustion häufig signifikante und unterschiedliche Publikationsverzerrungen festgestellt. Schlussfolgerungen Es bestanden zahlreiche Diskrepanzen zwischen den Informationen im Register und den veröffentlichten Berichten über Akupunktur und Moxibustion, die auch in anderen Bereichen häufig zu beobachten waren. Darüber hinaus wurde bei einem Großteil der registrierten Studien der Forschungsstand nicht zeitnah aktualisiert. Es sollten Anstrengungen unternommen werden, um die Qualität der Berichterstattung und die zeitnahe Aktualisierung zu verbessern.
Collapse
Affiliation(s)
- Yali Liu
- Center for Clinical Epidemiology and Evidence-based Medicine, Beijing Children's Hospital, Capital Medical University, National Center for Children's Health, Beijing, China
| | - Shu Wang
- Department of Neurosurgery, Beijing Tiantan Hospital, Capital Medical University, Beijing, China
- Department of Neurosurgery, SanBo Brain Hospital, Capital Medical University, Beijing, China
| | - Wenjie Chen
- Shantou University Medical College, Shantou, China
| | - Yingxin Tan
- The First Hospital of Lanzhou University, Lanzhou, China
| | - Wangqing Dun
- The First Hospital of Lanzhou University, Lanzhou, China
| | - Yuan Zhang
- Neonatal Center, Beijing Children's Hospital, Capital Medical University, National Center for Children's Health, Beijing, China
- Dalian Medical University, Dalian, China
- Department of Pediatric Hematology-Oncology, Dalian Municipal Women and Children's Medical Center, Dalian, China
| | - Tingting Lu
- Institution of Clinical Research and Evidence Based Medicine, The Gansu Provincial Hospital, Lanzhou, China
| | - Xuejing Hou
- The First Hospital of Lanzhou University, Lanzhou, China
| | - Jia Liu
- Institute of Basic Research in Clinical Medicine, China Academy of Chinese Medical Sciences, Beijing, China
| |
Collapse
|
11
|
Butcher NJ, Monsour A, Mew EJ, Chan AW, Moher D, Mayo-Wilson E, Terwee CB, Chee-A-Tow A, Baba A, Gavin F, Grimshaw JM, Kelly LE, Saeed L, Thabane L, Askie L, Smith M, Farid-Kapadia M, Williamson PR, Szatmari P, Tugwell P, Golub RM, Monga S, Vohra S, Marlin S, Ungar WJ, Offringa M. Guidelines for Reporting Outcomes in Trial Protocols: The SPIRIT-Outcomes 2022 Extension. JAMA 2022; 328:2345-2356. [PMID: 36512367 DOI: 10.1001/jama.2022.21243] [Citation(s) in RCA: 52] [Impact Index Per Article: 26.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 12/14/2022]
Abstract
IMPORTANCE Complete information in a trial protocol regarding study outcomes is crucial for obtaining regulatory approvals, ensuring standardized trial conduct, reducing research waste, and providing transparency of methods to facilitate trial replication, critical appraisal, accurate reporting and interpretation of trial results, and knowledge synthesis. However, recommendations on what outcome-specific information should be included are diverse and inconsistent. To improve reporting practices promoting transparent and reproducible outcome selection, assessment, and analysis, a need for specific and harmonized guidance as to what outcome-specific information should be addressed in clinical trial protocols exists. OBJECTIVE To develop harmonized, evidence- and consensus-based standards for describing outcomes in clinical trial protocols through integration with the Standard Protocol Items: Recommendations for Interventional Trials (SPIRIT) 2013 statement. EVIDENCE REVIEW Using the Enhancing the Quality and Transparency of Health Research (EQUATOR) methodological framework, the SPIRIT-Outcomes 2022 extension of the SPIRIT 2013 statement was developed by (1) generation and evaluation of candidate outcome reporting items via consultation with experts and a scoping review of existing guidance for reporting trial outcomes (published within the 10 years prior to March 19, 2018) identified through expert solicitation, electronic database searches of MEDLINE and the Cochrane Methodology Register, gray literature searches, and reference list searches; (2) a 3-round international Delphi voting process (November 2018-February 2019) completed by 124 panelists from 22 countries to rate and identify additional items; and (3) an in-person consensus meeting (April 9-10, 2019) attended by 25 panelists to identify essential items for outcome-specific reporting to be addressed in clinical trial protocols. FINDINGS The scoping review and consultation with experts identified 108 recommendations relevant to outcome-specific reporting to be addressed in trial protocols, the majority (72%) of which were not included in the SPIRIT 2013 statement. All recommendations were consolidated into 56 items for Delphi voting; after the Delphi survey process, 19 items met criteria for further evaluation at the consensus meeting and possible inclusion in the SPIRIT-Outcomes 2022 extension. The discussions during and after the consensus meeting yielded 9 items that elaborate on the SPIRIT 2013 statement checklist items and are related to completely defining and justifying the choice of primary, secondary, and other outcomes (SPIRIT 2013 statement checklist item 12) prospectively in the trial protocol, defining and justifying the target difference between treatment groups for the primary outcome used in the sample size calculations (SPIRIT 2013 statement checklist item 14), describing the responsiveness of the study instruments used to assess the outcome and providing details on the outcome assessors (SPIRIT 2013 statement checklist item 18a), and describing any planned methods to account for multiplicity relating to the analyses or interpretation of the results (SPIRIT 2013 statement checklist item 20a). CONCLUSIONS AND RELEVANCE This SPIRIT-Outcomes 2022 extension of the SPIRIT 2013 statement provides 9 outcome-specific items that should be addressed in all trial protocols and may help increase trial utility, replicability, and transparency and may minimize the risk of selective nonreporting of trial results.
Collapse
Affiliation(s)
- Nancy J Butcher
- Child Health Evaluative Sciences, The Hospital for Sick Children Research Institute, Toronto, Ontario, Canada
- Department of Psychiatry, University of Toronto, Toronto, Ontario, Canada
| | - Andrea Monsour
- Child Health Evaluative Sciences, The Hospital for Sick Children Research Institute, Toronto, Ontario, Canada
| | - Emma J Mew
- Child Health Evaluative Sciences, The Hospital for Sick Children Research Institute, Toronto, Ontario, Canada
- Department of Chronic Disease Epidemiology, School of Public Health, Yale University, New Haven, Connecticut
| | - An-Wen Chan
- Department of Medicine, Women's College Research Institute, University of Toronto, Toronto, Ontario, Canada
| | - David Moher
- Centre for Journalology, Clinical Epidemiology Program, Ottawa Hospital Research Institute, Ottawa, Ontario, Canada
- School of Epidemiology and Public Health, University of Ottawa, Ottawa, Ontario, Canada
| | - Evan Mayo-Wilson
- Department of Epidemiology, Gillings School of Global Public Health, University of North Carolina, Chapel Hill
| | - Caroline B Terwee
- Amsterdam University Medical Centers, Vrije Universiteit, Department of Epidemiology and Data Science, Amsterdam, the Netherlands
- Department of Methodology, Amsterdam Public Health Research Institute, Amsterdam, the Netherlands
| | - Alyssandra Chee-A-Tow
- Child Health Evaluative Sciences, The Hospital for Sick Children Research Institute, Toronto, Ontario, Canada
| | - Ami Baba
- Child Health Evaluative Sciences, The Hospital for Sick Children Research Institute, Toronto, Ontario, Canada
| | - Frank Gavin
- public panel member, Toronto, Ontario, Canada
| | - Jeremy M Grimshaw
- Clinical Epidemiology Program, Ottawa Hospital Research Institute, Ottawa, Ontario, Canada
- Department of Medicine, University of Ottawa, Ottawa, Ontario, Canada
| | - Lauren E Kelly
- Department of Pharmacology and Therapeutics, University of Manitoba, Winnipeg, Canada
- Children's Hospital Research Institute of Manitoba, Winnipeg, Canada
| | - Leena Saeed
- Child Health Evaluative Sciences, The Hospital for Sick Children Research Institute, Toronto, Ontario, Canada
| | - Lehana Thabane
- Health Research Methods, Evidence, and Impact, McMaster University, Hamilton, Ontario, Canada
| | - Lisa Askie
- NHMRC Clinical Trials Centre, University of Sydney, Sydney, New South Wales, Australia
| | | | - Mufiza Farid-Kapadia
- Child Health Evaluative Sciences, The Hospital for Sick Children Research Institute, Toronto, Ontario, Canada
| | - Paula R Williamson
- MRC-NIHR Trials Methodology Research Partnership, Department of Health Data Science, University of Liverpool, Liverpool, England
| | - Peter Szatmari
- Cundill Centre for Child and Youth Depression, Centre for Addiction and Mental Health, Toronto, Ontario, Canada
- Department of Psychiatry, The Hospital for Sick Children, Toronto, Ontario, Canada
| | - Peter Tugwell
- School of Epidemiology and Public Health, University of Ottawa, Ottawa, Ontario, Canada
- Department of Medicine, University of Ottawa, Ottawa, Ontario, Canada
- Bruyère Research Institute, Ottawa, Ontario, Canada
- Ottawa Hospital Research Institute, Ottawa, Ontario, Canada
| | - Robert M Golub
- Department of Medicine, Feinberg School of Medicine, Northwestern University, Chicago, Illinois
| | - Suneeta Monga
- Department of Psychiatry, University of Toronto, Toronto, Ontario, Canada
- Department of Psychiatry, The Hospital for Sick Children, Toronto, Ontario, Canada
| | - Sunita Vohra
- Departments of Pediatrics and Psychiatry, Faculty of Medicine and Dentistry, University of Alberta, Edmonton, Canada
| | - Susan Marlin
- Clinical Trials Ontario, Toronto, Canada
- Department of Public Health Sciences, Queen's University, Kingston, Ontario, Canada
| | - Wendy J Ungar
- Child Health Evaluative Sciences, The Hospital for Sick Children Research Institute, Toronto, Ontario, Canada
- Institute of Health Policy, Management, and Evaluation, University of Toronto, Toronto, Ontario, Canada
| | - Martin Offringa
- Child Health Evaluative Sciences, The Hospital for Sick Children Research Institute, Toronto, Ontario, Canada
- Institute of Health Policy, Management, and Evaluation, University of Toronto, Toronto, Ontario, Canada
- Division of Neonatology, The Hospital for Sick Children, Toronto, Ontario, Canada
| |
Collapse
|
12
|
Castellini G, Pillastrini P, Vanti C, Bargeri S, Giagio S, Bordignon E, Fasciani F, Marzioni F, Innocenti T, Chiarotto A, Gianola S, Bertozzi L. Some conservative interventions are more effective than others for people with chronic non-specific neck pain: a systematic review and network meta-analysis. J Physiother 2022; 68:244-254. [PMID: 36266185 DOI: 10.1016/j.jphys.2022.09.007] [Citation(s) in RCA: 10] [Impact Index Per Article: 5.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 10/25/2021] [Revised: 03/08/2022] [Accepted: 09/26/2022] [Indexed: 01/11/2023] Open
Abstract
QUESTION Which is the most effective conservative intervention for patients with non-specific chronic neck pain (CNSNP)? DESIGN A systematic review and network meta-analysis of randomised clinical trials. PARTICIPANTS Adults with CNSNP of at least 3 months duration. INTERVENTIONS All available pharmacological and non-pharmacological interventions. OUTCOME MEASURES The primary outcomes were pain intensity and disability. The secondary outcome was adverse events. RESULTS Overall, 119 RCTs (12,496 patients; 32 interventions) were included. Risk of bias was low in 50.4% of trials, unclear in 22.7% and high in 26.9%. Compared with inert treatment, a combination of active and/or passive multimodal non-pharmacological inventions (eg, exercise and manual therapy) were effective for pain on a 0-to-10 scale at 1 month (MD range 0.84 to 3.74) and at 3 to 6 months (MD range 1.06 to 1.49), and effective on disability on a 0-to-100 scale at 1 month (MD range 10.26 to 14.09) and 3 to 6 months (MD range 5.60 to 16.46). These effects ranged from possible to definite clinical relevance. Compared with inert treatment, anti-inflammatory drugs alone or in combination with another non-pharmacological treatment did not reduce pain at 1 month or 3 to 6 months. At 12 months, no superiority was found over inert treatment on both outcomes. Most mild adverse events were experienced following acupuncture/dry needling intervention. On average, the evidence varied from low to very low certainty. CONCLUSIONS While multimodal non-pharmacological interventions may reduce pain and disability for up to 3 to 6 months of follow-up when compared with inert treatment, the evidence was very uncertain about their effects. Better quality and larger trials are needed to improve the certainty of evidence. REGISTRATION PROSPERO CRD42019124501.
Collapse
Affiliation(s)
- Greta Castellini
- IRCCS Istituto Ortopedico Galeazzi, Unit of Clinical Epidemiology, Milan, Italy
| | - Paolo Pillastrini
- Department of Biomedical and Neuromotor Sciences (DIBINEM) - University of Bologna, Italy
| | - Carla Vanti
- IRCCS Istituto Ortopedico Galeazzi, Unit of Clinical Epidemiology, Milan, Italy
| | - Silvia Bargeri
- IRCCS Istituto Ortopedico Galeazzi, Unit of Clinical Epidemiology, Milan, Italy
| | - Silvia Giagio
- Department of Biomedical and Neuromotor Sciences (DIBINEM) - University of Bologna, Italy
| | | | - Francesco Fasciani
- Department of Biomedical and Neuromotor Sciences (DIBINEM) - University of Bologna, Italy
| | - Francesco Marzioni
- Department of Biomedical and Neuromotor Sciences (DIBINEM) - University of Bologna, Italy
| | - Tiziano Innocenti
- Department of Health Science - Faculty of Science - Vrije Universiteit Amsterdam, Amsterdam Movement Sciences, The Netherlands; GIMBE Foundation, Bologna, Italy
| | - Alessandro Chiarotto
- Department of Health Science - Faculty of Science - Vrije Universiteit Amsterdam, Amsterdam Movement Sciences, The Netherlands; Department of General Practice - Erasmus MC - University Medical Center Rotterdam, The Netherlands
| | - Silvia Gianola
- IRCCS Istituto Ortopedico Galeazzi, Unit of Clinical Epidemiology, Milan, Italy
| | - Lucia Bertozzi
- Division of Occupational Medicine, IRCCS Azienda Ospedaliero-Universitaria di Bologna, Bologna, Italy; Department of Biomedical and Neuromotor Sciences (DIBINEM), Alma Mater Studiorum University of Bologna, Bologna, Italy
| |
Collapse
|
13
|
Hutchinson N, Moyer H, Zarin DA, Kimmelman J. The proportion of randomized controlled trials that inform clinical practice. eLife 2022; 11:79491. [PMID: 35975784 PMCID: PMC9427100 DOI: 10.7554/elife.79491] [Citation(s) in RCA: 9] [Impact Index Per Article: 4.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/14/2022] [Accepted: 08/15/2022] [Indexed: 11/13/2022] Open
Abstract
Prior studies suggest that clinical trials are often hampered by problems in design, conduct and reporting that limit their uptake in clinical practice. We have described 'informativeness' as the ability of a trial to guide clinical, policy or research decisions. Little is known about the proportion of initiated trials that inform clinical practice. We created a cohort of randomized interventional clinical trials in three disease areas (ischemic heart disease, diabetes mellitus and lung cancer), that were initiated between 1 January 2009 and 31 December 2010 using ClinicalTrials.gov. We restricted inclusion to trials aimed at answering a clinical question related to the treatment or prevention of disease. Our primary outcome was the proportion of clinical trials fulfilling four conditions of informativeness: importance of the clinical question, trial design, feasibility, and reporting of results. Our study included 125 clinical trials. The proportion meeting four conditions for informativeness was 26.4% (95% CI 18.9 - 35.0). Sixty-seven percent of participants were enrolled in informative trials. The proportion of informative trials did not differ significantly between our three disease areas. Our results suggest that the majority of clinical trials designed to guide clinical practice possess features that may compromise their ability to do so. This highlights opportunities to improve the scientific vetting of clinical research.
Collapse
Affiliation(s)
- Nora Hutchinson
- Studies of Translation, Ethics and Medicine, McGill University, Montreal, Canada
| | - Hannah Moyer
- Studies of Translation, Ethics and Medicine, McGill University, Montreal, Canada
| | - Deborah A Zarin
- Multi-Regional Clinical Trials Center, Brigham and Women's Hospital, Boston, United States
| | - Jonathan Kimmelman
- Studies of Translation, Ethics and Medicine Research Group, Biomedical Ethics Unit, McGill University, Montreal, Canada
| |
Collapse
|
14
|
How informative were early SARS-CoV-2 treatment and prevention trials? a longitudinal cohort analysis of trials registered on ClinicalTrials.gov. PLoS One 2022; 17:e0262114. [PMID: 35061758 PMCID: PMC8782516 DOI: 10.1371/journal.pone.0262114] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/07/2021] [Accepted: 12/17/2021] [Indexed: 01/03/2023] Open
Abstract
Background
Early in the SARS-CoV-2 pandemic, commentators warned that some COVID trials were inadequately conceived, designed and reported. Here, we retrospectively assess the prevalence of informative COVID trials launched in the first 6 months of the pandemic.
Methods
Based on prespecified eligibility criteria, we created a cohort of Phase 1/2, Phase 2, Phase 2/3 and Phase 3 SARS-CoV-2 treatment and prevention efficacy trials that were initiated from 2020-01-01 to 2020-06-30 using ClinicalTrials.gov registration records. We excluded trials evaluating behavioural interventions and natural products, which are not regulated by the U.S. Food and Drug Administration (FDA). We evaluated trials on 3 criteria of informativeness: potential redundancy (comparing trial phase, type, patient-participant characteristics, treatment regimen, comparator arms and primary outcome), trials design (according to the recommendations set-out in the May 2020 FDA guidance document on SARS-CoV-2 treatment and prevention trials) and feasibility of patient-participant recruitment (based on timeliness and success of recruitment).
Results
We included all 500 eligible trials in our cohort, 58% of which were Phase 2 and 84.8% were directed towards the treatment of SARS-CoV-2. Close to one third of trials met all three criteria and were deemed informative (29.9% (95% Confidence Interval 23.7–36.9)). The proportion of potentially redundant trials in our cohort was 4.1%. Over half of the trials in our cohort (56.2%) did not meet our criteria for high quality trial design. The proportion of trials with infeasible patient-participant recruitment was 22.6%.
Conclusions
Less than one third of COVID-19 trials registered on ClinicalTrials.gov during the first six months met all three criteria for informativeness. Shortcomings in trial design, recruitment feasibility and redundancy reflect longstanding weaknesses in the clinical research enterprise that were likely amplified by the exceptional circumstances of a pandemic.
Collapse
|
15
|
Woolf B, Edwards P. Does pre-notification increase questionnaire response rates: a randomised controlled trial nested within a systematic review. BMC Med Res Methodol 2021; 21:259. [PMID: 34837937 PMCID: PMC8627620 DOI: 10.1186/s12874-021-01462-z] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/29/2021] [Accepted: 11/12/2021] [Indexed: 11/24/2022] Open
Abstract
BACKGROUND Missing outcome data can lead to bias in the results of systematic reviews. One way to address missing outcome data is by requesting the data from the trial authors, but non-response is common. One way to potentially improve response rates is by sending study participants advance communication. During the update of a systematic review examining the effect of pre-notification on response rates, study authors needed to be contacted for further information. This study was nested within the systematic review by randomising authors to receive a notification of the upcoming request for information. The objective was to test if pre-notification increased response rates. METHODS The participants were study authors included in the systematic review, whose studies were at unclear risk of bias. The intervention was a pre-notification of the request for further information, sent 1 day before the request. The outcome was defined as the proportion of authors who responded to the request for information. Authors were randomised by simple randomisation. Thirty three authors were randomised to the pre-notification arm, and 42 were randomised to the control arm. Authors were blinded to the possibility of an alternative condition. RESULTS All authors randomised were analysed. 14/33 (42.4%) authors in the pre-notification arm had returned responses to the questionnaire, and 18/42 (42.9%) in the control arm. There was no evidence of a difference between these groups (absolute difference = - 0.5, 95% CI (- 23.4 to 22.5%), p = 1). We received no complaints about receiving the pre-notification. CONCLUSIONS This study's results do not support the hypothesis that pre-notification increases response from study authors being contacted for a request for more information. However, the study has a low power, and the results may not generalise to other contexts, methods of administering a pre-notification, or study populations. TRIAL REGISTRATION Registration and protocol: This trial is not registered with any trial registry. However, the protocol was posted in advance on the Open Science Framework website and is available on the Open Science Framework website: DOI: https://doi.org/10.17605/OSF.IO/MSV2W or https://osf.io/msv2w/.
Collapse
Affiliation(s)
- Benjamin Woolf
- Department of Psychological Science, University of Bristol, 5 Priory Road, Bristol, UK.
- Medical Research Council Integrative Epidemiology Unit, University of Bristol, Bristol, UK.
- Faculty of Epidemiology and Population Health, London School of Hygiene and Tropical Medicine, London, UK.
| | - Phil Edwards
- Faculty of Epidemiology and Population Health, London School of Hygiene and Tropical Medicine, London, UK
| |
Collapse
|
16
|
Woodford J, Karlsson M, Hagström J, Hägg Sylvén Y, Norbäck K, Grönqvist H, von Essen L. Conducting Digital Health Care Research: Document Analysis of Challenges Experienced During Intervention Development and Feasibility Study Setup of an Internet-Administered Intervention for Parents of Children Treated for Cancer. JMIR Form Res 2021; 5:e26266. [PMID: 34433524 PMCID: PMC8538021 DOI: 10.2196/26266] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/04/2020] [Revised: 04/16/2021] [Accepted: 05/02/2021] [Indexed: 11/19/2022] Open
Abstract
Background The design and conduct of research to develop, test, and evaluate complex health care interventions is challenging. Although the existing literature describes key challenges associated with the design and conduct of definitive (evaluation) trials, there is a lack of information concerning specific challenges associated with the intervention development phase and setup of feasibility studies. In particular, the literature is scarce concerning the challenges associated with conducting digital health care research, such as research on internet-administered interventions and research using digital features to support the execution of study procedures (eg, recruitment, consent, retention, and data collection and management). This study is conducted in the context of the intervention development and feasibility study setup phases of an internet-administered, guided, low-intensity cognitive behavioral therapy–based intervention for parents of children previously treated for cancer. Objective The aim of this study is to explore the challenges experienced during the development phase of the internet-administered intervention and digital features to support the execution of the study procedures and a feasibility study setup. Methods To explore the key challenges experienced, we conducted a document analysis of written records from all study meetings held by the research team (meeting minutes) between June 7, 2018, and January 10, 2020, guided by a thematic analysis approach. Furthermore, discussion groups with members of the research team were held to develop a more detailed understanding of the key challenges experienced. Methods and results are reported in accordance with the relevant items from the Standards for Reporting Qualitative Research checklist. Results Six main themes were identified: decision-making and communication, expertise, external constraints, flexibility, planning and scheduling, and technical constraints. Conclusions Significant challenges were experienced during the intervention development and setup phases of the feasibility study. Implications are discussed to inform future design, conduct, and planning of internet-administered intervention development and feasibility studies, especially within the context of digital health care research.
Collapse
Affiliation(s)
- Joanne Woodford
- Healthcare Sciences and e-Health, Department of Women's and Children's Health, Uppsala University, Uppsala, Sweden
| | - Mathilda Karlsson
- Healthcare Sciences and e-Health, Department of Women's and Children's Health, Uppsala University, Uppsala, Sweden
| | - Josefin Hagström
- Healthcare Sciences and e-Health, Department of Women's and Children's Health, Uppsala University, Uppsala, Sweden
| | - Ylva Hägg Sylvén
- Healthcare Sciences and e-Health, Department of Women's and Children's Health, Uppsala University, Uppsala, Sweden
| | - Kajsa Norbäck
- Healthcare Sciences and e-Health, Department of Women's and Children's Health, Uppsala University, Uppsala, Sweden
| | - Helena Grönqvist
- Healthcare Sciences and e-Health, Department of Women's and Children's Health, Uppsala University, Uppsala, Sweden
| | - Louise von Essen
- Healthcare Sciences and e-Health, Department of Women's and Children's Health, Uppsala University, Uppsala, Sweden
| |
Collapse
|
17
|
Brustia R, Laurent A, Goumard C, Langella S, Cherqui D, Kawai T, Soubrane O, Cauchy F, Farges O, Menahem B, Hobeika C, Rhaiem R, Sommacale D, Okumura S, Hofmeyr S, Ferrero A, Pruvot FR, Regimbeau JM, Fuks D, Vibert E, Scatton O. Laparoscopic versus open liver resection for intrahepatic cholangiocarcinoma: Report of an international multicenter cohort study with propensity score matching. Surgery 2021; 171:1290-1302. [PMID: 34535270 DOI: 10.1016/j.surg.2021.08.015] [Citation(s) in RCA: 17] [Impact Index Per Article: 5.7] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/31/2021] [Revised: 08/09/2021] [Accepted: 08/12/2021] [Indexed: 11/09/2022]
Abstract
BACKGROUND Intrahepatic cholangiocarcinoma is a rare disease with a poor prognosis. In patients where surgical resection is possible, outcome is influenced by perioperative morbidity and lymph node status. Laparoscopic liver resection is associated with improved clinical and oncological outcomes in primary and metastatic liver cancer compared with open liver resection, but evidence on intrahepatic cholangiocarcinoma is still insufficient. The primary aim of this study was to compare overall survival for a large series of patients treated for intrahepatic cholangiocarcinoma by open or laparoscopic approach. Secondary objectives were to compare disease-free survival, predictors of death, and recurrence. METHODS Patients treated with laparoscopic or open liver resection for intrahepatic cholangiocarcinoma from 2000 to 2018 from 3 large international databases were analyzed retrospectively. Each patient in the laparoscopic resection group (case) was matched with 1 open resection control (1:1 ratio), through a propensity score calculated on clinically relevant preoperative covariates. Overall and disease-free survival were compared between the matched groups. Predictors of mortality and recurrence were analyzed with Cox regression, and the Textbook Outcomes were described. RESULTS During the study period, 855 patients met the inclusion criteria (open liver resection = 709, 82.9%; laparoscopic liver resection = 146, 17.1%). Two groups of 89 patients each were analyzed after propensity score matching, with no significant difference regarding pre- and postoperative variables. Overall survival at 1, 3, and 5 years was 92%, 75%, and 63% in the laparoscopic liver resection group versus 92%, 58%, and 49% in the open liver resection group (P = .0043). Adjusted Cox regression revealed severe postoperative complications (hazard ratio: 10.5, 95% confidence interval [1.01-109] P = .049) and steatosis (hazard ratio: 13.8, 95% confidence interval [1.23-154] P = .033) as predictors of death, and transfusion (hazard ratio: 19.2, 95% confidence interval [4.04-91.4] P < .001) and severe postoperative complications (hazard ratio: 4.07, 95% confidence interval [1.15-14.4] P = .030) as predictors of recurrence. CONCLUSION The survival advantage of laparoscopic liver resection over open liver resection for intrahepatic cholangiocarcinoma is equivocal, given historical bias and missing data.
Collapse
Affiliation(s)
- Raffaele Brustia
- Department of Digestive and Hepato-Pancreatic-Biliary Surgery, DMU CARE, Assistance Publique-Hôpitaux de Paris (AP-HP), Hôpitaux Universitaires Henri Mondor, Créteil, France. https://twitter.com/raffaele.brustia
| | - Alexis Laurent
- Univ Paris Est Creteil, Faculté de Santé, Créteil, France and Department of Digestive and Hepato-Pancreatic-Biliary Surgery, DMU CARE, Assistance Publique-Hôpitaux de Paris (AP-HP), Hôpitaux Universitaires Henri Mondor, Créteil, France
| | - Claire Goumard
- Department of Hepatobiliary and Liver Transplantation Surgery, AP-HP, Hôpital Pitié Salpêtrière, CRSA, Sorbonne Université, Paris, France
| | - Serena Langella
- Department of General and Oncological Surgery, Ospedale Mauriziano, Torino, Italy
| | - Daniel Cherqui
- Centre Hepato-Biliaire, AP-HP Paul Brousse Hospital, Paris-Saclay University, Villejuif, France
| | - Takayuki Kawai
- Department of Surgery, Graduate School of Medicine, Kyoto University, Kyoto, Japan
| | - Olivier Soubrane
- Department of Hepatobiliary Surgery, APHP, University Hospitals Paris Nord Val de Seine, Beaujon, Clichy, France
| | - Francois Cauchy
- Department of Hepatobiliary Surgery, APHP, University Hospitals Paris Nord Val de Seine, Beaujon, Clichy, France
| | - Olivier Farges
- Department of Hepatobiliary Surgery, APHP, University Hospitals Paris Nord Val de Seine, Beaujon, Clichy, France
| | - Benjamin Menahem
- Department of Digestive Surgery, University Hospital of Caen, France
| | - Christian Hobeika
- Department of Hepatobiliary and Liver Transplantation Surgery, AP-HP, Hôpital Pitié Salpêtrière, CRSA, Sorbonne Université, Paris, France
| | - Rami Rhaiem
- Department of Hepatobiliary, Pancreatic, and Digestive Surgery, Robert Debré University Hospital, Reims, France University Reims Champagne-Ardenne, France
| | - Daniele Sommacale
- Univ Paris Est Creteil, Faculté de Santé, Créteil, France and Department of Digestive and Hepato-Pancreatic-Biliary Surgery, DMU CARE, Assistance Publique-Hôpitaux de Paris (AP-HP), Hôpitaux Universitaires Henri Mondor, Créteil, France
| | - Shinya Okumura
- Department of Surgery, Graduate School of Medicine, Kyoto University, Kyoto, Japan
| | - Stefan Hofmeyr
- Division of Surgery, Faculty of Medicine and Health Sciences, Stellenbosch University and Tygerberg Academic Hospital, Cape Town, South Africa
| | - Alessandro Ferrero
- Department of General and Oncological Surgery, Ospedale Mauriziano, Torino, Italy
| | - François-René Pruvot
- Department of Digestive Surgery and Transplantation, Univ Lille, CHRU Lille, Lille, France
| | - Jean-Marc Regimbeau
- SSPC (Simplification of Surgical Patients Care), Clinical Research Unit, University of Picardie Jules Verne, Amiens, France and Department of Digestive Surgery, Amiens University Medical Center, Amiens, France
| | - David Fuks
- Department of Digestive, Oncologic, and Metabolic Surgery, Institut Mutualiste Montsouris, Université Paris Descartes, Paris, France
| | - Eric Vibert
- Centre Hepato-Biliaire, AP-HP Paul Brousse Hospital, Paris-Saclay University, Villejuif, France
| | - Olivier Scatton
- Department of Hepatobiliary and Liver Transplantation Surgery, AP-HP, Hôpital Pitié Salpêtrière, CRSA, Sorbonne Université, Paris, France.
| | | | | | | |
Collapse
|
18
|
Qian Y, Walters SJ, Jacques R, Flight L. Comprehensive review of statistical methods for analysing patient-reported outcomes (PROs) used as primary outcomes in randomised controlled trials (RCTs) published by the UK's Health Technology Assessment (HTA) journal (1997-2020). BMJ Open 2021; 11:e051673. [PMID: 34489292 PMCID: PMC8422492 DOI: 10.1136/bmjopen-2021-051673] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 12/23/2022] Open
Abstract
OBJECTIVES To identify how frequently patient-reported outcomes (PROs) are used as primary and/or secondary outcomes in randomised controlled trials (RCTs) and to summarise what statistical methods are used for the analysis of PROs. DESIGN Comprehensive review. SETTING RCTs funded and published by the United Kingdom's (UK) National Institute for Health Research (NIHR) Health Technology Assessment (HTA) Programme. DATA SOURCES AND ELIGIBILITY HTA reports of RCTs published between January 1997 and December 2020 were reviewed. DATA EXTRACTION Information relating to PRO use and analysis methods was extracted. PRIMARY AND SECONDARY OUTCOME MEASURES The frequency of using PROs as primary and/or secondary outcomes; statistical methods that were used for the analysis of PROs as primary outcomes. RESULTS In this review, 37.6% (114/303) of trials used PROs as primary outcomes, and 82.8% (251/303) of trials used PROs as secondary outcomes from 303 NIHR HTA reports of RCTs. In the 114 RCTs where the PRO was the primary outcome, the most used PRO was the Short-Form 36 (8/114); the most popular methods for multivariable analysis were linear mixed model (45/114), linear regression (29/114) and analysis of covariance (13/114); logistic regression was applied for binary and ordinal outcomes in 14/114 trials; and the repeated measures analysis was used in 39/114 trials. CONCLUSION The majority of trials used PROs as primary and/or secondary outcomes. Conventional methods such as linear regression are widely used, despite the potential violation of their assumptions. In recent years, there is an increasing trend of using complex models (eg, with mixed effects). Statistical methods developed to address these violations when analysing PROs, such as beta-binomial regression, are not routinely used in practice. Future research will focus on evaluating available statistical methods for the analysis of PROs.
Collapse
Affiliation(s)
- Yirui Qian
- School of Health and Related Research, The University of Sheffield, Sheffield, UK
| | - Stephen J Walters
- School of Health and Related Research, The University of Sheffield, Sheffield, UK
| | - Richard Jacques
- School of Health and Related Research, The University of Sheffield, Sheffield, UK
| | - Laura Flight
- School of Health and Related Research, The University of Sheffield, Sheffield, UK
| |
Collapse
|
19
|
Raittio L, Launonen A, Mattila VM, Reito A. Estimates of the mean difference in orthopaedic randomized trials: obligatory yet obscure. BMC Med Res Methodol 2021; 21:59. [PMID: 33761900 PMCID: PMC7992936 DOI: 10.1186/s12874-021-01249-2] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/24/2020] [Accepted: 03/08/2021] [Indexed: 11/10/2022] Open
Abstract
Background Randomized controlled trials in orthopaedics are powered to mainly find large effect sizes. A possible discrepancy between the estimated and the real mean difference is a challenge for statistical inference based on p-values. We explored the justifications of the mean difference estimates used in power calculations. The assessment of distribution of observations in the primary outcome and the possibility of ceiling effects were also assessed. Methods Systematic review of the randomized controlled trials with power calculations in eight clinical orthopaedic journals published between 2016 and 2019. Trials with one continuous primary outcome and 1:1 allocation were eligible. Rationales and references for the mean difference estimate were recorded from the Methods sections. The possibility of ceiling effect was addressed by the assessment of the weighted mean and standard deviation of the primary outcome and its elaboration in the Discussion section of each RCT where available. Results 264 trials were included in this study. Of these, 108 (41 %) trials provided some rationale or reference for the mean difference estimate. The most common rationales or references for the estimate of mean difference were minimal clinical important difference (16 %), observational studies on the same subject (8 %) and the ‘clinical relevance’ of the authors (6 %). In a third of the trials, the weighted mean plus 1 standard deviation of the primary outcome reached over the best value in the patient-reported outcome measure scale, indicating the possibility of ceiling effect in the outcome. Conclusions The chosen mean difference estimates in power calculations are rarely properly justified in orthopaedic trials. In general, trials with a patient-reported outcome measure as the primary outcome do not assess or report the possibility of the ceiling effect in the primary outcome or elaborate further in the Discussion section.
Collapse
Affiliation(s)
- Lauri Raittio
- The Faculty of Medicine and Health Technology, Tampere University, Arvo Ylpön katu 34, 33520, Tampere, Finland.
| | - Antti Launonen
- Department of Orthopaedics and Traumatology, Tampere University Hospital, Teiskontie 35, 33520, Tampere, Finland
| | - Ville M Mattila
- The Faculty of Medicine and Health Technology, Tampere University, Arvo Ylpön katu 34, 33520, Tampere, Finland.,Department of Orthopaedics and Traumatology, Tampere University Hospital, Teiskontie 35, 33520, Tampere, Finland
| | - Aleksi Reito
- Department of Orthopaedics and Traumatology, Tampere University Hospital, Teiskontie 35, 33520, Tampere, Finland
| |
Collapse
|
20
|
Indrayan A. Aleatory and epistemic uncertainties can completely derail medical research results. J Postgrad Med 2020; 66:94-98. [PMID: 32134004 PMCID: PMC7239410 DOI: 10.4103/jpgm.jpgm_585_19] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/08/2023] Open
Abstract
Aleatory uncertainties are generated by intrinsic factors such as studying a sample rather than the whole population and the source of epistemic uncertainties is extraneous such as limitations of knowledge. These uncertainties inflict all the findings in empirical medical research, but they are rarely appreciated. This article highlights these uncertainties and shows with the help of an example how apparently valid and reliable findings can completely derail due to these uncertainties. We conclude that aleatory and epistemic uncertainties should get due consideration while drawing conclusions and before the results are put into practice. Methods to reduce their impact on results are also presented.
Collapse
Affiliation(s)
- A Indrayan
- Department of Clinical Research, Max Healthcare Institute, Saket, New Delhi, India
| |
Collapse
|
21
|
Graña Possamai C, Ravaud P, Ghosn L, Tran VT. Use of wearable biometric monitoring devices to measure outcomes in randomized clinical trials: a methodological systematic review. BMC Med 2020; 18:310. [PMID: 33153462 PMCID: PMC7646072 DOI: 10.1186/s12916-020-01773-w] [Citation(s) in RCA: 9] [Impact Index Per Article: 2.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 03/18/2020] [Accepted: 09/01/2020] [Indexed: 12/28/2022] Open
Abstract
BACKGROUND Wearable biometric monitoring devices (BMDs) have the potential to transform the conduct of randomized controlled trials (RCTs) by shifting the collection of outcome data from single measurements at predefined time points to dense continuous measurements. METHODS Methodological systematic review to understand how recent RCTs used BMDs to measure outcomes and to describe the reporting of these RCTs. Electronic search was performed in the Cochrane Central Register of Controlled Trials, PubMed, and EMBASE and completed a page-by-page hand search in five leading medical journals between January 1, 2018, and December 31, 2018. Three reviewers independently extracted all primary and secondary outcomes collected using BMDs, and assessed (1) the definitions used to summarize BMD outcome data; (2) whether the validity, reliability, and responsiveness of sensors was reported; (3) the discrepancy with outcomes prespecified in public clinical trial registries; and (4) the methods used to manage missing and incomplete BMD outcome data. RESULTS Of the 4562 records screened, 75 RCTs were eligible. Among them, 24% tested a pharmacological intervention and 57% used an inertial measurement sensor to measure physical activity. Included trials involved 464 outcomes (average of 6 [SD = 8] outcomes per trial). In total, 35 trials used a BMD to measure a primary outcome. Several issues affected the value and transparency of trials using BMDs to measure outcomes. First, the definition of outcomes used in the trials was highly heterogeneous (e.g., 21 diabetes trials had 266 outcomes and 153 had different unique definitions to measure diabetes control), which limited the combination and comparison of results. Second, information on the validity, reliability, and responsiveness of sensors used was lacking in 74% of trials. Third, half (53%) of the outcomes measured with BMDs had not been prespecified, with a high risk of outcome reporting bias. Finally, reporting on the management of incomplete outcome data (e.g., due to suboptimal compliance with the BMD) was absent in 68% of RCTs. CONCLUSIONS Use of BMDs to measure outcomes is becoming the norm rather than the exception in many fields. Yet, trialists need to account for several methodological issues when specifying and conducting RCTs using these novel tools.
Collapse
Affiliation(s)
- Carolina Graña Possamai
- METHODS Team, Center for Research in Epidemiology and Statistics (CRESS), Université de Paris/INSERM (UMR 1153), 1 Place du Parvis Notre Dame, 75004, Paris, France
| | - Philippe Ravaud
- METHODS Team, Center for Research in Epidemiology and Statistics (CRESS), Université de Paris/INSERM (UMR 1153), 1 Place du Parvis Notre Dame, 75004, Paris, France.,Centre d'Epidémiologie Clinique, Hôpital Hôtel-Dieu (AP-HP), 1 Place du Parvis Notre Dame, 75004, Paris, France.,Department of Epidemiology, Columbia University Mailman School of Public Health, 22 W 168th St, New York, NY, USA
| | - Lina Ghosn
- METHODS Team, Center for Research in Epidemiology and Statistics (CRESS), Université de Paris/INSERM (UMR 1153), 1 Place du Parvis Notre Dame, 75004, Paris, France.,Centre d'Epidémiologie Clinique, Hôpital Hôtel-Dieu (AP-HP), 1 Place du Parvis Notre Dame, 75004, Paris, France
| | - Viet-Thi Tran
- METHODS Team, Center for Research in Epidemiology and Statistics (CRESS), Université de Paris/INSERM (UMR 1153), 1 Place du Parvis Notre Dame, 75004, Paris, France. .,Centre d'Epidémiologie Clinique, Hôpital Hôtel-Dieu (AP-HP), 1 Place du Parvis Notre Dame, 75004, Paris, France.
| |
Collapse
|
22
|
Articulating the "So, What?" in Clinical Research: Insight from the M-CHOIR Group. PLASTIC AND RECONSTRUCTIVE SURGERY-GLOBAL OPEN 2020; 8:e2848. [PMID: 33133904 PMCID: PMC7572177 DOI: 10.1097/gox.0000000000002848] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/04/2020] [Accepted: 03/23/2020] [Indexed: 11/25/2022]
Abstract
With the academic culture of "publish or perish," authors must ensure that they are delivering high-quality data with a meaningful impact on clinical practice. Even for physician-scientists at the top of their fields, establishing the relevance of a study to clinical practice is a challenge. Thus, it is essential that research proposals ask questions that are clinically important, use appropriate methodologies, and examine outcomes that are relevant to both the physicians and the patients. The question of "so, what?" or in other words, "who cares?" is one that can make or break a study's impact on clinical practice. Researchers should use models such as PICOS (Population, Intervention, Comparison, Outcomes, and Study design) and FINER (Feasible, Interesting, Novel, Ethical, Relevant) and ask why readers will care about their study's findings before the study is conducted. By doing so, researchers can ensure the successful execution of their study and a meaningful impact of their findings, in both academia and clinical practice. This Special Topic article aims to guide researchers in producing relevant, impactful conclusions of their studies by providing input and resources from the Michigan Center for Hand Outcomes and Innovation (M-CHOIR) group.
Collapse
|
23
|
Mott A, Mitchell A, McDaid C, Harden M, Grupping R, Dean A, Byrne A, Doherty L, Sharma H. Systematic review assessing the evidence for the use of stem cells in fracture healing. Bone Jt Open 2020; 1:628-638. [PMID: 33215094 PMCID: PMC7659646 DOI: 10.1302/2633-1462.110.bjo-2020-0129] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 12/20/2022] Open
Abstract
AIMS Bone demonstrates good healing capacity, with a variety of strategies being utilized to enhance this healing. One potential strategy that has been suggested is the use of stem cells to accelerate healing. METHODS The following databases were searched: MEDLINE, CENTRAL, EMBASE, Cochrane Database of Systematic Reviews, WHO-ICTRP, ClinicalTrials.gov, as well as reference checking of included studies. The inclusion criteria for the study were: population (any adults who have sustained a fracture, not including those with pre-existing bone defects); intervention (use of stem cells from any source in the fracture site by any mechanism); and control (fracture healing without the use of stem cells). Studies without a comparator were also included. The outcome was any reported outcomes. The study design was randomized controlled trials, non-randomized or observational studies, and case series. RESULTS In all, 94 eligible studies were identified. The clinical and methodological aspects of the studies were too heterogeneous for a meta-analysis to be undertaken. A narrative synthesis examined study characteristics, stem cell methods (source, aspiration, concentration, and application) and outcomes. CONCLUSION Insufficient high-quality evidence is available to determine the efficacy of stem cells for fracture healing. The studies were heterogeneous in population, methods, and outcomes. Work to address these issues and establish standards for future research should be undertaken.Cite this article: Bone Joint Open 2020;1-10:628-638.
Collapse
Affiliation(s)
- Andrew Mott
- York Trials Unit, Department of Health Sciences, University of York, York, UK
| | - Alex Mitchell
- York Trials Unit, Department of Health Sciences, University of York, York, UK
| | - Catriona McDaid
- York Trials Unit, Department of Health Sciences, University of York, York, UK
| | - Melissa Harden
- Centre for Reviews and Dissemination, University of York, York, UK
| | - Rachael Grupping
- Department of Trauma and Orthopaedics, Hull University Teaching Hospitals, Hull, UK
| | - Alexandra Dean
- York Trials Unit, Department of Health Sciences, University of York, York, UK
| | - Ailish Byrne
- York Trials Unit, Department of Health Sciences, University of York, York, UK
| | - Laura Doherty
- York Trials Unit, Department of Health Sciences, University of York, York, UK
| | - Hemant Sharma
- Department of Trauma and Orthopaedics, Hull University Teaching Hospitals, Hull, UK
| |
Collapse
|
24
|
Hébert M, Cartier R, Dagenais F, Langlois Y, Coutu M, Noiseux N, El-Hamamsy I, Stevens LM. Standardizing Postoperative Complications-Validating the Clavien-Dindo Complications Classification in Cardiac Surgery. Semin Thorac Cardiovasc Surg 2020; 33:443-451. [PMID: 32979483 DOI: 10.1053/j.semtcvs.2020.09.029] [Citation(s) in RCA: 7] [Impact Index Per Article: 1.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/12/2020] [Accepted: 09/19/2020] [Indexed: 11/11/2022]
Abstract
Cardiac surgery lacks a method for quantifying postoperative morbidities. The Clavien-Dindo Complications Classification (CDCC) and the Comprehensive Complication Index (CCI) were successfully implemented as outcome reporting methods in other surgical specialties. This study aims to validate these complication scales in cardiac surgery. Between 2010 and 2019, we prospectively collected data on 41,218 adult patients (73% men, mean age 67 ± 11 years) undergoing cardiac surgery at 6 university hospitals. Complications were graded using the CDCC based on the complication's treatment invasiveness with adaptations for common treatments in cardiac surgery. CCI were calculated, representing multiple complications on a scale of 0 (no complication) to 100 (death). Associations with predictors of poor outcome were assessed using mixed-effects models accounting for center as a random effect. CDCC grade was 0 in 23.0%, I in 11.4%, II in 35.3%, IIIa in 6.4%, IIIb in 2.6%, IVa in 16.1%, IVb in 2.1%, and V in 3.1%. Median CCI was 23 (9, 40). A change from lowest to highest observed CDCC grade was associated with an increase in the Society of Thoracic Surgeons mortality score from 1.1% to 4.7%, surgery duration from 177 to 233 minutes, and hospital stay from 5.2 to 17 days (all P < 0.0001). The CCI also increased with greater procedure complexity (P < 0.0001). Increase in CDCC/CCI is associated with greater comorbidities, surgery durations, lengths of stay, and procedure complexity, accurately reflecting the nuances of the adult cardiac surgery postoperative course. These have great potential for uniform outcome reporting and quality improvement initiatives.
Collapse
Affiliation(s)
- Mélanie Hébert
- Faculty of Medicine, Université de Montréal, Montreal, Canada
| | - Raymond Cartier
- Faculty of Medicine, Université de Montréal, Montreal, Canada; Division of Cardiac Surgery, Montreal Heart Institute, Montreal, Canada
| | - François Dagenais
- Division of Cardiac Surgery, Quebec Heart and Lung Institute, Québec, Canada
| | - Yves Langlois
- Division of Cardiac Surgery, Jewish General Hospital, Montreal, Canada
| | - Marianne Coutu
- Division of Cardiac Surgery, Centre Hospitalier Universitaire de Sherbrooke, Sherbrooke, Canada
| | - Nicolas Noiseux
- Faculty of Medicine, Université de Montréal, Montreal, Canada; Division of Cardiac Surgery, Centre Hospitalier Universitaire de l'Université de Montréal, Montreal, Canada
| | - Ismail El-Hamamsy
- Faculty of Medicine, Université de Montréal, Montreal, Canada; Division of Cardiac Surgery, Montreal Heart Institute, Montreal, Canada; Division of Cardiac Surgery, Hôpital du Sacré-Coeur de Montréal, Montreal, Canada
| | - Louis-Mathieu Stevens
- Faculty of Medicine, Université de Montréal, Montreal, Canada; Division of Cardiac Surgery, Centre Hospitalier Universitaire de l'Université de Montréal, Montreal, Canada.
| |
Collapse
|
25
|
Bindernagel R, Cook C, Wayant C, Jellison S, Vassar M. Assessing the quality of intervention reporting in dermatology randomized controlled trials using the TIDieR checklist. Br J Dermatol 2020; 183:1114-1115. [PMID: 32628774 DOI: 10.1111/bjd.19382] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/26/2019] [Revised: 06/24/2020] [Accepted: 06/26/2020] [Indexed: 12/01/2022]
Affiliation(s)
- R Bindernagel
- Kansas City University of Medicine and Biosciences, Kansas City, MO, USA
| | - C Cook
- Oklahoma State University Center for Health Sciences, Tulsa, OK, USA
| | - C Wayant
- Oklahoma State University Center for Health Sciences, Tulsa, OK, USA
| | - S Jellison
- Oklahoma State University Center for Health Sciences, Tulsa, OK, USA
| | - M Vassar
- Oklahoma State University Center for Health Sciences, Tulsa, OK, USA
| |
Collapse
|
26
|
Boutron I, Créquit P, Williams H, Meerpohl J, Craig JC, Ravaud P. Future of evidence ecosystem series: 1. Introduction Evidence synthesis ecosystem needs dramatic change. J Clin Epidemiol 2020; 123:135-142. [DOI: 10.1016/j.jclinepi.2020.01.024] [Citation(s) in RCA: 15] [Impact Index Per Article: 3.8] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/02/2019] [Revised: 12/23/2019] [Accepted: 01/07/2020] [Indexed: 12/13/2022]
|
27
|
Créquit P, Boutron I, Meerpohl J, Williams HC, Craig J, Ravaud P. Future of evidence ecosystem series: 2. current opportunities and need for better tools and methods. J Clin Epidemiol 2020; 123:143-152. [DOI: 10.1016/j.jclinepi.2020.01.023] [Citation(s) in RCA: 10] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/02/2019] [Revised: 12/26/2019] [Accepted: 01/07/2020] [Indexed: 02/06/2023]
|
28
|
|
29
|
Randomized clinical trials in ANCA-associated vasculitis: a systematic analysis of the WHO - International Clinical Trials Registry Platform. Orphanet J Rare Dis 2020; 15:130. [PMID: 32471463 PMCID: PMC7257545 DOI: 10.1186/s13023-020-01408-6] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/01/2020] [Accepted: 05/14/2020] [Indexed: 01/19/2023] Open
Abstract
BACKGROUND The analysis of the main features of randomized controlled trials (RCTs) on ANCA-associated vasculitis (AAV) can inform future study design. METHODS We searched within the International Clinical Trials Registry Platform all registered RCTs on AAV from October 2008 to December 2018. Two reviewers selected studies according to pre-specified eligibility criteria. We retrieved information including countries, funding, design, sample sizes, eligibility criteria, primary outcomes (POs), and treatments. RESULTS Among the 40 RCTs identified, 22 (55%) were conducted in Europe, 29 (72,5%) in a single country, 14 (35%) were industry-funded. The median number of patients planned to enrol was 68 (IQR 36-138). Only 28% of RCTs targeted a single vasculitis, and ANCA negative patients were not included in about 40% of studies. Interventions investigated were mainly drugs given to induce (40%) or maintain (32.5%) remission. Eighty-five percent of POs were considered being 'patient-important', but discrepancies in definition of disease states, such as remission or relapse were observed. Glucocorticoids use was part of the PO in < 25% of studies. The number of trials targeting a single disease, non-industry funded, incorporating glucocorticoids in PO, as well as the planned sample size increased over time. CONCLUSION Despite the important achievements in the field, a better harmonization of eligibility, and outcome criteria across studies is an important objective to pursue in next future.
Collapse
|
30
|
Reporting of Organ Support Outcomes in Septic Shock Randomized Controlled Trials: A Methodologic Review-The Sepsis Organ Support Study. Crit Care Med 2020; 47:984-992. [PMID: 30889023 DOI: 10.1097/ccm.0000000000003746] [Citation(s) in RCA: 8] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/23/2023]
Abstract
OBJECTIVES Many recent randomized controlled trials in the field of septic shock failed to demonstrate a benefit on mortality. Randomized controlled trials increasingly report organ support duration and organ support-free days as primary or secondary outcomes. We conducted a methodologic systematic review to assess how organ support outcomes were defined and reported in septic shock randomized controlled trials. DATA SOURCES MEDLINE via PubMed, Embase, Cochrane Central Register of Controlled Trials, and Web of Science. STUDY SELECTION We included randomized controlled trials published between January 2004 and March 2018 that involved septic shock adults and assessed organ support duration and/or organ support-free days for hemodynamic support, respiratory support, or renal replacement therapy. DATA EXTRACTION For each randomized controlled trial, we extracted the definitions of organ support duration and organ support-free days. We particularly evaluated how nonsurvivors were accounted for. Study authors were contacted to provide any missing information regarding these definitions. DATA SYNTHESIS We included 28 randomized controlled trials. Organ support duration and organ support-free days outcomes were reported in 17 and 15 randomized controlled trials, respectively, for hemodynamic support, 15 and 15 for respiratory support, and five and nine for renal replacement therapy. Nonsurvivors were included in the organ support duration calculation in 13 of 14 randomized controlled trials (93%) for hemodynamic support and nine of 10 (90%) for respiratory support. The organ support-free days definition for hemodynamic support, respiratory support, and renal replacement therapy was reported in six of 15 randomized controlled trials (40%), eight of 15 randomized controlled trials (53%), and six of nine randomized controlled trials (67%) reporting an organ support-free days outcome, respectively. Of these, one half assigned "0" to nonsurvivors, and the other half attributed one point per day alive free of organ support up to a predefined time point. CONCLUSIONS This study highlights the heterogeneity and infrequency of organ support duration/organ support-free days outcome reporting in septic shock trials. When reported, the definitions of these outcome measures and methods of calculation are also infrequently reported, in particular how nonsurvivors were accounted for, which may have an important impact on interpretation.
Collapse
|
31
|
Zhang MWB, Heng S, Amron SB, Mahreen Z, Song G, Fung DSS, Smith HE. Gamified M-Health Attention Bias Modification Intervention for Individuals with Opioid Use Disorder: Protocol for a Pilot Randomised Study. INTERNATIONAL JOURNAL OF ENVIRONMENTAL RESEARCH AND PUBLIC HEALTH 2020; 17:ijerph17030752. [PMID: 31991620 PMCID: PMC7037559 DOI: 10.3390/ijerph17030752] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 12/18/2019] [Revised: 01/22/2020] [Accepted: 01/23/2020] [Indexed: 12/11/2022]
Abstract
Introduction: Globally, there is an epidemic of opioid use disorders. Locally, in Singapore, there is an increase in the number of individuals abusing opioids. The advances in experimental psychology have highlighted the need to modify unconscious, automatic biases. These automatic, unconscious biases result in individuals having preferential attention to substance-related cues in their natural environment, thus leading to a slip or relapse back into their underlying addictive disorders. Prior studies have demonstrated not only the presence of robust attentional biases amongst individuals with opioid use disorder, even when maintained on methadone; and the effectiveness of bias modification amongst these individuals. There remains limited evaluation of attention bias modification amongst a treatment-seeking cohort of Asian individuals. The objective of this pilot is to ensure that the methods of the planned definitive randomized trial could be conducted. Methods and Analysis: A non-blinded pilot randomized trial will be conducted. A total of 30 participants will be randomized to receive either the conventional application or the newly designed co-designed application. In order to identify these 30 participants, 60 participants will be recruited and screened to determine if they have baseline biases. Participants will be recruited from the inpatient unit at the National Addictions Management Service (NAMS) Singapore. All participants who are enrolled into the trial will complete a baseline assessment task, and a bias modification assessment and modification task daily. They will have to complete a baseline demographic and clinical information questionnaire, as well as a cravings rating scale before and after the intervention daily. Perspectives—that of self-reported experiences—will be sought from the participants following their completion of three intervention tasks. Descriptive statistical analyses will be performed, and chi-square and ANOVA analyses will be performed. Qualitative analyses will be undertaken for the perspectives shared. Ethics and Dissemination: Ethical approval has been obtained from the National Healthcare Group’s Domain Specific Research Board (DSRB) (approval number that of 2019/00934). The findings arising from this study will be disseminated by means of conferences and publications.
Collapse
Affiliation(s)
- Melvyn W. B. Zhang
- National Addiction Management Service, Institute of Mental Health, Singapore 539747, Singapore; (S.H.); (S.B.A.); (Z.M.); (G.S.)
- Correspondence: ; Tel.: +65-3892504
| | - Sandor Heng
- National Addiction Management Service, Institute of Mental Health, Singapore 539747, Singapore; (S.H.); (S.B.A.); (Z.M.); (G.S.)
| | - Syidda B. Amron
- National Addiction Management Service, Institute of Mental Health, Singapore 539747, Singapore; (S.H.); (S.B.A.); (Z.M.); (G.S.)
| | - Zaakira Mahreen
- National Addiction Management Service, Institute of Mental Health, Singapore 539747, Singapore; (S.H.); (S.B.A.); (Z.M.); (G.S.)
| | - Guo Song
- National Addiction Management Service, Institute of Mental Health, Singapore 539747, Singapore; (S.H.); (S.B.A.); (Z.M.); (G.S.)
| | | | - Helen E. Smith
- Family Medicine and Primary Care, Lee Kong Chian School of Medicine, Nanyang Technological University Singapore, Singapore 308232, Singapore;
| |
Collapse
|
32
|
Lourenço TRM, Pergialiotis V, Durnea CM, Elfituri A, Haddad JM, Betschart C, Falconi G, Nygaard CC, Bergstrom L, Pattel M, Doumouchtsis SK. A systematic review of reported outcomes and outcome measures in randomized trials evaluating surgical interventions for posterior vaginal prolapse to aid development of a core outcome set. Int J Gynaecol Obstet 2020; 148:271-281. [PMID: 31814121 DOI: 10.1002/ijgo.13079] [Citation(s) in RCA: 13] [Impact Index Per Article: 3.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Revised: 05/05/2019] [Accepted: 12/06/2019] [Indexed: 12/12/2022]
Abstract
BACKGROUND Recent systematic reviews have demonstrated wide variations on outcome measure selection and outcome reporting in trials on surgical treatments for anterior, apical and mesh prolapse surgery. A systematic review of reported outcomes and outcome measures in posterior compartment vaginal prolapse interventions is highly warranted in the process of developing core outcome sets. OBJECTIVE To evaluate outcome and outcome measures reporting in posterior prolapse surgical trials. SEARCH STRATEGY We searched MEDLINE, EMBASE and the Cochrane Central Register of Controlled Trials (CENTRAL). SELECTION CRITERIA Randomized trials evaluating the efficacy and safety of different surgical interventions for posterior compartment vaginal prolapse. DATA COLLECTION AND ANALYSIS Two researchers independently assessed studies for inclusion, evaluated methodological quality, and extracted relevant data. Methodological quality, outcome reporting quality and publication characteristics were evaluated. MAIN RESULTS Twenty-seven interventional and four follow-up trials were included. Seventeen studies enrolled patients with posterior compartment surgery as the sole procedure and 14 with multicompartment procedures. Eighty-three reported outcomes and 45 outcome measures were identified. The most frequently reported outcomes were blood loss (20 studies, 74%), pain (18 studies, 66%) and infection (16 studies, 59%). CONCLUSIONS Wide variations in reported outcomes and outcome measures were found. Until a core outcome set is established, we propose an interim core outcome set that could include the three most commonly reported outcomes of the following domains: hospitalization; intraoperative, postoperative urinary, gastrointestinal, vaginal and sexual outcomes; clinical effectiveness. PROSPERO CRD42017062456.
Collapse
Affiliation(s)
- Thais R M Lourenço
- Department of Urogynecology, Clinical Hospital of the University of São Paulo School of Medicine, São Paulo University, São Paulo, Brazil
| | - Vasilis Pergialiotis
- Laboratory of Experimental Surgery and Surgical Research "N.S. Christeas", Athens University Medical School, Athens, Greece
| | - Constantin M Durnea
- Department of Obstetrics and Gynecology, Epsom and St Helier University Hospitals NHS Trust, London, UK.,Northwick Park Hospital, London North West University Healthcare NHS Trust, London, UK
| | - Abdullatif Elfituri
- Department of Obstetrics and Gynecology, Epsom and St Helier University Hospitals NHS Trust, London, UK
| | - Jorge M Haddad
- Department of Urogynecology, Clinical Hospital of the University of São Paulo School of Medicine, São Paulo University, São Paulo, Brazil
| | - Cornelia Betschart
- Department of Gynecology, University Hospital of Zurich, Zurich, Switzerland
| | - Gabriele Falconi
- Department of Obstetrics and Gynecology, San Bortolo Hospital, Vicenza, Italy
| | - Christiana C Nygaard
- Department of Obstetrics and Gynecology, Epsom and St Helier University Hospitals NHS Trust, London, UK.,Department of Obstetrics and Gynecology, Hospital São Lucas, Pontifical Catholic University of Rio Grande do Sul, Rio Grande do Sul, Brazil
| | | | | | - Stergios K Doumouchtsis
- Laboratory of Experimental Surgery and Surgical Research "N.S. Christeas", Athens University Medical School, Athens, Greece.,Department of Obstetrics and Gynecology, Epsom and St Helier University Hospitals NHS Trust, London, UK.,St George's University of London, London, UK
| | | |
Collapse
|