Avoidable waste of research related to outcome planning and reporting in clinical trials

Development and validation of observational and qualitative study protocol reporting checklists for novice researchers (ObsQual checklist)

INTRODUCTION

Currently, no reporting guidelines exist for observational and qualitative study protocols. In an effort to enhance the quality of research protocols, we introduce two study protocol reporting checklists that we have developed.

MAIN RECOMMENDATIONS

These checklists include educational components and examples intended to assist novice researchers. Through the analysis of 333 study protocols submitted for ethical review, our checklists have been developed and validated, demonstrating their applicability across various observational and qualitative study designs.

CHANGES IN MANAGEMENT

We provide insights into the systematic implementation of these checklists alongside complementary elements that support their effectiveness. We recommend longitudinal monitoring and evaluation of checklist utilization.

Selective outcome reporting in trials of behavioural health interventions in health psychology and behavioural medicine journals: a review

Selective outcome reporting can result in overestimation of treatment effects, research waste, and reduced openness and transparency. This review aimed to examine selective outcome reporting in trials of behavioural health interventions and determine potential outcome reporting bias. A review of nine health psychology and behavioural medicine journals was conducted to identify randomised controlled trials of behavioural health interventions published since 2019. Discrepancies in outcome reporting were observed in 90% of the 29 trials with corresponding registrations/protocols. Discrepancies included 72% of trials omitting prespecified outcomes; 55% of trials introduced new outcomes. Thirty-eight percent of trials omitted prespecified and introduced new outcomes. Three trials (10%) downgraded primary outcomes in registrations/protocols to secondary outcomes in final reports; downgraded outcomes were not statistically significant in two trials. Five trials (17%) upgraded secondary outcomes to primary outcomes; upgraded outcomes were statistically significant in all trials. In final reports, three trials (7%) omitted outcomes from the methods section; three trials (7%) introduced new outcomes in results that were not in the methods. These findings indicate that selective outcome reporting is a problem in behavioural health intervention trials. Journal- and trialist-level approaches are needed to minimise selective outcome reporting in health psychology and behavioural medicine.

Five aspects of research waste in biomedicine: A scoping review

BACKGROUND

The number of published journal articles has grown exponentially during the last 30 years, which may have led to some wasteful research. However, the terminology associated with research waste remains unclear. To address this, we aimed to identify, define, and categorize the aspects of research waste in published biomedical reports.

METHODS

In this scoping review, we systematically searched for biomedical literature reports from 1993 to 2023 in two databases, focusing on those addressing and defining research waste. Through data charting, we analyzed and categorized the aspects of research waste.

RESULTS

Based on 4285 initial records in the searches, a total of 832 reports were included in the analysis. The included reports were primarily narrative reviews (26%) and original reports (21%). We categorized research waste into five aspects: methodological, invisible, negligible, underreported, and structural (MINUS) research waste. More than half of the reports (56%) covered methodological research waste concerning flaws in study design, study conduct, or analysis. Invisible research waste covered nonpublication, discontinuation, and lack of data-sharing. Negligible research waste primarily concerned unnecessary repetition, for example, stemming from the absence of preceding a trial with a systematic review of the literature. Underreported research waste mainly included poor reporting, resulting in a lack of transparency. Structural research waste comprised inadequate management, collaboration, prioritization, implementation, and dissemination.

CONCLUSION

MINUS encapsulates the five main aspects of research waste. Recognizing these aspects of research waste is important for addressing and preventing further research waste and thereby ensuring efficient resource allocation and scientific integrity.

A Decade of Efforts to Add Value to Child Health Research Practices

OBJECTIVE

To identify practices that add value to improve the design, conduct, and reporting of child health research and reduce research waste.

STUDY DESIGN

In order to categorize the contributions of members of Standards for Research (StaR) in Child Health network, we developed a novel Child Health Improving Research Practices (CHIRP) framework comprised of 5 domains meant to counteract avoidable child health research waste and improve quality: 1) address research questions relevant to children, their families, clinicians, and researchers; 2) apply appropriate research design, conduct and analysis; 3) ensure efficient research oversight and regulation; 4) Provide accessible research protocols and reports; and 5) develop unbiased and usable research reports, including 17 responsible research practice recommendations. All child health research relevant publications by the 48 original StaR standards' authors over the last decade were identified, and main topic areas were categorized using this framework.

RESULTS

A total of 247 publications were included in the final sample: 100 publications (41%) in domain 1 (3 recommendations), 77 publications (31%) in domain 2 (3), 35 publications (14%) in domain 3 (4), 20 publications (8%) in domain 4 (4), and 15 publications (6%) in domain 5 (3). We identified readily implementable "responsible" research practices to counter child health research waste and improve quality, especially in the areas of patients and families' engagement throughout the research process, developing Core Outcome Sets, and addressing ethics and regulatory oversight issues.

CONCLUSION

While most of the practices are readily implementable, increased awareness of methodological issues and wider guideline uptake is needed to improve child health research. The CHIRP Framework can be used to guide responsible research practices that add value to child health research.

Gynecological trials frequently exclude people based on their symptoms rather than their condition: a systematic review of Cochrane reviews and their component trials

OBJECTIVES

To identify strategies used in recent randomized controlled trials (RCTs) and their associated Cochrane Reviews where patients with the same gynecological condition present with different symptoms but would plausibly benefit from a common intervention.

STUDY DESIGN AND SETTING

We searched the Cochrane library (February 2022) for reviews in polycystic ovarian syndrome (PCOS) and endometriosis. Reviews were included if the intervention was intended to treat all condition-specific symptoms. For each trial we recorded the strategy used and the number of potentially eligible participants excluded as a direct result of the chosen strategy. For each review we recorded the numbers of RCTs and participants excluded on the basis of symptoms experienced.

RESULTS

There were 89 distinct PCOS trials in 13 reviews, and 13 Endometriosis trials in 11 reviews. Most trials restricted their eligibility to participants with specific symptoms (55% PCOS, 46% endometriosis). The second most common strategy was to measure and analyze clinical outcomes that were not relevant to all participants (38% PCOS, 31% endometriosis). Reviews excluded 27% of trials in participants evaluating the same intervention in participants experiencing the same condition based on the outcomes measured in the trials.

CONCLUSION

Most gynecological trials exclude patients who could benefit from treatment or measure outcomes not relevant to all participants. We introduce a taxonomy to describe trial design strategies for conditions with heterogeneous symptoms.

Reasons for missing evidence in rehabilitation meta-analyses: a cross-sectional meta-research study

BACKGROUND

Systematic reviews of randomized controlled trials are the best evidence for informing on intervention effectiveness. Their results, however, can be biased due to omitted evidence in the quantitative analyses. We aimed to assess the proportion of randomized controlled trials omitted from meta-analyses in the rehabilitation field and explore related reasons.

METHODS

This is a cross-sectional meta-research study. For each systematic review included in a published selected sample in the rehabilitation field, we identified an index meta-analysis on the primary outcome and the main comparison. We then looked at all the studies considered eligible for the chosen comparison in the systematic review and identified those trials that have been omitted (i.e., not included) from each index meta-analysis. Reasons for omission were collected based on an eight-reason classification. We used descriptive statistics to describe the proportion of omitted trials overall and according to each reason.

RESULTS

Starting from a cohort of 827 systematic reviews, 131 index meta-analyses comprising a total of 1761 eligible trials were selected. Only 16 index meta-analyses included all eligible studies while 15 omitted studies without providing references. From the remaining 100 index meta-analyses, 717 trials (40,7%) were omitted overall. Specific reasons for omission were: "unable to distinguish between selective reporting and inadequate planning" (39,3%, N = 282), "inadequate planning" (17%, N = 122), "justified to be not included" (15,1%, N = 108), "incomplete reporting" (8,4%, N = 60), "selective reporting" (3,3%, N = 24) and other situations (e.g., outcome present but no motivation for omission) (5,2%, N = 37). The 11,7% (N = 84) of omitted trials were not assessed due to non-English language or full text not available.

CONCLUSIONS

Almost half of the eligible trials were omitted from their index meta-analyses. Better reporting, protocol registration, definition and adoption of core outcome sets are needed to prevent omission of evidence in systematic reviews.

Exploring the perspectives of selectors and collecters of trial outcome data: an international qualitative study

INTRODUCTION

Selecting and collecting data to support appropriate primary and secondary outcomes is a critical step in designing trials that can change clinical practice. In this study, we aimed to investigate who contributes to the process of selecting and collecting trial outcomes, and how these people are involved. This work serves two main purposes: (1) it provides the trials community with evidence to demonstrate how outcomes are currently selected and collected, and (2) it allows people involved in trial design and conduct to pick apart these processes to consider how efficiencies and improvements can be made.

METHODS

One-with-one semi-structured interviews, supported by a topic guide to ensure coverage of key content. The Framework approach was used for thematic analysis of data, and themes were linked through constant comparison of data both within and across participant groups. Interviews took place between July 2020 and January 2021. Participants were twenty-nine international trialists from various contributor groups, working primarily on designing and/or delivering phase III pragmatic effectiveness trials. Their experience spanned various funders, trial settings, clinical specialties, intervention types, and participant populations.

RESULTS

We identified three descriptive themes encompassing the process of primary and secondary outcome selection, collection, and the publication of outcome data. Within these themes, participants raised issues around the following: 1) Outcome selection: clarity of the research question; confidence in selecting trial outcomes and how confidence decreases with increased experience; interplay between different interested parties; how patients and the public are involved in outcome selection; perceived impact of poor outcome selection including poor recruitment and/or retention; and use of core outcome sets. 2) Outcome collection: disconnect between decisions made by outcome selectors and the practical work done by outcome collectors; potential impact of outcome measures on trial participants; potential impact on trial staff workload; and use of routinely collected data. 3) Publication of outcome data: difficulties in finding time to write and revise manuscripts for publication due to time and funding constraints. Participants overwhelmingly focused on the process of outcome selection, a topic they talked about unprompted. When prompted, participants do discuss outcome collection, but poor communication between selectors and collectors at the trial design stage means that outcome selection is rarely linked with the data collection workload it generates.

DISCUSSION

People involved in the design and conduct of trials fail to connect decisions around outcome selection with data collection workload. Publication of outcome data and effective dissemination of trial results are hindered due to the project-based culture of some academic clinical trial research.

Telehealth palliative care interventions for patients with advanced cancer: a scoping review

PURPOSE

Telehealth allows patients to maintain contact with healthcare providers without necessitating travel, and is becoming increasingly utilized. The purpose of this study is to describe the components of telehealth palliative care interventions for patients with advanced cancer before the COVID-19 pandemic; identify any intervention components associated with improvements in outcomes; and evaluate reporting of interventions.

METHODS

This scoping review was registered on the Open Science Framework. We searched 5 medical databases from inception to June 19, 2020. Inclusion criteria were: age ≥ 18, advanced cancer, asynchronous or synchronous telehealth intervention, and specialized palliative care interventions in any setting. We assessed the quality of intervention reporting using the Template for Intervention Description and Replication (TIDieR) checklist.

RESULTS

Twenty-three studies met the inclusion criteria: 15 (65%) quantitative (7 randomized controlled trials, 5 feasibility trials, 3 retrospective chart reviews); 4 (17%) mixed methods, and 4 (17%) qualitative. Most quantitative and mixed methods studies were conducted in North America (12/19, 63%), reported on hybrid (in-person and telehealth) interventions (9/19, 47%), and were delivered by nurses (12/19, 63%) in the home setting (14/19, 74%). In most studies that reported improvements in patient- or caregiver-reported outcomes, the content was psychoeducational and resulted in improvements for psychological symptoms. No study provided complete reporting on all 12 TIDieR checklist items.

CONCLUSION

Telehealth studies are needed that reflect palliative care's mission to provide multidisciplinary team-based care that improves quality of life in diverse settings, and that provide detailed reporting of interventions.

Heterogeneity in the definition of delirium in ICUs and association with the intervention effect in randomized controlled trials: a meta-epidemiological study

PURPOSE

To evaluate the heterogeneity in the definition of delirium in randomized controlled trials (RCTs) included in meta-analyses of delirium in intensive care units (ICUs) and to explore whether intervention effect depends on the definition used.

METHODS

We searched PubMed for meta-analyses including RCTs evaluating prevention or treatment strategies of delirium in ICU. The definition of delirium was collected from RCTs and classified as validated (DSM criteria, CAM-ICU, ICDSC, NEECHAM, DRS-R98) or non-validated (non-validated scales, set of symptoms, physician appreciation or not reported). We conducted a meta-epidemiological analysis to compare intervention effects between trials using or not a validated definition by a two-step method as primary analysis and a multilevel model as secondary analysis. A ratio of odds ratios (ROR) < 1 indicated larger intervention effects in trials using a non-validated definition.

RESULTS

Of 149 RCTs (41 meta-analyses), 109 (73.1%) used a validated definition and 40 (26.8%) did not (including 31 [20.8%] not reporting the definition). The primary analysis of 7 meta-analyses (30 RCTs) found no significant difference in intervention effects between trials using a validated definition and the others (ROR = 0.54, 95% CI 0.27-1.08), whereas the secondary multilevel analysis including 12 meta-analyses (67 RCTs) found significantly larger effects for trials using a non-validated versus a validated definition (ROR = 0.36, 95% CI 0.21-0.62).

CONCLUSION

The definition of delirium was heterogeneous across RCTs, with one-fifth not reporting how they evaluated delirium. We did not find a significant association with intervention effect in the primary analysis. The secondary analysis including more studies revealed significantly larger intervention effects in trials using a non-validated versus a validated definition.

The Consistency between Registered Acupuncture-Moxibustion Clinical Studies and Their Published Studies and Update Status of Registered Information

BACKGROUND

Few studies have analyzed the consistency between registered acupuncture-moxibustion clinical studies and their published research results as well as their update status of registered information.

METHODS

We searched for acupuncture-moxibustion clinical studies that were registered at the World Health Organization International Clinical Trials Registry Platform between 2013 and 2015 and collected data regarding their characteristics and update status. Published results of these registered studies were identified and compared with registered information.

RESULTS

A total of 425 registered acupuncture-moxibustion clinical studies were included; 379 (89.2%) of them were interventional studies, and the remaining 46 (10.8%) were observational studies. Forty-six studies (10.8%) were found to have published results, and 51 published articles were identified. Overall, 73.2% (311) of registered studies did not update the research status in time; 46.6% (198) stopped updating before recruiting; 21.6% (92) stopped updating after recruiting; and 4.9% (21) stopped updating after completion. Regarding the 46 studies with published results, 29 (63.0%) were considered to be affected by reporting bias. These reporting biases predominantly involved the omission of some predefined outcomes or endpoints (16 studies), contradictions regarding descriptions of sample sizes (9 studies), discrepancies in treatment measurements or group distribution (7 studies), and inconsistent treatment durations (4 studies). When compared with other studies, significant and various reporting biases could also be commonly found in fields other than acupuncture-moxibustion.

CONCLUSIONS

There were many discrepancies between registered information and published reports on acupuncture-moxibustion, which could also be commonly observed in other fields. Moreover, a large proportion of registered studies did not update their research status in time. Efforts should be made to improve the reporting quality and timely updates.

Hintergrund

Es gibt nur wenige Studien, in denen die Übereinstimmung zwischen den registrierten klinischen Studien zur Akupunktur und Moxibustion mit den veröffentlichten Studienergebnissen und dem Aktualisierungsstand der Informationen im Register untersucht wurde.

Methoden

Wir suchten nach klinischen Studien zur Akupunktur und Moxibustion, die zwischen 2013 und 2015 auf der International Clinical Trials Registry Platform der Weltgesundheitsorganisation registriert wurden, und erhoben Daten zu ihren Merkmalen und ihrem Aktualisierungsstand. Die veröffentlichten Ergebnisse der registrierten Studien wurden identifiziert und mit den Informationen im Register verglichen.

Ergebnisse

Insgesamt wurden 425 registrierte klinische Studien zur Akupunktur und Moxibustion eingeschlossen, davon waren 379 (89,2 %) Interventionsstudien und die restlichen 46 (10,8 %) waren Beobachtungsstudien. Es wurden 46 Studien (10,8 %) mit veröffentlichten Ergebnissen gefunden und 51 veröffentlichte Artikel identifiziert. Insgesamt wurde bei 73,2 % (311) der registrierten Studien der Forschungsstand nicht zeitnah aktualisiert; bei 46,6 % (198) wurde die Aktualisierung vor der Rekrutierung eingestellt; bei 21,6 % (92) wurde die Aktualisierung nach der Rekrutierung eingestellt und bei 4,9 % (21) wurde die Aktualisierung nach Abschluss der Studie eingestellt. Von den 46 Studien mit veröffentlichten Ergebnissen wurden 29 (63,0 %) als von Publikationsverzerrung betroffen angesehen. Diese Publikationsverzerrung betraf vor allem die Auslassung einiger vordefinierter Zielkriterien oder Endpunkte (16 Studien), Widersprüche bei der Beschreibung des Stichprobenumfangs (9 Studien), Diskrepanzen bei den Behandlungsmessungen oder der Gruppenverteilung (7 Studien) und Inkonsistenzen bei der Behandlungsdauer (4 Studien). Beim Vergleich mit anderen Studien wurden auch in anderen Bereichen als Akupunktur und Moxibustion häufig signifikante und unterschiedliche Publikationsverzerrungen festgestellt.

Schlussfolgerungen

Es bestanden zahlreiche Diskrepanzen zwischen den Informationen im Register und den veröffentlichten Berichten über Akupunktur und Moxibustion, die auch in anderen Bereichen häufig zu beobachten waren. Darüber hinaus wurde bei einem Großteil der registrierten Studien der Forschungsstand nicht zeitnah aktualisiert. Es sollten Anstrengungen unternommen werden, um die Qualität der Berichterstattung und die zeitnahe Aktualisierung zu verbessern.

Guidelines for Reporting Outcomes in Trial Protocols: The SPIRIT-Outcomes 2022 Extension

IMPORTANCE

Complete information in a trial protocol regarding study outcomes is crucial for obtaining regulatory approvals, ensuring standardized trial conduct, reducing research waste, and providing transparency of methods to facilitate trial replication, critical appraisal, accurate reporting and interpretation of trial results, and knowledge synthesis. However, recommendations on what outcome-specific information should be included are diverse and inconsistent. To improve reporting practices promoting transparent and reproducible outcome selection, assessment, and analysis, a need for specific and harmonized guidance as to what outcome-specific information should be addressed in clinical trial protocols exists.

OBJECTIVE

To develop harmonized, evidence- and consensus-based standards for describing outcomes in clinical trial protocols through integration with the Standard Protocol Items: Recommendations for Interventional Trials (SPIRIT) 2013 statement.

EVIDENCE REVIEW

Using the Enhancing the Quality and Transparency of Health Research (EQUATOR) methodological framework, the SPIRIT-Outcomes 2022 extension of the SPIRIT 2013 statement was developed by (1) generation and evaluation of candidate outcome reporting items via consultation with experts and a scoping review of existing guidance for reporting trial outcomes (published within the 10 years prior to March 19, 2018) identified through expert solicitation, electronic database searches of MEDLINE and the Cochrane Methodology Register, gray literature searches, and reference list searches; (2) a 3-round international Delphi voting process (November 2018-February 2019) completed by 124 panelists from 22 countries to rate and identify additional items; and (3) an in-person consensus meeting (April 9-10, 2019) attended by 25 panelists to identify essential items for outcome-specific reporting to be addressed in clinical trial protocols.

FINDINGS

The scoping review and consultation with experts identified 108 recommendations relevant to outcome-specific reporting to be addressed in trial protocols, the majority (72%) of which were not included in the SPIRIT 2013 statement. All recommendations were consolidated into 56 items for Delphi voting; after the Delphi survey process, 19 items met criteria for further evaluation at the consensus meeting and possible inclusion in the SPIRIT-Outcomes 2022 extension. The discussions during and after the consensus meeting yielded 9 items that elaborate on the SPIRIT 2013 statement checklist items and are related to completely defining and justifying the choice of primary, secondary, and other outcomes (SPIRIT 2013 statement checklist item 12) prospectively in the trial protocol, defining and justifying the target difference between treatment groups for the primary outcome used in the sample size calculations (SPIRIT 2013 statement checklist item 14), describing the responsiveness of the study instruments used to assess the outcome and providing details on the outcome assessors (SPIRIT 2013 statement checklist item 18a), and describing any planned methods to account for multiplicity relating to the analyses or interpretation of the results (SPIRIT 2013 statement checklist item 20a).

CONCLUSIONS AND RELEVANCE

This SPIRIT-Outcomes 2022 extension of the SPIRIT 2013 statement provides 9 outcome-specific items that should be addressed in all trial protocols and may help increase trial utility, replicability, and transparency and may minimize the risk of selective nonreporting of trial results.

Some conservative interventions are more effective than others for people with chronic non-specific neck pain: a systematic review and network meta-analysis

QUESTION

Which is the most effective conservative intervention for patients with non-specific chronic neck pain (CNSNP)?

DESIGN

A systematic review and network meta-analysis of randomised clinical trials.

PARTICIPANTS

Adults with CNSNP of at least 3 months duration.

INTERVENTIONS

All available pharmacological and non-pharmacological interventions.

OUTCOME MEASURES

The primary outcomes were pain intensity and disability. The secondary outcome was adverse events.

RESULTS

Overall, 119 RCTs (12,496 patients; 32 interventions) were included. Risk of bias was low in 50.4% of trials, unclear in 22.7% and high in 26.9%. Compared with inert treatment, a combination of active and/or passive multimodal non-pharmacological inventions (eg, exercise and manual therapy) were effective for pain on a 0-to-10 scale at 1 month (MD range 0.84 to 3.74) and at 3 to 6 months (MD range 1.06 to 1.49), and effective on disability on a 0-to-100 scale at 1 month (MD range 10.26 to 14.09) and 3 to 6 months (MD range 5.60 to 16.46). These effects ranged from possible to definite clinical relevance. Compared with inert treatment, anti-inflammatory drugs alone or in combination with another non-pharmacological treatment did not reduce pain at 1 month or 3 to 6 months. At 12 months, no superiority was found over inert treatment on both outcomes. Most mild adverse events were experienced following acupuncture/dry needling intervention. On average, the evidence varied from low to very low certainty.

CONCLUSIONS

While multimodal non-pharmacological interventions may reduce pain and disability for up to 3 to 6 months of follow-up when compared with inert treatment, the evidence was very uncertain about their effects. Better quality and larger trials are needed to improve the certainty of evidence.

REGISTRATION

PROSPERO CRD42019124501.

The proportion of randomized controlled trials that inform clinical practice

Prior studies suggest that clinical trials are often hampered by problems in design, conduct and reporting that limit their uptake in clinical practice. We have described 'informativeness' as the ability of a trial to guide clinical, policy or research decisions. Little is known about the proportion of initiated trials that inform clinical practice. We created a cohort of randomized interventional clinical trials in three disease areas (ischemic heart disease, diabetes mellitus and lung cancer), that were initiated between 1 January 2009 and 31 December 2010 using ClinicalTrials.gov. We restricted inclusion to trials aimed at answering a clinical question related to the treatment or prevention of disease. Our primary outcome was the proportion of clinical trials fulfilling four conditions of informativeness: importance of the clinical question, trial design, feasibility, and reporting of results. Our study included 125 clinical trials. The proportion meeting four conditions for informativeness was 26.4% (95% CI 18.9 - 35.0). Sixty-seven percent of participants were enrolled in informative trials. The proportion of informative trials did not differ significantly between our three disease areas. Our results suggest that the majority of clinical trials designed to guide clinical practice possess features that may compromise their ability to do so. This highlights opportunities to improve the scientific vetting of clinical research.

How informative were early SARS-CoV-2 treatment and prevention trials? a longitudinal cohort analysis of trials registered on ClinicalTrials.gov

Background

Early in the SARS-CoV-2 pandemic, commentators warned that some COVID trials were inadequately conceived, designed and reported. Here, we retrospectively assess the prevalence of informative COVID trials launched in the first 6 months of the pandemic.

Methods

Based on prespecified eligibility criteria, we created a cohort of Phase 1/2, Phase 2, Phase 2/3 and Phase 3 SARS-CoV-2 treatment and prevention efficacy trials that were initiated from 2020-01-01 to 2020-06-30 using ClinicalTrials.gov registration records. We excluded trials evaluating behavioural interventions and natural products, which are not regulated by the U.S. Food and Drug Administration (FDA). We evaluated trials on 3 criteria of informativeness: potential redundancy (comparing trial phase, type, patient-participant characteristics, treatment regimen, comparator arms and primary outcome), trials design (according to the recommendations set-out in the May 2020 FDA guidance document on SARS-CoV-2 treatment and prevention trials) and feasibility of patient-participant recruitment (based on timeliness and success of recruitment).

Results

We included all 500 eligible trials in our cohort, 58% of which were Phase 2 and 84.8% were directed towards the treatment of SARS-CoV-2. Close to one third of trials met all three criteria and were deemed informative (29.9% (95% Confidence Interval 23.7–36.9)). The proportion of potentially redundant trials in our cohort was 4.1%. Over half of the trials in our cohort (56.2%) did not meet our criteria for high quality trial design. The proportion of trials with infeasible patient-participant recruitment was 22.6%.

Conclusions

Less than one third of COVID-19 trials registered on ClinicalTrials.gov during the first six months met all three criteria for informativeness. Shortcomings in trial design, recruitment feasibility and redundancy reflect longstanding weaknesses in the clinical research enterprise that were likely amplified by the exceptional circumstances of a pandemic.

Does pre-notification increase questionnaire response rates: a randomised controlled trial nested within a systematic review

BACKGROUND

Missing outcome data can lead to bias in the results of systematic reviews. One way to address missing outcome data is by requesting the data from the trial authors, but non-response is common. One way to potentially improve response rates is by sending study participants advance communication. During the update of a systematic review examining the effect of pre-notification on response rates, study authors needed to be contacted for further information. This study was nested within the systematic review by randomising authors to receive a notification of the upcoming request for information. The objective was to test if pre-notification increased response rates.

METHODS

The participants were study authors included in the systematic review, whose studies were at unclear risk of bias. The intervention was a pre-notification of the request for further information, sent 1 day before the request. The outcome was defined as the proportion of authors who responded to the request for information. Authors were randomised by simple randomisation. Thirty three authors were randomised to the pre-notification arm, and 42 were randomised to the control arm. Authors were blinded to the possibility of an alternative condition.

RESULTS

All authors randomised were analysed. 14/33 (42.4%) authors in the pre-notification arm had returned responses to the questionnaire, and 18/42 (42.9%) in the control arm. There was no evidence of a difference between these groups (absolute difference = - 0.5, 95% CI (- 23.4 to 22.5%), p = 1). We received no complaints about receiving the pre-notification.

CONCLUSIONS

This study's results do not support the hypothesis that pre-notification increases response from study authors being contacted for a request for more information. However, the study has a low power, and the results may not generalise to other contexts, methods of administering a pre-notification, or study populations.

TRIAL REGISTRATION

Registration and protocol: This trial is not registered with any trial registry. However, the protocol was posted in advance on the Open Science Framework website and is available on the Open Science Framework website: DOI: https://doi.org/10.17605/OSF.IO/MSV2W or https://osf.io/msv2w/.

Conducting Digital Health Care Research: Document Analysis of Challenges Experienced During Intervention Development and Feasibility Study Setup of an Internet-Administered Intervention for Parents of Children Treated for Cancer

Background

The design and conduct of research to develop, test, and evaluate complex health care interventions is challenging. Although the existing literature describes key challenges associated with the design and conduct of definitive (evaluation) trials, there is a lack of information concerning specific challenges associated with the intervention development phase and setup of feasibility studies. In particular, the literature is scarce concerning the challenges associated with conducting digital health care research, such as research on internet-administered interventions and research using digital features to support the execution of study procedures (eg, recruitment, consent, retention, and data collection and management). This study is conducted in the context of the intervention development and feasibility study setup phases of an internet-administered, guided, low-intensity cognitive behavioral therapy–based intervention for parents of children previously treated for cancer.

Objective

The aim of this study is to explore the challenges experienced during the development phase of the internet-administered intervention and digital features to support the execution of the study procedures and a feasibility study setup.

Methods

To explore the key challenges experienced, we conducted a document analysis of written records from all study meetings held by the research team (meeting minutes) between June 7, 2018, and January 10, 2020, guided by a thematic analysis approach. Furthermore, discussion groups with members of the research team were held to develop a more detailed understanding of the key challenges experienced. Methods and results are reported in accordance with the relevant items from the Standards for Reporting Qualitative Research checklist.

Results

Six main themes were identified: decision-making and communication, expertise, external constraints, flexibility, planning and scheduling, and technical constraints.

Conclusions

Significant challenges were experienced during the intervention development and setup phases of the feasibility study. Implications are discussed to inform future design, conduct, and planning of internet-administered intervention development and feasibility studies, especially within the context of digital health care research.

Laparoscopic versus open liver resection for intrahepatic cholangiocarcinoma: Report of an international multicenter cohort study with propensity score matching

BACKGROUND

Intrahepatic cholangiocarcinoma is a rare disease with a poor prognosis. In patients where surgical resection is possible, outcome is influenced by perioperative morbidity and lymph node status. Laparoscopic liver resection is associated with improved clinical and oncological outcomes in primary and metastatic liver cancer compared with open liver resection, but evidence on intrahepatic cholangiocarcinoma is still insufficient. The primary aim of this study was to compare overall survival for a large series of patients treated for intrahepatic cholangiocarcinoma by open or laparoscopic approach. Secondary objectives were to compare disease-free survival, predictors of death, and recurrence.

METHODS

Patients treated with laparoscopic or open liver resection for intrahepatic cholangiocarcinoma from 2000 to 2018 from 3 large international databases were analyzed retrospectively. Each patient in the laparoscopic resection group (case) was matched with 1 open resection control (1:1 ratio), through a propensity score calculated on clinically relevant preoperative covariates. Overall and disease-free survival were compared between the matched groups. Predictors of mortality and recurrence were analyzed with Cox regression, and the Textbook Outcomes were described.

RESULTS

During the study period, 855 patients met the inclusion criteria (open liver resection = 709, 82.9%; laparoscopic liver resection = 146, 17.1%). Two groups of 89 patients each were analyzed after propensity score matching, with no significant difference regarding pre- and postoperative variables. Overall survival at 1, 3, and 5 years was 92%, 75%, and 63% in the laparoscopic liver resection group versus 92%, 58%, and 49% in the open liver resection group (P = .0043). Adjusted Cox regression revealed severe postoperative complications (hazard ratio: 10.5, 95% confidence interval [1.01-109] P = .049) and steatosis (hazard ratio: 13.8, 95% confidence interval [1.23-154] P = .033) as predictors of death, and transfusion (hazard ratio: 19.2, 95% confidence interval [4.04-91.4] P < .001) and severe postoperative complications (hazard ratio: 4.07, 95% confidence interval [1.15-14.4] P = .030) as predictors of recurrence.

CONCLUSION

The survival advantage of laparoscopic liver resection over open liver resection for intrahepatic cholangiocarcinoma is equivocal, given historical bias and missing data.

Comprehensive review of statistical methods for analysing patient-reported outcomes (PROs) used as primary outcomes in randomised controlled trials (RCTs) published by the UK's Health Technology Assessment (HTA) journal (1997-2020)

OBJECTIVES

To identify how frequently patient-reported outcomes (PROs) are used as primary and/or secondary outcomes in randomised controlled trials (RCTs) and to summarise what statistical methods are used for the analysis of PROs.

DESIGN

Comprehensive review.

SETTING

RCTs funded and published by the United Kingdom's (UK) National Institute for Health Research (NIHR) Health Technology Assessment (HTA) Programme.

DATA SOURCES AND ELIGIBILITY

HTA reports of RCTs published between January 1997 and December 2020 were reviewed.

DATA EXTRACTION

Information relating to PRO use and analysis methods was extracted.

PRIMARY AND SECONDARY OUTCOME MEASURES

The frequency of using PROs as primary and/or secondary outcomes; statistical methods that were used for the analysis of PROs as primary outcomes.

RESULTS

In this review, 37.6% (114/303) of trials used PROs as primary outcomes, and 82.8% (251/303) of trials used PROs as secondary outcomes from 303 NIHR HTA reports of RCTs. In the 114 RCTs where the PRO was the primary outcome, the most used PRO was the Short-Form 36 (8/114); the most popular methods for multivariable analysis were linear mixed model (45/114), linear regression (29/114) and analysis of covariance (13/114); logistic regression was applied for binary and ordinal outcomes in 14/114 trials; and the repeated measures analysis was used in 39/114 trials.

CONCLUSION

The majority of trials used PROs as primary and/or secondary outcomes. Conventional methods such as linear regression are widely used, despite the potential violation of their assumptions. In recent years, there is an increasing trend of using complex models (eg, with mixed effects). Statistical methods developed to address these violations when analysing PROs, such as beta-binomial regression, are not routinely used in practice. Future research will focus on evaluating available statistical methods for the analysis of PROs.

Estimates of the mean difference in orthopaedic randomized trials: obligatory yet obscure

Background

Randomized controlled trials in orthopaedics are powered to mainly find large effect sizes. A possible discrepancy between the estimated and the real mean difference is a challenge for statistical inference based on p-values. We explored the justifications of the mean difference estimates used in power calculations. The assessment of distribution of observations in the primary outcome and the possibility of ceiling effects were also assessed.

Methods

Systematic review of the randomized controlled trials with power calculations in eight clinical orthopaedic journals published between 2016 and 2019. Trials with one continuous primary outcome and 1:1 allocation were eligible. Rationales and references for the mean difference estimate were recorded from the Methods sections. The possibility of ceiling effect was addressed by the assessment of the weighted mean and standard deviation of the primary outcome and its elaboration in the Discussion section of each RCT where available.

Results

264 trials were included in this study. Of these, 108 (41 %) trials provided some rationale or reference for the mean difference estimate. The most common rationales or references for the estimate of mean difference were minimal clinical important difference (16 %), observational studies on the same subject (8 %) and the ‘clinical relevance’ of the authors (6 %). In a third of the trials, the weighted mean plus 1 standard deviation of the primary outcome reached over the best value in the patient-reported outcome measure scale, indicating the possibility of ceiling effect in the outcome.

Conclusions

The chosen mean difference estimates in power calculations are rarely properly justified in orthopaedic trials. In general, trials with a patient-reported outcome measure as the primary outcome do not assess or report the possibility of the ceiling effect in the primary outcome or elaborate further in the Discussion section.

Aleatory and epistemic uncertainties can completely derail medical research results

Aleatory uncertainties are generated by intrinsic factors such as studying a sample rather than the whole population and the source of epistemic uncertainties is extraneous such as limitations of knowledge. These uncertainties inflict all the findings in empirical medical research, but they are rarely appreciated. This article highlights these uncertainties and shows with the help of an example how apparently valid and reliable findings can completely derail due to these uncertainties. We conclude that aleatory and epistemic uncertainties should get due consideration while drawing conclusions and before the results are put into practice. Methods to reduce their impact on results are also presented.

Use of wearable biometric monitoring devices to measure outcomes in randomized clinical trials: a methodological systematic review

BACKGROUND

Wearable biometric monitoring devices (BMDs) have the potential to transform the conduct of randomized controlled trials (RCTs) by shifting the collection of outcome data from single measurements at predefined time points to dense continuous measurements.

METHODS

Methodological systematic review to understand how recent RCTs used BMDs to measure outcomes and to describe the reporting of these RCTs. Electronic search was performed in the Cochrane Central Register of Controlled Trials, PubMed, and EMBASE and completed a page-by-page hand search in five leading medical journals between January 1, 2018, and December 31, 2018. Three reviewers independently extracted all primary and secondary outcomes collected using BMDs, and assessed (1) the definitions used to summarize BMD outcome data; (2) whether the validity, reliability, and responsiveness of sensors was reported; (3) the discrepancy with outcomes prespecified in public clinical trial registries; and (4) the methods used to manage missing and incomplete BMD outcome data.

RESULTS

Of the 4562 records screened, 75 RCTs were eligible. Among them, 24% tested a pharmacological intervention and 57% used an inertial measurement sensor to measure physical activity. Included trials involved 464 outcomes (average of 6 [SD = 8] outcomes per trial). In total, 35 trials used a BMD to measure a primary outcome. Several issues affected the value and transparency of trials using BMDs to measure outcomes. First, the definition of outcomes used in the trials was highly heterogeneous (e.g., 21 diabetes trials had 266 outcomes and 153 had different unique definitions to measure diabetes control), which limited the combination and comparison of results. Second, information on the validity, reliability, and responsiveness of sensors used was lacking in 74% of trials. Third, half (53%) of the outcomes measured with BMDs had not been prespecified, with a high risk of outcome reporting bias. Finally, reporting on the management of incomplete outcome data (e.g., due to suboptimal compliance with the BMD) was absent in 68% of RCTs.

CONCLUSIONS

Use of BMDs to measure outcomes is becoming the norm rather than the exception in many fields. Yet, trialists need to account for several methodological issues when specifying and conducting RCTs using these novel tools.

Articulating the "So, What?" in Clinical Research: Insight from the M-CHOIR Group

With the academic culture of "publish or perish," authors must ensure that they are delivering high-quality data with a meaningful impact on clinical practice. Even for physician-scientists at the top of their fields, establishing the relevance of a study to clinical practice is a challenge. Thus, it is essential that research proposals ask questions that are clinically important, use appropriate methodologies, and examine outcomes that are relevant to both the physicians and the patients. The question of "so, what?" or in other words, "who cares?" is one that can make or break a study's impact on clinical practice. Researchers should use models such as PICOS (Population, Intervention, Comparison, Outcomes, and Study design) and FINER (Feasible, Interesting, Novel, Ethical, Relevant) and ask why readers will care about their study's findings before the study is conducted. By doing so, researchers can ensure the successful execution of their study and a meaningful impact of their findings, in both academia and clinical practice. This Special Topic article aims to guide researchers in producing relevant, impactful conclusions of their studies by providing input and resources from the Michigan Center for Hand Outcomes and Innovation (M-CHOIR) group.

Systematic review assessing the evidence for the use of stem cells in fracture healing

AIMS

Bone demonstrates good healing capacity, with a variety of strategies being utilized to enhance this healing. One potential strategy that has been suggested is the use of stem cells to accelerate healing.

METHODS

The following databases were searched: MEDLINE, CENTRAL, EMBASE, Cochrane Database of Systematic Reviews, WHO-ICTRP, ClinicalTrials.gov, as well as reference checking of included studies. The inclusion criteria for the study were: population (any adults who have sustained a fracture, not including those with pre-existing bone defects); intervention (use of stem cells from any source in the fracture site by any mechanism); and control (fracture healing without the use of stem cells). Studies without a comparator were also included. The outcome was any reported outcomes. The study design was randomized controlled trials, non-randomized or observational studies, and case series.

RESULTS

In all, 94 eligible studies were identified. The clinical and methodological aspects of the studies were too heterogeneous for a meta-analysis to be undertaken. A narrative synthesis examined study characteristics, stem cell methods (source, aspiration, concentration, and application) and outcomes.

CONCLUSION

Insufficient high-quality evidence is available to determine the efficacy of stem cells for fracture healing. The studies were heterogeneous in population, methods, and outcomes. Work to address these issues and establish standards for future research should be undertaken.Cite this article: Bone Joint Open 2020;1-10:628-638.

Standardizing Postoperative Complications-Validating the Clavien-Dindo Complications Classification in Cardiac Surgery

Cardiac surgery lacks a method for quantifying postoperative morbidities. The Clavien-Dindo Complications Classification (CDCC) and the Comprehensive Complication Index (CCI) were successfully implemented as outcome reporting methods in other surgical specialties. This study aims to validate these complication scales in cardiac surgery. Between 2010 and 2019, we prospectively collected data on 41,218 adult patients (73% men, mean age 67 ± 11 years) undergoing cardiac surgery at 6 university hospitals. Complications were graded using the CDCC based on the complication's treatment invasiveness with adaptations for common treatments in cardiac surgery. CCI were calculated, representing multiple complications on a scale of 0 (no complication) to 100 (death). Associations with predictors of poor outcome were assessed using mixed-effects models accounting for center as a random effect. CDCC grade was 0 in 23.0%, I in 11.4%, II in 35.3%, IIIa in 6.4%, IIIb in 2.6%, IVa in 16.1%, IVb in 2.1%, and V in 3.1%. Median CCI was 23 (9, 40). A change from lowest to highest observed CDCC grade was associated with an increase in the Society of Thoracic Surgeons mortality score from 1.1% to 4.7%, surgery duration from 177 to 233 minutes, and hospital stay from 5.2 to 17 days (all P < 0.0001). The CCI also increased with greater procedure complexity (P < 0.0001). Increase in CDCC/CCI is associated with greater comorbidities, surgery durations, lengths of stay, and procedure complexity, accurately reflecting the nuances of the adult cardiac surgery postoperative course. These have great potential for uniform outcome reporting and quality improvement initiatives.

Randomized clinical trials in ANCA-associated vasculitis: a systematic analysis of the WHO - International Clinical Trials Registry Platform

BACKGROUND

The analysis of the main features of randomized controlled trials (RCTs) on ANCA-associated vasculitis (AAV) can inform future study design.

METHODS

We searched within the International Clinical Trials Registry Platform all registered RCTs on AAV from October 2008 to December 2018. Two reviewers selected studies according to pre-specified eligibility criteria. We retrieved information including countries, funding, design, sample sizes, eligibility criteria, primary outcomes (POs), and treatments.

RESULTS

Among the 40 RCTs identified, 22 (55%) were conducted in Europe, 29 (72,5%) in a single country, 14 (35%) were industry-funded. The median number of patients planned to enrol was 68 (IQR 36-138). Only 28% of RCTs targeted a single vasculitis, and ANCA negative patients were not included in about 40% of studies. Interventions investigated were mainly drugs given to induce (40%) or maintain (32.5%) remission. Eighty-five percent of POs were considered being 'patient-important', but discrepancies in definition of disease states, such as remission or relapse were observed. Glucocorticoids use was part of the PO in < 25% of studies. The number of trials targeting a single disease, non-industry funded, incorporating glucocorticoids in PO, as well as the planned sample size increased over time.

CONCLUSION

Despite the important achievements in the field, a better harmonization of eligibility, and outcome criteria across studies is an important objective to pursue in next future.

Reporting of Organ Support Outcomes in Septic Shock Randomized Controlled Trials: A Methodologic Review-The Sepsis Organ Support Study

OBJECTIVES

Many recent randomized controlled trials in the field of septic shock failed to demonstrate a benefit on mortality. Randomized controlled trials increasingly report organ support duration and organ support-free days as primary or secondary outcomes. We conducted a methodologic systematic review to assess how organ support outcomes were defined and reported in septic shock randomized controlled trials.

DATA SOURCES

MEDLINE via PubMed, Embase, Cochrane Central Register of Controlled Trials, and Web of Science.

STUDY SELECTION

We included randomized controlled trials published between January 2004 and March 2018 that involved septic shock adults and assessed organ support duration and/or organ support-free days for hemodynamic support, respiratory support, or renal replacement therapy.

DATA EXTRACTION

For each randomized controlled trial, we extracted the definitions of organ support duration and organ support-free days. We particularly evaluated how nonsurvivors were accounted for. Study authors were contacted to provide any missing information regarding these definitions.

DATA SYNTHESIS

We included 28 randomized controlled trials. Organ support duration and organ support-free days outcomes were reported in 17 and 15 randomized controlled trials, respectively, for hemodynamic support, 15 and 15 for respiratory support, and five and nine for renal replacement therapy. Nonsurvivors were included in the organ support duration calculation in 13 of 14 randomized controlled trials (93%) for hemodynamic support and nine of 10 (90%) for respiratory support. The organ support-free days definition for hemodynamic support, respiratory support, and renal replacement therapy was reported in six of 15 randomized controlled trials (40%), eight of 15 randomized controlled trials (53%), and six of nine randomized controlled trials (67%) reporting an organ support-free days outcome, respectively. Of these, one half assigned "0" to nonsurvivors, and the other half attributed one point per day alive free of organ support up to a predefined time point.

CONCLUSIONS

This study highlights the heterogeneity and infrequency of organ support duration/organ support-free days outcome reporting in septic shock trials. When reported, the definitions of these outcome measures and methods of calculation are also infrequently reported, in particular how nonsurvivors were accounted for, which may have an important impact on interpretation.

Gamified M-Health Attention Bias Modification Intervention for Individuals with Opioid Use Disorder: Protocol for a Pilot Randomised Study

Introduction: Globally, there is an epidemic of opioid use disorders. Locally, in Singapore, there is an increase in the number of individuals abusing opioids. The advances in experimental psychology have highlighted the need to modify unconscious, automatic biases. These automatic, unconscious biases result in individuals having preferential attention to substance-related cues in their natural environment, thus leading to a slip or relapse back into their underlying addictive disorders. Prior studies have demonstrated not only the presence of robust attentional biases amongst individuals with opioid use disorder, even when maintained on methadone; and the effectiveness of bias modification amongst these individuals. There remains limited evaluation of attention bias modification amongst a treatment-seeking cohort of Asian individuals. The objective of this pilot is to ensure that the methods of the planned definitive randomized trial could be conducted. Methods and Analysis: A non-blinded pilot randomized trial will be conducted. A total of 30 participants will be randomized to receive either the conventional application or the newly designed co-designed application. In order to identify these 30 participants, 60 participants will be recruited and screened to determine if they have baseline biases. Participants will be recruited from the inpatient unit at the National Addictions Management Service (NAMS) Singapore. All participants who are enrolled into the trial will complete a baseline assessment task, and a bias modification assessment and modification task daily. They will have to complete a baseline demographic and clinical information questionnaire, as well as a cravings rating scale before and after the intervention daily. Perspectives—that of self-reported experiences—will be sought from the participants following their completion of three intervention tasks. Descriptive statistical analyses will be performed, and chi-square and ANOVA analyses will be performed. Qualitative analyses will be undertaken for the perspectives shared. Ethics and Dissemination: Ethical approval has been obtained from the National Healthcare Group’s Domain Specific Research Board (DSRB) (approval number that of 2019/00934). The findings arising from this study will be disseminated by means of conferences and publications.

A systematic review of reported outcomes and outcome measures in randomized trials evaluating surgical interventions for posterior vaginal prolapse to aid development of a core outcome set

BACKGROUND

Recent systematic reviews have demonstrated wide variations on outcome measure selection and outcome reporting in trials on surgical treatments for anterior, apical and mesh prolapse surgery. A systematic review of reported outcomes and outcome measures in posterior compartment vaginal prolapse interventions is highly warranted in the process of developing core outcome sets.

OBJECTIVE

To evaluate outcome and outcome measures reporting in posterior prolapse surgical trials.

SEARCH STRATEGY

We searched MEDLINE, EMBASE and the Cochrane Central Register of Controlled Trials (CENTRAL).

SELECTION CRITERIA

Randomized trials evaluating the efficacy and safety of different surgical interventions for posterior compartment vaginal prolapse.

DATA COLLECTION AND ANALYSIS

Two researchers independently assessed studies for inclusion, evaluated methodological quality, and extracted relevant data. Methodological quality, outcome reporting quality and publication characteristics were evaluated.

MAIN RESULTS

Twenty-seven interventional and four follow-up trials were included. Seventeen studies enrolled patients with posterior compartment surgery as the sole procedure and 14 with multicompartment procedures. Eighty-three reported outcomes and 45 outcome measures were identified. The most frequently reported outcomes were blood loss (20 studies, 74%), pain (18 studies, 66%) and infection (16 studies, 59%).

CONCLUSIONS

Wide variations in reported outcomes and outcome measures were found. Until a core outcome set is established, we propose an interim core outcome set that could include the three most commonly reported outcomes of the following domains: hospitalization; intraoperative, postoperative urinary, gastrointestinal, vaginal and sexual outcomes; clinical effectiveness.

PROSPERO

CRD42017062456.