The importance of decision intent within descriptions of pragmatic trials

Comments, suggestions, and criticisms of the Pragmatic Explanatory Continuum Indicator Summary-2 design tool: a citation analysis

INTRODUCTION

The pragmatic explanatory continuum indicator summary (PRECIS) tool, initially published in 2009 and revised in 2015, was created to assist trialists to align their design choices with the intended purpose of their randomised controlled trial (RCT): either to guide real-world decisions between alternative interventions (pragmatic) or to test hypotheses about intervention mechanisms by minimising sources of variation (explanatory). There have been many comments, suggestions, and criticisms of PRECIS-2. This summary will be used to facilitate the development of to the next revision, which is PRECIS-3.

METHODS

We used Web of Science to identify all publication types citing PRECIS-2, published between May 2015 and July 2023. Citations were eligible if they contained 'substantive' suggestions, comments, or criticism of the PRECIS-2 tool. We defined 'substantive' as comments explicitly referencing at least one PRECIS-2 domain or a concept directly linked to an existing or newly proposed domain. Two reviewers independently extracted comments, suggestions, and criticisms, noting their implications for the update. These were discussed among authors to achieve consensus on the interpretation of each comment and its implications for PRECIS-3.

RESULTS

The search yielded 885 publications, and after full-text review, 89 articles met the inclusion criteria. Comments pertained to new domains, changes in existing domains, or were relevant across several or all domains. Proposed new domains included assessment of the comparator arm and a domain to describe blinding. There were concerns about scoring eligibility and recruitment domains for cluster trials. Suggested areas for improvement across domains included the need for more scoring guidance for explanatory design choices.

DISCUSSION

Published comments recognise PRECIS-2's success in aiding trialists with pragmatic or explanatory design choices. Enhancing its implementation and widespread use will involve adding new domains, refining domain definitions, and addressing overall tool issues. This citation review offers valuable user feedback, pivotal for shaping the upcoming version of the PRECIS tool, PRECIS-3.

Lack of pragmatic attitude of self-labelled pragmatic trials on manual therapy: a methodological review

BACKGROUND

Pragmatic randomized controlled trials are getting more interest to improve trials' external validity. This study aimed to assess how pragmatic the design of the self-labelled pragmatic randomised controlled trials in the manual therapy field is.

METHODS

We searched MEDLINE and the Cochrane Central Register of Controlled Trials for self-labelled pragmatic randomised controlled trials in the manual therapy field until January 2024 were included. Two independent reviewers collected and extracted data related to the intention of the trial, the rationale for the intervention, and specific features of the trial and performed an assessment using the PRECIS-2 tool.

RESULTS

Of 39 self-labelled pragmatic trials, the mean PRECIS-2 score was 3.5 (SD: 0.6). Choice of outcome measures, how the interventions were performed, the follow-up of the participants and how all the available data were included in the statistical analysis were the domains rated as most 'pragmatic'. Participants' eligibility, recruitment, and setting obtained lower scores. Less than 25% of the trials claimed that the aim was to investigate an intervention under real-world conditions and to make clinical decisions about its effectiveness. In the 21% of the sample the authors described neither the proof-of-concept of the intervention nor the state of previous studies addressing related research questions.

CONCLUSIONS

Self-labelled pragmatic randomised controlled trials showed a moderately pragmatic attitude. Beyond the label 'pragmatic', the description of the intention of the trial and the context of every PRECIS-2 domain is crucial to understanding the real pragmatism of a trial.

Methods for pragmatic randomized clinical trials of pain therapies: IMMPACT statement

ABSTRACT

Pragmatic, randomized, controlled trials hold the potential to directly inform clinical decision making and health policy regarding the treatment of people experiencing pain. Pragmatic trials are designed to replicate or are embedded within routine clinical care and are increasingly valued to bridge the gap between trial research and clinical practice, especially in multidimensional conditions, such as pain and in nonpharmacological intervention research. To maximize the potential of pragmatic trials in pain research, the careful consideration of each methodological decision is required. Trials aligned with routine practice pose several challenges, such as determining and enrolling appropriate study participants, deciding on the appropriate level of flexibility in treatment delivery, integrating information on concomitant treatments and adherence, and choosing comparator conditions and outcome measures. Ensuring data quality in real-world clinical settings is another challenging goal. Furthermore, current trials in the field would benefit from analysis methods that allow for a differentiated understanding of effects across patient subgroups and improved reporting of methods and context, which is required to assess the generalizability of findings. At the same time, a range of novel methodological approaches provide opportunities for enhanced efficiency and relevance of pragmatic trials to stakeholders and clinical decision making. In this study, best-practice considerations for these and other concerns in pragmatic trials of pain treatments are offered and a number of promising solutions discussed. The basis of these recommendations was an Initiative on Methods, Measurement, and Pain Assessment in Clinical Trials (IMMPACT) meeting organized by the Analgesic, Anesthetic, and Addiction Clinical Trial Translations, Innovations, Opportunities, and Networks.

Effectiveness and Safety of Pharmacopuncture on Inpatients with Tension Headache Caused by Traffic Accidents: A Pragmatic Randomized Controlled Trial

Background: This study investigated the effectiveness and safety of pharmacopuncture for pain relief and functional improvement in patients with traffic accident (TA)-induced acute tension headaches. Methods: The study employed a parallel, single-centered, pragmatic, randomized controlled trial design. Eighty patients complaining of acute tension headaches were randomized into the integrative Korean medicine treatment (IKM treatment) group and the pharmacopuncture group on suboccipital muscles (suboccipital muscles pharmacopuncture + IKM treatment), with 40 participants assigned to each group. The patients in the pharmacopuncture group underwent pharmacopuncture as an add-on therapy, consisting of three sessions. Both groups were reassessed 2 months post-intervention. To assess the outcomes, the Numeric Rating Scale (NRS) for Headache, NRS for Neck Pain, Headache Disability Index, Headache Impact Test-6, EuroQol 5-Dimension, and Patient Global Impression of Change were used. Results: The improvement in the outcomes of the pharmacopuncture group was significantly greater than that of the comparison group on day 4 of hospitalization in terms of pain (difference in NRS of headache -2.59, 95% CI -3.06 to -2.12; NRS of Neck pain -1.05, 95% CI -1.50 to -0.59) and function (difference in HDI -24.78, 95% CI, -31.79 to -17.76; HIT-6 -6.13, 95% CI, -9.47 to -2.78). Additionally, in 2 months of follow-up, the recovery rate of headache was significantly higher in the pharmacopuncture group than in the comparison group. Conclusions: The pharmacopuncture group demonstrated superior outcomes in symptom improvement than the comparison group did, providing insights into novel and useful applications of pharmacopuncture in the clinical practice of Korean medicine.

Research objectives and general considerations for pragmatic clinical trials of pain treatments: IMMPACT statement

ABSTRACT

Many questions regarding the clinical management of people experiencing pain and related health policy decision-making may best be answered by pragmatic controlled trials. To generate clinically relevant and widely applicable findings, such trials aim to reproduce elements of routine clinical care or are embedded within clinical workflows. In contrast with traditional efficacy trials, pragmatic trials are intended to address a broader set of external validity questions critical for stakeholders (clinicians, healthcare leaders, policymakers, insurers, and patients) in considering the adoption and use of evidence-based treatments in daily clinical care. This article summarizes methodological considerations for pragmatic trials, mainly concerning methods of fundamental importance to the internal validity of trials. The relationship between these methods and common pragmatic trials methods and goals is considered, recognizing that the resulting trial designs are highly dependent on the specific research question under investigation. The basis of this statement was an Initiative on Methods, Measurement, and Pain Assessment in Clinical Trials (IMMPACT) systematic review of methods and a consensus meeting. The meeting was organized by the Analgesic, Anesthetic, and Addiction Clinical Trial Translations, Innovations, Opportunities, and Networks (ACTTION) public-private partnership. The consensus process was informed by expert presentations, panel and consensus discussions, and a preparatory systematic review. In the context of pragmatic trials of pain treatments, we present fundamental considerations for the planning phase of pragmatic trials, including the specification of trial objectives, the selection of adequate designs, and methods to enhance internal validity while maintaining the ability to answer pragmatic research questions.

An analysis of published trials found that current use of pragmatic trial labels is uninformative

OBJECTIVES

Randomized trials labelled as "pragmatic" are attractive to funders, patients, and clinicians as the label implies that the results are directly applicable to clinical care. We examined how authors justify use of the label (e.g., by referring to one or more PRECIS [PRagmatic Explanatory Continuum Indicator Summary]-2 domains).

STUDY DESIGN AND SETTING

We reviewed primary trial reports published 2014-2019, registered in ClinicalTrials.gov and using the pragmatic label anywhere in the report.

RESULTS

Among 415 trials, the label was justified by reference to at least one design element in 282 (68.0%); of these, 240 (85.1%) referenced trial characteristics that can be mapped to one or more of the PRECIS-2 domains, most commonly eligibility (91, 32.3%), setting (90, 31.9%), flexibility delivery (89, 31.6%), and organization (75, 26.6%); 42 (14.9%) referenced characteristics that are not PRECIS-2 domains, most commonly type of intervention/comparator (48, 17%), recruitment without consent (22, 7.8%), routinely collected data (22, 7.8%), and cluster randomization (20, 7.1%). Most reports referenced only one or two design elements. Overall, 9/415 (2.2%) provided PRECIS wheels.

CONCLUSION

Current use of pragmatic labels is uninformative. Authors should clarify the decision the trial is intended to support and include a PRECIS-2 table to make the design transparent.

Ethical considerations within pragmatic randomized controlled trials in dementia: Results from a literature survey

Introduction

This review aims to describe the landscape of pragmatic randomized controlled trials (RCTs) in the context of Alzheimer's disease (AD) and related dementias with respect to ethical considerations.

Methods

Searches of MEDLINE were performed from January 2014 until April 2019. Extracted information included: trial setting, interventions, data collection, study population, and ethical protections (including ethics approvals, capacity assessment, and informed consent).

Results

We identified 62 eligible reports. More than two-thirds (69%) included caregivers or health-care professionals as research participants. Fifty-eight (94%) explicitly identified at least one vulnerable group. Two studies did not report ethics approval. Of 57 studies in which patients were participants, 55 (96%) reported that consent was obtained but in 37 studies (67%) no mention was made regarding assessment of the patients' capacity to consent to research participation.

Discussion

Few studies reported protections implemented when vulnerable participants were included. Shortcomings remain when reporting consent approaches and capacity assessment.

Pragmatic trials of pain therapies: a systematic review of methods

ABSTRACT

Pragmatic randomised clinical trials aim to directly inform clinical or health policy decision making. Here, we systematically review methods and design of pragmatic trials of pain therapies to examine methods, identify common challenges, and areas for improvement. Seven databases were searched for pragmatic randomised controlled clinical trials that assessed pain treatment in a clinical population of adults reporting pain. All screening steps and data extractions were performed twice. Data were synthesised descriptively, and correlation analyses between prespecified trial features and PRECIS-2 (PRagmatic-Explanatory Continuum Indicator Summary 2) ratings and attrition were performed. Protocol registration: PROSPERO-ID CRD42020178954. Of 57 included trials, only 21% assessed pharmacological interventions, the remainder physical, surgical, psychological, or self-management pain therapies. Three-quarters of the trials were comparative effectiveness designs, often conducted in multiple centres (median: 5; Q1/3: 1, 9.25) and with a median sample size of 234 patients at randomization (Q1/3: 135.5; 363.5). Although most trials recruited patients with chronic pain, reporting of pain duration was poor and not well described. Reporting was comprehensive for most general items, while often deficient for specific pragmatic aspects. Average ratings for pragmatism were highest for treatment adherence flexibility and clinical relevance of outcome measures. They were lowest for patient recruitment methods and extent of follow-up measurements and appointments. Current practice in pragmatic trials of pain treatments can be improved in areas such as patient recruitment and reporting of methods, analysis, and interpretation of data. These improvements will facilitate translatability to other real-world settings-the purpose of pragmatic trials.

Methodological challenges in pragmatic trials in Alzheimer's disease and related dementias: Opportunities for improvement

BACKGROUND AND AIMS

We need more pragmatic trials of interventions to improve care and outcomes for people living with Alzheimer's disease and related dementias. However, these trials present unique methodological challenges in their design, analysis, and reporting-often, due to the presence of one or more sources of clustering. Failure to account for clustering in the design and analysis can lead to increased risks of Type I and Type II errors. We conducted a review to describe key methodological characteristics and obtain a "baseline assessment" of methodological quality of pragmatic trials in dementia research, with a view to developing new methods and practical guidance to support investigators and methodologists conducting pragmatic trials in this field.

METHODS

We used a published search filter in MEDLINE to identify trials more likely to be pragmatic and identified a subset that focused on people living with Alzheimer's disease or other dementias or included them as a defined subgroup. Pairs of reviewers extracted descriptive information and key methodological quality indicators from each trial.

RESULTS

We identified N = 62 eligible primary trial reports published across 36 different journals. There were 15 (24%) individually randomized, 38 (61%) cluster randomized, and 9 (15%) individually randomized group treatment designs; 54 (87%) trials used repeated measures on the same individual and/or cluster over time and 17 (27%) had a multivariate primary outcome (e.g. due to measuring an outcome on both the patient and their caregiver). Of the 38 cluster randomized trials, 16 (42%) did not report sample size calculations accounting for the intracluster correlation and 13 (34%) did not account for intracluster correlation in the analysis. Of the 9 individually randomized group treatment trials, 6 (67%) did not report sample size calculations accounting for intracluster correlation and 8 (89%) did not account for it in the analysis. Of the 54 trials with repeated measurements, 45 (83%) did not report sample size calculations accounting for repeated measurements and 19 (35%) did not utilize at least some of the repeated measures in the analysis. No trials accounted for the multivariate nature of their primary outcomes in sample size calculation; only one did so in the analysis.

CONCLUSION

There is a need and opportunity to improve the design, analysis, and reporting of pragmatic trials in dementia research. Investigators should pay attention to the potential presence of one or more sources of clustering. While methods for longitudinal and cluster randomized trials are well developed, accessible resources and new methods for dealing with multiple sources of clustering are required. Involvement of a statistician with expertise in longitudinal and clustered designs is recommended.

A review of pragmatic trials found a high degree of diversity in design and scope, deficiencies in reporting and trial registry data, and poor indexing

OBJECTIVE

We established a large database of trials to serve as a resource for future methodological and ethical analyses. Here, we use meta-data to describe the broad landscape of pragmatic trials including research areas, identification as pragmatic, quality of trial registry data and enrolment.

STUDY DESIGN AND SETTING

Trials were identified by a validated search filter and included if a primary report of a health-related randomized trial published January 2014-April 2019. Data were collated from MEDLINE, Web of Science, ClinicalTrials.gov, and full text.

RESULTS

4337 eligible trials were identified from 13,065 records, of which 1988 were registered in ClinicalTrials.gov. Research areas were diverse, with the most common being general and internal medicine; public, environmental and occupational health; and health care sciences and services. The term "pragmatic" was seldom used in titles or abstracts. Several domains in ClinicalTrials.gov had questionable data quality. We estimated that one-fifth of trials under-accrued by at least 15%.

CONCLUSION

There is a need to improve reporting of pragmatic trials and quality of trial registry data. Under accrual remains a challenge in pragmatic RCTs despite calls for more streamlined recruitment approaches. The diversity of pragmatic trials should be reflected in future ethical analyses.

Designing provider-focused implementation trials with purpose and intent: introducing the PRECIS-2-PS tool

Background

First articulated by Schwartz and Lellouch (1967), randomized controlled trials (RCTs) can be conceptualized along a continuum from more explanatory to more pragmatic. The purpose and intent of the former is to test interventions under ideal contexts, and the purpose and intent of the latter is to test interventions in real-world contexts. The PRagmatic Explanatory Continuum Indicator Summary-2 (PRECIS-2) is a validated tool that helps researchers make decisions about the elements of the trial to match the overall purpose and intent of the trial along the continuum. The PRECIS-2 tool has guided the design of hundreds of RCTs. However, a few aspects of the tool would benefit from greater clarity, including its application to provider-focused implementation trials rather than patient-focused intervention trials.

Main text

We describe the newly developed PRECIS-2-Provider Strategies (PRECIS-2-PS) tool, an extension of the PRECIS-2 tool, which has been adapted for trials testing provider-focused strategies. We elaborate on nine domains that can make a provider-focused trial more explanatory or more pragmatic, including eligibility, recruitment, setting, implementation resources, flexibility of provider strategies, flexibility of intervention, data collection, primary outcome, and primary analysis. We detail the complementary roles that researchers and stakeholders play in the trial design phase, with implications for generalizability of trial results to the contexts in which they are intended to be applied.

Conclusions

The PRECIS-2-PS tool is designed to help research and practice teams plan for provider-focused trials that reflect the overall intent and purpose of the trial. The tool has potential to help advance the science of provider-focused strategies across a range of trials, with the ultimate goal of facilitating the adoption, integration, and sustainability of provider-focused strategies outside the context of trials.

Supplementary Information

The online version contains supplementary material available at 10.1186/s13012-020-01075-y.