The design can limit PRECIS-2 retrospective assessment of the clinical trial explanatory/pragmatic features

Methods for pragmatic randomized clinical trials of pain therapies: IMMPACT statement

ABSTRACT

Pragmatic, randomized, controlled trials hold the potential to directly inform clinical decision making and health policy regarding the treatment of people experiencing pain. Pragmatic trials are designed to replicate or are embedded within routine clinical care and are increasingly valued to bridge the gap between trial research and clinical practice, especially in multidimensional conditions, such as pain and in nonpharmacological intervention research. To maximize the potential of pragmatic trials in pain research, the careful consideration of each methodological decision is required. Trials aligned with routine practice pose several challenges, such as determining and enrolling appropriate study participants, deciding on the appropriate level of flexibility in treatment delivery, integrating information on concomitant treatments and adherence, and choosing comparator conditions and outcome measures. Ensuring data quality in real-world clinical settings is another challenging goal. Furthermore, current trials in the field would benefit from analysis methods that allow for a differentiated understanding of effects across patient subgroups and improved reporting of methods and context, which is required to assess the generalizability of findings. At the same time, a range of novel methodological approaches provide opportunities for enhanced efficiency and relevance of pragmatic trials to stakeholders and clinical decision making. In this study, best-practice considerations for these and other concerns in pragmatic trials of pain treatments are offered and a number of promising solutions discussed. The basis of these recommendations was an Initiative on Methods, Measurement, and Pain Assessment in Clinical Trials (IMMPACT) meeting organized by the Analgesic, Anesthetic, and Addiction Clinical Trial Translations, Innovations, Opportunities, and Networks.

Identification of weight loss interventions for translation among endometrial cancer survivors: A RE-AIM analysis

Interventions for obesity-related cancers that combine nutrition and physical activity for weight loss exist; however, their application to survivors of endometrial cancer is unknown. Furthermore, little is known about pre-implementation perceptions of existing programs from a variety of interested persons (physicians, researchers) who may be part of the implementation team. Adapting an existing intervention rather than developing a new intervention may speed the translational lag time as long as intervention characteristics and fit within the delivery system are considered during the planning phase. To describe the process of determining the core elements of obesity-related interventions for cancer survivors and determine which one might be best delivered by an urban healthcare system that predominantly serves individuals who live in rural areas of Virginia and West Virginia. A pragmatic review of the literature was conducted via PubMed and Google Scholar with broad search terms of cancer survivor AND weight loss AND health intervention. Identified interventions were scored related to the Practical, Robust Implementation and Sustainability Model-which is an extension of RE-AIM framework to guide the understanding of who, what, where, when, and how the intervention was conducted. Intervention characteristics are reported. In addition, ratings from three independent reviewers on the validated 5-point Likert scale of an intervention's acceptability, appropriateness, and feasibility in the intended delivery system were collected and summarized. Twelve interventions were identified with an average sample size of 241(±195) and a range of 48-683 participants. Target populations included survivors of colorectal, breast, and endometrial cancers as well as general cancer survivors and included both men and women or only women. Most participants (74%) identified as white/Caucasian and average age ranged from 47.1 to 65.9 years. Program duration ranged from 4 weeks to 18 months, with an average duration of 32 weeks. Intervention dosage ranged from three times a week to once a month. Intervention acceptability, appropriateness, and feasibility had average and standard deviation ratings of 3.52(±0.46), 3.41(±0.45), and 3.21(±0.46), respectively, out of 5. The four interventions with the highest combined acceptable, appropriate, and feasible scores are being considered for potential use as an obesity-related intervention for survivors of endometrial cancer. Future work is needed to determine relevant adaptations and efficacy among survivors of endometrial cancer with obesity. Our approach may be beneficial for other interventionists aiming to speed intervention development and implementation.

Randomized controlled trials comparing gastric bypass, gastric band, and sleeve gastrectomy: A systematic review examining validity and applicability to wider clinical practice

Consideration of how applicable the results of surgical trials are to clinical practice is important to inform decision-making. Randomized controlled trials comparing at least two surgical interventions (of gastric bypass, gastric band, and sleeve gastrectomy) for severe and complex obesity were examined using the PRagmatic Explanatory Continuum Indicator Summary-2 tool, to consider how applicable the trial results are to clinical practice, and the Risk of Bias 2 tool, to examine validity. MEDLINE, Embase, and CENTRAL databases were searched for studies published between November 2013 and June 2021, and 15 were identified. Using the PRagmatic Explanatory Continuum Indicator Summary-2 tool, three were classified as pragmatic, with good applicability to clinical practice. Ten had more explanatory domains but did include some pragmatic characteristics, and two were predominantly explanatory. This was due to some trial design features that would not be considered applicable to the wider clinical setting, including being single-centered, having prescribed intervention delivery methods, and intensive follow-up regimens. Only two trials had low risk of bias, of which one was considered pragmatic. Three had high risk of bias. Overall, few trials in bariatric surgery are pragmatic with low risk of bias. Well-designed pragmatic trials are needed to inform practice and reduce research waste.

Analysis of the characteristics and the degree of pragmatism exhibited by pragmatic-labelled trials of antineoplastic treatments

BACKGROUND

Pragmatic clinical trials (PCTs) are designed to reflect how an investigational treatment would be applied in clinical practice. As such, unlike their explanatory counterparts, they measure therapeutic effectiveness and are capable of generating high-quality real-world evidence. However, the conduct of PCTs remains extremely rare. The scarcity of such studies has contributed to the emergence of the efficacy-effectiveness gap and has led to calls for launching more of them, including in the field of oncology. This analysis aimed to identify self-labelled pragmatic trials of antineoplastic interventions and to evaluate whether their use of this label was justified.

METHODS

We searched PubMed® and Embase® for publications corresponding with studies that investigated antitumor therapies and that were tagged as pragmatic in their titles, abstracts and/or index terms. Subsequently, we consulted all available source documents for the included trials and extracted relevant information from them. The data collected were then used to appraise the degree of pragmatism displayed by the PCTs with the help of the validated PRECIS-2 tool.

RESULTS

The literature search returned 803 unique records, of which 46 were retained upon conclusion of the screening process. This ultimately resulted in the identification of 42 distinct trials that carried the 'pragmatic' label. These studies examined eight different categories of neoplasms and were mostly randomized, open-label, multicentric, single-country trials sponsored by non-commercial parties. On a scale of one (very explanatory) to five (very pragmatic), the median PCT had a PRECIS-2 score per domain of 3.13 (interquartile range: 2.57-3.53). The most and least pragmatic studies in the sample had a score of 4.44 and 1.57, respectively. Only a minority of trials were described in sufficient detail to allow them to be graded across all domains of the PRECIS-2 instrument. Many of the studies examined also had features that arguably precluded them from being pragmatic altogether, such as being monocentric or placebo-controlled in nature.

CONCLUSION

PCTs of antineoplastic treatments are generally no more pragmatic than they are explanatory.

Blinding and sham control methods in trials of physical, psychological, and self-management interventions for pain (article I): a systematic review and description of methods

ABSTRACT

Blinding is challenging in randomised controlled trials of physical, psychological, and self-management therapies for pain, mainly because of their complex and participatory nature. To develop standards for the design, implementation, and reporting of control interventions in efficacy and mechanistic trials, a systematic overview of currently used sham interventions and other blinding methods was required. Twelve databases were searched for placebo or sham-controlled randomised clinical trials of physical, psychological, and self-management treatments in a clinical pain population. Screening and data extraction were performed in duplicate, and trial features, description of control methods, and their similarity to the active intervention under investigation were extracted (protocol registration ID: CRD42020206590). The review included 198 unique control interventions, published between 2008 and December 2021. Most trials studied people with chronic pain, and more than half were manual therapy trials. The described control interventions ranged from clearly modelled based on the active treatment to largely dissimilar control interventions. Similarity between control and active interventions was more frequent for certain aspects (eg, duration and frequency of treatments) than others (eg, physical treatment procedures and patient sensory experiences). We also provide an overview of additional, potentially useful methods to enhance blinding, as well as the reporting of processes involved in developing control interventions. A comprehensive picture of prevalent blinding methods is provided, including a detailed assessment of the resemblance between active and control interventions. These findings can inform future developments of control interventions in efficacy and mechanistic trials and best-practice recommendations.

Generalizability of randomized controlled trials in primary health care: Applying the PRECIS-2 tool on published protocols

RATIONALE

Pragmatic design may facilitate the generalizability of effectiveness of randomized controlled trials (RCTs) in primary health care (PHC).

AIMS AND OBJECTIVES

The aim of this study was to investigate whether published protocols in PHC were designed pragmatically and to explore whether specific trial characteristics may be related to a pragmatic design.

METHODS

Using the Pragmatic Explanatory Continuum Indicator Summary-2 (PRECIS-2), we assessed pragmatism for 123 published RCT protocols. For each domain, we calculated the mean score with the 95% confidence interval (95% CI). Interrater reliability was assessed by weighted κ-coefficient with 95% CI. We examined potential associations of published protocol characteristics with overall pragmatism by performing univariate and multivariate analyses.

RESULTS

We observed the highest score for primary analysis (4.66, 95% CI: 4.51, 4.82). The eligibility score was intermediate (3.16, 95% CI: 3.01, 3.32). Domains with scores towards the explanatory side included organization (2.50, 95% CI: 2.36, 2.63), flexibility of delivery (2.74, 95% CI: 2.60, 2.88) and flexibility of adherence (3.00, 95% CI: 2.83, 3.17). Interrater agreement was good (κ = 0.61; 95% CI: 0.34, 0.80; p < 0.001). Higher sample sizes were correlated to a pragmatic design (odds ratio: 6.86, 95% CI: 1.64, 28.75; p = 0.04).

CONCLUSION

Most RCT protocols were rated as intermediate in the pragmatic-explanatory continuum. Future research may guide all stakeholders on how best to incorporate the level of pragmatism in the interpretation of the results so that the trials are more likely to be applicable in real-world settings.

Pragmatic trials of pain therapies: a systematic review of methods

ABSTRACT

Pragmatic randomised clinical trials aim to directly inform clinical or health policy decision making. Here, we systematically review methods and design of pragmatic trials of pain therapies to examine methods, identify common challenges, and areas for improvement. Seven databases were searched for pragmatic randomised controlled clinical trials that assessed pain treatment in a clinical population of adults reporting pain. All screening steps and data extractions were performed twice. Data were synthesised descriptively, and correlation analyses between prespecified trial features and PRECIS-2 (PRagmatic-Explanatory Continuum Indicator Summary 2) ratings and attrition were performed. Protocol registration: PROSPERO-ID CRD42020178954. Of 57 included trials, only 21% assessed pharmacological interventions, the remainder physical, surgical, psychological, or self-management pain therapies. Three-quarters of the trials were comparative effectiveness designs, often conducted in multiple centres (median: 5; Q1/3: 1, 9.25) and with a median sample size of 234 patients at randomization (Q1/3: 135.5; 363.5). Although most trials recruited patients with chronic pain, reporting of pain duration was poor and not well described. Reporting was comprehensive for most general items, while often deficient for specific pragmatic aspects. Average ratings for pragmatism were highest for treatment adherence flexibility and clinical relevance of outcome measures. They were lowest for patient recruitment methods and extent of follow-up measurements and appointments. Current practice in pragmatic trials of pain treatments can be improved in areas such as patient recruitment and reporting of methods, analysis, and interpretation of data. These improvements will facilitate translatability to other real-world settings-the purpose of pragmatic trials.

Robustness and pragmatism of the evidence supporting the European Society for Medical Oncology guidelines for the diagnosis, treatment, and follow-up of follicular lymphoma

Background: Results of randomized clinical trials may not be entirely applicable to clinical practice. The present manuscript aims to explore the pragmatism and robustness of the evidence that supports the European Society for Medical Oncology (ESMO) follicular lymphoma (FL) guidelines.Methods & design: Analysis of all trials used to support positive, therapeutic, oncological recommendations in the 2020 ESMO FL guidelines. Predefined data points were extracted from each trial. Pragmatism was assessed by means of the PRECIS-2 tool, the difference in overall survival in the interventions compared and the source of funding. Robustness was assessed by means of the fragility index and the p value.Results: 28 trials were included. The full protocol or a protocol summary was provided for 12 (43%). Based on the PRECIS-2 domains, trials were considered pragmatic in organization, analysis and flexibility and explanatory in eligibility. Robustness was high, with 4/24 (17%) trials with p values between 0.05 and 0.005 and a median fragility index of 18.Conclusions: Results of trials to support ESMO recommendations in FL were robust. Pragmatism was high in some domains but modest to low in others and the pattern was similar across trials. Transparency in the publication of trial protocols was suboptimal.

A review of pragmatic trials found a high degree of diversity in design and scope, deficiencies in reporting and trial registry data, and poor indexing

OBJECTIVE

We established a large database of trials to serve as a resource for future methodological and ethical analyses. Here, we use meta-data to describe the broad landscape of pragmatic trials including research areas, identification as pragmatic, quality of trial registry data and enrolment.

STUDY DESIGN AND SETTING

Trials were identified by a validated search filter and included if a primary report of a health-related randomized trial published January 2014-April 2019. Data were collated from MEDLINE, Web of Science, ClinicalTrials.gov, and full text.

RESULTS

4337 eligible trials were identified from 13,065 records, of which 1988 were registered in ClinicalTrials.gov. Research areas were diverse, with the most common being general and internal medicine; public, environmental and occupational health; and health care sciences and services. The term "pragmatic" was seldom used in titles or abstracts. Several domains in ClinicalTrials.gov had questionable data quality. We estimated that one-fifth of trials under-accrued by at least 15%.

CONCLUSION

There is a need to improve reporting of pragmatic trials and quality of trial registry data. Under accrual remains a challenge in pragmatic RCTs despite calls for more streamlined recruitment approaches. The diversity of pragmatic trials should be reflected in future ethical analyses.

The PRECIS-2 tool seems not to be useful to discriminate the degree of pragmatism of medicine masked trials from that of open-label trials

PURPOSE

To assess, with all available trial information, whether the assessment of the PRECIS-2 nine domains could provide a clear distinction between medicine masked pragmatic randomized controlled trials (pRCTs) and open-label pRCTs.

METHODS

A search was conducted of participant-level pRCTs on medicines published on 25 influential medical journals in July 2018-December 2019. All pre-licensing (phases 1-3) and cluster pRCTs were excluded. All trials' available reports were searched through the published article information, Google Scholar, and trial websites. Instead of providing a score to each PRECIS-2 domain, these were classified as E (explanatory), N (neutral), or P (pragmatic).

RESULTS

Of 128 pRCTs, 18 (14%) were participant-level pRCTs on medicines. The full trial protocol was available for 14 trials; 12 had published the protocol and nine had additional reports published. All trials were prospectively registered, and none was funded by industry. Ten and eight were masked and open-label trials, respectively. Masked pRCTS had 34% of pragmatic and 60% of explanatory domains; open-label pRCTS had 45% pragmatic and 45% explanatory domains. Among the 10 masked trials, only one had a majority of five pragmatic domains; among the eight open-label trials, four had a majority of six or five pragmatic domains. "Follow-up" was considered explanatory in the 18 pRCTs; "primary analysis" was pragmatic in 17 pRCTs.

CONCLUSION

The PRECIS-2 tool seems not to be sensitive enough to clearly discriminate between medicine masked pRCTs and open-label pRCTs. When conducting systematic reviews, it is suggested that the PRECIS-2 tool should not be used to support placing masked trials in the pragmatic side of the explanatory/pragmatic continuum.

Articles provided insufficient information to conduct an appropriate retrospective assessment of the pragmatic/explanatory features of medicine trials with the PRECIS-2 tool

PURPOSE

To assess whether, in the retrospective assessment of the pragmatic/explanatory features of pragmatic randomized controlled trials (pRCTs), the nine PRECIS-2 domain scores using the information provided in articles were modified after using the information reported in other publicly available sources.

METHODS

This is a cross-sectional study of participant-level pRCTs published in July 2018 to December 2019 in the four highest-impact general medicine journals. The articles described the main results of pRCTs assessing medicines in one or more arms that were not in the pre-licensing phases. The information reported in trial full protocols, published protocols, and other publications, registries, and trial websites were assessed and scored, and compared with that previously obtained after reviewing the information reported in the articles.

RESULTS

Out of 76 articles on pRCTs, 13 (17%) were included in the analysis. All were two-arm trials, assessing medicines only (n = 7), medicine vs device (n = 2), medicine vs surgery (n = 1), or medicine vs placebo (n = 3). Seven were open-label trials, and six had any type of masking. All except one had the full protocol available and/or published protocol; seven had other types of publication available. The assessment of the nine PRECIS-2 domains with the information reported in the 13 articles was changed in all trials after using the information included in other additional available sources. Between one (n = 1 article) and six (n = 2) domains were modified in each pRCT. The domains that most commonly changed were "organization" (n = 12), "recruitment" (n = 11), and "follow-up" (n = 8). "Primary outcome" and "primary analysis" were not modified in any trial. Eight percent of all domains could not be assessed due to inadequate or lack of information in seven articles; those were "recruitment" (n = 3), "organization" (n = 3), "setting" (n = 2), and "flexibility:adherence" (n = 1).

CONCLUSION

Articles describing the trial main results are usually insufficient for the appropriate retrospective assessment of the pragmatic/explanatory features of a pRCT by authors not involved in the conduct of the trial. To address this issue, editors should require the submission of the original full protocol and final full protocol with the history of amendments to be published as supplementary material to the article.