The role for pragmatic randomized controlled trials (pRCTs) in comparative effectiveness research

The majority of clinical trials assessing mobilization and manipulation for neck pain lack a pragmatic approach: a systematic review of 174 trials

BACKGROUND

Disorders of the cervical spine are some of the costliest musculoskeletal conditions to manage globally. Joint mobilization and manipulation have been shown to be an effective treatment for neck pain. However, the generalizability and clinical translation depends on the nature of the trial designs that inform its use. The extent to which randomized control trials (RCTs) assessing manual therapy treatments for cervical spine disorders fall on the efficacy (explanatory) -effectiveness (pragmatic) spectrum often informs how the findings are translated into clinical practice.

OBJECTIVE

The aim of this systematic review was to determine where RCTs of manual therapy for neck disorders fall on the efficacy-effectiveness spectrum.

METHODS

A search of three electronic databases including PubMed, CINAHL, and CENTRAL were completed for trials published from inception to May 2023. RCTs in which joint mobilization or manipulation were used to treat cervical spine disorders were assessed on the effectiveness-efficacy spectrum using the Rating of Included Trials on the Efficacy-Effectiveness Spectrum (RITES) tool and risk of bias using the Revised Cochrane Risk of Bias tool.

RESULTS

A total of 174 trials met eligibility. RITES domain two trial setting (71.3% vs 16.1%), domain three flexibility of intervention(s) (62.1% vs 23%), and domain four clinical relevance of experimental and comparison intervention(s) (51.7% vs 29.3%) all favored efficacy over effectiveness. Domain one participant characteristic(s) had a slightly greater emphasis on effectiveness compared to efficacy (36.8% vs 44.8%). Most studies (96%) had at least some risk of bias.

CONCLUSION

Over half of the RCTs assessing the treatment effect of joint mobilization and manipulation for neck pain favor efficacy (explanatory) over effectiveness (pragmatic) designs. Future RCTs on this topic should consider a greater emphasis on pragmatic trial design components in order to better reflect real-world translation to clinical practice.

Implementing a pragmatic clinical trial to tailor opioids for chronic pain on behalf of the IGNITE ADOPT PGx investigators

Chronic pain is a prevalent condition with enormous economic burden. Opioids such as tramadol, codeine, and hydrocodone are commonly used to treat chronic pain; these drugs are activated to more potent opioid receptor agonists by the hepatic CYP2D6 enzyme. Results from clinical studies and mechanistic understandings suggest that CYP2D6-guided therapy will improve pain control and reduce adverse drug events. However, CYP2D6 is rarely used in clinical practice due in part to the demand for additional clinical trial evidence. Thus, we designed the ADOPT-PGx (A Depression and Opioid Pragmatic Trial in Pharmacogenetics) chronic pain study, a multicenter, pragmatic, randomized controlled clinical trial, to assess the effect of CYP2D6 testing on pain management. The study enrolled 1048 participants who are taking or being considered for treatment with CYP2D6-impacted opioids for their chronic pain. Participants were randomized to receive immediate or delayed (by 6 months) genotyping of CYP2D6 with clinical decision support (CDS). CDS encouraged the providers to follow the CYP2D6-guided trial recommendations. The primary study outcome is the 3-month absolute change in the composite pain intensity score assessed using Patient-Reported Outcomes Measurement Information System (PROMIS) measures. Follow-up will be completed in July 2024. Herein, we describe the design of this trial along with challenges encountered during enrollment.

The Oral Health Statistics Guidelines for Reporting Observational Studies and Clinical Trials in Oral Health Research: Explanation and Elaboration

Adequate and transparent reporting is necessary for critically appraising research. Yet, evidence suggests that the design, conduct, analysis, interpretation, and reporting of oral health research could be greatly improved. Accordingly, the Task Force on Design and Analysis in Oral Health Research-statisticians and trialists from academia and industry-empaneled a group of authors to develop methodological and statistical reporting guidelines identifying the minimum information needed to document and evaluate observational studies and clinical trials in oral health: the Oral Health Statistics Guidelines. Drafts were circulated to the editors of 85 oral health journals and to task force members and sponsors and discussed at a December 2020 workshop attended by 49 researchers. The final version was subsequently approved by the task force in September 2021, submitted for journal review in 2022, and revised in 2023. The checklist consists of 48 guidelines: 5 for introductory information, 17 for methods, 13 for statistical analysis, 6 for results, and 7 for interpretation; 7 are specific to clinical trials. Each of these guidelines identifies relevant information, explains its importance, and often describes best practices. The checklist was published in multiple journals. The article was published simultaneously in Journal of Dental Research Clinical and Translational Research, the Journal of the American Dental Association, and the Journal of Oral and Maxillofacial Surgery. Completed checklists should accompany manuscripts submitted for publication to these and other oral health journals to help authors, journal editors, and reviewers verify that the manuscript provides the information necessary to adequately document and evaluate the research.

The OHStat Guidelines for Reporting Observational Studies and Clinical Trials in Oral Health Research: explanation and elaboration

Adequate and transparent reporting is necessary for critically appraising research. Yet, evidence suggests that the design, conduct, analysis, interpretation, and reporting of oral health research could be greatly improved. Accordingly, the Task Force on Design and Analysis in Oral Health Research-statisticians and trialists from academia and industry-empaneled a group of authors to develop methodological and statistical reporting guidelines identifying the minimum information needed to document and evaluate observational studies and clinical trials in oral health: the OHstat Guidelines. Drafts were circulated to the editors of 85 oral health journals and to Task Force members and sponsors and discussed at a December 2020 workshop attended by 49 researchers. The final version was subsequently approved by the Task Force in September 2021, submitted for journal review in 2022, and revised in 2023. The checklist consists of 48 guidelines: 5 for introductory information, 17 for methods, 13 for statistical analysis, 6 for results, and 7 for interpretation; 7 are specific to clinical trials. Each of these guidelines identifies relevant information, explains its importance, and often describes best practices. The checklist was published in multiple journals. The article was published simultaneously in JDR Clinical and Translational Research, the Journal of the American Dental Association, and the Journal of Oral and Maxillofacial Surgery. Completed checklists should accompany manuscripts submitted for publication to these and other oral health journals to help authors, journal editors, and reviewers verify that the manuscript provides the information necessary to adequately document and evaluate the research.

The OHStat Guidelines for Reporting Observational Studies and Clinical Trials in Oral Health Research: Explanation and Elaboration

Adequate and transparent reporting is necessary for critically appraising research. Yet, evidence suggests that the design, conduct, analysis, interpretation, and reporting of oral health research could be greatly improved. Accordingly, the Task Force on Design and Analysis in Oral Health Research-statisticians and trialists from academia and industry-empaneled a group of authors to develop methodological and statistical reporting guidelines identifying the minimum information needed to document and evaluate observational studies and clinical trials in oral health: the OHstat Guidelines. Drafts were circulated to the editors of 85 oral health journals and to Task Force members and sponsors and discussed at a December 2020 workshop attended by 49 researchers. The final version was subsequently approved by the Task Force in September 2021, submitted for journal review in 2022, and revised in 2023. The checklist consists of 48 guidelines: 5 for introductory information, 17 for methods, 13 for statistical analysis, 6 for results, and 7 for interpretation; 7 are specific to clinical trials. Each of these guidelines identifies relevant information, explains its importance, and often describes best practices. The checklist was published in multiple journals. The article was published simultaneously in JDR Clinical and Translational Research, the Journal of the American Dental Association, and the Journal of Oral and Maxillofacial Surgery. Completed checklists should accompany manuscripts submitted for publication to these and other oral health journals to help authors, journal editors, and reviewers verify that the manuscript provides the information necessary to adequately document and evaluate the research.

Analysis of the characteristics and the degree of pragmatism exhibited by pragmatic-labelled trials of antineoplastic treatments

BACKGROUND

Pragmatic clinical trials (PCTs) are designed to reflect how an investigational treatment would be applied in clinical practice. As such, unlike their explanatory counterparts, they measure therapeutic effectiveness and are capable of generating high-quality real-world evidence. However, the conduct of PCTs remains extremely rare. The scarcity of such studies has contributed to the emergence of the efficacy-effectiveness gap and has led to calls for launching more of them, including in the field of oncology. This analysis aimed to identify self-labelled pragmatic trials of antineoplastic interventions and to evaluate whether their use of this label was justified.

METHODS

We searched PubMed® and Embase® for publications corresponding with studies that investigated antitumor therapies and that were tagged as pragmatic in their titles, abstracts and/or index terms. Subsequently, we consulted all available source documents for the included trials and extracted relevant information from them. The data collected were then used to appraise the degree of pragmatism displayed by the PCTs with the help of the validated PRECIS-2 tool.

RESULTS

The literature search returned 803 unique records, of which 46 were retained upon conclusion of the screening process. This ultimately resulted in the identification of 42 distinct trials that carried the 'pragmatic' label. These studies examined eight different categories of neoplasms and were mostly randomized, open-label, multicentric, single-country trials sponsored by non-commercial parties. On a scale of one (very explanatory) to five (very pragmatic), the median PCT had a PRECIS-2 score per domain of 3.13 (interquartile range: 2.57-3.53). The most and least pragmatic studies in the sample had a score of 4.44 and 1.57, respectively. Only a minority of trials were described in sufficient detail to allow them to be graded across all domains of the PRECIS-2 instrument. Many of the studies examined also had features that arguably precluded them from being pragmatic altogether, such as being monocentric or placebo-controlled in nature.

CONCLUSION

PCTs of antineoplastic treatments are generally no more pragmatic than they are explanatory.

Pragmatic Comparative Effectiveness Trials and Learning Health Systems in Pain Medicine: Opportunities and Challenges

Large randomized clinical trials or aggregates of clinical trials represent the highest levels of clinical evidence because they minimize different sources of confounding and bias. The current review provides an in-depth discussion of the challenges faced and methods we can use to overcome these obstacles to tailor novel designs of pragmatic effectiveness trials to pain medicine. The authors describe their experiences with an open-source learning health system to collect high-quality evidence and conduct pragmatic clinical trials within a busy academic pain center.

The Cognitive Orientation to daily Occupational Performance (CO-OP) Approach is superior to ordinary treatment for achievement of goals and transfer effects in children with cerebral palsy and spina bifida - a randomized controlled trial

PURPOSE

Children with cerebral palsy (CP) or spina bifida (SB) often have executive dysfunction affecting activity performance. With the Cognitive Orientation to daily Occupational Performance (CO-OP) Approach, children find their own way to perform activities, using problem-solving strategies and meta-cognitive thinking. The present study aimed to investigate the effectiveness of the CO-OP Approach in children with CP or SB, compared with conventional rehabilitation, in achieving self-identified activity goals, and to explore any generalization and transfer effects.

METHOD

Randomized controlled trial, CO-OP versus treatment as usual, 38 children (7-16 years) participated. Each child identified four goals (to study generalization and transfer, one remained untrained). Primary outcomes: Canadian Occupational Performance Measure (COPM) and Performance Quality Rating Scale (PQRS). Secondary outcomes assessed executive functions and self-rated everyday-life competence.

RESULTS

Self-rated goal attainment (COPM) was significantly greater for both trained and untrained goals in the CO-OP group compared with the control group. The rating of observed performance (PQRS) was significantly higher for trained goals in the CO-OP group. The CO-OP group experienced fewer problems in everyday life after treatment. Executive functions did not differ significantly between groups.

CONCLUSION

CO-OP is more effective than ordinary treatment in achieving both trained and untrained goals.IMPLICATIONS FOR REHABILITATIONCO-OP enables children with CP (MACS levels I-III) or SB without intellectual disabilities to reach self-identified goals.CO-OP shows transfer effects to new activities and situations, which may enhance children's self-efficacy.CO-OP is an important complement to conventional rehabilitation services for children with CP and SB.

Evaluation of droplet digital PCR rapid detection method and precise diagnosis and treatment for suspected sepsis (PROGRESS): a study protocol for a multi-center pragmatic randomized controlled trial

Background

Sepsis is still a major public health concern and a medical emergency due to its high morbidity and mortality. Accurate and timely etiology diagnosis is crucial for sepsis management. As an emerging rapid and sensitive pathogen detection tool, digital droplet PCR (ddPCR) has shown promising potential in rapid identification of pathogens and antimicrobial resistance genes. However, the diagnostic value and clinical impact of ddPCR tests remains to be studied in patients with suspected sepsis. PROGRESS trial is aimed to evaluate the clinical effectiveness of a novel ddPCR assay compared with standard practice.

Methods

PROGRESS is a multicenter, open-label, pragmatic randomized controlled trial (pRCT) set in ten hospitals, including departments of infectious disease and intensive care units. In this study, a total of 2292 patients with suspected sepsis will be randomly assigned to two arms: the ddPCR group and the control group with a ratio of 3:1. The primary outcome is the diagnostic efficacy, that is, the sensitivity and specificity of the ddPCR assay compared with the synchronous blood culture. Secondary outcomes include the mortality rates and the mean Sequential Organ Failure Assessment (SOFA) score at follow-up time points, the length of stay in the hospital, the time to directed antimicrobial therapy, duration of broad-spectrum antibiotic use, and the EQ-5D-5L score on day 90.

Discussion

It is the first multicenter pragmatic RCT to explore the diagnostic efficacy and clinical impact of the ddPCR assay in patients with suspected sepsis, taking advantage of both RCT’s ability to establish causality and the feasibility of pragmatic approaches in real-world studies (RWS). This trial will help us to get a comprehensive view of the assay’s capacity for precise diagnosis and treatment of sepsis. It has the potential to monitor the pathogen load change and to guide the antimicrobial therapy, making a beneficial impact on the prognosis of sepsis patients.

Trial registration: ClinicalTrial.gov, NCT05190861. Registered January 13, 2022—‘Retrospectively registered’, https://clinicaltrials.gov/ct2/show/NCT05190861.

Challenges Encountered in Surgical Traumatic Brain Injury Research: A Need for Methodological Improvement of Future Studies

BACKGROUND

Investigating neurosurgical interventions for traumatic brain injury (TBI) involves complex methodological and practical challenges. In the present report, we have provided an overview of the current state of neurosurgical TBI research and discussed the key challenges and possible solutions.

METHODS

The content of our report was based on an extensive literature review and personal knowledge and expert opinions of senior neurosurgeon researchers and epidemiologists.

RESULTS

Current best practice research strategies include randomized controlled trials (RCTs) and comparative effectiveness research. The performance of RCTs has been complicated by the heterogeneity of TBI patient populations with the associated sample size requirements, the traditional eminence-based neurosurgical culture, inadequate research budgets, and the often acutely life-threatening setting of severe TBI. Statistical corrections can mitigate the effects of heterogeneity, and increasing awareness of clinical equipoise and informed consent alternatives can improve trial efficiency. The substantial confounding by indication, which limits the interpretability of observational research, can be circumvented by using an instrumental variable analysis. Traditional TBI outcome measures remain relevant but do not adequately capture the subtleties of well-being, suggesting a need for multidimensional approaches to outcome assessments.

CONCLUSIONS

In settings in which traditional RCTs are difficult to conduct and substantial confounding by indication can be present, observational studies using an instrumental variable analysis and "pragmatic" RCTs are promising alternatives. Embedding TBI research into standard clinical practice should be more frequently considered but will require fundamental modifications to the current health care system. Finally, multimodality outcome assessment will be key to improving future surgical and nonsurgical TBI research.

The effect on relapse rate and psychiatric symptomatology: Switching a combination of first- and second-generation antipsychotic polypharmacy to antipsychotic monotherapy in long-term inpatients with schizophrenia and related disorders. A pragmatic randomized open-label trial (SwAP trial)

BACKGROUND

There is little evidence to support the use of antipsychotic polypharmacy, and there are concerns about safety and side effects. Nonetheless, it is commonly used in the treatment of long-term inpatients with schizophrenia. This study investigated the effects of switching from a combination of first- and second-generation antipsychotics (FGA and SGA) to monotherapy (FGA or SGA) on relapse rates and psychiatric symptomatology.

METHODS

Institutionalized patients with chronic psychotic disorders using a combination of SGA and FGA (n = 136) participated in a randomized open-label trial. The SWITCH group discontinued either FGA or SGA, the STAY group continued combination treatment. Relapse and psychotic symptoms were measured at baseline and during follow-up at 3, 6, and 9 months. Psychiatric symptomatology was measured using the Brief Psychiatric Rating Scale (BPRS). Relapse was defined as (i) an increase in BPRS score of at least 2 points on any item, or (ii) an increase of at least 4 points in total BPRS score and an adjustment of antipsychotics.

RESULTS

A logistic regression model, corrected for sex, showed that the probability of relapse was significantly lower in the SWITCH group: 0.29 (95% CI 0.13-0.62). The protective effect of switching to monotherapy was attributable to patients continuing clozapine as monotherapy. For patients who did not experience a relapse nor dropped out, BPRS total scores decreased significantly more in the SWITCH group (p = 0.0001).

CONCLUSION

Switching from a combination of FGA and SGA to monotherapy in long-term inpatients does not increase the relapse rate and may even reduce it.

Electro-acupuncture for protracted amphetamine abstinence syndrome: study protocol for a pragmatic randomized controlled trial

BACKGROUND

Protracted amphetamine abstinence syndrome is one of the primary causes of relapse for amphetamine-type drug abusers during withdrawal. However, the importance of the management of protracted amphetamine abstinence syndrome is underestimated. Electro-acupuncture may be a safe and effective alternative therapy for protracted amphetamine abstinence syndrome, but the evidence is limited.

METHODS

The study is a prospective, two-center, randomized, waitlist controlled, open-label pragmatic trial. A total of 300 patients with protracted amphetamine abstinence syndrome will be recruited. All participants will be randomly assigned to an electro-acupuncture group or a waitlist group in a 1:1 ratio. Participants in the electro-acupuncture group will receive the electrical-acupuncture treatment. Waitlist group participants will not receive electro-acupuncture treatment but will be assessed at each visit. Treatments will be administered twice a week for a total of 4 consecutive weeks. The primary outcome in this study is the change in the ACSA between baseline (week 0) and the completion of treatment (week 4), and the secondary outcomes are changes in the Hamilton Depression Scale (HAMD), the visual analog scale (VAS), the Hamilton Anxiety Scale (HAMA), the Pittsburgh Sleep Quality Index (PSQI), the Montreal Cognitive Assessment (MoCA), and the Medical Outcomes Study 36-item Short-Form Health Survey (SF-36).

DISCUSSION

This study will assess the effectiveness of acupuncture in PAAS in real-world settings to provide support for clinical decisions and a basis for subsequent trials comparing acupuncture with other positive regimens.

TRIAL REGISTRATION

ClinicalTrials.gov ChiCTR2000040619 . Registered on 3 December 2020.

Measuring implementation fidelity in a cluster-randomized pragmatic trial: development and use of a quantitative multi-component approach

BACKGROUND

In pragmatic trials, on-site partners, rather than researchers, lead intervention delivery, which may result in implementation variation. There is a need to quantitatively measure this variation. Applying the Framework for Implementation Fidelity (FIF), we develop an approach for measuring variability in site-level implementation fidelity. This approach is then applied to measure site-level fidelity in a cluster-randomized pragmatic trial of Music & MemorySM (M&M), a personalized music intervention targeting agitated behaviors in residents living with dementia, in US nursing homes (NHs).

METHODS

Intervention NHs (N = 27) implemented M&M using a standardized manual, utilizing provided staff trainings and iPods for participating residents. Quantitative implementation data, including iPod metadata (i.e., song title, duration, number of plays), were collected during baseline, 4-month, and 8-month site visits. Three researchers developed four FIF adherence dimension scores. For Details of Content, we independently reviewed the implementation manual and reached consensus on six core M&M components. Coverage was the total number of residents exposed to the music at each NH. Frequency was the percent of participating residents in each NH exposed to M&M at least weekly. Duration was the median minutes of music received per resident day exposed. Data elements were scaled and summed to generate dimension-level NH scores, which were then summed to create a Composite adherence score. NHs were grouped by tercile (low-, medium-, high-fidelity).

RESULTS

The 27 NHs differed in size, resident composition, and publicly reported quality rating. The Composite score demonstrated significant variation across NHs, ranging from 4.0 to 12.0 [8.0, standard deviation (SD) 2.1]. Scaled dimension scores were significantly correlated with the Composite score. However, dimension scores were not highly correlated with each other; for example, the correlation of the Details of Content score with Coverage was τb = 0.11 (p = 0.59) and with Duration was τb = - 0.05 (p = 0.78). The Composite score correlated with CMS quality star rating and presence of an Alzheimer's unit, suggesting face validity.

CONCLUSIONS

Guided by the FIF, we developed and used an approach to quantitatively measure overall site-level fidelity in a multi-site pragmatic trial. Future pragmatic trials, particularly in the long-term care environment, may benefit from this approach.

TRIAL REGISTRATION

Clinicaltrials.gov NCT03821844. Registered on 30 January 2019, https://clinicaltrials.gov/ct2/show/NCT03821844 .

Pragmatic trials of pain therapies: a systematic review of methods

ABSTRACT

Pragmatic randomised clinical trials aim to directly inform clinical or health policy decision making. Here, we systematically review methods and design of pragmatic trials of pain therapies to examine methods, identify common challenges, and areas for improvement. Seven databases were searched for pragmatic randomised controlled clinical trials that assessed pain treatment in a clinical population of adults reporting pain. All screening steps and data extractions were performed twice. Data were synthesised descriptively, and correlation analyses between prespecified trial features and PRECIS-2 (PRagmatic-Explanatory Continuum Indicator Summary 2) ratings and attrition were performed. Protocol registration: PROSPERO-ID CRD42020178954. Of 57 included trials, only 21% assessed pharmacological interventions, the remainder physical, surgical, psychological, or self-management pain therapies. Three-quarters of the trials were comparative effectiveness designs, often conducted in multiple centres (median: 5; Q1/3: 1, 9.25) and with a median sample size of 234 patients at randomization (Q1/3: 135.5; 363.5). Although most trials recruited patients with chronic pain, reporting of pain duration was poor and not well described. Reporting was comprehensive for most general items, while often deficient for specific pragmatic aspects. Average ratings for pragmatism were highest for treatment adherence flexibility and clinical relevance of outcome measures. They were lowest for patient recruitment methods and extent of follow-up measurements and appointments. Current practice in pragmatic trials of pain treatments can be improved in areas such as patient recruitment and reporting of methods, analysis, and interpretation of data. These improvements will facilitate translatability to other real-world settings-the purpose of pragmatic trials.

The GetReal Trial Tool: Design, Assess and Discuss Clinical Drug Trials in Light of RWE Generation

Methodologies incorporating Real World Elements into clinical trial design (also called pragmatic trials) offer an attractive opportunity to assess the effect of a treatment strategy in routine care and as such guide decision making in practice. Uptake of these methods is slow for several reasons, including uncertainty about acceptability of trial results, lack of experience with the methodology and operational challenges. We developed the 'Get Real Trial Tool', an easy-to-use interface, which allows users to assess the impact of design choices on generalisability to routine clinical practice, while taking into account risk of bias, precision, acceptability and operational feasibility. The tool is grounded in the scientific literature on pragmatic trials combined with knowledge of experts from academia, pharmaceutical companies, HTA bodies, patient organisations, and regulators. The aim is to help researchers optimise trial design and facilitate translation of evidence from pragmatic trials to clinical practice. In this paper we describe the development, structure and application of the GetReal Trial Tool.

Utilization of evidence-based guidelines for prevention of obesity and hypercholesterolemia among primary healthcare physicians in southwest of Saudi Arabia

To evaluate knowledge and use of evidence-based guidelines for prevention of obesity and hypercholesterolemia among primary healthcare physicians in Jazan, Saudi Arabia.This cross-sectional study targeted 170 primary healthcare centres (PHCs) in Jazan. Measurement of study's variables was completed during personal interviews. The content of the study instrument was based on The Saudi Guidelines on the Prevention and Management of Obesity and The Cholesterol Management Guide for Healthcare Practitioners.A total of 234 physicians participated in this study. The age of the physicians varied between 25 and 65 years and 51.7% of them were females. Knowledge of the physicians about the eat-well plate recommendations was very low, with none of the physicians reporting the recommended daily portions of fruits and vegetables. Less than 20% of the physicians knew the cutoff points for considering central obesity among male and female individuals. Only 21% of the physicians reported adherence concerning screening for obesity and 42% reported adherence to the guidelines concerning screening for hypercholesterolemia. Only 9% of the physicians were adherent to the guidelines concerning reducing the risk of obesity and only 13% reported adherence to the guidelines related to the reduction of hypercholesterolemia risk.The current investigation detected a low level of knowledge and adherence concerning the evidence-based practice related to prevention of the obesity and hypercholesterolemia and therefore limited role of PHC physicians in the prevention of obesity or hypercholesterolemia.

Acupuncture combined with opioids for cancer pain: a pilot pragmatic randomized controlled trial

OBJECTIVE

Given the existing evidence for the analgesic effect of acupuncture, the current study aimed to assess whether acupuncture could be feasible and manageable as an adjunctive therapy for cancer pain in a real-world hospital setting.

METHODS

Thirty patients in an Oncology department with moderate or severe pain were recruited and randomized to an adjunctive acupuncture group or control group, who received pharmacotherapy for pain management without acupuncture. The duration of the treatment course was 1 week with a 2-week follow-up. In total, four acupuncture sessions were administered, on days 1/2/4/6 of the trial. Pain intensity was measured using a numerical rating scale (NRS) and the daily opioid dose was recorded.

RESULTS

The overall trends favored acupuncture for both pain intensity and daily opioid consumption. The proportion of participants experiencing at least a 2-point reduction in the NRS at the end of the treatment was 93% (n = 14/15) for the acupuncture group and 57% (n = 8/14) for the control group (risk difference (RD) 36.1%, 95% confidence interval (CI) [7.4%-65.0%]; relative risk (RR) 1.63, 95% CI [1.02-2.62]; p = 0.04). There were no serious adverse events and no dropouts during the treatment.

CONCLUSION

This pilot study showed that adding acupuncture to routine analgesia for patients with cancer pain was feasible and acceptable to patients. The clinical effects of adding acupuncture as an adjunctive therapy need to be further evaluated.

CLINICAL TRIAL REGISTRATION NUMBER

ChiCTR1800017023 (Chinese Clinical Trial Registry).

[Benefit assessment of digital health applications-challenges and opportunities]

Digital health applications promise to improve patient health and medical care. This analysis provides a brief overview of evidence-based benefit assessment and the challenges to the underlying evidence as prerequisites for optimal patient-oriented decision making. Classical concepts in study design, recent developments, and innovative approaches are described with the aim of highlighting future areas of development in innovative study designs and strategic evaluation concepts for digital health applications. A special focus is on pragmatic study designs.Evidence-based benefit assessment has fundamental requirements and criteria regardless of the type of treatments evaluated. Reliable evidence is essential. Fast, efficient, reliable, and practice-relevant evaluation of digital health applications is not achieved by turning to nonrandomized trials, but rather by better pragmatic randomized trials. They are feasible and combine the characteristics of digital health applications, classical methodological concepts, and new approaches to study conduct. Routinely collected data, low-contact study conduct (remote trials, virtual trials), and digital biomarkers promote useful randomized real-world evidence as solid evidence base for digital health applications. Continuous learning evaluation with randomized designs embedded in routine care is key to sustainable and efficient benefit assessment of digital health applications and may be crucial for strategic improvement of healthcare.

Safety data in randomised real-world evidence studies: Salford Lung Study learnings

Evidence to support clinical decision making must be based on safety data that have been captured, analysed and interpreted in a robust and reliable way. Randomised real-world evidence (RRWE) studies provide the opportunity to evaluate the use of medicines in patients and settings representative of routine clinical practice. However, elements that underpin the design of RRWE studies can have a significant impact upon the analysis, interpretation and implications of safety data. In this narrative review, we use data from the Salford Lung Study; two prospective, 12-month, open-label, parallel-group, phase III randomised controlled trials conducted in primary care in the UK; to highlight the importance of capturing treatment modifications when attempting to evaluate safety events according to actual treatment exposure. We demonstrate that analysing safety data by actual treatment received (i.e. accounting for the treatment modifications that occur routinely in the primary care setting) provides additional insight beyond analysing according to randomised treatment strategy only. It is therefore proposed that understanding of safety data from RRWE trials can be optimised by analysing both by randomised group and by actual treatment received.

Population diversity of cardiovascular outcome trials and real-world patients with diabetes in a Chinese tertiary hospital

BACKGROUND

Recent cardiovascular outcome trials (CVOTs) changed the therapeutic strategy of guidelines for type 2 diabetes. We compared the characteristics of patients from real-world hospital settings with those of participants in recent pragmatic randomized trials.

METHODS

This electronic medical record (EMR)-based retrospective observational study investigated the data of patients with diabetes from inpatient and outpatient settings in West China Hospital of Sichuan University from January 1, 2011, to June 30, 2019. We identified patients meeting the inclusion criteria of a pragmatic randomized trial (EMPA-REG OUTCOME) based on EMRs and compared their baseline characteristics with those of the trial participants. The cutoff for the clinical significance of each characteristic was set as its minimal clinically important difference based on expert consultation.

RESULTS

We included 48,257 inpatients and 36,857 outpatients with diabetes and found that 8389 (17.4%) inpatients and 2646 (7.2%) outpatients met the inclusion criteria for the EMPA-REG OUTCOME trial. Compared with the trial population, the real-world inpatients meeting the eligibility criteria of the EMPA-REG OUTCOME had similar age, blood pressure, and lipid profiles but comprised of fewer males, metformin users, anti-hypertensive drug users, and aspirin users, and had a lower body mass index. The group of outpatients meeting the eligibility criteria had fewer males, similar age, fewer metformin users, fewer insulin users, fewer anti-hypertensive drug users, and fewer aspirin users compared with the trial population.

CONCLUSIONS

The trial population in EMPA-REG OUTCOME represents only a small portion of patients with diabetes from the inpatient and outpatient departments of a Chinese tertiary medical center. Evidence localization in different clinical settings and validation are essential to enabling extrapolation of the results from CVOTs in patients with diabetes to Chinese clinical practice.

What Factors Predict Willingness to Join Low-Risk Pragmatic Clinical Trials?

Pragmatic clinical trials (PCTs) may improve the efficiency, relevance, and representativeness of research. While prior research has indicated that willingness to join a PCT is high, it is not universal among those asked in surveys exploring attitudes toward hypothetical PCTs. The objective of this study was to examine what factors predict willingness to join a hypothetical low-risk PCT comparing two blood pressure medicines. In our study, 2,618 respondents, recruited from three populations (adult patients from an academic health system, adult patients from an integrated delivery system, and adults from an online nationally representative panel), completed an online survey. Most respondents (90%) expressed willingness to participate in the hypothetical PCT. The two key predictors of expressed willingness to join low-risk PCTs were respondents' understanding of key features of PCTs, including how they differ from traditional research, and the degree of importance respondents perceived comparative research to have. Increasing awareness of the rationale for PCTs and understanding of these trials, including how they differ from explanatory trials, may increase prospective participants' willingness to contribute to this effort.

KEPT-app trial: a pragmatic, single-blind, parallel, cluster-randomised effectiveness study of pelvic floor muscle training among incontinent pregnant women: study protocol

INTRODUCTION

Pelvic floor muscle training (PFMT) strongly recommended to incontinent pregnant women. The Kegel Exercise Pregnancy Training-app trial is a multicentre cluster-randomised study aims to assess the effectiveness and its cost-effectiveness of the mobile app guidance in PFMT among incontinent pregnant women.

METHODS AND ANALYSIS

370 pregnant women (aged 18 years old and above) will be recruited with International Consultation on Incontinence Questionnaire-Urinary Incontinence-Short Form. Ten clusters (primary care clinics) will be randomly assigned to either PFMT or usual care in a 1:1 ratio by an independent researcher (sealed envelope). The primary outcome will be urinary incontinence, and the secondary outcomes (quality of life; PFMT adherence, psychological status and mobile apps' usability) will be assessed at four measurement time points (t0: baseline) and postintervention (t1: 4 weeks, t2: 8 weeks and t3: 8 weeks postnatal). T-test analysis will determine any significant differences at the baseline between the control and intervention groups. The mixed-model analysis will determine the effectiveness of the intervention at the population-average level for both the primary and secondary outcomes. For the cost-effectiveness analysis, expenditures during the study and 6 months after the intervention will be compared between the groups using the multiway sensitivity analysis. The recruitment planned will be in December 2020, and the planned end of the study will be in August 2021.

ETHICS AND DISSEMINATION

This study protocol was approved by the Ethics Committee for Research Involving Human Subjects, Universiti Putra Malaysia (JKEUPM-2019-368) and Medical Research and Ethics Committee (MREC), Ministry of Health Malaysia, NMRR-19-412-47116 (IIR) with the ANZCTR registration. This study will obtain informed written consent from all the study participants. The results which conform with the Consolidated Standards of Reporting Trials and the Recommendations for Interventional Trials will be published for dissemination in peer-reviewed journals and conference proceedings.

TRIAL REGISTRATION NUMBER

ACTRN12619000379112.

Feasibility of using Clinical Practice Research Datalink data to identify patients with chronic obstructive pulmonary disease to enrol into real-world trials

PURPOSE

To assess the feasibility of using Clinical Practice Research Datalink (CPRD) data for identifying populations of patients with chronic obstructive pulmonary disease (COPD) eligible for a hypothetical pragmatic trial.

METHODS

A retrospective multidatabase cohort study using CPRD primary care and linked secondary care data to describe the characteristics of populations of patients with COPD. Patients' demographic and lifestyle factors, comorbidity profile, spirometry measurements and treatment changes were evaluated, as was the distribution of follow-up time and types of losses during follow-up. Characteristics were evaluated using descriptive statistics.

RESULTS

A total of 322 991 patients from 1148 primary care practices in the United Kingdom across two CPRD primary care databases, CPRD GOLD and CPRD Aurum, were potentially eligible to participate in a hypothetical trial using CPRD, starting on 31 December 2017. Patients with COPD in CPRD GOLD and CPRD Aurum were comparable in terms of age (median age 70 vs. 68 years), gender (50% vs. 52% male), disease severity (e.g., 25% vs. 24% Medical Research Council [MRC] dyspnoea score grades 3-5) and history of respiratory conditions (e.g., 43% vs. 38% asthma). High proportions of patients with COPD in CPRD GOLD and CPRD Aurum were available on 31 December 2012 for follow-up at 1, 2, and 5 years (92%, 85% and 67%, respectively).

CONCLUSIONS

Patients and data from CPRD GOLD and CPRD Aurum were comparable across key aspects relevant to COPD trials. A pragmatic trial using CPRD to recruit patients with COPD is scientifically feasible.

The impact of fluticasone furoate/vilanterol on healthcare resource utilisation in the Salford Lung Study in chronic obstructive pulmonary disease

AIM

The Salford Lung Study (SLS) in chronic obstructive pulmonary disease (COPD) was a randomised controlled trial evaluating the effectiveness and safety of initiating fluticasone furoate/vilanterol (FF/VI) 100/25 µg versus continuing usual care (UC) in patients with COPD and a history of exacerbations. Here, we investigate the impact of initiating FF/VI on healthcare resource utilisation (HRU) in SLS COPD.

METHODS

HRU and interventions were determined from patients' electronic health records. Annual rates of on-treatment all-cause and COPD-related secondary care contacts (SCCs) and primary care contacts (PCCs) for FF/VI versus UC were analysed using a general linear model. Costs were derived from national data sources.

RESULTS

Least-squares (LS) mean annual rates of all-cause (9.81 versus 9.36) and COPD-related (1.57 versus 1.48) SCCs were similar for FF/VI and UC, as were rates of all-cause hospitalisations (0.87 versus 0.82). Mean duration of hospital stay/patient was 4.5 and 4.2 days, respectively. COPD-related SCC mean total cost/patient was £484 FF/VI and £475 UC. LS mean annual rates of all-cause PCCs were significantly higher for FF/VI (21.20 versus 18.88 UC; p < 0.001). LS mean annual rates of COPD-related PCCs were similar for FF/VI and UC (2.42 versus 2.46). All-cause PCC mean total cost/patient was £900 FF/VI versus £811 UC, but COPD-related PCC costs were similar (£116 versus £114). Direct COPD-related total medical costs/patient were significantly lower for FF/VI (LS geometric mean £806 versus £963 UC; p < 0.001).

DISCUSSION

In patients with COPD and exacerbation history, FF/VI may represent a less costly alternative to current therapies.GlaxoSmithKline plc. study HZC115151; ClinicalTrials.gov NCT01551758.The reviews of this paper are available via the supplemental material section.

Clinical Evidence for Association of Acupuncture and Acupressure With Improved Cancer Pain: A Systematic Review and Meta-Analysis

Question

Is the use of acupuncture and acupressure associated with improved cancer pain management compared with sham intervention and/or analgesic therapy alone?

Findings

In this systematic review of 17 randomized clinical trials and meta-analysis of 14 trials in the current English-language and Chinese-language literature, a significant association was found between real (compared with sham) acupuncture and reduced pain, and acupuncture combined with analgesic therapy was associated with decreased analgesic use. However, heterogeneity lowered the level of certainty of the evidence.

Meaning

This study found a moderate level of evidence that acupuncture and/or acupressure was significantly associated with lower pain intensity in patients with cancer compared with a sham control, which suggests a potential for a combination of acupuncture and acupressure to help reduce opioid doses in patients with cancer.

Importance

Research into acupuncture and acupressure and their application for cancer pain has been growing, but the findings have been inconsistent.

Objective

To evaluate the existing randomized clinical trials (RCTs) for evidence of the association of acupuncture and acupressure with reduction in cancer pain.

Data Sources

Three English-language databases (PubMed, Embase, and CINAHL) and 4 Chinese-language biomedical databases (Chinese Biomedical Literature Database, VIP Database for Chinese Technical Periodicals, China National Knowledge Infrastructure, and Wanfang) were searched for RCTs published from database inception through March 31, 2019.

Study Selection

Randomized clinical trials that compared acupuncture and acupressure with a sham control, analgesic therapy, or usual care for managing cancer pain were included.

Data Extraction and Synthesis

Data were screened and extracted independently using predesigned forms. The quality of RCTs was appraised with the Cochrane Collaboration risk of bias tool. Random-effects modeling was used to calculate the effect sizes of included RCTs. The quality of evidence was evaluated with the Grading of Recommendations Assessment, Development and Evaluation approach.

Main Outcomes and Measures

The primary outcome was pain intensity measured by the Brief Pain Inventory, Numerical Rating Scale, Visual Analog Scale, or Verbal Rating Scale.

Results

A total of 17 RCTs (with 1111 patients) were included in the systematic review, and data from 14 RCTs (with 920 patients) were used in the meta-analysis. Seven sham-controlled RCTs (35%) were notable for their high quality, being judged to have a low risk of bias for all of their domains, and showed that real (compared with sham) acupuncture was associated with reduced pain intensity (mean difference [MD], −1.38 points; 95% CI, −2.13 to −0.64 points; I² = 81%). A favorable association was also seen when acupuncture and acupressure were combined with analgesic therapy in 6 RCTs for reducing pain intensity (MD, −1.44 points; 95% CI, −1.98 to −0.89; I² = 92%) and in 2 RCTs for reducing opioid dose (MD, −30.00 mg morphine equivalent daily dose; 95% CI, −37.5 mg to −22.5 mg). The evidence grade was moderate because of the substantial heterogeneity among studies.

Conclusions and Relevance

This systematic review and meta-analysis found that acupuncture and/or acupressure was significantly associated with reduced cancer pain and decreased use of analgesics, although the evidence level was moderate. This finding suggests that more rigorous trials are needed to identify the association of acupuncture and acupressure with specific types of cancer pain and to integrate such evidence into clinical care to reduce opioid use.

Real-world data for health technology assessment for reimbursement decisions in Asia: current landscape and a way forward

There is growing interest globally in using real-world data (RWD) and real-world evidence (RWE) for health technology assessment (HTA). Optimal collection, analysis, and use of RWD/RWE to inform HTA requires a conceptual framework to standardize processes and ensure consistency. However, such framework is currently lacking in Asia, a region that is likely to benefit from RWD/RWE for at least two reasons. First, there is often limited Asian representation in clinical trials unless specifically conducted in Asian populations, and RWD may help to fill the evidence gap. Second, in a few Asian health systems, reimbursement decisions are not made at market entry; thus, allowing RWD/RWE to be collected to give more certainty about the effectiveness of technologies in the local setting and inform their appropriate use. Furthermore, an alignment of RWD/RWE policies across Asia would equip decision makers with context-relevant evidence, and improve timely patient access to new technologies. Using data collected from eleven health systems in Asia, this paper provides a review of the current landscape of RWD/RWE in Asia to inform HTA and explores a way forward to align policies within the region. This paper concludes with a proposal to establish an international collaboration among academics and HTA agencies in the region: the REAL World Data In ASia for HEalth Technology Assessment in Reimbursement (REALISE) working group, which seeks to develop a non-binding guidance document on the use of RWD/RWE to inform HTA for decision making in Asia.

Balancing Internal and External Validity Using Precis-2 and Re-Aim: Case Exemplars

The National Institutes of Health estimated that $3-$4 billion is spent supporting clinical trials annually, yet an average of 17 years is required for research evidence to be implemented into practice. Study designs that balance external and internal validity, based on the goals of the study and relative to the state of the science, may accelerate knowledge translation. The purpose of this case study was to evaluate two randomized control trials on the effectiveness-efficacy continuum using the PRagmatic Explanatory Continuum Indicator Summary and the Research Effectiveness Adoption Implementation Maintenance model. Findings provided insight into strategies that may lead to more balanced approaches to research design. Incorporating tools such as PRECIS-2 and RE-AIM when designing and implementing research interventions may accelerate knowledge translation to close the gap between scientific knowledge and clinical practice.

The importance of decision intent within descriptions of pragmatic trials

OBJECTIVE

It is now more than 50 years since the concepts of explanatory and pragmatic attitudes toward trials were first discussed by Schwartz and Lellouch in their influential 1967 paper. Since then, there has been increasing focus on design aspects that may be consistent with more pragmatic attitudes within clinical trials, and a number of tools developed to assist investigators prospectively think about their trial design. Researchers have subsequently expressed interest in using these tools retrospectively to characterize trials as pragmatic or explanatory.

RESULTS

We suggest that recent attempts to retrospectively dichotomize trials solely on the basis of quantitative scoring of trial design features are flawed. Instead, we argue that there is a need to consider both the intent and design when assessing the degree of pragmatism within a trial.

CONCLUSION

The practical implication of our suggestion for trial reporting is that investigators should explicitly state the intent of the trial through a clear articulation of the decision that they hope will be informed by the trial results. This should be coupled with a completed PRagmatic-Explanatory Continuum Indicator Summary 2 assessment (or similar) with an explanation of study design choices to appropriately assess whether the study design is consistent with the study intent. We believe this will assist reviewers and knowledge users in making assessments of trials.

Cluster over individual randomization: are study design choices appropriately justified? Review of a random sample of trials

Background:

Novel rationales for randomizing clusters rather than individuals appear to be emerging from the push for more pragmatic trials, for example, to facilitate trial recruitment, reduce the costs of research, and improve external validity. Such rationales may be driven by a mistaken perception that choosing cluster randomization lessens the need for informed consent. We reviewed a random sample of published cluster randomized trials involving only individual-level health care interventions to determine (a) the prevalence of reporting a rationale for the choice of cluster randomization; (b) the types of explicit, or if absent, apparent rationales for the use of cluster randomization; (c) the prevalence of reporting patient informed consent for study interventions; and (d) the types of justifications provided for waivers of consent. We considered cluster randomized trials for evaluating exclusively the individual-level health care interventions to focus on clinical trials where individual randomization is only theoretically possible and where there is a general expectation of informed consent.

Methods:

A random sample of 40 cluster randomized trials were identified by implementing a validated electronic search filter in two electronic databases (Ovid MEDLINE and Embase), with two reviewers independently extracting information from each trial. Inclusion criteria were the following: primary report of a cluster randomized trial, evaluating exclusively an individual-level health care intervention, published between 2007 and 2016, and conducted in Canada, the United States, European Union, Australia, or low- and middle-income country settings.

Results:

Twenty-five trials (62.5%, 95% confidence interval = 47.5%–77.5%) reported an explicit rationale for the use of cluster randomization. The most commonly reported rationales were those with logistical or administrative convenience (15 trials, 60%) and those that need to avoid contamination (13 trials, 52%); five trials (20%) were cited rationales related to the push for more pragmatic trials. Twenty-one trials (52.5%, 95% confidence interval = 37%–68%) reported written informed consent for the intervention, two (5%) reported verbal consent, and eight (20%) reported waivers of consent, while in nine trials (22.5%) consent was unclear or not mentioned. Reported justifications for waivers of consent included that study interventions were already used in clinical practice, patients were not randomized individually, and the need to facilitate the pragmatic nature of the trial. Only one trial reported an explicit and appropriate justification for waiver of consent based on minimum criteria in international research ethics guidelines, namely, infeasibility and minimal risk.

Conclusion:

Rationales for adopting cluster over individual randomization and for adopting consent waivers are emerging, related to the need to facilitate pragmatic trials. Greater attention to clear reporting of study design rationales, informed consent procedures, as well as justification for waivers is needed to ensure that such trials meet appropriate ethical standards.

Addressing identification bias in the design and analysis of cluster-randomized pragmatic trials: a case study

Background

Pragmatic trials provide the opportunity to study the effectiveness of health interventions to improve care in real-world settings. However, use of open-cohort designs with patients becoming eligible after randomization and reliance on electronic health records (EHRs) to identify participants may lead to a form of selection bias referred to as identification bias. This bias can occur when individuals identified as a result of the treatment group assignment are included in analyses.

Methods

To demonstrate the importance of identification bias and how it can be addressed, we consider a motivating case study, the PRimary care Opioid Use Disorders treatment (PROUD) Trial. PROUD is an ongoing pragmatic, cluster-randomized implementation trial in six health systems to evaluate a program for increasing medication treatment of opioid use disorders (OUDs). A main study objective is to evaluate whether the PROUD intervention decreases acute care utilization among patients with OUD (effectiveness aim). Identification bias is a particular concern, because OUD is underdiagnosed in the EHR at baseline, and because the intervention is expected to increase OUD diagnosis among current patients and attract new patients with OUD to the intervention site. We propose a framework for addressing this source of bias in the statistical design and analysis.

Results

The statistical design sought to balance the competing goals of fully capturing intervention effects and mitigating identification bias, while maximizing power. For the primary analysis of the effectiveness aim, identification bias was avoided by defining the study sample using pre-randomization data (pre-trial modeling demonstrated that the optimal approach was to use individuals with a prior OUD diagnosis). To expand generalizability of study findings, secondary analyses were planned that also included patients newly diagnosed post-randomization, with analytic methods to account for identification bias.

Conclusion

As more studies seek to leverage existing data sources, such as EHRs, to make clinical trials more affordable and generalizable and to apply novel open-cohort study designs, the potential for identification bias is likely to become increasingly common. This case study highlights how this bias can be addressed in the statistical study design and analysis.

Trial registration

ClinicalTrials.gov, NCT03407638. Registered on 23 January 2018.

Using predictive analytics to improve pragmatic trial design

Clinical trials embedded in health systems can randomize large populations using automated data sources to determine trial eligibility and assess outcomes. The suicide prevention outreach trial used real-world data for trial design and randomized 18,868 individuals in four health systems using patient-reported thoughts of death or self-harm (Patient Health Questionnaire item 9). This took 3.5 years. We consider if using predictive analytics, that is, suicide risk estimates based on prediction models, could improve trial "efficiency." We used data on mental health outpatient visits between 1 January 2009 and 30 September 2017 in seven health systems (HealthPartners; Henry Ford Health System; and Colorado, Hawaii, Northwest, Southern California, and Washington Kaiser Permanente regions). We used a suicide risk prediction model developed in these same systems. We compared five trial designs with different eligibility criteria: a response of a 2 or 3 on Patient Health Questionnaire item 9, a response of a 3, suicide risk score above 90th, 95th, or 99th percentile. We compared the sample that met each criterion, 90-day suicide attempt rate following first eligible visit, and necessary sample sizes to detect a 15%, 25%, and 35% relative reduction in the suicide attempt rate, assuming 90% power, for each eligibility criterion. Our sample included 24,355,599 outpatient visits. Despite wide-spread use of Patient Health Questionnaire, 21,026,985 (86.3%) visits did not have a recorded Patient Health Questionnaire. Of the 2,928,927 individuals in our sample, 109,861 had a recorded Patient Health Questionnaire item 9 response of a 2 or 3 over the study years with a 1.40% 90-day suicide attempt rate and 50,047 had a response of a 3 (suicide attempt rate 1.98%). More patients met criteria requiring a certain risk score or higher: 331,273 had a 90th percentile risk score or higher (suicide attempt rate: 1.36%); 182,316 a 95th percentile or higher (suicide attempt rate 2.16%), and 78,655 a 99th percentile or higher (suicide attempt rate: 3.95%). Eligibility criterion of a Patient Health Questionnaire item 9 response of a 2 or 3 would require randomizing 44,081 individuals (40.2% of eligible population in our sample); eligibility criterion of a 3 would require 31,024 individuals (62.0% of eligible population). Eligibility criterion of a suicide risk score of 90th percentile or higher would require 45,675 individuals (13.8% of eligible population), 95th percentile 28,699 individuals (15.7% of eligible population), and 99th percentile 15,509 (19.7% of eligible population). A suicide risk prediction calculator could improve trial "efficiency"; identifying more individuals at increased suicide risk than relying on patient-report. It is an open scientific question if individuals identified using predictive analytics would respond differently to interventions than those identified by more traditional means.

Real-World Data and Randomised Controlled Trials: The Salford Lung Study

Traditional efficacy double-blind randomised controlled trials (DBRCTs) measure the benefit a treatment produces under near-ideal test conditions in highly selected patient populations; however, the behaviour of patients and investigators in such trials is highly controlled, highly compliant and adherent, and non-representative of routine clinical practice. Pragmatic effectiveness trials measure the benefit a treatment produces in patients in everyday "real-world" practice. Ideally, effectiveness trials should recruit patients as similar as possible to those who will ultimately be prescribed the medicine, and create freedom within the study design to allow normal behaviours of patients and healthcare professionals (HCPs) to be expressed. The Salford Lung Study (SLS) was a world-first, prospective, phase III, pragmatic randomised controlled trial (RCT) programme in patients with chronic obstructive pulmonary disease and asthma to evaluate the effectiveness of a pre-licensed medication (fluticasone furoate/vilanterol) in real-world practice using electronic health records and through collaboratively engaging general practitioners and community pharmacists in clinical research. The real-world aspect of SLS was unique, requiring careful planning and attention to the goals of maximising the external validity of the trials while maintaining scientific rigour and securing suitable electronic processes for proper interpretation of safety data. Key learnings from SLS that may inform the design of future pragmatic effectiveness RCTs include: (1) ensuring the trial setting and operational infrastructure are aligned with routine clinical care; (2) recruiting a broad patient population with characteristics as close as possible to patients in routine clinical practice, to maximise the generalisability and applicability of trial results; (3) ensuring that patients and HCPs are suitably engaged in the trial, to maximise the chances of successful trial delivery; and (4) careful study design, incorporating outcomes of value to patients, HCPs, policymakers and payers, and using pre-planned analyses to address scientifically valid research hypotheses to ensure robustness of the trial data.

Pragmatic Trials in Genomic Medicine: The Integrating Pharmacogenetics In Clinical Care (I-PICC) Study

Pragmatic clinical trials (PCTs) have an established presence in clinical research and yet have only recently garnered attention within the landscape of genomic medicine. Using the PRagmatic-Explanatory Continuum Indicator Summary 2 (PRECIS-2) as a framework, this paper illustrates the application of PCT principles to The Integrating Pharmacogenetics In Clinical Care (I-PICC) Study, a trial of pharmacogenetic testing prior to statin initiation for cardiovascular disease prevention in primary care. The trial achieved high engagement with providers (85% enrolled of those approached) and enrolled a representative sample of participants for which statin therapy would be recommended. The I-PICC Study has a high level of pragmatism, which should enhance the generalizability of its findings. The PRECIS-2 may be useful in the design and evaluation of PCTs of genomic medicine interventions, contributing to the generation of evidence that can bridge the gap between genomics innovation and clinical adoption.

Quality improvement and practice-based research in sleep medicine using structured clinical documentation in the electronic medical record

Background:

We developed and implemented a structured clinical documentation support (SCDS) toolkit within the electronic medical record, to optimize patient care, facilitate documentation, and capture data at office visits in a sleep medicine/neurology clinic for patient care and research collaboration internally and with other centers.

Methods:

To build our SCDS toolkit, physicians met frequently to develop content, define the cohort, select outcome measures, and delineate factors known to modify disease progression. We assigned tasks to the care team and mapped data elements to the progress note. Programmer analysts built and tested the SCDS toolkit, which included several score tests. Auto scored and interpreted tests included the Generalized Anxiety Disorder 7-item, Center for Epidemiological Studies Depression Scale, Epworth Sleepiness Scale, Pittsburgh Sleep Quality Index, Insomnia Severity Index, and the International Restless Legs Syndrome Study Group Rating Scale. The SCDS toolkits also provided clinical decision support (untreated anxiety or depression) and prompted enrollment of patients in a DNA biobank.

Results:

The structured clinical documentation toolkit captures hundreds of fields of discrete data at each office visit. This data can be displayed in tables or graphical form. Best practice advisories within the toolkit alert physicians when a quality improvement opportunity exists. As of May 1, 2019, we have used the toolkit to evaluate 18,105 sleep patients at initial visit. We are also collecting longitudinal data on patients who return for annual visits using the standardized toolkits. We provide a description of our development process and screenshots of our toolkits.

Conclusions:

The electronic medical record can be structured to standardize Sleep Medicine office visits, capture data, and support multicenter quality improvement and practice-based research initiatives for sleep patients at the point of care.

The ethical challenges raised in the design and conduct of pragmatic trials: an interview study with key stakeholders

Background

There is a concern that the apparent effectiveness of interventions tested in clinical trials may not be an accurate reflection of their actual effectiveness in usual practice. Pragmatic randomized controlled trials (RCTs) are designed with the intent of addressing this discrepancy. While pragmatic RCTs may increase the relevance of research findings to practice they may also raise new ethical concerns (even while reducing others). To explore this question, we interviewed key stakeholders with the aim of identifying potential ethical challenges in the design and conduct of pragmatic RCTs with a view to developing future guidance on these issues.

Methods

Interviews were conducted with clinical investigators, methodologists, patient partners, ethicists, and other knowledge users (e.g., regulators). Interviews covered experiences with pragmatic RCTs, ethical issues relevant to pragmatic RCTs, and perspectives on the appropriate oversight of pragmatic RCTs. Interviews were coded inductively by two coders. Interim and final analyses were presented to the broader team for comment and discussion before the analytic framework was finalized.

Results

We conducted 45 interviews between April and September 2018. Interviewees represented a range of disciplines and jurisdictions as well as varying content expertise. Issues of importance in pragmatic RCTs were (1) identification of relevant risks from trial participation and determination of what constitutes minimal risk; (2) determining when alterations to traditional informed consent approaches are appropriate; (3) the distinction between research, quality improvement, and practice; (4) the potential for broader populations to be affected by the trial and what protections they might be owed; (5) the broader range of trial stakeholders in pragmatic RCTs, and determining their roles and responsibilities; and (6) determining what constitutes “usual care” and implications for trial reporting.

Conclusions

Our findings suggest both the need to discuss familiar ethical topics in new ways and that there are new ethical issues in pragmatic RCTs that need greater attention. Addressing the highlighted issues and developing guidance will require multidisciplinary input, including patient and community members, within a broader and more comprehensive analysis that extends beyond consent and attends to the identified considerations relating to risk and stakeholder roles and responsibilities.

Acupuncture for cancer pain: protocol for a pilot pragmatic randomised controlled trial

INTRODUCTION

Although acupuncture has been recommended for alleviating cancer pain by clinical guidelines, the level of the supporting evidence needs to be improved. A pragmatic randomised controlled trial (pRCT) in a hospital setting would provide real-world assessments of the overall clinical effects of acupuncture. This pilot trial aims to explore the feasibility and provide data for sample size calculations for a pRCT evaluating the effectiveness of acupuncture as an adjunctive therapy to routine medical care for cancer pain.

METHODS AND ANALYSIS

Thirty patients with cancer admitted to the oncology department with moderate or severe pain will be recruited. Participants will be randomised at a ratio of 1:1 to the adjunctive acupuncture group or a control group which receives routine pain management without acupuncture. The standardised section of the acupuncture protocol will be developed based on the results of reviews of the literature, recommendations in clinical guidelines and interviews with clinical experts. The acupuncturist will be allowed to tailor the protocol according to the individual situation of each participant. Primary outcomes relevant to the feasibility of conducting a fully powered trial include: numbers and proportions of participants recruited, screened, consented and randomised; numbers and reasons for withdrawals and dropouts; numbers and types of adverse events; feasibility of implementing the trial procedures; evaluation of the comprehensiveness and ease-of-use of the case report form. Secondary outcomes are clinical measurements of the effectiveness of the treatment that are intended for use in the full-scale trial. Analysis of feasibility will be descriptive and pain intensity measures will be analysed using mixed-effects regression.

ETHICS AND DISSEMINATION

Ethics approval was obtained from the Institutional Ethics Committee of Guangdong Provincial Hospital of Chinese Medicine (approval no: Z2017-184-01) and RMIT University Human Research Ethics Committee (reference no: 21361). Results will be disseminated in a peer-reviewed journal, and trial participants will be informed via email and/or phone calls.

TRIAL REGISTRATION NUMBER

ChiCTR1800017023.

Trial design and baseline data for LIRA-PRIME: A randomized trial investigating the efficacy of liraglutide in controlling glycaemia in type 2 diabetes in a primary care setting

AIMS

Using a pragmatic approach, the LIRA-PRIME trial aims to address a knowledge gap by comparing efficacy in controlling glycaemia with glucagon-like peptide-1 analog liraglutide vs oral antidiabetic drugs (OADs) in patients with type 2 diabetes (T2D) uncontrolled with metformin monotherapy in primary care practice. We report the study design and patient baseline characteristics.

MATERIALS AND METHODS

This 104-week, two-arm, open-label, active-controlled trial is active in 219 primary care practices across nine countries. At screening, eligible patients with T2D were at least 18 years of age, had been using a stable daily dose of metformin ≥1500 mg or the maximum tolerated dose for ≥60 days, and had a glycated haemoglobin (HbA1c) of 7.5% to 9.0%, measured ≤90 days before screening. Patients were randomized (1:1) to liraglutide or OAD, both in addition to pre-trial metformin. Individual OADs were chosen by the treating physician based on local guidelines. The primary endpoint is time to inadequate glycaemic control, defined as HbA1c above 7.0% at two scheduled consecutive visits after the first 26 weeks of treatment.

RESULTS

The trial randomized 1997 patients with a mean (standard deviation) age of 56.9 (10.8) years, T2D duration of 7.2 (5.9) years (range, <1-47 years), and HbA1c of 8.2%. One-fifth of patients had a history of diabetes complications, and most were overweight (24.8%) or had obesity (65.3%).

CONCLUSIONS

This pragmatically designed, large-scale, multinational, randomized clinical trial will help guide treatment decisions for patients with T2D who are inadequately controlled with metformin monotherapy and treated in primary care.

From randomised controlled trial to real world implementation of a novel home-based heart failure tool: pooled and comparative analyses of two clinical controlled trials

Objectives

A home-based tool for heart failure (HF) patients, was evaluated in a specialist setting as a randomised controlled trial (RCT) and also in a validation cohort in a primary care setting in a clinical controlled trial (CCT). The tool provides education, symptom monitoring and titration of diuretics. The aim of this study was thus to extend validity of the previous RCT findings in order to describe applicability of the tool in clinical practice.

Methods

Data from both trials were analysed separately, as well as a pooled data set (n=172). Data were analysed with respect to HF related in-hospital days, self-care behaviour and system adherence, during a 6-month intervention. The analysis of in-hospital days for the pooled data was adjusted for baseline differences between the two study cohorts, relating to disease state.

Results

In the RCT (n=72) the intervention group (IG) consisted of 32 patients and the control group (CG) of 40 patients. The risk ratio (RR) for in-hospital days was RR: 0.72, 95% CI 0.61 to 0.84, p<0.05 in favour of the IG. In the CCT (n=100) both the IG and the CG consisted of 50 patients and the IG had fewer in-hospitals days, comparable to the RCT findings with RR: 0.67; 95% CI 0.45 to 0.99; p<0.05. For the pooled data set made up of 172 patients, the groups were well balanced but with a higher prevalence of hypertension in the CG. The RR relating to in-hospital days for the pooled data set was 0.71; 95% CI 0.61 to 0.82; p<0.05 in favour of the IG. There was a statistically significant improvement in self-care by 27% and the median system adherence was 94%.

Conclusions

These analyses suggest that the evaluated tool might reduce HF related in-hospital days in the general HF population, which adds to the external validity of previous findings.

Clinical Trial Registration

NCT03655496.

Which design to evaluate complex interventions? Toward a methodological framework through a systematic review

BACKGROUND

Evaluation of complex interventions (CI) is challenging for health researchers and requires innovative approaches. The objective of this work is to present the main methods used to evaluate CI.

METHODS

A systematic review of the scientific literature was conducted to identify methods used for the evaluation of CI. We searched MEDLINE via PubMed databases for articles including an evaluation or a pilot study of a complex intervention, published in a ten-year period. Key-words of this research were ("complex intervention*" AND "evaluation").

RESULTS

Among 445 identified articles, 100 research results or protocols were included. Among them, 5 presented 2 different types of design in the same publication, thus our work included 105 designs. Individual randomized controlled trials (IRCT) represented 21.9% (n = 23) of evaluation designs, randomized clinical trials adaptations 44.8% (n = 47), quasi -experimental designs and cohort study 19.0% (n = 20), realist evaluation 6.7% (n = 7) and other cases studies and other approaches 8.6% (n = 9). A process/mechanisms analysis was included in 80% (n = 84) of these designs.

CONCLUSION

A range of methods can be used successively or combined at various steps of the evaluation approach. A framework is proposed to situate each of the designs with respect to evaluation questions. The growing interest of researchers in alternative methods and the development of their use must be accompanied by conceptual and methodological research in order to more clearly define their principles of use.

Challenges and Opportunities for Using Big Health Care Data to Advance Medical Science and Public Health

Methodological advancements in epidemiology, biostatistics, and data science have strengthened the research world's ability to use data captured from electronic health records (EHRs) to address pressing medical questions, but gaps remain. We describe methods investments that are needed to curate EHR data toward research quality and to integrate complementary data sources when EHR data alone are insufficient for research goals. We highlight new methods and directions for improving the integrity of medical evidence generated from pragmatic trials, observational studies, and predictive modeling. We also discuss needed methods contributions to further ease data sharing across multisite EHR data networks. Throughout, we identify opportunities for training and for bolstering collaboration among subject matter experts, methodologists, practicing clinicians, and health system leaders to help ensure that methods problems are identified and resulting advances are translated into mainstream research practice more quickly.

Improving pragmatic clinical trial design using real-world data

BACKGROUND

Pragmatic clinical trials often use automated data sources such as electronic health records, claims, or registries to identify eligible individuals and collect outcome information. A specific advantage that this automated data collection often yields is having data on potential participants when design decisions are being made. We outline how this data can be used to inform trial design.

METHODS

Our work is motivated by a pragmatic clinical trial evaluating the impact of suicide-prevention outreach interventions on fatal and non-fatal suicide attempts in the 18 months after randomization. We illustrate our recommended approaches for designing pragmatic clinical trials using historical data from the health systems participating in this study. Specifically, we illustrate how electronic health record data can be used to inform the selection of trial eligibility requirements, to estimate the distribution of participant characteristics over the course of the trial, and to conduct power and sample size calculations.

RESULTS

Data from 122,873 people with patient health questionnaire (PHQ) responses, recorded in their electronic health records between 1 July 2010 and 31 March 2012, were used to show that the suicide attempt rate in the 18 months following completion of the questionnaire varies by response to item nine of the PHQ. We estimated that the proportion of individuals with a prior recorded elevated PHQ (i.e. history of suicidal ideation) would decrease from approximately 50% at the beginning of a trial to about 5%, 50 weeks later. Using electronic health record data, we conducted simulations to estimate the power to detect a 25% reduction in suicide attempts. Simulation-based power calculations estimated that randomizing 8000 participants per randomization arm would allow 90% power to detect a 25% reduction in the suicide attempt rate in the intervention arm compared to usual care at an alpha rate of 0.05.

CONCLUSIONS

Historical data can be used to inform the design of pragmatic clinical trials, a strength of trials that use automated data collection for randomizing participants and assessing outcomes. In particular, realistic sample size calculations can be conducted using real-world data from the health systems in which the trial will be conducted. Data-informed trial design should yield more realistic estimates of statistical power and maximize efficiency of trial recruitment.

Role of local treatment including radiotherapy in Barcelona Clinic of Liver Cancer stage C patients: a nationwide cohort analysis in South Korea

Purpose

Sorafenib is recommended as the standard treatment for hepatocellular carcinoma (HCC) of Barcelona Clinic of Liver Cancer stage C (BCLC C). However, local treatment including radiation therapy (LRT) is also widely administered in practice. The aim of our study was to define the role of LRT among BCLC C patients.

Patients and methods

Of the patients with HCC enrolled the Korean Central Cancer Registry from 2008 to 2014, the Korean Liver Cancer Study Group randomly extracted 10,580 patient data from ~50 hospitals nationwide. Among them, 3,401 patients were categorized to have BCLC C HCC. Among them, patients with information on initial treatments, defined as the first and secondary treatment within 60 days after the first treatment, were selected and classified into three initial treatment groups: LRT, sorafenib, and no treatment.

Results

Among 3,401 BCLC C HCC patients, 1,486 were included in the study and the remaining patients were excluded as they did not meet the criteria (eg, underwent local treatments without radiotherapy [RT] or received chemotherapy other than sorafenib). Of these, 266 were assigned to LRT (17.9%), 316 to sorafenib (21.3%), and 904 to no treatment group (60.8%). Median survival time of the sorafenib group was shorter than that of the LRT group (3.8 vs 7.6 months, P<0.001). In multivariable analysis, sorafenib group showed significantly higher risk related to mortality compared to LRT group, not only among all patients (HR: 1.50, 95% CI: 1.23-1.84) but also between subgroup cohorts with portal invasions (1.55, 1.23-1.84), with lymph node metastases (2.42, 1.53-3.83), without distant metastases (1.43, 1.10-1.87), and with distant metastases (1.57, 1.13-2.19). Additionally, no treatment group showed the worst survival among the three treatment groups not only in all patients, but also in all subgroups of patients (P<0.001 in all).

Conclusion

LRT as an initial treatment showed survival benefit as compared to sorafenib in HCC patients of BCLC C.

Mindfulness Training Enhances Self-Regulation and Facilitates Health Behavior Change for Primary Care Patients: a Randomized Controlled Trial

BACKGROUND

Self-management of health is important for improving health outcomes among primary care patients with chronic disease. Anxiety and depressive disorders are common and interfere with self-regulation, which is required for disease self-management. An insurance-reimbursable mindfulness intervention integrated within primary care may be effective for enhancing chronic disease self-management behaviors among primary care patients with anxiety, depression, trauma, and stress-related and adjustment disorders compared with the increasingly standard practice of referring patients to outside mindfulness resources.

OBJECTIVE

Mindfulness Training for Primary Care (MTPC) is an 8-week, referral-based, insurance-reimbursable program integrated into safety-net health system patient-centered medical homes. We hypothesized that MTPC would be more effective for catalyzing chronic disease self-management action plan initiation within 2 weeks, versus a low-dose comparator (LDC) consisting of a 60-min mindfulness introduction, referral to community and digital resources, and addition to a 6-month waitlist for MTPC.

PARTICIPANTS

Primary care providers (PCPs) and mental health clinicians referred 465 patients over 12 months. All participants had a DSM-V diagnosis.

DESIGN AND INTERVENTIONS

Participants (N = 136) were randomized in a 2:1 allocation to MTPC (n = 92) or LDC (n = 44) in a randomized controlled comparative effectiveness trial. MTPC incorporates mindfulness, self-compassion, and mindfulness-oriented behavior change skills and is delivered as insurance-reimbursable visits within primary care. Participants took part in a chronic disease self-management action planning protocol at week 7.

MAIN MEASURES

Level of self-reported action plan initiation on the action plan initiation survey by week 9.

KEY RESULTS

Participants randomized to MTPC, relative to LDC, had significantly higher adjusted odds of self-management action plan initiation in an intention-to-treat analysis (OR = 2.28; 95% CI = 1.02 to 5.06, p = 0.025).

CONCLUSIONS

An 8-week dose of mindfulness training is more effective than a low-dose mindfulness comparator in facilitating chronic disease self-management behavior change among primary care patients.

Stakeholder views regarding ethical issues in the design and conduct of pragmatic trials: study protocol

Background

Randomized controlled trial (RCT) trial designs exist on an explanatory-pragmatic spectrum, depending on the degree to which a study aims to address a question of efficacy or effectiveness. As conceptualized by Schwartz and Lellouch in 1967, an explanatory approach to trial design emphasizes hypothesis testing about the mechanisms of action of treatments under ideal conditions (efficacy), whereas a pragmatic approach emphasizes testing effectiveness of two or more available treatments in real-world conditions. Interest in, and the number of, pragmatic trials has grown substantially in recent years, with increased recognition by funders and stakeholders worldwide of the need for credible evidence to inform clinical decision-making. This increase has been accompanied by the onset of learning healthcare systems, as well as an increasing focus on patient-oriented research. However, pragmatic trials have ethical challenges that have not yet been identified or adequately characterized. The present study aims to explore the views of key stakeholders with respect to ethical issues raised by the design and conduct of pragmatic trials. It is embedded within a large, four-year project that seeks to develop guidance for the ethical design and conduct of pragmatic trials. As a first step, this study will address important gaps in the current empirical literature with respect to identifying a comprehensive range of ethical issues arising from the design and conduct of pragmatic trials. By opening up a broad range of topics for consideration within our parallel ethical analysis, we will extend the current debate, which has largely emphasized issues of consent, to the range of ethical considerations that may flow from specific design choices.

Methods

Semi-structured interviews with key stakeholders (e.g. trialists, methodologists, lay members of study teams, bioethicists, and research ethics committee members), across multiple jurisdictions, identified based on their known experience and/or expertise with pragmatic trials.

Discussion

We expect that the study outputs will be of interest to a wide range of knowledge users including trialists, ethicists, research ethics committees, journal editors, regulators, healthcare policymakers, research funders and patient groups. All publications will adhere to the Tri-Agency Open Access Policy on Publications.

Electronic supplementary material

The online version of this article (10.1186/s12910-018-0332-z) contains supplementary material, which is available to authorized users.