Rationale and design of the THIRST Alert feasibility study: a pragmatic, single-centre, parallel-group randomised controlled trial of an interruptive alert for oral fluid restriction in patients treated with intravenous furosemide

INTRODUCTION

Acute heart failure (HF) is a major cause of unplanned hospitalisation characterised by excess body water. A restriction in oral fluid intake is commonly imposed on patients as an adjunct to pharmacological therapy with loop diuretics, but there is a lack of evidence from traditional randomised controlled trials (RCTs) to support the safety and effectiveness of this intervention in the acute setting.This study aims to explore the feasibility of using computer alerts within the electronic health record (EHR) system to invite clinical care teams to enrol patients into a pragmatic RCT at the time of clinical decision-making. It will additionally assess the effectiveness of using an alert to help address the clinical research question of whether oral fluid restriction is a safe and effective adjunct to pharmacological therapy for patients admitted with fluid overload.

METHODS AND ANALYSIS

THIRST (Randomised Controlled Trial within the electronic Health record of an Interruptive alert displaying a fluid Restriction Suggestion in patients with the treatable Trait of congestion) Alert is a single-centre, parallel-group, open-label pragmatic RCT embedded in the EHR system that will be conducted as a feasibility study at an National Health Service (NHS) hospital in London. The clinical care team will be invited to enrol suitable patients in the study using a point-of-care alert with a target sample size of 50 patients. Enrolled patients will then be randomised to either restricted or unrestricted oral fluid intake. Two primary outcomes will be explored (1) the proportion of eligible patients enrolled in the study and (2) the mean difference in oral fluid intake between randomised groups. A series of secondary outcomes are specified to evaluate the effectiveness of the alert, adherence to the randomised treatment allocation and the quality of data generated from routine care, relevant to the outcomes of interest.

ETHICS AND DISSEMINATION

This study was approved by Riverside Research Ethics Committee (Ref: 22/LO/0889) and will be published on completion.

TRIAL REGISTRATION NUMBER

NCT05869656.

Patient-reported outcomes and target effect sizes in pragmatic randomized trials in ClinicalTrials.gov: A cross-sectional analysis

BACKGROUND

Use of patient-reported outcomes (PROs) and patient and public engagement are critical ingredients of pragmatic trials, which are intended to be patient centered. Engagement of patients and members of the public in selecting the primary trial outcome and determining the target difference can better ensure that the trial is designed to inform the decisions of those who ultimately stand to benefit. However, to the best of our knowledge, the use and reporting of PROs and patient and public engagement in pragmatic trials have not been described. The objectives of this study were to review a sample of pragmatic trials to describe (1) the prevalence of reporting patient and public engagement; (2) the prevalence and types of PROs used; (3) how its use varies across trial characteristics; and (4) how sample sizes and target differences are determined for trials with primary PROs.

METHODS AND FINDINGS

This was a methodological review of primary reports of pragmatic trials. We used a published electronic search filter in MEDLINE to identify pragmatic trials, published in English between January 1, 2014 and April 3, 2019; we identified the subset that were registered in ClinicalTrials.gov and explicitly labeled as pragmatic. Trial descriptors were downloaded from ClinicalTrials.gov; information about PROs and sample size calculations were extracted from the manuscript. Chi-squared, Cochran-Armitage, and Wilcoxon rank sum tests were used to examine associations between trial characteristics and use of PROs. Among 4,337 identified primary trial reports, 1,988 were registered in CT.gov, of which 415 were explicitly labeled as pragmatic. Use of patient and public engagement was identified in 39 (9.4%). PROs were measured in 235 (56.6%): 144 (34.7%) used PROs as primary outcomes and 91 (21.9%) as only secondary outcomes. Primary PROs were symptoms (64; 44%), health behaviors (36; 25.0%), quality of life (17; 11.8%), functional status (16; 11.1%), and patient experience (10; 6.9%). Trial characteristics with lower prevalence of use of PROs included being conducted exclusively in children or adults over age 65 years, cluster randomization, recruitment in low- and middle-income countries, and primary purpose of prevention; trials conducted in Europe had the highest prevalence of PROs. For the 144 trials with a primary PRO, 117 (81.3%) reported a sample size calculation for that outcome; of these, 71 (60.7%) justified the choice of target difference, most commonly, using estimates from pilot studies (31; 26.5%), standardized effect sizes (20; 17.1%), or evidence reviews (16; 13.7%); patient or stakeholder opinions were used to justify the target difference in 8 (6.8%). Limitations of this study are the need for trials to be registered in ClinicalTrials.gov, which may have reduced generalizability, and extracting information only from the primary trial report.

CONCLUSIONS

In this study, we observed that pragmatic trials rarely report patient and public engagement and do not commonly use PROs as primary outcomes. When provided, target differences are often not justified and rarely informed by patients and stakeholders. Research funders, scientific journals, and institutions should support trialists to incorporate patient engagement to fulfill the mandate of pragmatic trials to be patient centered.

Feasibility of Using Real-world Data to Emulate Postapproval Confirmatory Clinical Trials of Therapeutic Agents Granted US Food and Drug Administration Accelerated Approval

This cross-sectional study examines the feasibility of using real-world data, such as billing, claims, and electronic health records, to emulate US Food and Drug Administration–required confirmatory clinical trials for the 50 new therapeutic agents that received accelerated approval between 2009 and 2018.

Real-world effectiveness of BNT162b2 mRNA vaccine: a meta-analysis of large observational studies

This paper aims to summarize through meta-analyses the overall vaccine effectiveness of the BNT162b2 mRNA vaccine from observational studies. A systematic literature search with no language restriction was performed in electronic databases to identify eligible observational studies which reported the adjusted effectiveness of the BNT162b2 mRNA vaccine to prevent RT-PCR confirmed COVID-19. Meta-analyses with the random-effects model were used to calculate the pooled hazard ratio (HR) and pooled incidence rate ratio (IRR) at 95% confidence intervals, and the vaccine effectiveness was indicated as (pooled HR - 1)/HR or (pooled IRR - 1)/IRR. Nineteen studies were included for this meta-analysis. The meta-analysis revealed significant protective effect against RT-PCR confirmed COVID-19 ≥ 14 days after the first dose, with vaccine effectiveness of 53% (95% confidence interval 32-68%), and ≥ 7 days after the second dose, with vaccine effectiveness of 95% (95% confidence interval: 96-97%). Despite its effectiveness, reporting vaccine safety data by relevant stakeholders should be encouraged as BNT162b2 mRNA is a new vaccine that has not gained full approval. There have been limited data about vaccine effectiveness among immunocompromised patients; thus, the vaccine should be used cautiously in this patient population.

Emulating Randomized Clinical Trials With Nonrandomized Real-World Evidence Studies: First Results From the RCT DUPLICATE Initiative

BACKGROUND

Regulators are evaluating the use of noninterventional real-world evidence (RWE) studies to assess the effectiveness of medical products. The RCT DUPLICATE initiative (Randomized, Controlled Trials Duplicated Using Prospective Longitudinal Insurance Claims: Applying Techniques of Epidemiology) uses a structured process to design RWE studies emulating randomized, controlled trials (RCTs) and compare results. We report findings of the first 10 trial emulations, evaluating cardiovascular outcomes of antidiabetic or antiplatelet medications.

METHODS

We selected 3 active-controlled and 7 placebo-controlled RCTs for replication. Using patient-level claims data from US commercial and Medicare payers, we implemented inclusion and exclusion criteria, selected primary end points, and comparator populations to emulate those of each corresponding RCT. Within the trial-mimicking populations, we conducted propensity score matching to control for >120 preexposure confounders. All study measures were prospectively defined and protocols registered before hazard ratios and 95% CIs were computed. Success criteria for the primary analysis were prespecified for each replication.

RESULTS

Despite attempts to emulate RCT design as closely as possible, differences between the RCT and corresponding RWE study populations remained. The regulatory conclusions were equivalent in 6 of 10. The RWE emulations achieved a hazard ratio estimate that was within the 95% CI from the corresponding RCT in 8 of 10 studies. In 9 of 10, either the regulatory or estimate agreement success criteria were fulfilled. The largest differences in effect estimates were found for RCTs where second-generation sulfonylureas were used as a proxy for placebo regarding cardiovascular effects. Nine of 10 replications had a standardized difference between effect estimates of <2, which suggests differences within expected random variation.

CONCLUSIONS

Agreement between RCT and RWE findings varies depending on which agreement metric is used. Interim findings indicate that selection of active comparator therapies with similar indications and use patterns enhances the validity of RWE. Even in the context of active comparators, concordance between RCT and RWE findings is not guaranteed, partially because trials are not emulated exactly. More trial emulations are needed to understand how often and in what contexts RWE findings match RCTs. Registration: URL: https://www.clinicaltrials.gov; Unique identifiers: NCT03936049, NCT04215523, NCT04215536, NCT03936010, NCT03936036, NCT03936062, NCT03936023, NCT03648424, NCT04237935, NCT04237922.

Burden of diabetes mellitus and its impact on COVID-19 patients: A meta-analysis of real-world evidence

BACKGROUND & AIMS

Coronavirus disease 2019 (COVID-19) spreads rapidly and within no time, it has been declared a pandemic by the World Health Organization. Evidence suggests diabetes to be a risk factor for the progression and poor prognosis of COVID-19. Therefore, we aimed to understand the pooled prevalence of diabetes in patients infected with COVID-19. We also aimed to compute the risk of mortality and ICU admissions in COVID-19 patients with and without diabetes.

METHODS

A comprehensive literature search was performed in PubMed to identify the articles reporting the diabetes prevalence and risk of mortality or ICU admission in COVID-19 patients. The primary outcome was to compute the pooled prevalence of diabetes in COVID-19 patients. Secondary outcomes included risk of mortality and ICU admissions in COVID-19 patients with diabetes compared to patients without diabetes.

RESULTS

This meta-analysis was based on a total of 23007 patients from 43 studies. The pooled prevalence of diabetes in patients infected with COVID-19 was found to be 15% (95% CI: 12%-18%), p = <0.0001. Mortality risk was found to be significantly higher in COVID-19 patients with diabetes as compared to COVID-19 patients without diabetes with a pooled risk ratio of 1.61 (95% CI: 1.16-2.25%), p = 0.005. Likewise, risk of ICU admission rate was significantly higher in COVID-19 patients with diabetes as compared to COVID-19 patients without diabetes with a pooled risk ratio of 1.88 (1.20%-2.93%), p = 0.006.

CONCLUSION

This meta-analysis found a high prevalence of diabetes and higher mortality and ICU admission risk in COVID-19 patients with diabetes.

Use of Real-World Evidence to Support FDA Approval of Oncology Drugs

OBJECTIVES

Real-world evidence (RWE) has gained increased attention in recent years as a complement to traditional clinical trials. The use of RWE to establish the efficacy of oncology drugs for Food and Drug Administration (FDA) approval has not been described. In this paper, we review 5 recent examples where RWE was submitted in support of the FDA approvals of original or supplementary indications for oncology drugs.

METHODS

To identify cases where RWE was used, we reviewed drug approval packages available at Drugs@FDA for oncology drugs approved between 2017 and 2019. Five cases were selected to present a broad overview of different types of RWE, different circumstances under which RWE has been used for regulatory approvals, and how FDA evaluated the data in each case. The type of RWE submitted, the indication, limitations identified by FDA reviewers, and the outcome of the submission are discussed.

RESULTS

RWE, particularly historical controls for rare or orphan indications, has been used to support both original and supplementary oncology drug approvals. Types of RWE included data from electronic health records, claims, post-marketing safety reports, retrospective medical record reviews, and expanded access studies. Small sample sizes, data quality, and methodological issues were among concerns cited by FDA reviewers.

CONCLUSION

By bridging the gap between the constraints of the trial setting and the realities of clinical practice, RWE can add value to a regulatory submission. These early examples provide insight into how regulators evaluated RWE submitted as evidence of efficacy for oncology drugs.

PCORnet® 2020: current state, accomplishments, and future directions

OBJECTIVE

To describe PCORnet, a clinical research network developed for patient-centered outcomes research on a national scale.

STUDY DESIGN AND SETTING

Descriptive study of the current state and future directions for PCORnet. We conducted cross-sectional analyses of the health systems and patient populations of the 9 Clinical Research Networks and 2 Health Plan Research Networks that are part of PCORnet.

RESULTS

Within the Clinical Research Networks, electronic health data are currently collected from 337 hospitals, 169,695 physicians, 3,564 primary care practices, 338 emergency departments, and 1,024 community clinics. Patients can be recruited for prospective studies from any of these clinical sites. The Clinical Research Networks have accumulated data from 80 million patients with at least one visit from 2009 to 2018. The PCORnet Health Plan Research Network population of individuals with a valid enrollment segment from 2009 to 2019 exceeds 60 million individuals, who on average have 2.63 years of follow-up.

CONCLUSION

PCORnet's infrastructure comprises clinical data from a diverse cohort of patients and has the capacity to rapidly access these patient populations for pragmatic clinical trials, epidemiological research, and patient-centered research on rare diseases.

Cost-effectiveness of direct oral anticoagulants versus vitamin K antagonist in atrial fibrillation: A study protocol using Real-World Data from Catalonia (FantasTIC Study)

BACKGROUND

Anticoagulant therapy is used for stroke prevention and proved to be effective and safe in the long term. The study aims to analyse the cost-effectiveness relationship of using of direct-acting oral anticoagulants vs vitamin K antagonists to prevent ischaemic stroke in patients with nonvalvular atrial fibrillation, including all the active ingredients marketed in Spain, prescribed for 2 years in the Primary Care service of the Institut Català de la Salut.

METHODS

Population-based cohort study, in which the cost of the 2 treatment groups will be evaluated. Direct costs (pharmacy, primary care, emergency and hospitalization) and indirect costs (lost productivity) will be included from a social perspective. Effectiveness (assessed as the occurrence of a health event, the 1 of primary interest being stroke) will be determined, with a 2-year time horizon and a 3% discount rate. The average cost of the 2 groups of drugs will be compared using a regression model to determine the factors with the greatest influence on determining costs. We will carry out a univariate ('one-way') deterministic sensitivity analysis.

DISCUSSION

We hope to provide relevant information about direct and indirect costs of oral anticoagulants, which, together with aspects of effectiveness and safety, could help shape the consensual decision-making of evaluating bodies.

Real-world treatment patterns and survival in stage IV non-small-cell lung cancer in Canada

Background

Almost half of all patients with non-small-cell lung cancer (nsclc) present with stage iv disease. The objective of the present study was to characterize treatment patterns and survival outcomes in patients with advanced nsclc.

Methods

We conducted a longitudinal population-level study in patients diagnosed with stage iv nsclc in Ontario between 1 April 2010 and 31 March 2015, with follow-up to 31 March 2017 for overall survival and treatment sequence. Patients were stratified as nonsquamous or squamous histology. A sub-analysis was conducted for patients with nonsquamous histology who received targeted therapies, on the assumption that their tumours were EGFR mutation-positive (EGFRm+). Treatment patterns were determined, and survival was calculated from date of diagnosis to death or censoring.

Results

Of 24,729 nsclc cases identified, stage iv disease was diagnosed in 49.2%, histology was nonsquamous in 10,103, and EGFRm+ was assumed in 508. Median patient age ranged from 69 to 72 years for the three cohorts. For patients with nonsquamous histology, palliative radiotherapy was the most frequently used first-line treatment (44.4%), followed by no treatment (26.7%) and chemotherapy (14.9%). In the EGFRm+ cohort, 75.6% received gefitinib as first- or second-line therapy, and almost half (47.4%) the 473 patients with squamous histology treated with first-line chemotherapy received cisplatin or carboplatin with gemcitabine. Median overall survival in the nonsquamous and squamous cohorts was 4.9 and 4.6 months respectively; it was 17.6 months for patients who were EGFRm+.

Conclusions

Survival of patients with stage iv nsclc remains poor, with the exception of patients who are EGFRm+. Only 14.9% of patients received first-line chemotherapy; the mainstay of treatment was palliative radiotherapy.

Real-World Clinical and Economic Outcomes Associated with Palbociclib for HR-Positive/HER2 Negative Metastatic Breast Cancer: A Commentary

Despite the achieved advancement in pharmacological cancer treatments, the majority of postmenopausal women with hormone receptor-positive metastatic breast cancer (mBC) will experience disease progression. Research into alternative therapies with improved efficacy and reduced side effects has led to the development of a new class of oral anticancer medications, the cyclin-dependent kinase (CDK) 4/6 inhibitors, which include palbociclib, ribociclib, and abemaciclib. Nonetheless, there is growing evidence that the effectiveness of oral anticancer medications is sub-optimal, being influenced by low adherence, sociodemographic factors, and adverse effect profiles. In addition, there is a disconnect between the high price tags of CDK 4/6 inhibitors and their observed effectiveness, raising questions about their value. Currently, the existing knowledge base on the effectiveness and cost-effectiveness of newer oral anticancer medications in understudied populations with possible health disparities is scant. This commentary discusses what is known about palbociclib's clinical effectiveness, safety, and adherence and suggests the need for further studies of real-world effectiveness and cost-effectiveness to help establish the value of newer oncologic drugs, such as palbociclib. DISCLOSURES: No funding supported the writing of this article. The authors have nothing to disclose.

Consensus statement of the Italian society of pediatric allergy and immunology for the pragmatic management of children and adolescents with allergic or immunological diseases during the COVID-19 pandemic

The COVID-19 pandemic has surprised the entire population. The world has had to face an unprecedented pandemic. Only, Spanish flu had similar disastrous consequences. As a result, drastic measures (lockdown) have been adopted worldwide. Healthcare service has been overwhelmed by the extraordinary influx of patients, often requiring high intensity of care. Mortality has been associated with severe comorbidities, including chronic diseases. Patients with frailty were, therefore, the victim of the SARS-COV-2 infection. Allergy and asthma are the most prevalent chronic disorders in children and adolescents, so they need careful attention and, if necessary, an adaptation of their regular treatment plans. Fortunately, at present, young people are less suffering from COVID-19, both as incidence and severity. However, any age, including infancy, could be affected by the pandemic.Based on this background, the Italian Society of Pediatric Allergy and Immunology has felt it necessary to provide a Consensus Statement. This expert panel consensus document offers a rationale to help guide decision-making in the management of children and adolescents with allergic or immunologic diseases.

Are Novel, Nonrandomized Analytic Methods Fit for Decision Making? The Need for Prospective, Controlled, and Transparent Validation

Real-world data and patient-level data from completed randomized controlled trials are becoming available for secondary analysis on an unprecedented scale. A range of novel methodologies and study designs have been proposed for their analysis or combination. However, to make novel analytical methods acceptable for regulators and other decision makers will require their testing and validation in broadly the same way one would evaluate a new drug: prospectively, well-controlled, and according to a pre-agreed plan. From a European regulators' perspective, the established methods qualification advice procedure with active participation of patient groups and other decision makers is an efficient and transparent platform for the development and validation of novel study designs.

What does digitalization hold for the creation of real-world evidence?

Health-related information is increasingly being collected and stored digitally. These data, either structured or unstructured, are becoming the ubiquitous assets that might enable us to comprehensively map out a patient's health journey from an asymptomatic state of wellness to disease onset and its trajectory. These new data could provide rich real-world evidence for better clinical care and research, if they can be accessed, linked and analyzed-all of which are possible. In this review, these opportunities will be explored through a case vignette of a patient with OA, followed by discussion on how this digitalized real-world evidence could best be utilized, as well as the challenges of data access, quality and maintaining public trust.

Pragmatic trial evaluating the effectiveness of a patient navigator to decrease emergency room utilisation in transition age youth with chronic conditions: the Transition Navigator Trial protocol

INTRODUCTION

Transition to adult care is a challenging and complex process for youth with special healthcare needs. We aim to compare effectiveness of a patient navigator service in reducing emergency room (ER) use among adolescents with chronic health conditions transitioning to adult care.

METHODS AND ANALYSIS

Pragmatic randomised controlled trial parallel group design comparing ER visit rates between patients with access to a personalised navigator intervention compared with usual care. Unit of randomisation is the patient. Treatment assignment will not be blinded. Embedded qualitative study to understand navigator's role and cost analysis attributable to the intervention will be performed. Patients aged 16-21 years, followed within a chronic disease clinic, expected to be transferred to adult care within 12 months and residing in Alberta during study period will be recruited from three tertiary care paediatric hospitals. Sample size will be 300 in each arm. Navigator intervention over 24 months is designed to assist participants in four domains: transition preparation, health system brokering, socioeconomic determinants of health and self-management. Primary outcome is ER visit rate during observation period. Secondary outcomes are ambulatory and inpatient care utilisation measures, as well as Transition Readiness Assessment Questionnaire score, and Short-Form Health Survey 12 (SF-12) score at 6 and 18 months post-randomisation. Poisson regression will compare rates of ER/urgent care visits between navigator and control participants, using intention to treat principle. Cost analysis of the intervention will be conducted. Thematic analysis will be used to identify perceptions of stakeholders regarding the role of navigators.

ETHICS AND DISSEMINATION

Ethics approval was obtained from the University of Calgary Conjoint Health Research Ethics Board (REB #162561) and the University of Alberta Health Research Ethics Board (Pro00077325). Our team is composed of diverse stakeholders who are committed to improving transition of care who will assist with dissemination of results.

TRIAL REGISTRATION NUMBER

NCT03342495.

Protocolized Post-Extubation Respiratory Support to prevent reintubation: protocol and statistical analysis plan for a clinical trial

INTRODUCTION

Following extubation from invasive mechanical ventilation, nearly one in seven critically ill adults requires reintubation. Reintubation is independently associated with increased mortality. Postextubation respiratory support (non-invasive ventilation or high-flow nasal cannula applied at the time of extubation) has been reported in small-to-moderate-sized trials to reduce reintubation rates among hypercapnic patients, high-risk patients without hypercapnia and low-risk patients without hypercapnia. It is unknown whether protocolised provision of postextubation respiratory support to every patient undergoing extubation would reduce the overall reintubation rate, compared with usual care.

METHODS AND ANALYSIS

The Protocolized Post-Extubation Respiratory Support (PROPER) trial is a pragmatic, cluster cross-over trial being conducted between 1 October 2017 and 31 March 2019 in the medical intensive care unit of Vanderbilt University Medical Center. PROPER compares usual care versus protocolized post-extubation respiratory support (a respiratory therapist-driven protocol that advises the provision of non-invasive ventilation or high-flow nasal cannula based on patient characteristics). For the duration of the trial, the unit is divided into two clusters. One cluster receives protocolised support and the other receives usual care. Each cluster crosses over between treatment group assignments every 3 months. All adults undergoing extubation from invasive mechanical ventilation are enrolled except those who received less than 12 hours of mechanical ventilation, have 'Do Not Intubate' orders, or have been previously reintubated during the hospitalisation. The anticipated enrolment is approximately 630 patients. The primary outcome is reintubation within 96 hours of extubation.

ETHICS AND DISSEMINATION

The trial was approved by the Vanderbilt Institutional Review Board. The results will be submitted for publication in a peer-reviewed journal and presented at one or more scientific conferences.

TRIAL REGISTRATION NUMBER

NCT03288311.

Research-grade data in the real world: challenges and opportunities in data quality from a pragmatic trial in community-based practices

Randomized controlled trials face cost, logistic, and generalizability limitations, including difficulty engaging racial/ethnic minorities. Real-world data (RWD) from pragmatic trials, including electronic health record (EHR) data, may produce intervention evaluation findings generalizable to diverse populations. This case study of Project IMPACT describes unique barriers and facilitators of optimizing RWD to improve health outcomes and advance health equity in small immigrant-serving community-based practices. Project IMPACT tested the effect of an EHR-based health information technology intervention on hypertension control among small urban practices serving South Asian patients. Challenges in acquiring accurate RWD included EHR field availability and registry capabilities, cross-sector communication, and financial, personnel, and space resources. Although using RWD from community-based practices can inform health equity initiatives, it requires multidisciplinary collaborations, clinic support, procedures for data input (including social determinants), and standardized field logic/rules across EHR platforms.

Determining the Comparative Value of Pharmaceutical Risk-Sharing Policies in Non-Small Cell Lung Cancer Using Real-World Data

BACKGROUND

Risk-sharing arrangements (RSAs) can be used to mitigate uncertainty about the value of a drug by sharing the financial risk between payer and pharmaceutical company. We evaluated the projected impact of alternative RSAs for non-small cell lung cancer (NSCLC) therapies based on real-world data.

METHODS

Data on treatment patterns of Dutch NSCLC patients from four different hospitals were used to perform "what-if" analyses, evaluating the costs and benefits likely associated with various RSAs. In the scenarios, drug costs or refunds were based on response evaluation criteria in solid tumors (RECIST) response, survival compared to the pivotal trial, treatment duration, or a fixed cost per patient. Analyses were done for erlotinib, gemcitabine/cisplatin, and pemetrexed/platinum for metastatic NSCLC, and gemcitabine/cisplatin, pemetrexed/cisplatin, and vinorelbine/cisplatin for nonmetastatic NSCLC.

RESULTS

Money-back guarantees led to moderate cost reductions to the payer. For conditional treatment continuation schemes, costs and outcomes associated with the different treatments were dispersed. When price was linked to the outcome, the payer's drug costs reduced by 2.5% to 26.7%. Discounted treatment initiation schemes yielded large cost reductions. Utilization caps mainly reduced the costs of erlotinib treatment (by 16%). Given a fixed cost per patient based on projected average use of the drug, risk sharing was unfavorable to the payer because of the lower than projected use. The impact of RSAs on a national scale was dispersed.

CONCLUSIONS

For erlotinib and pemetrexed/platinum, large cost reductions were observed with risk sharing. RSAs can mitigate uncertainty around the incremental cost-effectiveness or budget impact of drugs, but only when the type of arrangement matches the setting and type of uncertainty.

OneFlorida Clinical Research Consortium: Linking a Clinical and Translational Science Institute With a Community-Based Distributive Medical Education Model

PROBLEM

Developing a national pragmatic clinical trial infrastructure is central to understanding the effectiveness of interventions applied under usual conditions and where people receive health care. To address this challenge, three Florida universities-the University of Florida Clinical and Translational Science Institute, Florida State University (with its community-based distributive medical education model), and the University of Miami-created (2010-2013) a statewide consortium, the OneFlorida Clinical Research Consortium, to support the conduct of pragmatic clinical trials and provide mentored research experiences for medical and graduate students in real-world practice settings.

APPROACH

OneFlorida has four programs, which report to a steering committee with membership from each partner, community members, and the state Medicaid agency and Department of Health to ensure shared governance. The Clinical Research Program provides support to conduct research in the network and uses champions to engage community clinicians. The Citizen Scientist Program has community members who provide input on health topics of importance to them, study design, recruitment and retention strategies, and the interpretation of findings. The Data Trust Program contains electronic health record and health care claims data for 10.6 million Floridians. The Minority Education Program, in collaboration with three historically black colleges and universities, offers minority junior faculty mentoring in pragmatic clinical trials and implementation science.

OUTCOMES

OneFlorida has implemented 27 studies with diverse patient populations and in diverse community practice settings.

NEXT STEPS

To identify evidence-based best practices from the clinical trials conducted in the network, foster their implementation, and expand research training opportunities.

What's the uptake? Pragmatic RCTs may be used to estimate uptake, and thereby population impact of interventions, but better reporting of trial recruitment processes is needed

BACKGROUND

Effectiveness of interventions in pragmatic trials may not translate directly into population impact, because of limited uptake by clinicians and/or the public. Uptake of an intervention is influenced by a number of factors.

METHODS

We propose a method for calculating population impact of clinical interventions that accounts for the intervention uptake. We suggest that population impact may be estimated by multiplying the two key components: (1) the effectiveness of the intervention in pragmatic trials (trial effect); and, (2) its uptake in clinical practice. We argue that participation rates in trials may be a valid proxy for uptake in clinical practice and, in combination with trial effectiveness estimates, be used to rank interventions by their likely population impact. We illustrate the method using the example of four interventions to decrease antibiotic prescription for acute respiratory infections in primary care: delayed prescribing, procalcitonin test, C-Reactive Protein, shared decision making.

RESULTS

In order to estimate uptake of interventions from trial data we need detailed reporting on the recruitment processes used for clinician participation in the trials. In the antibiotic prescribing example, between 75 and 91% of the population would still be prescribed or consume antibiotics because effective interventions were not taken up. Of the four interventions considered, we found that delayed prescribing would have the highest population impact and shared decision making the lowest.

CONCLUSION

Estimates of uptake and population impact of an intervention may be possible from pragmatic RCTs, provided the recruitment processes for these trials are adequately reported (which currently few of them are). Further validation of this method using empirical data on intervention uptake in the real world would support use of this method to decide on public funding of interventions.

Willingness to participate in pragmatic dialysis trials: the importance of physician decisional autonomy and consent approach

BACKGROUND

Pragmatic clinical trials embedded in routine delivery of clinical care can lead to improvements in quality of care, but often have design features that raise ethical concerns.

METHODS

We performed a discrete choice experiment and used conjoint analysis to assess how specific attributes of pragmatic dialysis trials influenced patients' and physicians' willingness to have their dialysis facility participate in a hypothetical trial of hypertension management. Electronic survey data were collected from 200 patients enrolled from 11 outpatient hemodialysis units and from 203 nephrologists. The three attributes studied were physicians' treatment autonomy, participants' research burden, and the approach to consent. The influence of each attribute was quantified using mixed-effects logistic regression.

RESULTS

Similar proportions of patients were willing to have their facilities participate in a trial with high vs. low physician autonomy (77% vs. 79%; p = 0.13) and research burden (76% vs. 80%; p = 0.06). Opt-in, opt-out, and notification-only consent approaches were acceptable to most patients (84%, 82%, and 81%, respectively), but compared to each of these consent approaches, fewer patients (66%) were willing to have their facility participate in a trial that used no notification (p < 0.001 for each 2-way comparison). Among the physicians, similar proportions were willing to participate in trials with high and low physician autonomy (61% and 61%, respectively, p = 0.96) or with low and high burden (60 and 61%, respectively, p = 0.79). However, as for the patients, the consent approach influenced trial acceptability with 77%, 69%, and 62% willing to participate using opt-in, opt-out, and notification-only, respectively, compared to no notification (36%) (p < 0.001 for each 2-way comparison).

CONCLUSIONS

Curtailing physician's treatment autonomy and increasing the burden associated with participation did not influence patients' or physicians' willingness to participate in the hypothetical research, suggesting that pragmatic dialysis trials are generally acceptable to patients and physicians. Both patients and physicians preferred consent approaches that include at least some level of patient notification, but the majority of patients were still willing to participate in trials that did not notify patients of the research.

Pragmatic clinical trials embedded in healthcare systems: generalizable lessons from the NIH Collaboratory

BACKGROUND

The clinical research enterprise is not producing the evidence decision makers arguably need in a timely and cost effective manner; research currently involves the use of labor-intensive parallel systems that are separate from clinical care. The emergence of pragmatic clinical trials (PCTs) poses a possible solution: these large-scale trials are embedded within routine clinical care and often involve cluster randomization of hospitals, clinics, primary care providers, etc. Interventions can be implemented by health system personnel through usual communication channels and quality improvement infrastructure, and data collected as part of routine clinical care. However, experience with these trials is nascent and best practices regarding design operational, analytic, and reporting methodologies are undeveloped.

METHODS

To strengthen the national capacity to implement cost-effective, large-scale PCTs, the Common Fund of the National Institutes of Health created the Health Care Systems Research Collaboratory (Collaboratory) to support the design, execution, and dissemination of a series of demonstration projects using a pragmatic research design.

RESULTS

In this article, we will describe the Collaboratory, highlight some of the challenges encountered and solutions developed thus far, and discuss remaining barriers and opportunities for large-scale evidence generation using PCTs.

CONCLUSION

A planning phase is critical, and even with careful planning, new challenges arise during execution; comparisons between arms can be complicated by unanticipated changes. Early and ongoing engagement with both health care system leaders and front-line clinicians is critical for success. There is also marked uncertainty when applying existing ethical and regulatory frameworks to PCTS, and using existing electronic health records for data capture adds complexity.

The role of pragmatism in explaining heterogeneity in meta-analyses of randomised trials: a protocol for a cross-sectional methodological review

INTRODUCTION

There has been increasing interest in pragmatic trials methodology. As a result, tools such as the Pragmatic-Explanatory Continuum Indicator Summary-2 (PRECIS-2) are being used prospectively to help researchers design randomised controlled trials (RCTs) within the pragmatic-explanatory continuum. There may be value in applying the PRECIS-2 tool retrospectively in a systematic review setting as it could provide important information about how to pool data based on the degree of pragmatism.

OBJECTIVES

To investigate the role of pragmatism as a source of heterogeneity in systematic reviews by (1) identifying systematic reviews with meta-analyses of RCTs that have moderate to high heterogeneity, (2) applying PRECIS-2 to RCTs of systematic reviews, (3) evaluating the inter-rater reliability of PRECIS-2, (4) determining how much of this heterogeneity may be explained by pragmatism.

METHODS

A cross-sectional methodological review will be conducted on systematic reviews of RCTs published in the Cochrane Library from 1 January 2014 to 1 January 2017. Included systematic reviews will have a minimum of 10 RCTs in the meta-analysis of the primary outcome and moderate to substantial heterogeneity (I²≥50%). Of the eligible systematic reviews, a random selection of 10 will be included for quantitative evaluation. In each systematic review, RCTs will be scored using the PRECIS-2 tool, in duplicate. Agreement between raters will be measured using the intraclass correlation coefficient. Subgroup analyses and meta-regression will be used to evaluate how much variability in the primary outcome may be due to pragmatism.

DISSEMINATION

This review will be among the first to evaluate the PRECIS-2 tool in a systematic review setting. Results from this research will provide inter-rater reliability information about PRECIS-2 and may be used to provide methodological guidance when dealing with pragmatism in systematic reviews and subgroup considerations. On completion, this review will be submitted to a peer-reviewed journal for publication.

Good practices for real-world data studies of treatment and/or comparative effectiveness: Recommendations from the joint ISPOR-ISPE Special Task Force on real-world evidence in health care decision making

PURPOSE

Real-world evidence (RWE) includes data from retrospective or prospective observational studies and observational registries and provides insights beyond those addressed by randomized controlled trials. RWE studies aim to improve health care decision making.

METHODS

The International Society for Pharmacoeconomics and Outcomes Research (ISPOR) and the International Society for Pharmacoepidemiology (ISPE) created a task force to make recommendations regarding good procedural practices that would enhance decision makers' confidence in evidence derived from RWD studies. Peer review by ISPOR/ISPE members and task force participants provided a consensus-building iterative process for the topics and framing of recommendations.

RESULTS

The ISPOR/ISPE Task Force recommendations cover seven topics such as study registration, replicability, and stakeholder involvement in RWE studies. These recommendations, in concert with earlier recommendations about study methodology, provide a trustworthy foundation for the expanded use of RWE in health care decision making.

CONCLUSION

The focus of these recommendations is good procedural practices for studies that test a specific hypothesis in a specific population. We recognize that some of the recommendations in this report may not be widely adopted without appropriate incentives from decision makers, journal editors, and other key stakeholders.

Using routinely recorded data in the UK to assess outcomes in a randomised controlled trial: The Trials of Access

BACKGROUND

In the UK, routinely recorded data may benefit prospective studies including randomised controlled trials (RCTs). In an on-going study, we aim to assess the feasibility of access and agreement of routinely recorded clinical and non-clinical data compared to data collected during a RCT using standard prospective methods. This paper will summarise available UK routinely recorded data sources and discuss our experience with the feasibility of accessing routinely recorded data for participants of a RCT before finally proposing recommendations for improving the access and implementation of routinely recorded data in RCTs.

METHODS

Setting: the case study RCT is the Standard and New Antiepileptic Drugs II (SANAD II) trial, a pragmatic, UK, multicentre, phase IV RCT assessing the clinical and cost-effectiveness of antiepileptic drug treatments for newly diagnosed epilepsy.

PARTICIPANTS

98 participants have provided written consent to permit the request of routinely recorded data. Study procedures: routinely recorded clinical and non-clinical data were identified and data requested through formal applications from available data holders for the duration that participants have been recruited into SANAD II. The feasibility of accessing routinely recorded data during a RCT is assessed and recommendations for improving access proposed.

RESULTS

Secondary-care clinical and socioeconomic data is recorded on a national basis and can be accessed, although there are limitations in the application process. Primary-care data are recorded by a number of organisations on a de-identified basis but access for specific individuals has not been feasible. Access to data recorded by non-clinical sources, including The Department for Work and Pensions and The Driving and Vehicle Licensing Agency, was not successful.

CONCLUSIONS

Recommendations discussed include further research to assess the attributes of routinely recorded data, an assessment of public perceptions and the development of strategies to collaboratively improve access to routinely recorded data for research.

TRIAL REGISTRATION

International Standard Randomised Controlled Trials, ISRCTN30294119 . Registered on 3 July 2012. EudraCT No: 2012-001884-64. Registered on 9 May 2012.

Series: Pragmatic trials and real world evidence: Paper 3. Patient selection challenges and consequences

This paper addresses challenges of identifying, enrolling, and retaining participants in a trial conducted within a routine care setting. All patients who are potential candidates for the treatments in routine clinical practice should be considered eligible for a pragmatic trial. To ensure generalizability, the recruited sample should have a similar distribution of the treatment effect modifiers as the target population. In practice, this can be best achieved by including-within the selected sites-all patients without further selection. If relevant heterogeneity between subgroups is expected, increasing the relative proportion of the subgroup of patients in the heterogeneous trial could be considered (oversampling) or a separate trial in this subgroup can be planned. Selection will nevertheless occur. Low enrollment and loss to follow-up can introduce selection and can jeopardize validity as well as generalizability. Pragmatic trials are conducted in clinical practice rather than in a dedicated research setting, which could reduce recruitment rates. However, if a trial poses a minimal burden to the physician and the patient and routine clinical practice is maximally adhered to, the participation rate may be high and loss to follow-up will not be a specific problem for pragmatic trials.

Pragmatic study designs for older adults with cancer: Report from the U13 conference

Cancer is a disease occurring disproportionately in older adults. However, the evidence base regarding how best to care for these patients remains limited due to their underrepresentation in cancer clinical trials. Pragmatic clinical trials represent a promising approach for enhancing the evidence base in geriatric oncology by allowing investigators to enroll older, frailer patients onto cancer clinical trials. These trials are more accessible, less resource intensive, and place minimal additional burden on participating patients. Additionally, these trials can be designed to measure endpoints directly relevant to older adults, such as quality of life, functional independence and treatment tolerability which are often not addressed in standard clinical trials. Therefore, pragmatic clinical trials allow researchers to include patients for whom the treatment will ultimately be applied and to utilize meaningful endpoints. Examples of pragmatic studies include both large, simple trials and cluster randomized trials. These study designs allow investigators to conduct clinical trials within the context of everyday practice. Further, researchers can devise these studies to place minimal burden on the patient, the treating clinicians and the participating institutions. In order to be successful, pragmatic trials must efficiently utilize the electronic medical record for data capture while also maximizing patient recruitment, enrollment and retention. Additionally, by strategically utilizing pragmatic clinical trials to test therapies and interventions that have previously shown efficacy in younger, fitter patients, these trials represent a potential mechanism to improve the evidence base in geriatric oncology and enhance care for older adults with cancer.

Meeting pragmatism halfway: making a pragmatic clinical trial protocol

Pragmatic clinical trials (PCTs) are today an increasingly prominent means of measuring the 'effectiveness' of healthcare interventions in 'real world' clinical settings, in order to produce evidence on which to base regulatory and clinical decision-making. Although several sociological studies have shown persuasively how PCTs are co-constructed within particular healthcare systems in which they are based, they have tended to focus on relatively later stages in careers of trials. The paper contributes to literature by considering how the 'real world' of the UK National Health Service (NHS) is incorporated into the design of a research protocol. Drawing on a meeting held just prior to patient recruitment for a PCT in maternal health, the paper analyses a trial collective's efforts to purify the messy domain of NHS clinical care into the orderly confines of the protocol (Law 2004), which meant satisfying demands for both scientific and social robustness (c.f. Nowotny et al. 2001). The findings show how efforts to inscribe robustness into the PCT protocol were themselves mediated through epistemic and regulatory conventions surrounding protocols as devices in healthcare research. Finally it is argued that meetings constitute an important epistemic instrument through which to settle various emerging tensions in PCT protocol design.

Are explanatory trials ethical? Shifting the burden of justification in clinical trial design

Most phase III clinical trials today are explanatory. Because explanatory, or efficacy, trials test hypotheses under "ideal" conditions, they are not well suited to providing guidance on decisions made in most clinical care contexts. Pragmatic trials, which test hypotheses under "usual" conditions, are often better suited to this task. Yet, pragmatic, or effectiveness, trials are infrequently carried out. This mismatch between the design of clinical trials and the needs of health care professionals is frustrating for everyone involved, and explains some of the challenges inherent in attempts to enhance knowledge translation and encourage evidence-based practice. The situation is more than simply frustrating, however; it is potentially unethical. Clinical trials must be socially valuable in order to (1) warrant the risks they impose on human research subjects and (2) fairly and efficiently assess new clinical interventions. Most bioethicists would agree that trials that have no social value, for instance, because their results do not have the potential to advance clinical care, should not be performed. What is less widely appreciated is that given limited research resources, trials that are more socially valuable should be preferred to trials that are less socially valuable when all else is equal. With respect to clinical trial design, I argue that while explanatory trials often have some social value, many have less social value than their pragmatic counterparts. On the basis of this general ethical assessment, I provide a preliminary defense of the position that clinical researchers should aim to conduct pragmatic trials, that is, that researchers face a burden of justification related to any idealizing elements added to trial designs.

Designing a valid randomized pragmatic primary care implementation trial: the my own health report (MOHR) project

BACKGROUND

There is a pressing need for greater attention to patient-centered health behavior and psychosocial issues in primary care, and for practical tools, study designs and results of clinical and policy relevance. Our goal is to design a scientifically rigorous and valid pragmatic trial to test whether primary care practices can systematically implement the collection of patient-reported information and provide patients needed advice, goal setting, and counseling in response.

METHODS

This manuscript reports on the iterative design of the My Own Health Report (MOHR) study, a cluster randomized delayed intervention trial. Nine pairs of diverse primary care practices will be randomized to early or delayed intervention four months later. The intervention consists of fielding the MOHR assessment--addresses 10 domains of health behaviors and psychosocial issues--and subsequent provision of needed counseling and support for patients presenting for wellness or chronic care. As a pragmatic participatory trial, stakeholder groups including practice partners and patients have been engaged throughout the study design to account for local resources and characteristics. Participatory tasks include identifying MOHR assessment content, refining the study design, providing input on outcomes measures, and designing the implementation workflow. Study outcomes include the intervention reach (percent of patients offered and completing the MOHR assessment), effectiveness (patients reporting being asked about topics, setting change goals, and receiving assistance in early versus delayed intervention practices), contextual factors influencing outcomes, and intervention costs.

DISCUSSION

The MOHR study shows how a participatory design can be used to promote the consistent collection and use of patient-reported health behavior and psychosocial assessments in a broad range of primary care settings. While pragmatic in nature, the study design will allow valid comparisons to answer the posed research question, and findings will be broadly generalizable to a range of primary care settings. Per the pragmatic explanatory continuum indicator summary (PRECIS) framework, the study design is substantially more pragmatic than other published trials. The methods and findings should be of interest to researchers, practitioners, and policy makers attempting to make healthcare more patient-centered and relevant.

TRIAL REGISTRATION

Clinicaltrials.gov: NCT01825746.