Pre-study feasibility and identifying sensitivity analyses for protocol pre-specification in comparative effectiveness research

Healthy Eating and Active Lifestyle after Bowel Cancer (HEAL ABC)-feasibility randomised controlled trial

BACKGROUND

Evidence from cohort studies indicates that a healthy lifestyle can improve cancer survival but evidence from randomised controlled trials (RCT) is lacking. Thus, this study tested the feasibility of conducting a lifestyle intervention in patients after colorectal cancer (CRC) treatment.

METHODS

An intervention was developed based on World Cancer Research Fund and American Institute for Cancer Research (WCRF/AICR) recommendations, the Health Action Process Approach, Motivational Interviewing and tested a feasibility, mixed-methods RCT. Participants were allocated to a three-month telephone-based intervention versus standard care control group. The follow up period was six months. Data on feasibility and secondary outcomes were collected and analysed using Stata (V15, StataCorp LLC) and NVivo 12 (QSR International Pty Ltd., Doncaster, VIC).

RESULTS

Recruitment was challenging (31 ineligible, 37 declined; recruitment rate = 48.6%.). In total, 34/35 participants completed the intervention, and 31 (89%) completed follow up; all 31 completers participated in six telephone calls during intervention and six months follow up. Study retention was 97% (34/35) and 89% (31/35) at three and six months, respectively. Data completion rates were high (>90%). Intervention was acceptable to participants, met their needs and kept them accountable towards their goals. Participants in the intervention group showed significant improvement in WCRF/AICR, Diet Quality Index-International score and a 10% reduction in ultra-processed food consumption.

CONCLUSIONS

The HEAL ABC intervention was feasible for 87% of intervention participants, supporting them in healthy lifestyle changes. However, alternative recruitment strategies are needed for a fully powered RCT to determine the effectiveness of the intervention.

Bridging the divide between clinical research and clinical care in oncology: An integrated real-world evidence generation platform

Real world data (RWD) are data relating to patient health status and/or the delivery of health care routinely collected from a variety of sources; real-world evidence (RWE) generated by RWD analyses can become an important component of drug development programs and, potentially, regulatory decision-making. As a RWD source, electronic health records (EHRs) can now provide patient-level data at unparalleled depth and granularity. We propose a RWE generation framework that could maximize the synergy between RWD and prospective clinical trials by capitalizing on an emerging data curation infrastructure that may be applied to both retrospective and prospective research. In this platform, centralized data collection and monitoring could be enabled via routine EHR use, and seamlessly integrated with select intentional data capture during prospective study periods. By bridging the divide between routine care and clinical research, this integrated platform aggregates retrospective and prospective data, collected both routinely and intentionally. This approach makes clinical trial participation more available to patients, increasing the potential depth of data, representativeness and efficiency of clinical research.

Real-world experiences with generating real-world evidence: Case Studies from PCORI's pragmatic clinical Studies program

BACKGROUND

Over the last decade, randomized studies evaluating outcomes of health care interventions conducted in real-world settings-often termed "pragmatic trials"-have come to be seen as an important means of obtaining relevant, actionable evidence to guide health care decisions. Despite extensive writing on methodological considerations in pragmatic trial design, limited information exists regarding the practical and logistical challenges encountered in carrying out rigorous randomized evaluations in highly representative, real-world contexts.

METHODS

The Patient Centered Outcomes Research Institute (PCORI) convened an expert panel in 2017 to examine common tradeoffs in study design and implementation through 3 "case studies" of in-progress, PCORI-funded pragmatic trials. This paper summarizes the findings of this panel, using the 3 examples to illustrate common implementation challenges encountered in pragmatic trials.

RESULTS

Investigators aimed to generate highly generalizable findings that could address real-world clinical decisions; however, practical considerations required that each study incorporate traditionally "explanatory" elements to achieve a "fit-for-purpose" approach to design and implementation. Within individual studies, efforts to balance pragmatic versus explanatory perspectives often involved multiple, diverse aspects of trial design and implementation, and the aspects of design and implementation where investigators reported encountering such tradeoffs varied across the three cases we examined.

CONCLUSIONS

Efforts to generate rigorous evidence that is generalizable to "real-world" practice require continuous and iterative efforts to balance "pragmatic" and "explanatory" perspectives. In each study examined, these tradeoffs were guided both by an overriding effort to maintain pragmatism and practical considerations that varied depending on the research question and study context.

Evaluating the Feasibility of Electronic Health Records and Claims Data Sources for Specific Research Purposes

Data collected in real-world clinical settings are increasingly being used to evaluate therapeutic options. While in its infancy for research assessing effectiveness, especially comparative effectiveness in the regulatory environment, electronic health records (EHR) and administrative insurance claims data are used extensively by both manufacturers and regulators to evaluate post-marketing safety of products in the real world. The feasibility of using these data for analysis in a research study depends on the specific research question and the availability, quality and relevance of the collected data to address the scientific question. It is unlikely that any specific database could be 'qualified' for use across all research questions, even within a specific therapeutic area, due to dependence of feasibility on the elements of the specific research question. This paper describes considerations for determining whether EHR or claims data can be used for specific research purposes. A new structured approach for assessing the feasibility of these data in research is proposed. The framework builds on and considers whether each element of the PICOTS framework for well-structured research questions is adequately captured to allow for viable reliance on EHR and claims data for that specific scientific question. Practical examples and discussion of the limitations of RWD for research are given along with approaches for interpretation of analyses using RWD.

A tool for empirical equipoise assessment in multigroup comparative effectiveness research

PURPOSE

In observational research, equipoise concerns whether groups being compared are similar enough for valid inference. Empirical equipoise was previously proposed as a tool to assess patient similarity based on propensity scores (PS). We extended this work for multigroup observational studies.

METHODS

We modified the tool to allow for multinomial exposures such that the proposed definition reduces to the original when there are only two groups. We illustrated how the tool can be used as a method to assess study design within three-group clinical examples. We then conducted three-group simulations to assess how the tool performed in a setting with residual confounding after PS weighting.

RESULTS

In a clinical example based on rheumatoid arthritis, 44.5% of the sample fell within the region of empirical equipoise when considering first-line biologics, whereas 57.7% did so for second-line biologics, consistent with the expectation that a second-line design results in better equipoise. In a simulation where the unmeasured confounder had the same magnitude of association with the treatment as the measured confounders and a 25% greater association with the outcome, the tool crossed the proposed threshold for empirical equipoise at a residual confounding of 20% on the ratio scale. When the unmeasured variable had a twice larger association with treatment, the tool became less sensitive and crossed the threshold at a residual confounding of 30%.

CONCLUSION

Our proposed tool may be useful in guiding cohort identification in multigroup observational studies, particularly with similar effects of unmeasured and measured covariates on treatment and outcome.

Multinomial Extension of Propensity Score Trimming Methods: A Simulation Study

Crump et al. (Biometrika. 2009;96(1):187-199), Stürmer et al. (Am J Epidemiol. 2010;172(7):843-854), and Walker et al. (Comp Eff Res. 2013;2013(3):11-20) proposed propensity score (PS) trimming methods as a means to improve efficiency (Crump) or reduce confounding (Stürmer and Walker). We generalized the trimming definitions by considering multinomial PSs, one for each treatment, and proved that these proposed definitions reduce to the original binary definitions when we have only 2 treatment groups. We then examined the performance of the proposed multinomial trimming methods in the setting of 3 treatment groups, in which subjects with extreme PSs more likely had unmeasured confounders. Inverse probability of treatment weights, matching weights, and overlap weights were used to control for measured confounders. All 3 methods reduced bias regardless of the weighting methods in most scenarios. Multinomial Stürmer and Walker trimming were more successful in bias reduction when the 3 treatment groups had very different sizes (10:10:80). Variance reduction, seen in all methods with inverse probability of treatment weights but not with matching weights or overlap weights, was more successful with multinomial Crump and Stürmer trimming. In conclusion, our proposed definitions of multinomial PS trimming methods were beneficial within our simulation settings that focused on the influence of unmeasured confounders.

Persistence of oral antidiabetic treatment for type 2 diabetes characterized by drug class, patient characteristics and severity of renal impairment: A Japanese database analysis

AIM

To evaluate the persistence with oral antidiabetic drug (OAD) treatment characterized by drug class, patient characteristics and severity of renal impairment (RI) in patients with type 2 diabetes (T2DM) in Japan.

MATERIALS AND METHODS

This retrospective, observational study extracted data from a large-scale hospital database (April 2008 to September 2016). Patients with T2DM aged ≥40 years on the day of their first prescription (index date) of any OAD (biguanides [BGs], thiazolidinediones [TZDs], sulphonylureas [SUs], glinides, dipeptidyl peptidase-4 [DPP-4] inhibitors, or α-glucosidase inhibitors [α-GIs]) available between January 1, 2014 and September 30, 2016 were identified. Sodium-glucose co-transporter-2 inhibitors were not available at study initiation. Treatment persistence was assessed by Kaplan-Meier survival curves. Patients were also categorized by RI status using estimated glomerular filtration rate: ≥90 mL/min/1.73 m2 (G1); 60 to <90 mL/min/1.73 m2 (G2); 30 to <60 mL/min/1.73 m2 (G3); and <30 mL/min/1.73 m2 (G4+).

RESULTS

We identified 206 406 index dates from 162 116 eligible patients. The largest number of index dates (91634) was observed for DPP-4 inhibitors, followed by BGs, SUs, α-GIs, glinides and TZDs. Treatment persistence was longest for DPP-4 inhibitors (median 17.0 months, 95% confidence interval [CI] 16.4-17.5) and BGs (median 17.3 months, 95% CI 16.6-18.2), and shortest for α-GIs (median 5.6 months, 95% CI 5.4-5.9) and SUs (median 4.3 months, 95% CI 4.2-4.6). Persistence was longest with DPP-4 inhibitors at all RI stages (G1-G4+), followed by BGs at stages G1/G2.

CONCLUSIONS

The longest OAD persistence was observed for BGs and DPP-4 inhibitors at RI stages G1/G2, and for DPP-4 inhibitors at RI stages G3/G4+.

Ethical Considerations for Increased Transparency and Reproducibility in the Retrospective Analysis of Health Care Data

In the field of health care, researchers and decision makers are increasingly turning toward retrospective observational studies of administrative claims and electronic health record databases to improve outcomes for patients. For many important questions, randomized studies have not been conducted, and even when they have been, such studies often inadequately reflect the realities of patients' lives or care. However, use of retrospective studies not only increases methodological complexity but also requires more subjectivity for those attempting to perform statistical analysis. The hurdles for establishing the reproducibility of such research to ensure accuracy and generalizability are therefore also higher, as are the requirements for transparency to limit the impact of bias. The ethical statistical practitioner will therefore need to take additional steps to enable results to be interpreted and acted upon with confidence. These include increased transparency regarding the impact of database selection, database quality, database content, and design decisions on the robustness of statistical conclusions. A number of approaches to increase the reproducibility of retrospective health care research are also presented, along with some discussion regarding responsibilities of data owners, statistical practitioners, publishers, and users of results.