A national strategy to develop pragmatic clinical trials infrastructure

Fidelity to the acupuncture intervention protocol in the ACUpuncture In The EmergencY department for pain management (ACUITY) trial: Expanding the gold standard of STRICTA and CONSORT guidelines

Background

Acupuncture shows promise as an effective nonpharmacologic option for reduction of acute pain in the emergency department (ED). Following CONSORT and STRICTA guidelines, randomized controlled trials (RCTs) generally report intervention details and acupoint options, but fidelity to acupuncture interventions, critical to reliability in intervention research, is rarely reported.

Methods

ACUITY is an NCCIH-funded, multi-site feasibility RCT of acupuncture in 3 EDs (Cleveland, Nashville, and San Diego). ACUITY acupuncturists were trained in study design, responsive acupuncture manualization protocol, logistics and real-time recording of session details via REDCap forms created to track fidelity.

Results

Across 3 recruiting sites, 79 participants received acupuncture: 51 % women, 43 % Black/African American, with heterogeneous acute pain sites at baseline: 32 % low back, 22 % extremity, 20 % abdominal, 10 % head. Pragmatically, participants were treated in ED common areas (52 %), private rooms (39 %), and semi-private rooms (9 %). Objective tracking found 98 % adherence to the six components of the acupuncture manualization protocol: staging, number of insertion points (M = 13.2, range 2-22), needle retention time (M = 23.5 min, range 4-52), session length (M = 40.3 min, range 20-66), whether general recommendations were provided and completion of the session form.

Conclusion

To the best of our knowledge, this is the first RCT to assess and report fidelity to an acupuncture protocol. Fidelity monitoring will be fundamental for ACUITY2, which would be a future definitive, multi-site RCT. Furthermore, we recommend that fidelity to acupuncture interventions be added to CONSORT and STRICTA reporting guidelines in future RCTs.

Protocol registration

The protocol of this study is registered at clinicaltrials.gov: NCT04880733.

Partnering with social service staff to implement pragmatic clinical trials: an interim analysis of implementation strategies

BACKGROUND

With recent growth in the conduct of pragmatic clinical trials, the reliance on frontline staff to contribute to trial-related activities has grown as well. Active partnerships with staff members are often critical to pragmatic trial implementation, but rarely do research teams track and evaluate the specific "implementation strategies" used to support staff's involvement in trial procedures (e.g., participant recruitment). Accordingly, we adapted implementation science methodologies and conducted an interim analysis of the strategies deployed with social service staff involved in one multi-site pragmatic clinical trial.

METHODS

We used a naturalistic, observational study design to characterize strategies our research team deployed with staff during monthly, virtual meetings. Data were drawn from meeting notes and recordings from the trial's 4-month Preparation phase and 8-month Implementation phase. Strategies were mapped to the Expert Recommendations for Implementing Change taxonomy and categorized into nine implementation clusters. Survey data were also collected from staff to identify the most useful strategies the research team should deploy when onboarding new staff members in the trial's second year.

RESULTS

A total of 287 strategies were deployed. Strategies in the develop stakeholder interrelationships cluster predominated in both the Preparation (35%) and Implementation (31%) phases, followed by strategies in the use iterative and evaluative approaches cluster, though these were more prevalent during trial Preparation (24%) as compared to trial Implementation (18%). When surveyed on strategy usefulness, strategies in the provide interactive assistance, use financial approaches, and support staff clusters were most useful, per staff responses.

CONCLUSIONS

While strategies to develop stakeholder interrelationships were used most frequently during trial Preparation and Implementation, program staff perceived strategies that provided technical assistance, supported clinicians, and used financial approaches to be most useful and should be deployed when onboarding new staff members. Research teams are encouraged to adapt and apply implementation strategy tracking methods when partnering with social service staff and deploy practical strategies that support pragmatic trial success given staff needs and preferences.

TRIAL REGISTRATION

NCT05357261. May 2, 2022.

Design, implementation, and inferential issues associated with clinical trials that rely on data in electronic medical records: a narrative review

Real world evidence is now accepted by authorities charged with assessing the benefits and harms of new therapies. Clinical trials based on real world evidence are much less expensive than randomized clinical trials that do not rely on "real world evidence" such as contained in electronic health records (EHR). Consequently, we can expect an increase in the number of reports of these types of trials, which we identify here as 'EHR-sourced trials.' 'In this selected literature review, we discuss the various designs and the ethical issues they raise. EHR-sourced trials have the potential to improve/increase common data elements and other aspects of the EHR and related systems. Caution is advised, however, in drawing causal inferences about the relationships among EHR variables. Nevertheless, we anticipate that EHR-CTs will play a central role in answering research and regulatory questions.

Acupuncture in the emergency department for pain management: A BraveNet multi-center feasibility study

PURPOSE

Pain accounts for up to 78% of emergency department (ED) patient visits and opioids remain a primary method of treatment despite risks of addiction and adverse effects. While prior acupuncture studies are promising as an alternative opioid-sparing approach to pain reduction, successful conduct of a multi-center pilot study is needed to prepare for a future definitive randomized control trial (RCT).

METHODS

Acupuncture in the Emergency Department for Pain Management (ACUITY) is funded by the National Center for Complementary and Integrative Health. The objectives are to: conduct a multi-center feasibility RCT, examine feasibility of data collection, develop/deploy a manualized acupuncture intervention and assess feasibility/implementation (barrier/facilitators) in 3 EDs affiliated with the BraveNet Practice Based Research Network.Adults presenting to a recruiting ED with acute non-emergent pain (e.g., musculoskeletal, back, pelvic, noncardiac chest, abdominal, flank or head) of ≥4 on a 0-10-point Numeric Rating Scale will be eligible. ED participants (n = 165) will be equally randomized to Acupuncture or Usual Care.At pre-, post-, and discharge time-points, patients will self-assess pain and anxiety using the Numeric Rating Scale. Pain, anxiety, post-ED opioid use and adverse events will be assessed at 1 and 4 weeks. Opioid utilization in the ED and discharge prescriptions will be extracted from patients' electronic medical records.Acupuncture recipients will asked to participate in a brief qualitative interview about 3 weeks after their discharge. ED providers and staff will also be interviewed about their general perspectives/experiences related to acupuncture in the ED and implementation of acupuncture in ACUITY.

RESULTS

Recruitment began on 5/3/21. As of 12/7/21: 84 patients have enrolled, the responsive acupuncture intervention has been developed and deployed, and 26 qualitative interviews have been conducted.

CONCLUSION

Successful conduct of ACUITY will provide the necessary framework for conducting a future, multi-center, definitive RCT of acupuncture in the ED.

CLINICAL TRIALSGOV

NCT04880733 https://clinicaltrials.gov/ct2/show/NCT04880733.

CONNECT: A pragmatic clinical trial testing a remotely provided linkage to service coordination after hospitalization for TBI

OBJECTIVE

To test whether a complex behavioral intervention delivered remotely to connect individuals to clinical resources after hospitalization for TBI improved their quality of life.

DESIGN/METHODS

Community-based randomized pragmatic clinical trial. Main measures TBI-QOL, Activity Measure for Post-Acute Care (AM-PAC), Clinical Satisfaction and Competency Rating Scale.

RESULTS

332 individuals ≥18 years-old hospitalized for TBI in four upper Midwest states were randomized to Remote (n = 166) and Usual Care (n = 166) groups. The groups were equivalent and representative of their state population's racial and ethnic composition, age, and proportion living in rural communities. There were no significant differences within or between experimental groups over the study period in TBI-QOL t-scores. There was a significant improvement in AM-PAC Daily Activities within the Remote group and a significant between-group improvement in clinical satisfaction for the Remote group.

CONCLUSION

Enrolling a representative, regional community-based sample of individuals with TBI can be successful, and delivering a customized complex behavioral intervention remotely is feasible. The overall lack of intervention effectiveness was likely due to enrolling individuals without pre-identified clinical needs, initiating intervention after the immediate post-acute phase when needs are often highest, inability to provide direct clinical care remotely, and potential lack of outcome measure responsiveness in our sample.

Sample size requirements for detecting treatment effect heterogeneity in cluster randomized trials

Cluster randomized trials (CRTs) refer to experiments with randomization carried out at the cluster or the group level. While numerous statistical methods have been developed for the design and analysis of CRTs, most of the existing methods focused on testing the overall treatment effect across the population characteristics, with few discussions on the differential treatment effect among subpopulations. In addition, the sample size and power requirements for detecting differential treatment effect in CRTs remain unclear, but are helpful for studies planned with such an objective. In this article, we develop a new sample size formula for detecting treatment effect heterogeneity in two-level CRTs for continuous outcomes, continuous or binary covariates measured at cluster or individual level. We also investigate the roles of two intraclass correlation coefficients (ICCs): the adjusted ICC for the outcome of interest and the marginal ICC for the covariate of interest. We further derive a closed-form design effect formula to facilitate the application of the proposed method, and provide extensions to accommodate multiple covariates. Extensive simulations are carried out to validate the proposed formula in finite samples. We find that the empirical power agrees well with the prediction across a range of parameter constellations, when data are analyzed by a linear mixed effects model with a treatment-by-covariate interaction. Finally, we use data from the HF-ACTION study to illustrate the proposed sample size procedure for detecting heterogeneous treatment effects.

Dental practice-based research networks (D-PBRN) worldwide: A scoping review

OBJECTIVES

For more than twenty years, dental practice-based research networks (D-PBRN) have helped to structure clinical research in private practice. They bring together practitioners working in several structures and may include a greater number of subjects. The aims of this study were thus to systematically explore the scientific production from dental private practices in general and to map and describe the D-PBRN activity worldwide.

DATA SOURCES

Two research procedures were carried out in parallel. The first was conducted as a scoping review to examine peer-reviewed literature indexed in the PubMed database and the second was performed on the World Wide Web to identify the main characteristics of the networks (location, scientific production…).

STUDY SELECTION

368 publications were identified among which 202 were published by PBRN members and the others by private practitioners not affiliated to any network. 210 (57 % of the included articles) were produced in the USA. A higher number of diverse centers are involved in each study when it is conducted by a PBRN (59.06 ± 66.59 vs. 13.51 ± 31.58 for networks and independent teams, respectively; p < 0.01). 24 D-PBRN were identified, a majority being based in the USA and 8 in Europe.

CONCLUSIONS

Although dental practice-based research has grown over the years, the number of D-PBRN worldwide remains low. Even if it requires some investment to produce research in dental offices, this type of networks helps to fill the gap between private practice and research and to improve knowledge on oral health.

RELEVANCE

The mapping of all the dental PBRN together with the research topics studied throughout the world make the relevance of this article. The ways to improve practice-based research in dentistry are also discussed in the paper.

Can Electronic Health Records Validly Estimate the Effects of Health System Interventions Aimed at Controlling Body Weight?

OBJECTIVE

This study aimed to compare weight collected at clinics and recorded in the electronic health record (EHR) with primary study-collected trial weights to assess the validity of using EHR data in future pragmatic weight loss or weight gain prevention trials.

METHODS

For both the Track and Shape obesity intervention randomized trials, clinic EHR weight data were compared with primary trial weight data over the same time period. In analyzing the EHR weights, intervention effects were estimated on the primary outcome of weight (in kilograms) with EHR data, using linear mixed effects models.

RESULTS

EHR weight measurements were higher on average and more variable than trial weight measurements. The mean difference and 95% CI were similar at all time points between the estimates using EHR and study-collected weights.

CONCLUSIONS

The results of this study can be used to help guide the planning of future pragmatic weight-related trials. This study provides evidence that body weight measurements abstracted from the EHR can provide valid, efficient, and cost-effective data to estimate treatment effects from randomized clinical weight loss and weight management trials. However, care should be taken to properly understand the data-generating process and any mechanisms that may affect the validity of these estimates.

Defining key design elements of registry-based randomised controlled trials: a scoping review

Background

Traditional randomised controlled trials remain the gold standard for improving clinical care but they do have their limitations, including their associated high costs, high failure rate and limited external validity. An alternative methodology is the newly defined, prospective, registry-based randomised controlled trial (RRCT), where treatment and outcome data is collected in an existing registry. This scoping review explores the current literature regarding RRCTs to help identify the key design elements of RRCTs and the characteristics of clinical registries on which they are reliant on.

Methods

A scoping review methodology conducted in accordance with the Joanna Briggs Institute guidelines was performed. Four databases were searched for articles published from inception to June 2018: Medline; Embase; the Cumulative Index to Nursing and Allied Health Literature and; Scopus. The search strategy included MeSH and text words related to RRCT.

Results

We identified 2369 articles of which 75 were selected for full-text screening. Of these, only 17 articles satisfied our inclusion criteria. All studies were published between 1996 and 2017 and all were investigator-initiated. Study designs were mainly multi-site comparative/effectiveness studies incorporating the use of disease registries (n = 8), procedure registries (n = 8) and a health services registry (n = 1). The low cost, reduced administrative burden and enhanced external validity of RRCTs make them an attractive research methodology which can be used to address questions of public health importance. We identified that that there are variable definitions of what constituted a RRCT and that issues related to ethical conduct and data integrity, completeness, timeliness, validation and endpoint adjudication need to be carefully addressed.

Conclusion

RRCTs potentially have an important role to play in informing best clinical practice and health policy. There are a number of issues that need to be addressed to optimise the utility of this approach, including establishing universally accepted criteria for the definition of a RRCT.

Value-Based Care for Musculoskeletal Pain: Are Physical Therapists Ready to Deliver?

Early physical therapy models hold great promise for delivering high-value care for individuals with musculoskeletal pain. However, existing physical therapist practice and research standards are misaligned with value-based principles, which limits the potential for growth and sustainability of these models. This Perspective describes how the value proposition of early physical therapy can be improved by redefining harm, embracing a prognostic approach to clinical decision making, and advocating for system-wide guideline-adherent pain care. It also outlines the need to adopt a common language to describe these models and embrace new, rigorous study designs and analytical approaches to better understand where and how early physical therapy delivers value. The goal is to define a clear path forward to ensure physical therapists are aligned within health care systems to deliver on the American Physical Therapy Association's vision of high-value care in a rapidly changing health care environment.

Cochrane reviews on acupuncture therapy for pain: A snapshot of the current evidence

Cochrane is an international non-profit organization established in 1993 to produce and disseminate high quality and unbiased systematic reviews of evidence on health care interventions. At the forefront of systematic review methodology, Cochrane is generally accepted to be among the most carefully prepared and rigorous sources of systematic review evidence. There are numerous Cochrane reviews on nonpharmacologic interventions for pain and multiple Cochrane reviews evaluating acupuncture therapy in pain conditions. But how complete and up to date are those reviews relative to other rigorous systematic reviews with meta-analyses of acupuncture therapy for pain published in the literature? In this 'snapshot' overview, we found 22 relevant Cochrane reviews, some concluding that acupuncture therapy is probably useful for treating specific pain conditions. However, many of the conditions for which acupuncture is most commonly used are either not represented in Cochrane reviews or the existing Cochrane reviews are seriously outdated and do not reflect current evidence. This creates confusion with the risks of adverse effects and addiction liability associated with pain medications, the prevalence of chronic pain, the ongoing opioid epidemic and the need for evidence-based options for pain as part of comprehensive pain care. Clinicians and patients want clarification on safe and effective options to treat pain. Issues involving reviewed trials' inadequate use of sham comparators, of acupuncture as a complex intervention with interactive components and a shift in research focus from efficacy trials to real-world pragmatic trials are discussed in relation to updating Cochrane reviews of acupuncture therapy for pain.

Informed consent and community engagement in open field research: lessons for gene drive science

Background

The development of the CRISPR/Cas9 gene editing system has generated new possibilities for the use of gene drive constructs to reduce or suppress mosquito populations to levels that do not support disease transmission. Despite this prospect, social resistance to genetically modified organisms remains high. Gene drive open field research thus raises important questions regarding what is owed to those who may not consent to such research, or those could be affected by the proposed research, but whose consent is not solicited. The precise circumstances under which informed consent must be obtained, and from whom, requires careful consideration. Furthermore, appropriate engagement processes should be central to any introduction of genetically modified mosquitos in proposed target settings.

Discussion

In this work, international guidance documents on informed consent and engagement are reviewed and applied to the genetically modified mosquito research context. Five analogous research endeavours that involve area-wide / open field experiments are reviewed. The approach of each in respect to the solicitation of individual informed consent and community engagement are highlighted.

Conclusions

While the solicitation of individual informed consent in host settings of gene drive field trials may not be possible or feasible in some instances, local community and stakeholder engagement will be key to building trust towards the proposed conduct of such research. In this regard, the approaches taken by investigators and sponsors of political science field research and weather modification field research should be avoided. Rather, proponents of gene drive field research should look to the Eliminate Dengue field trials, cluster randomised trials, and pragmatic clinical trials for guidance regarding how the solicitation of individual informed consent of host communities ought to be managed, and how these communities ought to be engaged.

OneFlorida Clinical Research Consortium: Linking a Clinical and Translational Science Institute With a Community-Based Distributive Medical Education Model

PROBLEM

Developing a national pragmatic clinical trial infrastructure is central to understanding the effectiveness of interventions applied under usual conditions and where people receive health care. To address this challenge, three Florida universities-the University of Florida Clinical and Translational Science Institute, Florida State University (with its community-based distributive medical education model), and the University of Miami-created (2010-2013) a statewide consortium, the OneFlorida Clinical Research Consortium, to support the conduct of pragmatic clinical trials and provide mentored research experiences for medical and graduate students in real-world practice settings.

APPROACH

OneFlorida has four programs, which report to a steering committee with membership from each partner, community members, and the state Medicaid agency and Department of Health to ensure shared governance. The Clinical Research Program provides support to conduct research in the network and uses champions to engage community clinicians. The Citizen Scientist Program has community members who provide input on health topics of importance to them, study design, recruitment and retention strategies, and the interpretation of findings. The Data Trust Program contains electronic health record and health care claims data for 10.6 million Floridians. The Minority Education Program, in collaboration with three historically black colleges and universities, offers minority junior faculty mentoring in pragmatic clinical trials and implementation science.

OUTCOMES

OneFlorida has implemented 27 studies with diverse patient populations and in diverse community practice settings.

NEXT STEPS

To identify evidence-based best practices from the clinical trials conducted in the network, foster their implementation, and expand research training opportunities.

Pragmatic clinical trials and the consent process

Pragmatic clinical trials (PCTs) are a relatively new methodological approach to the execution of clinical research that can increase research efficiency and provide access to unique data. Some have suggested that the costs and delays associated with obtaining informed consent could make PCTs difficult or even impossible to execute. Alternative consent models have been proposed, some of which lower standards of disclosure, delay consent, or waive it altogether. We analyze the permissibility of changes to informed consent in the context of Canadian research ethics policies, legislation, common law, professional codes of ethics, and professional standards of practice. We find that Canadian law and policy relating to informed consent clearly applies to any clinician who might be involved in a PCT. In addition, existing consent norms seem unable to accommodate alternative consent models for pragmatic research if such models would involve lowering the standard of disclosure. The strong emphasis on the primacy of individual rights that exist in law and in research ethics norms cannot easily coexist with strategies that involve either waiver of consent requirements or the provision of incomplete information about the research prior to enrolment. If Canadian policy-makers wish to create the regulatory flexibility necessary to accommodate altered consent and disclosure, it is likely this will require the alteration of existing health information legislation, national research ethics policy, and professional standards.

Better Together: The Making and Maturation of the Palliative Care Research Cooperative Group

OBJECTIVE

To describe the growth and outcomes of the Palliative Care Research Cooperative Group (PCRC).

BACKGROUND

Despite advances, significant gaps remain in the evidence base to inform care for people with serious illness. To generate this needed evidence and bolster research capacity, the Palliative Care Research Cooperative (PCRC) group was formed.

METHODS

The PCRC supports investigators in the conduct of multisite clinical studies. After developing a governance structure and completing a proof of concept demonstration study, the PCRC expanded its infrastructure to include additional resource cores (Clinical Studies; Measurement; Data Informatics and Statistics; and Caregiver Studies). The PCRC also supports an Investigator Development Center as many palliative care investigators valued opportunities to advance their skills. Additional key aspects of PCRC resources include a Scientific Review Committee, a Publications Committee, and initiatives to purposefully engage investigators in a community of palliative care science.

RESULTS

The PCRC has grown to over 300 members representing more than 130 distinct sites. To date, the PCRC has supported the submission of 51 research applications and has engaged in 27 studies. The PCRC supports investigator research development needs through webinars and clinical trials "intensives." To foster a sense of community, the PCRC has convened biannual meetings, developed special interest groups, and regularly communicates via a newsletter and its website.

CONCLUSION

With a particular focus on facilitating conduct of rigorous multisite clinical studies, the PCRC fosters an engaged multidisciplinary research community, filling an important void in generating and disseminating evidence that informs the provision of high-quality care to people with serious illness.

Validation of clinic weights from electronic health records against standardized weight measurements in weight loss trials

OBJECTIVE

To validate clinic weights in electronic health records against researcher-measured weights for outcome assessment in weight loss trials.

METHODS

Clinic and researcher-measured weights from a published trial (BE WELL) were compared using Lin's concordance correlation coefficient, Bland and Altman's limits of agreement, and polynomial regression model. Changes in clinic and researcher-measured weights in BE WELL and another trial, E-LITE, were analyzed using growth curve modeling.

RESULTS

Among BE WELL (n = 330) and E-LITE (n = 241) participants, 96% and 90% had clinic weights (mean [SD] of 5.8 [6.1] and 3.7 [3.9] records) over 12 and 15 months of follow-up, respectively. The concordance correlation coefficient was 0.99, and limits of agreement plots showed no pattern between or within treatment groups, suggesting overall good agreement between researcher-measured and nearest-in-time clinic weights up to 3 months. The 95% confidence intervals for predicted percent differences fell within ±3% for clinic weights within 3 months of the researcher-measured weights. Furthermore, the growth curve slopes for clinic and researcher-measured weights by treatment group did not differ significantly, suggesting similar inferences about treatment effects over time, in both trials.

CONCLUSIONS

Compared with researcher-measured weights, close-in-time clinic weights showed high agreement and inference validity. Clinic weights could be a valid pragmatic outcome measure in weight loss studies.

Practice-based Research Networks (PBRNs) in the Era of Integrated Delivery Systems

Practice-based research networks (PBRNs) have been in existence for several decades, and they provide one mechanism to conduct research outside of academic research centers. Two transformative changes to the practice environment pose significant challenges to the manner in which PBRNs have functioned in the past and require changes to their current activities. The widespread introduction of electronic health records and the organization of practices into often hospital-dominated integrated delivery systems change the manner in which medicine is practiced, administered, and financed. Research funders are committed to extending research into communities, although we have yet to learn how to conduct these activities efficiently. We describe a number of operational challenges to this transformation, and we also propose ways to address these challenges and improve the quality and efficiency through which research is conducted. PBRNs can ensure their relevance in the research environment by adapting to this new era.

Guidance for researchers developing and conducting clinical trials in practice-based research networks (PBRNs)

BACKGROUND

There is increased interest nationally in multicenter clinical trials to answer questions about clinical effectiveness, comparative effectiveness, and safety in real-world community settings. Primary care practice-based research networks (PBRNs), comprising community- and/or academically affiliated practices committed to improving medical care for a range of health problems, offer ideal settings for these trials, especially pragmatic clinical trials. However, many researchers are not familiar with working with PBRNs.

METHODS

Experts in practice-based research identified solutions to challenges that researchers and PBRN personnel experience when collaborating on clinical trials in PBRNs. These were organized as frequently asked questions in a draft document presented at a 2013 Agency for Health care Research and Quality PBRN conference workshop, revised based on participant feedback, then shared with additional experts from the DARTNet Institute, Clinical Translational Science Award PBRN, and North American Primary Care Research Group PBRN workgroups for further input and modification.

RESULTS

The "Toolkit for Developing and Conducting Multi-site Clinical Trials in Practice-Based Research Networks" offers guidance in the areas of recruiting and engaging practices, budgeting, project management, and communication, as well as templates and examples of tools important in developing and conducting clinical trials.

CONCLUSION

Ensuring the successful development and conduct of clinical trials in PBRNs requires a highly collaborative approach between academic research and PBRN teams.