Scientific and organizational collaboration in comparative effectiveness research: the VA cooperative studies program model

Clinical trial facilitators: A novel approach to support the execution of clinical research at the study site level

In the summer of 2020, multiple efforts were undertaken to establish safe and effective vaccines to combat the spread of the coronavirus disease (COVID-19). In the United States (U.S.), Operation Warp Speed (OWS) was the program designated to coordinate such efforts. OWS was a partnership between the Department of Health and Human Services (HHS), the Department of Defense (DOD), and the private sector, that aimed to help accelerate control of the COVID-19 pandemic by advancing development, manufacturing, and distribution of vaccines, therapeutics, and diagnostics. The U.S. Department of Veterans Affairs’ (VA) was identified as a potential collaborator in several large-scale OWS Phase III clinical trial efforts designed to evaluate the safety and efficacy of various vaccines that were in development. Given the global importance of these trials, it was recognized that there would be a need for a coordinated, centralized effort within VA to ensure that its medical centers (sites) would be ready and able to efficiently initiate, recruit, and enroll into these trials.

The manuscript outlines the partnership and start-up activities led by two key divisions of the VA's Office of Research and Development's clinical research enterprise. These efforts focused on site and enterprise-level requirements for multiple trials, with one trial serving as the most prominently featured of these studies within the VA. As a result, several best practices arose that included designating clinical trial facilitators to study sites to support study initiation activities and successful study enrollment at these locations in an efficient and timely fashion.

A Qualitative Scoping Review of Early-Terminated Clinical Trials Sponsored by the Department of Veterans Affairs Cooperative Studies Program From 2010 to 2020

Increasing attention has been paid to the risks and benefits of terminating large clinical trials before reaching prespecified targets, because such decisions can greatly affect the implementation of findings. The Department of Veterans Affairs (VA) Cooperative Studies Program (CSP) is a research infrastructure dedicated to conducting high-quality clinical research. A scoping review was performed to characterize barriers preventing the attainment of prespecified recruitment, statistical power, or sample-size targets in VA CSP trials. A trial was eligible for inclusion if the trial was sponsored by the VA CSP, primary findings were published within the last 10 years, and a decision was made to terminate enrollment or follow-up before meeting a priori recruitment or endpoint targets. In 11 of 29 included trials (37.9%), a decision was made to terminate the trial early. The most common reason for early termination was related to under-recruitment (n = 5). Other reasons included early detection of safety signals (n = 2), futility (n = 1), and benefit (n = 1). This review highlights recruitment as a critical facet of trial conduct that may hinder the production of high-quality data and thus warrant additional attention. Solutions to enhance recruitment now implemented by the VA CSP, including dedicated enrollment infrastructure and screening facilitated by informatics approaches, show promise in reducing this cause for early termination.

Clinical trials proposed for the VA Cooperative Studies Program: Success rates and factors impacting approval

The process by which funding organizations select among the myriad number of proposals they receive is a matter of significant concern for researchers and the public alike. Despite an extensive literature on the topic of peer review and publications on criteria by which clinical investigations are reviewed, publications analyzing peer review and other processes leading to government funding decisions on large multi-site clinical trials proposals are sparse. To partially address this gap, we reviewed the outcomes of scientific and programmatic evaluation for all letters of intent (LOIs) received by the Department of Veterans Affairs (VA) Cooperative Studies Program (CSP) between July 4, 2008, and November 28, 2016. If accepted, these LOIs represented initial steps towards later full proposals that also underwent scientific peer review. Twenty-two of 87 LOIs were ultimately funded and executed as CSP projects, for an overall success rate of 25%. Most proposals which received a negative decision did so prior to submission of a full proposal. Common reasons for negative scientific review of LOIs included investigator inexperience, perceived lack of major scientific impact, lack of preliminary data and flawed or confused experimental design, while the most common reasons for negative reviews of final proposals included questions of scientific impact and issues of study design, including outcome measures, randomization, and stratification. Completed projects have been published in high impact clinical journals. Findings highlight several factors leading to successfully obtaining funding support for clinical trials. While our analysis is restricted to trials proposed for CSP, the similarities in review processes with those employed by the National Institutes of Health and the Patient Centered Outcomes Research Institute suggest the possibility that they may also be important in a broader context.

Development and implementation of standardized study performance metrics for a VA healthcare system clinical research consortium

The cost of conducting clinical trials is continuously increasing and is driven in large part by the time and resources required to activate trials and reach accrual targets. The impact of low enrollment in a clinical trial can negatively affect the validity of study results and delay its generalizability to the broader population. Quality is a multidimensional concept which could relate to the design, conduct, and analysis of a trial, its clinical relevance, protection/safety of study participants, or quality of reporting. Furthermore, the quality of controlled trials is of obvious relevance to systematic reviews and if the "raw material" or "data" is flawed then the conclusions of systematic reviews cannot be trusted. To date, the literature surrounding the establishment of standardized study enrollment and quality metrics to assess site performance in clinical trial consortiums is scarce. The lack of these metrics presents challenges to study site teams, sponsors, and other clinical research enterprise key stakeholders for adequately monitoring and evaluating study site performance as it relates to fulfilling trial enrollment and quality goals. The Department of Veterans Affairs (VA) Cooperative Studies Program (CSP) Network of Dedicated Enrollment Sites (NODES) undertook an effort to determine the feasibility of establishing and implementing standardized study enrollment and quality metrics for a clinical research consortium (NODES) as a tool to evaluate its performance. In this manuscript, we describe the development and implementation of standardized study enrollment and quality metrics to assess site performance across studies in our clinical research consortium.

The design of a randomized control trial of exoskeletal-assisted walking in the home and community on quality of life in persons with chronic spinal cord injury

There are more than 300,000 estimated cases of spinal cord injury (SCI) in the United States, and approximately 27,000 of these are Veterans. Immobilization from SCI results in adverse secondary medical conditions and reduced quality of life. Veterans with SCI who have completed rehabilitation after injury and are unable to ambulate receive a wheelchair as standard of care. Powered exoskeletons are a technology that offers an alternative form of limited mobility by enabling over-ground walking through an external framework for support and computer-controlled motorized hip and knee joints. Few studies have reported the safety and efficacy for use of these devices in the home and community environments, and none evaluated their impact on patient-centered outcomes through a randomized clinical trial (RCT). Absence of reported RCTs for powered exoskeletons may be due to a range of challenges, including designing, statistically powering, and conducting such a trial within an appropriate experimental framework. An RCT for the study of exoskeletal-assisted walking in the home and community environments also requires the need to address key factors such as: avoiding selection bias, participant recruitment and retention, training, and safety concerns, particularly in the home environment. These points are described here in the context of a national, multisite Department of Veterans Affairs Cooperative Studies Program-sponsored trial. The rationale and methods for the study design were focused on providing a template for future studies that use powered exoskeletons or other strategies for walking and mobility in people with immobilization due to SCI.

Increasing access to clinical research using an innovative mobile recruitment approach: The (MoRe) concept

Background

Access to healthcare delivery programs and systems is a primary correlate to the overall health and well-being of Veterans and the general population. Participation in clinical research is a gateway to novel therapies that are intended to address current global health issues. Meeting or exceeding recruitment goals in clinical research is one of the key determinants of the timely and successful completion of a study. The travel and time burdens experienced by study participants are often considered barriers to their enrollment into clinical research. The Department of Veterans Affairs (VA) Cooperative Studies Program (CSP) established a consortium of nine VA medical centers (VAMCs) called the Network of Dedicated Enrollment Sites (NODES). The NODES program provides study site-level expertise and innovative approaches that address challenges to clinical research execution. In alignment with our mission, our program developed an approach to increase study participant access to clinical research through implementing “Mobile Recruitment (MoRe)” units. This manuscript describes the utility and challenges associated with employing this strategy to address three common barriers to clinical research participation: 1) research participant travel burden, 2) participant access to study opportunities, and 3) low participant enrollment.

Methods

A plan to introduce the Mobile Recruitment (MoRe) unit as a recruitment strategy was piloted for a high-volume, observational cohort study and mega biobank in the VA health care system, the “Million Veteran Program (MVP)”. MoRe is a recruitment strategy for CSP research integrating mobile technology and atypical research recruitment locations. Recruitment locations include primary or main VA hospitals and their assigned VA Community-Based Outpatient Clinics (CBOCs). Each Node site (n = 9) received components of the MoRe unit including a laptop, printer, portable cart with storage space, cooler/ice packs for specimen storage and transport. Each site's usage of these components varied based on its respective needs. Activities focused on both VA main facilities and CBOC facilities for recruitment.

Results

Seven of the nine Node sites compared the effectiveness of the MoRe unit on MVP study enrollment outcomes over three-time points: pre-intervention period, intervention period, and post-intervention period. The utilization of MoRe in the intervention period demonstrated a 36.9% increase in enrollment compared to the previous six months (pre-intervention period). There was a 2% enrollment increase at the six-month post-intervention period as compared to the intervention period. When comparing the pre-intervention period to the post-intervention period (duration of eighteen months), enrollment increased by 38.9%.

Conclusion

Five of the seven sites experienced an increase in enrollment during the intervention and post-intervention periods. The two sites without an increase in enrollment experienced various extenuating factors. Characteristics of sites using MoRe included the ability to utilize a smaller, unconventional space, i.e. not a traditional clinical research exam space for recruitment. MoRe was utilized in hospital laboratory space, CBOCs, primary care clinics, and other subspecialty clinics that allowed recruitment activities but did not have dedicated space to offer the research teams for that purpose. This initiative successfully demonstrated the benefit of deploying the unit, proving its utility in cases in where there was a lack of space or alternative workstations for research activities. The implementation of MoRe by NODES as a recruitment strategy for MVP may be transferable to other VA clinical research studies, as well as to other healthcare settings executing similar clinical research activities.

A multicenter randomized placebo controlled trial of rifampin to reduce pedal amputations for osteomyelitis in veterans with diabetes (VA INTREPID)

Background

The prevalence of diabetes mellitus continues to inexorably rise in the United States and throughout the world. Lower limb amputations are a devastating comorbid complication of diabetes mellitus. Osteomyelitis increases the risk of amputation fourfold and commonly presages death.

Antimicrobial therapy for diabetic foot osteomyelitis (DFO) varies greatly, indicating that high quality data are needed to inform clinical decision making. Several small trials have indicated that the addition of rifampin to backbone antimicrobial regimens for osteomyelitis outside the setting of the diabetic foot results in 28 to 42% higher cure rates.

Methods/design

This is a prospective, randomized, double-blind investigation of the addition of 6 weeks of rifampin, 600 mg daily, vs. matched placebo (riboflavin) to standard-of-care, backbone antimicrobial therapy for DFO. The study population are patients enrolled in Veteran Health Administration (VHA), ages ≥18 and ≤ 89 years with diabetes mellitus and definite or probable osteomyelitis of the foot for whom an extended course of oral or intravenous antibiotics is planned. The primary endpoint is amputation-free survival. The primary hypothesis is that using rifampin as adjunctive therapy will lower the hazard rate compared with the group that does not use rifampin as adjunctive therapy. The primary hypothesis will be tested by means of a two-sided log-rank test with a 5% significance level. The test has 90% power to detect a hazard ratio of 0.67 or lower with a total of 880 study participants followed on average for 1.8 years.

Discussion

VA INTREPID will test if a rifampin-adjunctive antibiotic regimen increases amputation-free survival in patients seeking care in the VHA with DFO. A positive finding and its adoption by clinicians would reduce lower extremity amputations and their associated physical and emotional impact and reduce mortality for Veterans and for the general population with diabetic foot osteomyelitis. Given that rifampin-adjunctive regimens are currently employed for therapy for the majority of DFO cases in Europe, and only in a small minority of cases in the United States, the trial results will impact therapeutic decisions, even if the null hypothesis is not rejected.

Trial registration

Registered January 6, 2017 at ClinicalTrials.gov, NCT03012529.

Research Lifecycle to Increase the Substantial Real-world Impact of Research: Accelerating Innovations to Application

BACKGROUND

US health care systems face a growing demand to incorporate innovations that improve patient outcomes at a lower cost. Funding agencies increasingly must demonstrate the impact of research investments on public health. The Learning Health System promotes continuous institutional innovation, yet specific processes to develop innovations for further research and implementation into real-world health care settings to maximize health impacts have not been specified.

OBJECTIVE

We describe the Research Lifecycle and how it leverages institutional priorities to support the translation of research discoveries to clinical application, serving as a broader operational approach to enhance the Learning Health System.

METHODS

Developed by the US Department of Veterans Affairs Office of Research and Development Research-to-Real-World Workgroup, the Research Lifecycle incorporates frameworks from product development, translational science, and implementation science methods. The Lifecycle is based on Workgroup recommendations to overcome barriers to more direct translation of innovations to clinical application and support practice implementation and sustainability.

RESULTS

The Research Lifecycle posits 5 phases which support a seamless pathway from discovery to implementation: prioritization (leadership priority alignment), discovery (innovation development), validation (clinical, operational feasibility), scale-up and spread (implementation strategies, performance monitoring), and sustainability (business case, workforce training). An example of how the Research Lifecycle has been applied within a health system is provided.

CONCLUSIONS

The Research Lifecycle aligns research and health system investments to maximize real-world practice impact via a feasible pathway, where priority-driven innovations are adapted for effective clinical use and supported through implementation strategies, leading to continuous improvement in real-world health care.

Impact of Department of Veterans Affairs Cooperative Studies Program clinical trials on practice guidelines for high blood pressure management

Knowing the extent to which a clinical trial's findings translate into clinical practice can be challenging. One practical approach to estimating a trial's influence on clinical practice can be achieved by assessing how the trial informed relevant clinical practice guidelines (CPGs). Accordingly, the objectives of this study were to provide an overview of all the clinical trials involving the Department of Veterans Affairs (VA) Cooperative Studies Program (CSP) that aimed at informing or resulted in informing the management of high blood pressure and to identify and describe the extent to which these trials informed CPGs for the management of high blood pressure. A total of 26 clinical trials involving the VA CSP were identified. Using bibliographic information, 21 CPGs for the management of hypertension representing over 40 years of treatment recommendations from eight collectives were evaluated to determine how they were informed by trials involving the VA CSP. From 1977 to 2018, 13 of the 26 trials (50.0%) were found to have informed 19 of the 21 CPGs (90.5%) a total of 54 times (mean = 2.6 trial citations per CPG, SD ± 1.8). Clinical trials involving the VA CSP have informed a sizeable proportion of CPGs for the management of high blood pressure over the past 40 years. Because of this impact on the CPGs, these trials are also likely to have had at least moderate influence on clinical practice.

An initiative using informatics to facilitate clinical research planning and recruitment in the VA health care system

Background

Randomized clinical trials are the gold standard for evaluating healthcare interventions and, more generally, add to the medical knowledge related to the treatment, diagnosis and prevention of diseases and conditions. Recent literature continues to identify health informatics methods that can help improve study efficiency throughout the life cycle of a clinical trial. Electronic medical record (EMR) data provides a mechanism to facilitate clinical trial research during the study planning and execution phases, and ultimately, can be utilized to enhance recruitment. The Department of Veterans Affairs (VA) has a strong history of clinical and epidemiological research with over four decades of data collected from Veterans it has served nationwide. The VA Informatics and Computing Infrastructure (VINCI) provides VA research investigators with a nationwide view of high-value VA patient data. Within VA, the Cooperative Studies Program (CSP) Network of Dedicated Enrollment Sites (NODES) is a consortium of nine sites that are part of an embedded clinical research infrastructure intended to provide systematic site-level solutions to issues that arise during the conduct of VA CSP clinical research. This paper describes the collaboration initiated by the Salt Lake City (SLC) node site to bring informatics and clinical trials together to enhance study planning and recruitment within the VA.

Methods

The SLC VA Medical Center physically houses both VINCI and a node site and the co-location of these two groups prompted a natural collaboration on both a local and national level. One of the functions of the SLC NODES is to enhance recruitment and promote the success of CSP projects. VINCI supports these efforts by providing VA researchers access to potential population pools. VINCI can provide 1) feasibility data during study planning, and 2) active patient lists during recruitment. The process for CSP study teams to utilize these services involves regulatory documentation, development of queries, revisions to the initial data request, and ongoing communications with several key study personnel including the requesting research team, study statisticians, and VINCI data managers.

Results

The early efforts of SLC NODES and VINCI aimed to provide patient lists exclusively to the SLC CSP study teams for the following purposes: 1) increasing recruitment for trials that were struggling to meet their respective enrollment goals, and 2) decreasing the time required by study coordinators to complete chart review activities. This effort was expanded to include multiple CSP sites and studies. To date, SLC NODES has facilitated the delivery of these VINCI services to nine active CSP studies.

Conclusion

The ability of clinical trial study teams to successfully plan and execute their respective trials is contingent upon their proficiency in obtaining data that will help them efficiently and effectively recruit and enroll eligible participants. This collaboration demonstrates that the utilization of a model that partners two distinct entities, with similar objectives, was effective in the provision of feasibility and patient lists to clinical trial study teams and facilitation of clinical trial research within a large, integrated healthcare system.

A cross-cutting approach to enhancing clinical trial site success: The Department of Veterans Affairs' Network of Dedicated Enrollment Sites (NODES) model

Background

Recruitment into clinical trials remains a key determinant to study completion and success. While various strategies have been proposed, it is unclear how they apply across different populations, diseases, and/or study goals. The ability to effectively overcome challenges may require different approaches that more broadly focus on addressing obstacles among sites that cannot be overcome by individual studies.

Methods

The Department of Veterans Affairs (VA) Cooperative Studies Program (CSP) established the Network of Dedicated Enrollment Sites (NODES) as a consortium of sites to generate systematic site-level solutions to more efficiently recruit in CSP studies. Initial activities identified priorities and developed approaches through team-based efforts. Metrics were also developed to assess overall network performance.

Results

Network efforts produced several new strategies and best practices for common problems in CSP research. Recruitment strategies included bringing studies to patients and developing data programs using algorithms for finding eligible patients. Efficiency efforts focused on cross-training and standardizing performance reports.

Conclusion

NODES addressed site challenges in clinical trial recruitment and management by taking an overall approach that looked at the system rather than individual studies. Practices and operational changes were implemented for CSP research related to recruitment, staff training and research methodology. The network activities suggest that team-based development of tools and insights may help better identify targets and increase efficiencies for clinical trials recruitment.

The past, present, and future of comparative effectiveness research in the US Department of Veterans Affairs

A particular challenge for the healthcare provider and the patient is to choose among competing therapeutic approaches for a particular condition. Often, the relative benefits and risks of potential therapies are not uniformly available from the existing scientific information. Many have pointed to the need for more comparative effectiveness research (CER) to aide in these decisions. The US Department of Veterans Affairs (VA) has a long history of conducting CER. The success of the VA CER program has been facilitated by several important aspects of scientific infrastructure related to (1) research question refinement, (2) study design, planning and coordination, (3) evidence synthesis, and (4) implementation research. In publications that had VA coauthors in 2 major medical journals, 25% of the published studies were classified as CER. The most frequent categories of study were pharmaceutical and behavioral interventions. In the future, the CER enterprise will move toward increased input from clinicians in research topic choice and enhanced consideration of other methodologies besides the randomized controlled trial.