Role of practice-based research networks in comparative effectiveness research

Integration of Improvement and Implementation Science in Practice-Based Research Networks: a Longitudinal, Comparative Case Study

BACKGROUND

Implementation science (IS) and quality improvement (QI) inhabit distinct areas of scholarly literature, but are often blended in practice. Because practice-based research networks (PBRNs) draw from both traditions, their experience could inform opportunities for strategic IS-QI alignment.

OBJECTIVE

To systematically examine IS, QI, and IS/QI projects conducted within a PBRN over time to identify similarities, differences, and synergies.

DESIGN

Longitudinal, comparative case study of projects conducted in the Oregon Rural Practice-based Research Network (ORPRN) from January 2007 to January 2019.

APPROACH

We reviewed documents and conducted staff interviews. We classified projects as IS, QI, IS/QI, or other using established criteria. We abstracted project details (e.g., objective, setting, theoretical framework) and used qualitative synthesis to compare projects by classification and to identify the contributions of IS and QI within the same project.

KEY RESULTS

Almost 30% (26/99) of ORPRN's projects included IS or QI elements; 54% (14/26) were classified as IS/QI. All 26 projects used an evidence-based intervention and shared many similarities in relation to objective and setting. Over half of the IS and IS/QI projects used randomized designs and theoretical frameworks, while no QI projects did. Projects displayed an upward trend in complexity over time. Project used a similar number of practice change strategies; however, projects classified as IS predominantly employed education/training while all IS/QI and most QI projects used practice facilitation. Projects including IS/QI elements demonstrated the following contributions: QI provides the mechanism by which the principles of IS are operationalized in order to support local practice change and IS in turn provides theories to inform implementation and evaluation to produce generalizable knowledge.

CONCLUSIONS

Our review of projects conducted over a 12-year period in one PBRN demonstrates key synergies for IS and QI. Strategic alignment of IS/QI within projects may help improve care quality and bridge the research-practice gap.

Applying a Commercialization-Readiness Framework to Optimize Value for Achieving Sustainability of an Electronic Health Data Research Network and Its Data Capabilities: The SAFTINet Experience

CONTEXT

Sustaining electronic health data networks and maximizing return on federal investment in their development is essential for achieving national data insight goals for transforming health care. However, crossing the business model chasm from grant funding to self-sustaining viability is challenging.

CASE DESCRIPTION

This paper presents lessons learned in seeking the sustainability of the Scalable Architecture for Federated Translational Inquiries Network (SAFTINet), and electronic health data network involving over 50 primary care practices in three states. SAFTINet was developed with funding from the Agency for Healthcare Research and Quality to create a multi-state network for comparative effectiveness research (CER) involving safety-net patients.

METHODS

Three analyses were performed: (1) a product gap analysis of alternative data sources; (2) a Strengths-Weaknesses-Opportunities-Threat (SWOT) analysis of SAFTINet in the context of competing alternatives; and (3) a customer discovery process involving approximately 150 SAFTINet stakeholders to identify SAFTINet's sustaining value proposition for health services researchers, clinical data partners, and policy makers.

FINDINGS

The results of this business model analysis informed SAFTINet's sustainability strategy. The fundamental high-level product needs were similar between the three primary customer segments: credible data, efficient and easy to use, and relevance to their daily work or 'jobs to be done'. However, how these benefits needed to be minimally demonstrated varied by customer such that different supporting evidence was required.

MAJOR THEMES

The SAFTINet experience illustrates that commercialization-readiness and business model methods can be used to identify multi-sided value propositions for sustaining electronic health data networks and their data capabilities as drivers of health care transformation.

Factors affecting engagement between academic faculty and decision-makers: learnings and priorities for a school of public health

Background

Schools of public health (SPHs) are increasingly being recognised as important contributors of human, social and intellectual capital relevant to health policy and decision-making. Few studies within the implementation science literature have systematically examined knowledge exchange experiences within this specific organisational context. The purpose of this study was therefore to elicit whether documented facilitators and barriers to engaging with government decision-makers resonates within an academic SPH context. We sought to understand the variations in such experiences at four different levels of government decision-making. Furthermore, we sought to elicit intervention priorities as identified by faculty.

Methods

Between May and December 2016, 211 (34%) of 627 eligible full-time faculty across one SPH in the United States of America participated in a survey on engagement with decision-makers at the city, state, federal and global government levels. Surveys were administered face-to-face or via Skype. Descriptive data as well as tests of association and logistic regression analyses were conducted using STATA.

Results

Over three-quarters of respondents identified colleagues with ties to decision-makers, institutional affiliation and conducting policy-relevant research as the highest facilitators. Several identified time constraints, academic incentives and financial support as important contributors to engagement. Faculty characteristics, such as research areas of expertise, career track and faculty rank, were found to be statistically significantly associated with facilitators. The top three intervention priorities that emerged were (1) creating incentives for engagement, (2) providing funding for engagement and (3) inculcating an institutional culture around engagement.

Conclusions

The data suggest that five principal categories of factors – individual characteristics, institutional environment, relational dynamics, research focus and funder policies – affect the willingness and ability of academic faculty to engage with government decision-makers. This study suggests that SPHs could enhance the relevance of their role in health policy decision-making by (1) periodically measuring engagement with decision-makers; (2) enhancing individual capacity in knowledge translation and communication, taking faculty characteristics into account; (3) institutionalising a culture that supports policies and practices for engagement in decision-making processes; and (4) creating a strategy to expand and nurture trusted, relevant networks and relationships with decision-makers.

Practice-Based Research Networks and the Mandate for Real-World Evidence

The 21^st Century Cures Act encourages the Food and Drug Administration to consider "real-world evidence" in its regulation of the safety and efficacy of drugs and devices. Many have interpreted this mandate to focus on non-randomized observational research. However, we suggest that regulatory science must also move from rarefied academic hospitals to community-based settings, where the vast majority of patients in fact receive care in the fragmented U.S. healthcare system. This move is especially important if innovations are to reach, and be validated in, more diverse populations. A solution can be found in the 183 Practiced-Based Research Networks ("PBRN"), i.e., groups of primary care clinicians and practices in all 50 states working to improve clinical care and translate research findings into practice. This symposium contribution seeks to (1) describe some of the common shortcomings of clinical trials, (2) explore the opportunities and challenges posed by use of real-world evidence as a basis for drug and device regulation, (3) briefly describe the history and evolution of PBRNs, and (4) articulate the challenges and opportunities for using PBRNs to fulfill the 21^st Century Cures Act mandate for real-world evidence.

IMPACCT Kids' Care: a real-world example of stakeholder involvement in comparative effectiveness research

The Patient-Centered Outcomes Research Institute has accelerated conversations about the importance of actively engaging stakeholders in all aspects of comparative effectiveness research (CER). Other scientific disciplines have a history of stakeholder engagement, yet few empirical examples exist of how these stakeholders can inform and enrich CER. Here we present a case study which includes the methods used to engage stakeholders, what we learned from them, and how we incorporated their ideas in a CER project. We selected stakeholders from key groups, built relationships with them and collected their feedback through interviews, observation and ongoing meetings during the four research process phases: proposal development, adapting study methods, understanding the context and information technology tool design and refinement.

A national data infrastructure for patient-centered outcomes research

Concerted efforts are underway to improve healthcare decision-making through patient-centered outcomes research. These efforts are supported by the Patient-Centered Outcomes Research Trust Fund, which was established within the Patient Protection and Affordable Care Act. This article focuses on describing national data infrastructure efforts that support patient-centered outcomes research. A national data infrastructure has the potential to decrease research costs and improve research throughput. We describe early and current efforts that demonstrated this potential, how the national effort is utilizing the lessons learned from these predecessor efforts and remaining challenges.

Clinical Comparative Effectiveness Research Through the Lens of Healthcare Decisionmakers

BACKGROUND

Healthcare expenditures in the United States exceed the healthcare expenditures of other countries, yet relatively unfavorable health outcomes persist. Despite the emergence of numerous evidence-based interventions, wide variations in clinical care have caused disparities in quality of care and cost. Comparative effectiveness and cost effectiveness research may better guide healthcare decisionmakers in determining which interventions work best, for which populations, under which conditions, and at what cost.

METHODS

This article reviews national health policies that promote comparative effectiveness research (CER), healthcare decisionmaker roles in CER, methodological approaches to CER, and future implications of CER.

RESULTS

This article provides a brief summary of CER health policy up to the Patient Protection and Affordable Care Act and its establishment of the Patient-Centered Outcomes Research Institute (PCORI). Through PCORI, participatory methods for engaging healthcare decisionmakers in the entire CER process have gained momentum as a strategy for improving the relevance of research and expediting the translation of research into practice. Well-designed, methodologically rigorous observational studies and randomized trials conducted in real-world settings have the potential to improve the quality, generalizability, and transferability of study findings.

CONCLUSION

Learning health systems and practice-based research networks provide the infrastructure for advancing CER methods, generating local solutions to high-quality cost-effective care, and transitioning research into implementation and dissemination science-all of which will ultimately guide health policy on clinical care, payment for care, and population health.

A systematic review of stakeholder engagement in comparative effectiveness and patient-centered outcomes research

OBJECTIVES

We conducted a review of the peer-reviewed literature since 2003 to catalogue reported methods of stakeholder engagement in comparative effectiveness research and patient-centered outcomes research.

METHODS AND RESULTS

We worked with stakeholders before, during and after the review was conducted to: define the primary and key research questions; conduct the literature search; screen titles, abstracts and articles; abstract data from the articles; and analyze the data. The literature search yielded 2,062 abstracts. The review was conducted on 70 articles that reported on stakeholder engagement in individual research projects or programs.

FINDINGS

Reports of stakeholder engagement are highly variable in content and quality. We found frequent engagement with patients, modestly frequent engagement with clinicians, and infrequent engagement with stakeholders in other key decision-making groups across the healthcare system. Stakeholder engagement was more common in earlier (prioritization) than in later (implementation and dissemination) stages of research. The roles and activities of stakeholders were highly variable across research and program reports.

RECOMMENDATIONS

To improve on the quality and content of reporting, we developed a 7-Item Stakeholder Engagement Reporting Questionnaire. We recommend three directions for future research: 1) descriptive research on stakeholder-engagement in research; 2) evaluative research on the impact of stakeholder engagement on the relevance, transparency and adoption of research; and 3) development and validation of tools that can be used to support stakeholder engagement in future work.

Welcome to the Journal of Comparative Effectiveness Research

As the Senior Editors of the Journal of Comparative Effectiveness Research (JCER) we are delighted to present you with the first issue of this exciting new publication, focusing on the socially and clinically impactful field of comparative effectiveness research (CER).

Implementation of a health data-sharing infrastructure across diverse primary care organizations

Practice-based research networks bring together academic researchers and primary care clinicians to conduct research that improves health outcomes in real-world settings. The Washington, Wyoming, Alaska, Montana, and Idaho region Practice and Research Network implemented a health data-sharing infrastructure across 9 clinics in 3 primary care organizations. Following implementation, we identified challenges and solutions. Challenges included working with diverse primary care organizations, adoption of health information data-sharing technology in a rapidly changing local and national landscape, and limited resources for implementation. Overarching solutions included working with a multidisciplinary academic implementation team, maintaining flexibility, and starting with an established network for primary care organizations. Approaches outlined may generalize to similar initiatives and facilitate adoption of health data sharing in other practice-based research networks.

A national strategy to develop pragmatic clinical trials infrastructure

An important challenge in comparative effectiveness research is the lack of infrastructure to support pragmatic clinical trials, which compare interventions in usual practice settings and subjects. These trials present challenges that differ from those of classical efficacy trials, which are conducted under ideal circumstances, in patients selected for their suitability, and with highly controlled protocols. In 2012, we launched a 1-year learning network to identify high-priority pragmatic clinical trials and to deploy research infrastructure through the NIH Clinical and Translational Science Awards Consortium that could be used to launch and sustain them. The network and infrastructure were initiated as a learning ground and shared resource for investigators and communities interested in developing pragmatic clinical trials. We followed a three-stage process of developing the network, prioritizing proposed trials, and implementing learning exercises that culminated in a 1-day network meeting at the end of the year. The year-long project resulted in five recommendations related to developing the network, enhancing community engagement, addressing regulatory challenges, advancing information technology, and developing research methods. The recommendations can be implemented within 24 months and are designed to lead toward a sustained national infrastructure for pragmatic trials.

Characteristics and lessons learned from practice-based research networks (PBRNs) in the United States

Practice-based research networks (PBRNs) are organizations that involve practicing clinicians in asking and answering clinically relevant research questions. This review explores the origins, characteristics, funding, and lessons learned through practice-based research in the United States. Primary care PBRNs emerged in the USA in the 1970s. Early studies explored the etiology of common problems encountered in primary care practices (eg, headache, miscarriage), demonstrating the gap between research conducted in controlled specialty settings and real-world practices. Over time, national initiatives and an evolving funding climate have shaped PBRN development, contributing to larger networks, a push for shared electronic health records, and the use of a broad range of research methodologies (eg, observational studies, pragmatic randomized controlled trials, continuous quality improvement, participatory methods). Today, there are over 160 active networks registered with the Agency for Healthcare Research and Quality's PBRN Resource Center that engage primary care clinicians, pharmacists, dentists, and other health care professionals in research and quality-improvement initiatives. PBRNs provide an important laboratory for encouraging collaborative research partnerships between academicians and practices or communities to improve population health, conduct comparative effectiveness and patient-centered outcomes research, and study health policy reform. PBRNs continue to face critical challenges that include: (1) adapting to a changing landscape; (2) recruiting and retaining membership; (3) securing infrastructure support; (4) straddling two worlds (academia and community) and managing expectations; and (5) preparing for workforce transitions.

Characteristics and lessons learned from practice-based research networks (PBRNs) in the United States

Practice-based research networks (PBRNs) are organizations that involve practicing clinicians in asking and answering clinically relevant research questions. This review explores the origins, characteristics, funding, and lessons learned through practice-based research in the United States. Primary care PBRNs emerged in the USA in the 1970s. Early studies explored the etiology of common problems encountered in primary care practices (eg, headache, miscarriage), demonstrating the gap between research conducted in controlled specialty settings and real-world practices. Over time, national initiatives and an evolving funding climate have shaped PBRN development, contributing to larger networks, a push for shared electronic health records, and the use of a broad range of research methodologies (eg, observational studies, pragmatic randomized controlled trials, continuous quality improvement, participatory methods). Today, there are over 160 active networks registered with the Agency for Healthcare Research and Quality's PBRN Resource Center that engage primary care clinicians, pharmacists, dentists, and other health care professionals in research and quality-improvement initiatives. PBRNs provide an important laboratory for encouraging collaborative research partnerships between academicians and practices or communities to improve population health, conduct comparative effectiveness and patient-centered outcomes research, and study health policy reform. PBRNs continue to face critical challenges that include: (1) adapting to a changing landscape; (2) recruiting and retaining membership; (3) securing infrastructure support; (4) straddling two worlds (academia and community) and managing expectations; and (5) preparing for workforce transitions.