Strategies for Building Delivery Science in an Integrated Health Care System

Codesign Use in Palliative Care Intervention Development: A Systematic Review

CONTEXT

Codesign is a methodology that includes active collaboration between stakeholders in designing solutions and has been used in the development and implementation of palliative care (PC) interventions.

OBJECTIVES

To synthesize the state of evidence for codesign in the development of PC interventions.

METHODS

We searched PubMed, EMBASE, and CINAHL for peer-reviewed studies published after 1995 that reported evidence of codesigned interventions and outcomes in patients receiving palliative, hospice, or end-of-life care. We screened studies through independent and blinded dual review within Covidence and assessed study quality with the 2018 Mixed Methods Appraisal Tool. We narratively synthesized codesign duration, engagement approach, stakeholders involved, intervention designs, follow-ups, and outcomes, comparing among codesigns reporting meaningful improvement in outcomes. We created a best practice checklist which we used to evaluate codesign use in each study.

RESULTS

About 1,036 abstracts and 54 full text articles were screened. Twenty-eight studies met inclusion criteria and were abstracted. Feedback collection modalities ranged from iterative drafting, pilot testing, advisory panels, workshops, focus groups, and interviews. Thirteen studies applied pretesting/prototyping through pretest post-test, focus groups, prototypes, alpha and beta testing, and mock-ups. Eleven studies reported improved outcomes, eight of which utilized iterative codesign. All the studies reporting improved outcomes mentioned meeting with stakeholders at least twice. Two studies met all criteria in our codesign best practice checklist.

CONCLUSION

Codesigned PC interventions demonstrate high variance in the modality of acquiring feedback and application of codesign. Successful codesign leading to improvement in outcomes is achieved by involving patients, caregivers, and providers in iterating intervention design.

Effective policy research of county and township health sector integration in China: Empirical evidence from the difference-in-differences model

Medical service fragmentation is a common problem worldwide, and many countries have adopted integration to solve the difficulty. Contrary to developed countries, developing countries such as China must consider how to implement integration under a relatively weak medical foundation. This study aims to evaluate the effect of the "Compact Union of County and Township Health Sectors" policy on the medical service capacity of a typical integration model represented by Shanxi Province in China and determine the path the policy followed. By using Shanxi's county-level medical integration as a quasi-natural experiment, this study establishes a difference-in-differences model to investigate the effect of the policy using official data. A series of tests are conducted to verify the robustness of the result. Finally, the policy pathway is tested. The results show that the third-level surgeries and outpatient service utilization of leading hospitals and township institutions increased. Still, inpatient service utilization and fourth-level surgeries did not show a significant change in either type of institution. Moreover, the enhancement of leading hospitals' service capacity comes mainly through improving asset efficiency and personal income, while the improvement of township institutions' capacity comes primarily through increased personal income. Compact integration of county-level medical institutions can stimulate and improve service capacity by improving asset efficiency and personal income, even with a weak medical foundation. However, to achieve continuous service capacity improvement, the professional level of county-level institutions must be strengthened with a superior hospital's assistance, and personnel's enthusiasm for active innovation must be cultivated.

Development and use of a novel tool for assessing and improving researcher embeddedness in learning health systems and applied system improvements

This paper outlines the development, deployment and use, and testing of a tool for measuring and improving healthcare researcher embeddedness - i.e., being connected to and engaged with key leverage points and stakeholders in a health system. Despite the widely acknowledged importance of embeddedness for learning health systems and late-stage translational research, we were not aware of useful tools for addressing and improving embeddedness in scholar training programs. We developed the MN-LHS Embeddedness Tool covering connections to committees, working groups, leadership, and other points of contact across four domains: patients and caregivers; local practice (e.g., operations and workflows); local institutional research (e.g., research committees and agenda- or initiative-setting groups); and national (strategic connections within professional groups, conferences, etc.). We used qualitative patterns and narrative findings from 11 learning health system training program scholars to explore variation in scholar trajectories and the embeddedness tool's usefulness in scholar professional development. Tool characteristics showed moderate evidence of construct validity; secondarily, we found significant differences in embeddedness, as a score, from baseline through program completion. The tool has demonstrated simple, practical utility in making embeddedness an explicit (rather than hidden) part of applied and learning health system researcher training, alongside emerging evidence for validity.

Generalizability of an EHR-network dataset to the United States for cardiovascular disease conditions: Comparison of Cerner real world data with the national inpatient sample

BACKGROUND

Electronic Health Record (EHR) data from health systems are increasingly being combined for clinical research purposes. Yet, it remains unclear whether these large EHR data sources provide a representative assessment of national disease prevalence and treatment. To evaluate this, we compared Cerner RealWorldData (CRWD), a large EHR data source, to those seen in the National Inpatient Sample (NIS) for 3 cardiovascular conditions (myocardial infarction (MI), congestive heart failure (CHF), and stroke.

METHODS

Adult patients (age ≥18 years) hospitalized with MI, CHF, and stroke were identified in both CRWD (86 health systems) and the NIS (4,782 hospitals). Patient demographics, comorbidities, procedures, outcomes (length of stay and in-hospital mortality) and hospital type (teaching or nonteaching) were compared between NIS and CRWD patients.

RESULTS

Of 86 health systems participating in CRWD, 33 were excluded for potential data quality issues which accounted for about 11% of hospitalizations in the dataset, leaving 53 for inclusion in analysis which accounted for about 89% of hospitalizations in the dataset. Between January 1, 2017 and December 31, 2018, 116,956 MI, 188,107 CHF, and 93,968 stroke hospitalizations were identified in CRWD vs 2,245,300 MI, 4,310,745 CHF, and 1,333,480 stroke hospitalizations in the NIS. Patient demographics were similar among patients in CWRD and the NIS for all 3 cardiovascular groups except for ethnicity, with underrepresentation of Hispanic individuals in CRWD vs the NIS. Patients hospitalized in CRWD had a slightly higher proportion of coded co-morbidities compared with NIS hospitalizations due to a longer potential look-back period. For patients with MI, hospital mortality, length of stay, coronary artery bypass graft (CABG) rates, and percutaneous coronary intervention (PCI) rates were similar between CRWD and NIS. Additionally, there was similar in hospital mortality and length of stay for those with CHF and stroke hospitalizations between CRWD and NIS.

CONCLUSIONS

On aggregate, characteristics of hospitalizations for MI, CHF, and stroke using EHR data from one nationwide EHR-derived database, CRWD, appears similar to characteristics of hospitalizations in the nationally representative NIS. Important limitations of CRWD include lack of geographic representativeness, under-representation of Hispanic adults, and the need to exclude health systems for missing data.

The implementation checklist: A pragmatic instrument for accelerating research-to-implementation cycles

Introduction

Learning health systems require rapid-cycle research and nimble implementation processes to maximize innovation across disparate specialties and operations. Existing detailed research-to-implementation frameworks require extensive time commitments and can be overwhelming for physician-researchers with clinical and operational responsibilities, inhibiting their widespread adoption. The creation of a short, pragmatic checklist to inform implementation processes may substantially improve uptake and implementation efficiency across a variety of health systems.

Methods

We conducted a systematic review of existing implementation frameworks to identify core concepts. Utilizing comprehensive stakeholder engagement with 25 operational leaders, embedded physician-researchers, and delivery scientists, concepts were iteratively integrated to create and implement a final concise instrument.

Results

A systematic review identified 894 publications describing implementation frameworks, which included 15 systematic reviews. Among these, domains were extracted from three commonly utilized instruments: the Quality Implementation Framework (QIF), the Consolidated Framework for Implementation Research (CFIR), and the Reach, Effectiveness, Adoption, Implementation, and Maintenance (RE-AIM) framework. Iterative testing and stakeholder engagement revision of a four-page draft implementation document with five domains resulted in a concise, one-page implementation planning instrument to be used at project outset and periodically throughout project implementation planning. The instrument addresses end-user feasibility concerns while retaining the main goals of more complex tools. This instrument was then systematically integrated into projects within the Kaiser Permanente Northern California Delivery Science and Applied Research program to address stakeholder engagement, efficiency, project planning, and operational implementation of study results.

Conclusion

A streamlined one-page implementation planning instrument, incorporating core concepts of existing frameworks, provides a pragmatic, robust framework for evidence-based healthcare innovation cycles that is being broadly implemented within a learning health system. These streamlined processes could inform other settings needing a best practice rapid-cycle research-to-implementation tool for large numbers of diverse projects.

Changing the research paradigm for digital transformation in healthcare delivery

The growing focus on healthcare transformation (i.e., new healthcare delivery models) raises interesting issues related to research design, methodology, and funding. More than 20 years have passed since the Institute of Medicine first called for the transition to digital health with a focus on system-wide change. Yet progress in healthcare delivery system change has been painfully slow. A knowledge gap exists; research has been inadequate and critical information is lacking. Despite calls by the National Academies of Science, Engineering, and Medicine for convergent, team-based transdisciplinary research with societal impact, the preponderance of healthcare research and funding continues to support more traditional siloed discipline research approaches. The lack of impact on healthcare delivery suggests that it is time to step back and consider differences between traditional science research methods and the realities of research in the domain of transformational change. The proposed new concepts in research design, methodologies, and funding are a needed step to advance the science. The Introduction looks at the growing gap in expectations for transdisciplinary convergent research and prevalent practices in research design, methodologies, and funding. The second section summarizes current expectations and drivers related to digital health transformation and the complex system problem of healthcare fragmentation. The third section then discusses strengths and weaknesses of current research and practice with the goal of identifying gaps. The fourth section introduces the emerging science of healthcare delivery and associated research methodologies with a focus on closing the gaps between research and translation at the frontlines. The final section concludes by proposing new transformational science research methodologies and offers evidence that suggests how and why they better align with the aims of digital transformation in healthcare delivery and could significantly accelerate progress in achieving them. It includes a discussion of challenges related to grant funding for non-traditional research design and methods. The findings have implications broadly beyond healthcare to any research that seeks to achieve high societal impact.

Innovation Centers in Health Care Delivery Systems: Structures for Success

The need to support innovation in health care delivery was prompted by payment reforms and access to digital tools and has been accelerated by the shift to virtual care as part of the COVID-19 pandemic response. Prior to the pandemic, a growing number of health systems set up innovation centers to focus on creating new services and exploring new business models relevant to value-based care. This is distinct from process improvement or implementation science, and often needs a different set of incentives to succeed within a large organization. We used a national survey to identify a diverse sample of innovation centers, and interviewed leaders to describe their aims, organizational structures, and activities. They all aim to improve patient outcomes and experience while reducing costs, but their strategic focus may differ. The centers also vary in their reporting structure, how they build internal capacity, and how they measure success. We highlight the range of strategies through examples of projects that improve quality, reduce costs, and generate new revenue. While the optimal forms and impact of innovation centers are still emerging, the fiscal pressures and the rapid uptake of digital technologies present opportunities for the redesign of health services in the postpandemic era. The experiences of these centers illustrate a set of approaches to increase any organization’s capacity for innovation.

Embedded Health Systems Science as a driver of care improvement within an integrated delivery organization

BACKGROUND

Embedded Health Systems Science (HSS) has the potential to reduce gaps between research and delivery of evidence-based medicine. Models are needed to guide the development of embedded HSS in health care delivery organizations particularly with the rise of value-based care.

METHODS

The development of HSS infrastructure at a large nonprofit health care delivery organization is described, along with an embedded HSS diabetes study to illustrate the integration of program specific data, electronic health records, and health care system data infrastructure. To compare diabetes outcomes across four evidenced-based programs, a control group was developed from EHR data using propensity score matching. Mixed effect adjusted models were used to estimate reductions in hemoglobin A1c (HbA1c) and body weight.

RESULTS

Adjusted analyses using an EHR derived comparison group demonstrated significantly different findings than unadjusted pre to post analyses. The embedded HSS study indicates that appropriate statistical methods, staff with required expertise, and integration with health system data infrastructure are needed to develop timely and rigorous HSS outcomes that effectively improve patient care.

CONCLUSIONS

Embedded HSS has the potential to inform value-based care models and contribute to evidence-based medicine approaches that improve patient care. Although developing system wide integrated data structures and staff with the appropriate skills requires substantial effort, the outcome is more reliable evaluations that lead to higher quality and higher value care.

IMPLICATIONS

Health care delivery organizations can improve patient care by dedicating resources to embed HSS into its routine operations.

A prognostic information system for real-time personalized care: Lessons for embedded researchers

Embedded researchers could play a central role in developing tools to personalize care using electronic medical records (EMRs). However, few studies have described the steps involved in developing such tools, or evaluated the key factors in success and failure. This case study describes how we used an EMR-derived data warehouse to develop a prototype informatics tool to help oncologists counsel patients with pancreatic cancer about their prognosis. The tool generated real-time prognostic information based on tumor type and stage, age, comorbidity status and lab tests. Our multidisciplinary team included embedded researchers, application developers, user experience experts, and an oncologist leader.This prototype succeeded in establishing proof of principle, but did not reach adoption into actual practice. In pilot testing, oncologists succeeded in generating prognostic information in real time. A few found it helpful in patient encounters, but all identified critical areas for further development before implementation. Generalizable lessons included the need to (1) include a wide range of potential use cases and stakeholders when selecting use cases for such tools; (2) develop talking points for clinicians to explain results from predictive tools to patients; (3) develop ways to reduce lag time between events and data availability; and (4) keep the options presented in the user interface very simple. This case demonstrates that embedded researchers can lead collaborations using EMR-derived data to create systems for real-time personalized patient counseling, and highlights challenges that such teams can anticipate.

The Primary Care Analytics Team: Integrating research and clinical care within the Veterans Health Administration Office of Primary Care

By designing and evaluating health system improvements and providing evidence to clinical decision-makers, embedded researchers are a critical part of a Learning Health System (LHS). In this article, we describe the evolution and mission of the Primary Care Analytics Team (PCAT), an integrated research team within the Veterans Health Administration Office of Primary Care. We discuss challenges and strategies for success in working with clinical operations partners and provide recommendations for other Learning Health Systems units embedded in large integrated health care organizations.

Accelerating learning healthcare system development through embedded research: Career trajectories, training needs, and strategies for managing and supporting embedded researchers

BACKGROUND

Health systems and organizations seeking to achieve learning healthcare system principles are increasingly relying on embedded research teams to optimize delivery of evidence-based, high-quality care that improves patient and staff experience alike. However, building organizational capacity to conduct and benefit from embedded research may be challenging in the absence of clearer guidance on career pathways and training, as well as strategies for managing and supporting this unique workforce.

METHODS

In February 2018, 115 attendees from multiple agencies, institutions and professional societies participated in a conference to accelerate development of learning healthcare systems through embedded research. Workgroups engaged in structured brainstorming discussions of key domains; 21 diverse members focused on strengthening the embedded research community through more explicit development and support of multilevel career trajectories.

RESULTS

Emphasis emerged on the need for training that goes beyond traditional curricula in rigorous scientific methods to include leadership, communication, and other organizational and business skills rarely offered in research training programs. These skills are required for effective engagement of multilevel stakeholders supporting evidence-based changes in routine care. Improving readiness of other stakeholders to effectively act on evidence was noted as equally crucial, as was creation of mid-career development opportunities for researchers and implementers.

CONCLUSIONS

Further development and support of the embedded research workforce will require explicit attention to novel training programs and support of researchers and the stakeholders in the systems they aim to improve.

IMPLICATIONS

Strategies for improving career entry and mastery of skills that foster effective multilevel stakeholder engagement hold promise for strengthening the embedded research community and their contributions to systematic improvements in health and health care.

Embedded Research in the Learning Healthcare System: Ongoing Challenges and Recommendations for Researchers, Clinicians, and Health System Leaders

Embedded research is an innovative means to improve performance in the learning healthcare system (LHS). However, few descriptions of successful embedded research programs have been published. In this perspective, we describe the Care Improvement Research Team, a mature partnership between researchers and clinicians at Kaiser Permanente Southern California. The program supports a core team of researchers and staff with dedicated resources to partner with health system leaders and practicing clinicians, using diverse methods to identify and rectify gaps in clinical practice. For example, recent projects helped clinicians to provide better care by reducing prescribing of unnecessary antibiotics for acute sinusitis and by preventing readmissions among the elderly. Embedded in operational workgroups, the team helps formulate research questions and enhances the rigor and relevance of data collection and analysis. A recent business-case analysis cited savings to the organization of over $10 million. We conclude that embedded research programs can play a key role in fulfilling the promise of the LHS. Program success depends on dedicated funding, robust data systems, and strong relationships between researchers and clinical stakeholders. Embedded researchers must be responsive to health system priorities and timelines, while clinicians should embrace researchers as partners in problem solving.

The Stanford Lightning Report Method: A comparison of rapid qualitative synthesis results across four implementation evaluations

BACKGROUND

Current evaluation methods are mismatched with the speed of health care innovation and needs of health care delivery partners. We introduce a qualitative approach called the lightning report method and its specific product-the "Lightning Report." We compare implementation evaluation results across four projects to explore report sensitivity and the potential depth and breadth of lightning report method findings.

METHODS

The lightning report method was refined over 2.5 years across four projects: team-based primary care, cancer center transformation, precision health in primary care, and a national life-sustaining decisions initiative. The novelty of the lightning report method is the application of Plus/Delta/Insight debriefing to dynamic implementation evaluation. This analytic structure captures Plus ("what works"), Delta ("what needs to be changed"), and Insights (participant or evaluator insights, ideas, and recommendations). We used structured coding based on implementation science barriers and facilitators outlined in the Consolidated Framework for Implementation Research (CFIR) applied to 17 Lightning Reports from four projects.

RESULTS

Health care partners reported that Lighting Reports were valuable, easy to understand, and they implied reports supported "corrective action" for implementations. Comparative analysis revealed cross-project emphasis on the domains of Inner Setting and Intervention Characteristics, with themes of communication, resources/staffing, feedback/reflection, alignment with simultaneous interventions and traditional care, and team cohesion. In three of the four assessed projects, the largest proportion of coding was to the clinic-level domain of Inner Setting-ranging from 39% for the cancer center project to a high of 56% for the life-sustaining decisions project.

CONCLUSIONS

The lightning report method can fill a gap in rapid qualitative approaches and is generalizable with consistent but flexible core methods. Comparative analysis suggests it is a sensitive tool, capable of uncovering differences and insights in implementation across projects. The Lightning Report facilitates partnered evaluation and communication with stakeholders by providing real-time, actionable insights in dynamic health care implementations.

Implementation outcomes and strategies for depression interventions in low- and middle-income countries: a systematic review

BACKGROUND

We systematically reviewed implementation research targeting depression interventions in low- and middle-income countries (LMICs) to assess gaps in methodological coverage.

METHODS

PubMed, CINAHL, PsycINFO, and EMBASE were searched for evaluations of depression interventions in LMICs reporting at least one implementation outcome published through March 2019.

RESULTS

A total of 8714 studies were screened, 759 were assessed for eligibility, and 79 studies met inclusion criteria. Common implementation outcomes reported were acceptability (n = 50; 63.3%), feasibility (n = 28; 35.4%), and fidelity (n = 18; 22.8%). Only four studies (5.1%) reported adoption or penetration, and three (3.8%) reported sustainability. The Sub-Saharan Africa region (n = 29; 36.7%) had the most studies. The majority of studies (n = 59; 74.7%) reported outcomes for a depression intervention implemented in pilot researcher-controlled settings. Studies commonly focused on Hybrid Type-1 effectiveness-implementation designs (n = 53; 67.1), followed by Hybrid Type-3 (n = 16; 20.3%). Only 21 studies (26.6%) tested an implementation strategy, with the most common being revising professional roles (n = 10; 47.6%). The most common intervention modality was individual psychotherapy (n = 30; 38.0%). Common study designs were mixed methods (n = 27; 34.2%), quasi-experimental uncontrolled pre-post (n = 17; 21.5%), and individual randomized trials (n = 16; 20.3).

CONCLUSIONS

Existing research has focused on early-stage implementation outcomes. Most studies have utilized Hybrid Type-1 designs, with the primary aim to test intervention effectiveness delivered in researcher-controlled settings. Future research should focus on testing and optimizing implementation strategies to promote scale-up of evidence-based depression interventions in routine care. These studies should use high-quality pragmatic designs and focus on later-stage implementation outcomes such as cost, penetration, and sustainability.