Caring Wisely: A Program to Support Frontline Clinicians and Staff in Improving Healthcare Delivery and Reducing Costs

Employee-Driven Innovation in Health Organizations: Insights From a Scoping Review

BACKGROUND

Employee-driven innovation (EDI) occurs when frontline actors in health organizations use their firsthand experience to spur new ideas to transform care. Despite its increasing prevalence in health organizations, the organizational conditions under which EDI is operationalized have received little scholarly attention.

METHODS

This scoping review identifies gaps and assembles existing knowledge on four questions: What is EDI in health organizations and which frontline actors are involved? What are the characteristics of the EDI process? What contextual factors enable or impede EDI? And what benefits does EDI bring to health organizations? We searched seven databases with keywords related to EDI in health organizations. After screening 1580 studies by title and abstract, we undertook full-text review of 453 articles, retaining 60 for analysis. We performed a descriptive and an inductive thematic analysis guided by the four questions.

RESULTS

Findings reveal an heterogeneous literature. Most articles are descriptive (n = 41). Few studies are conceptual and empirical (n = 15) and four are conference papers. EDI was often described as a participatory, learning innovation process involving frontline clinical and non-clinical staff and managers. Majority EDI were top-down, often driven by the organization's focus on participatory improvement and innovation and research-based initiatives. Five categories of methods is used in top-down EDI, two thirds of which includes a learning, a team and/or a digital component. Hybrid EDI often involves a team-based component. Bottom-up EDI emerged spontaneously from the work of frontline actors. Enablers, barriers, and benefits of EDI are seen at macro, organizational, team and individual levels; some benefits spread to other health organizations and health systems.

CONCLUSION

This scoping review provides a comprehensive understanding of the organizational conditions under which EDI is operationalized. It offers insights for researchers, health organizations, and policy-makers about how and why frontline actors' involvement is crucial for the transformation of care.

Training the next generation of learning health system scientists

Introduction

The learning health system (LHS) aligns science, informatics, incentives, stakeholders, and culture for continuous improvement and innovation. The Agency for Healthcare Research and Quality and the Patient‐Centered Outcomes Research Institute designed a K12 initiative to grow the number of LHS scientists. We describe approaches developed by 11 funded centers of excellence (COEs) to promote partnerships between scholars and health system leaders and to provide mentored research training.

Methods

Since 2018, the COEs have enlisted faculty, secured institutional resources, partnered with health systems, developed and implemented curricula, recruited scholars, and provided mentored training. Program directors for each COE provided descriptive data on program context, scholar characteristics, stakeholder engagement, scholar experiences with health system partnerships, roles following program completion, and key training challenges.

Results

To date, the 11 COEs have partnered with health systems to train 110 scholars. Nine (82%) programs partner with a Veterans Affairs health system and 9 (82%) partner with safety net providers. Clinically trained scholars (n = 87; 79%) include 70 physicians and 17 scholars in other clinical disciplines. Non‐clinicians (n = 29; 26%) represent diverse fields, dominated by population health sciences. Stakeholder engagement helps scholars understand health system and patient/family needs and priorities, enabling opportunities to conduct embedded research, improve outcomes, and grow skills in translating research methods and findings into practice. Challenges include supporting scholars through roadblocks that threaten to derail projects during their limited program time, ranging from delays in access to data to COVID‐19‐related impediments and shifts in organizational priorities.

Conclusions

Four years into this novel training program, there is evidence of scholars' accomplishments, both in traditional academic terms and in terms of moving along career trajectories that hold the potential to lead and accelerate transformational health system change. Future LHS training efforts should focus on sustainability, including organizational support for scholar activities.

Changing the research paradigm for digital transformation in healthcare delivery

The growing focus on healthcare transformation (i.e., new healthcare delivery models) raises interesting issues related to research design, methodology, and funding. More than 20 years have passed since the Institute of Medicine first called for the transition to digital health with a focus on system-wide change. Yet progress in healthcare delivery system change has been painfully slow. A knowledge gap exists; research has been inadequate and critical information is lacking. Despite calls by the National Academies of Science, Engineering, and Medicine for convergent, team-based transdisciplinary research with societal impact, the preponderance of healthcare research and funding continues to support more traditional siloed discipline research approaches. The lack of impact on healthcare delivery suggests that it is time to step back and consider differences between traditional science research methods and the realities of research in the domain of transformational change. The proposed new concepts in research design, methodologies, and funding are a needed step to advance the science. The Introduction looks at the growing gap in expectations for transdisciplinary convergent research and prevalent practices in research design, methodologies, and funding. The second section summarizes current expectations and drivers related to digital health transformation and the complex system problem of healthcare fragmentation. The third section then discusses strengths and weaknesses of current research and practice with the goal of identifying gaps. The fourth section introduces the emerging science of healthcare delivery and associated research methodologies with a focus on closing the gaps between research and translation at the frontlines. The final section concludes by proposing new transformational science research methodologies and offers evidence that suggests how and why they better align with the aims of digital transformation in healthcare delivery and could significantly accelerate progress in achieving them. It includes a discussion of challenges related to grant funding for non-traditional research design and methods. The findings have implications broadly beyond healthcare to any research that seeks to achieve high societal impact.

A roadmap to operationalize and evaluate impact in a learning health system

BACKGROUND

Many health systems invest in initiatives to accelerate translation of knowledge into practice. However, organizations lack guidance on how to develop and operationalize such Learning Health System (LHS) programs and evaluate their impact. Kaiser Permanente Washington (KPWA) launched our LHS program in June 2017 and developed a logic model as a foundation to evaluate the program's impact.

OBJECTIVE

To develop a roadmap for organizations that want to establish an LHS program, understand how LHS core components relate to one another when operationalized in practice, and evaluate and improve their progress.

METHODS

We conducted a narrative review on LHS models, key model components, and measurement approaches.

RESULTS

The KPWA LHS Logic Model provides a broad set of constructs relevant to LHS programs, depicts their relationship to LHS operations, harmonizes terms across models, and offers measurable operationalizations of each construct to guide other health systems. The model identifies essential LHS inputs, provides transparency into LHS activities, and defines key outcomes to evaluate LHS processes and impact. We provide reflections on the most helpful components of the model and identify areas that need further improvement using illustrative examples from deployment of the LHS model during the COVID-19 pandemic.

CONCLUSION

The KPWA LHS Logic Model is a starting point for future LHS implementation research and a practical guide for healthcare organizations that are building, operationalizing, and evaluating LHS initiatives.

Embedded Research in the Learning Healthcare System: Ongoing Challenges and Recommendations for Researchers, Clinicians, and Health System Leaders

Embedded research is an innovative means to improve performance in the learning healthcare system (LHS). However, few descriptions of successful embedded research programs have been published. In this perspective, we describe the Care Improvement Research Team, a mature partnership between researchers and clinicians at Kaiser Permanente Southern California. The program supports a core team of researchers and staff with dedicated resources to partner with health system leaders and practicing clinicians, using diverse methods to identify and rectify gaps in clinical practice. For example, recent projects helped clinicians to provide better care by reducing prescribing of unnecessary antibiotics for acute sinusitis and by preventing readmissions among the elderly. Embedded in operational workgroups, the team helps formulate research questions and enhances the rigor and relevance of data collection and analysis. A recent business-case analysis cited savings to the organization of over $10 million. We conclude that embedded research programs can play a key role in fulfilling the promise of the LHS. Program success depends on dedicated funding, robust data systems, and strong relationships between researchers and clinical stakeholders. Embedded researchers must be responsive to health system priorities and timelines, while clinicians should embrace researchers as partners in problem solving.

Strategies for Building Delivery Science in an Integrated Health Care System

Health systems today have increasing opportunities and imperatives to conduct delivery science, which is applied research that evaluates clinical or organizational practices that systems can implement or encourage. Examples include research on eliminating racial/ethnic disparities in hypertension management and on identifying the types of patients who can successfully use video visits. Clinical leaders and researchers often face barriers to delivery science, including limited funding, insufficient leadership support, lack of engagement between operational and research leaders, limited pools of research expertise, and lack of pathways to identify and develop ideas. We describe five key strategies we employed to address these barriers and develop a portfolio of delivery science programs in Kaiser Permanente Northern California. This portfolio now includes small and medium-sized grant programs, training programs for postdoctoral research fellows and experienced physician researchers, and a dedicated team that partners with clinicians to develop high-priority ideas and conduct small projects. Most of our approaches are consistent with frameworks used to develop delivery science by other health systems; some are innovative. Most of these strategies are adaptable by other health systems prepared to make long-range organizational commitments to mechanisms that foster partnerships between clinical leaders and researchers.

Assessment of a Simulated Case-Based Measurement of Physician Diagnostic Performance

IMPORTANCE

Diagnostic acumen is a fundamental skill in the practice of medicine. Scalable, practical, and objective tools to assess diagnostic performance are lacking.

OBJECTIVE

To validate a new method of assessing diagnostic performance that uses automated techniques to assess physicians' diagnostic performance on brief, open-ended case simulations.

DESIGN, SETTING, AND PARTICIPANTS

Retrospective cohort study of 11 023 unique attempts to solve case simulations on an online software platform, The Human Diagnosis Project (Human Dx). A total of 1738 practicing physicians, residents (internal medicine, family medicine, and emergency medicine), and medical students throughout the United States voluntarily used Human Dx software between January 21, 2016, and January 15, 2017.

MAIN OUTCOMES AND MEASURES

Internal structure validity was assessed by 3 measures of diagnostic performance: accuracy, efficiency, and a combined score (Diagnostic Acumen Precision Performance [DAPP]). These were each analyzed by level of training. Association with other variables' validity evidence was evaluated by correlating diagnostic performance and affiliation with an institution ranked in the top 25 medical schools by US News and World Report.

RESULTS

Data were analyzed for 239 attending physicians, 926 resident physicians, 347 intern physicians, and 226 medical students. Attending physicians had higher mean accuracy scores than medical students (difference, 8.1; 95% CI, 4.2-12.0; P < .001), as did residents (difference, 8.0; 95% CI, 4.8-11.2; P < .001) and interns (difference, 5.9; 95% CI, 2.3-9.6; P < .001). Attending physicians had higher mean efficiency compared with residents (difference, 4.8; 95% CI, 1.8-7.8; P < .001), interns (difference, 5.0; 95% CI, 1.5-8.4; P = .001), and medical students (difference, 5.4; 95% CI, 1.4-9.3; P = .003). Attending physicians also had significantly higher mean DAPP scores than residents (difference, 2.6; 95% CI, 0.0-5.2; P = .05), interns (difference, 3.6; 95% CI, 0.6-6.6; P = .01), and medical students (difference, 6.7; 95% CI, 3.3-10.2; P < .001). Attending physicians affiliated with a US News and World Report-ranked institution had higher mean DAPP scores compared with nonaffiliated attending physicians (80 [95% CI, 77-83] vs 72 [95% CI, 70-74], respectively; P < .001). Resident physicians affiliated with an institution ranked in the top 25 medical schools by US News and World Report also had higher mean DAPP scores compared with nonaffiliated peers (75 [95% CI, 73-77] vs 71 [95% CI, 69-72], respectively; P < .001).

CONCLUSIONS AND RELEVANCE

The data suggest that diagnostic performance is higher in those with more training and that DAPP scores may be a valid measure to appraise diagnostic performance. This diagnostic assessment tool allows individuals to receive immediate feedback on performance through an openly accessible online platform.