Kaiser Permanente's performance improvement system, Part 4: Creating a learning organization

Learning health systems in low-income and middle-income countries: exploring evidence and expert insights

Introduction

Learning health systems (LHS) is a multifaceted subject. This paper reviewed current concepts as well as real-world experiences of LHS, drawing on published and unpublished knowledge in order to identify and describe important principles and practices that characterise LHS in low/middle-income country (LMIC) settings.

Methods

We adopted an exploratory approach to the literature review, recognising there are limited studies that focus specifically on system-wide learning in LMICs, but a vast set of connected bodies of literature. 116 studies were included, drawn from an electronic literature search of published and grey literature. In addition, 17 interviews were conducted with health policy and research experts to gain experiential knowledge.

Results

The findings were structured by eight domains on learning enablers. All of these interact with one another and influence actors from community to international levels.

We found that learning comes from the connection between information, deliberation, and action. Moreover, these processes occur at different levels. It is therefore important to consider experiential knowledge from multiple levels and experiences. Creating spaces and providing resources for communities, staff and managers to deliberate on their challenges and find solutions has political implications, however, and is challenging, particularly when resources are constrained, funding and accountability are fragmented and the focus is short-term and narrow. Nevertheless, we can learn from countries that have managed to develop institutional mechanisms and human capacities which help health systems respond to changing environments with ‘best fit’ solutions.

Conclusion

Health systems are knowledge producers, but learning is not automatic. It needs to be valued and facilitated. Everyday governance of health systems can create spaces for reflective practice and learning within routine processes at different levels. This article highlights important enablers, but there remains much work to be done on developing this field of knowledge.

Aiming for quality: a global compass for national learning systems

INTRODUCTION

Transforming a health system into a learning one is increasingly recognized as necessary to support the implementation of a national strategic direction on quality with a focus on frontline experience. The approach to a learning system that bridges the gap between practice and policy requires active exploration.

METHODS

This scoping review adapted the methodological framework for scoping studies from Arksey and O'Malley. The central research question focused on common themes for learning to improve the quality of health services at all levels of the national health system, from government policy to point-of-care delivery.

RESULTS

A total of 3507 records were screened, resulting in 101 articles on strategic learning across the health system: health professional level (19%), health organizational level (15%), subnational/national level (26%), multiple levels (35%), and global level (6%). Thirty-five of these articles focused on learning systems at multiple levels of the health system. A national learning system requires attention at the organizational, subnational, and national levels guided by the needs of patients, families, and the community. The compass of the national learning system is centred on four cross-cutting themes across the health system: alignment of priorities, systemwide collaboration, transparency and accountability, and knowledge sharing of real-world evidence generated at the point of care.

CONCLUSION

This paper proposes an approach for building a national learning system to improve the quality of health services. Future research is needed to validate the application of these guiding principles and make improvements based on the findings.

Relational interventions for organizational learning: An experience report

INTRODUCTION

Quality improvement and implementation science practitioners identify relational issues as important obstacles to success. Relational interventions may be important for successful performance improvement and fostering Learning Health Systems.

METHODS

This case report describes the experience and lessons learned from implementing a relational approach to organizational change, informed by Relational Coordination Theory, in a health system. Structured interviews were used to obtain qualitative participant feedback. Relational Coordination was measured serially using a validated seven-item survey.

RESULTS

An initial, relational intervention on one unit promoted increased participant engagement, self-efficacy, and motivation that led to the spontaneous, emergent dissemination of relational change, and learning into other parts of the health system. Staff involved in the intervention reported increased systems thinking, enhanced focus on communication and relationships as key drivers for improvement and learning, and greater awareness of organizational change as something co-created by staff and executives.

CONCLUSIONS

This experience supports the hypothesis that relational interventions are important for fostering the development of Learning Health Systems.

Validation of Dimensions of Learning Organization Questionnaire (DLOQ) in health care setting in Greece

Background: In health care systems the organizational learning is a continuous process to improve actions through better knowledge and detect or correct errors. This study examines the validity and reliability of the Dimensions of Learning Organization Questionnaire in addition to the applicability of the instrument in a Greek health care cultural context. Design and methods: A cross-sectional study was carried out in 6 general hospitals of Attica and the sampling scheme was the stratified sampling. Questionnaires were distributed to 487 healthcare professionals and 380 valid questionnaires were returned. The research tool used in this survey is the extensive form of DLOQ, which has been adapted and translated into Greek language. Data analysis was carried out with SPSS 25. Results: Three leading experts of the health sector examined the face validity assessment of the translated DLOQ and stated that it is characterized by high face validity. As regard construct validity of the DOLQ throughout Multitrait-Multimethod Matrix, was proved that all the variables of the same factors are statistically significantly correlated (p<0.001), and their correlation coefficients have moderate to high power ranging between 0.563 and 0.798. Moreover, the discriminant validity was demonstrated as certain correlation coefficients between variables of different factors were found to be higher than of certain correlation coefficients between variables of the same factors. The internal consistency among the items of the DOLQ range between 0.842 and 0.977 and they are considered to be good to excellent. Conclusions: Our results confirmed that DLOQ is a valuable tool in measuring Learning Organizational in Greek public hospitals.

Multi-level analysis of the learning health system: Integrating contributions from research on organizations and implementation

Introduction

Organizations and systems that deliver health care may better adapt to rapid change in their environments by acting as learning organizations and learning health systems (LHSs). Despite widespread recognition that multilevel forces shape capacity for learning within care delivery organizations, there is no agreed-on, comprehensive, multilevel framework to inform LHS research and practice.

Methods

We develop such a framework, which can enhance both research on LHSs and practical steps toward their development. We draw on existing frameworks and research within organization and implementation science and synthesize contributions from three influential frameworks: the Consolidated Framework for Implementation Research, the social-ecological framework, and the organizational change framework. These frameworks come, respectively, from the fields of implementation science, public health, and organization science.

Results

Our proposed integrative framework includes both intraorganizational levels (individual, team, mid-management, organization) and the operating and general environments in which delivery organizations operate. We stress the importance of examining interactions among influential factors both within and across system levels and focus on the effects of leadership, incentives, and culture. Additionally, we indicate that organizational learning depends substantially on internal and cross-level alignment of these factors. We illustrate the contribution of our multilevel perspective by applying it to the analysis of three diverse implementation initiatives that aimed at specific care improvements and enduring system learning.

Conclusions

The framework and perspective developed here can help investigators and practitioners broadly scan and then investigate forces influencing improvement and learning and may point to otherwise unnoticed interactions among influential factors. The framework can also be used as a planning tool by managers and practitioners.

An Organization-Specific and Modifiable Inpatient Safety Composite Measure

In early 2013, seeking to apply the principles of value-based purchasing to all Kaiser Permanente hospitals as part of an existing organizationwide value-based performance incentive plan, Kaiser Permanente developed an inpatient safety composite measure that tracks hospital-level performance improvement related to 10 key inpatient safety events. The elements of the composite are weighted equally, and the tool draws on scoring methodologies used by the National Committee for Quality Assurance and the Centers for Medicare & Medicaid Services Hospital Inpatient Value-Based Purchasing Program. Two years after implementation of the composite measure, hospitals experienced improvement across 9 of the 10 adverse events assessed, though only one improvement achieved statistical significance. The measure successfully distinguishes four levels of improvement and is broadly applicable to hospitals and hospital systems.

Building an oncology community of practice to improve cancer care

Background

Communities of practice (cops) have been shown to be effective models for achieving quality outcomes in health care.

Objective

Here, we describe the application of the cop model to the Canadian oncology context.

Methods

We established an oncology cop at our urban community hospital and its networks. Goals were to decrease barriers to access, foster collaboration, and improve knowledge of guidelines in cancer care. We hosted 6 in-person multidisciplinary meetings, focusing on screening, diagnosis, and management of common solid tumours. Health care providers affiliated with our hospital were invited to attend and to complete post-meeting surveys. Likert scales assessed whether cop goals were realized.

Results

Meetings attracted a mean of 57 attendees (range: 48-65 attendees), with a mean of 84% completing the surveys and consenting to the analysis. Attendees included family physicians (mean: 41%), specialist physicians (mean: 24%), nurses (mean: 10%), and allied health care providers (mean: 22%). Repeat attendance increased during the series, with 85% of attendees at the final meeting having attended 1 or more prior meetings. Across the series, most participants agreed or strongly agreed that the cop reduced barriers (mean: 76.0% ± 7.9%) and improved access to cancer care services (mean: 82.4% ± 8.1%) and subject matter experts (mean: 91.7% ± 4.2%); fostered teamwork (mean: 84.5% ± 6.8%) and a culture of collaboration (mean: 94.8% ± 4.2%); improved knowledge of cancer care services (mean: 93.3% ± 4.8%), standards of practice (mean: 92.3% ± 3.1%), and quality indicators (mean: 77.5% ± 6.3%); and improved cancer-related practice (mean: 88.8% ± 4.6%) and satisfaction in caring for cancer patients (mean: 82.9% ± 6.8%). Participant feedback carried a potential for bias.

Conclusions

We demonstrated the feasibility of oncology cops and found that participants perceived their value in reducing barriers to access, fostering collaboration, and improving knowledge of guidelines in cancer care.

Learning from implementation setbacks: Identifying and responding to contextual challenges

Introduction

We address organizational learning about implementation context during setbacks to primary care redesign in an ambulatory system. The redesign expanded care teams and added a medical assistant assigned administrative and coordination tasks. The redesign was expected to improve care efficiency, prevention, and continuity. In response to setbacks, redesign and system leaders used understanding of context to plan system-wide changes, as well as program adjustments. Doing so enhanced the redesign's prospects and contributed to system learning.

Methods

We conducted a 33-month, mixed-methods study. Qualitative data included quarterly calls with the redesign leaders and 63 activity log entries. There were three site visits; 73 interviews with practice leaders, providers, and medical assistants. Data analysis used categories from an implementation research framework; these were refined and then expanded inductively using log reports, debriefings with change leaders, and documents. Quantitative analysis used system operational data on chronic care, prevention, efficiency, productivity, and patient access.

Results

Redesigned teams were not implemented as widely or rapidly as anticipated and did not deliver hoped-for gains in operational metrics. Interviews reported that team redesign was leading to improvements in chronic care and prevention and eased provider burden. Besides making small adjustments to cope with setbacks, redesign and system leaders engaged in more thorough organizational learning. They examined contextual challenges underlying setbacks and posing risks to the delivery system as a whole. Their responses to challenges helped strengthen the redesign's prospects, improved the delivery system's position in its labor market, and helped the system prepare to meet emerging requirements for value-based care and population health.

Conclusions

This case points to benefits for both health care researchers and change practitioners of paying closer attention to how context affects implementation of organizational change, and to opportunities and conditions for learning from setbacks during change.

Scaling up improvements more quickly and effectively

Faster and more widespread implementation could help more patients to benefit more quickly from known effective treatments. So could more effective implementation of better assessment methods, service delivery models, treatments and services. Implementation at scale and 'descaling' are ways for hospitals and health systems to respond to rising demands and costs. The paper proposes ways to provide leaders with the information that would help them to decide whether and how to scale up a proven improvement. We draw on our knowledge of the improvement and implementation literature on the subject and on our experience of scale up programs in Kaiser Permanente, in Swedish county health systems, and in international health. We describe a '3S' scale up infrastructure and other ingredients that appear necessary for successful widespread improvement, and list the resources that we have found useful for developing scale up programs. The paper aims to encourage more actionable research into scale up, and shows the opportunities for researchers to both advance implementation and improvement science and contribute to reducing suffering and costs in a more timely and effective way.

Telemedicine in Primary Care for Patients With Chronic Conditions: The ValCrònic Quasi-Experimental Study

Background

The increase of chronic diseases prevalence has created the need to adapt care models and to provide greater home supervision.

Objective

The objective of our study was to evaluate the impact of telemonitoring on patients with long-term conditions at high risk for rehospitalization or an emergency department visit, in terms of target disease control (diabetes, hypertension, heart failure, and chronic obstructive pulmonary disease).

Methods

We conducted a quasi-experimental study with a before-and-after analysis to assess the effectiveness of the ValCrònic program after 1 year of primary care monitoring. The study included high-risk patients with 1 or more of the following conditions: diabetes, high blood pressure, heart failure, and chronic obstructive pulmonary disease. We assessed risk according to the Community Assessment Risk Screen. Participants used an electronic device (tablet) to self-report relevant health information, which was then automatically entered into their eHealth record for consultation.

Results

The total sample size was 521 patients. Compared with the preintervention year, there were significant reductions in weight (82.3 kg before vs 80.1 kg after; P=.001) and in the proportion of people with high systolic (≥140 mmHg; 190, 36.5% vs 170, 32.6%; P=.001) and diastolic (≥90 mmHg; 72, 13.8% vs 40, 7.7%; P=.01) blood pressures, and hemoglobin A_1c ≥8% (186, 35.7% vs 104, 20.0%; P=.001). There was also a decrease in the proportion of participants who used emergency services in primary care (68, 13.1% vs 33, 6.3%; P<.001) and in hospital (98, 18.8% vs 67, 12.8%; P<.001). Likewise, fewer participants required hospital admission due to an emergency (105, 20.2% vs 71, 13.6%; P<.001) or disease exacerbation (55, 10.5% vs 42, 8.1%; P<.001).

Conclusions

The ValCrònic telemonitoring program in patients at high risk for rehospitalization or an emergency department visit appears to be useful to improve target disease control and to reduce the use of resources.

Analytical Methods for a Learning Health System: 1. Framing the Research Question

Learning health systems use routinely collected electronic health data (EHD) to advance knowledge and support continuous learning. Even without randomization, observational studies can play a central role as the nation’s health care system embraces comparative effectiveness research and patient-centered outcomes research. However, neither the breadth, timeliness, volume of the available information, nor sophisticated analytics, allow analysts to confidently infer causal relationships from observational data. However, depending on the research question, careful study design and appropriate analytical methods can improve the utility of EHD.

The introduction to a series of four papers, this review begins with a discussion of the kind of research questions that EHD can help address, noting how different evidence and assumptions are needed for each. We argue that when the question involves describing the current (and likely future) state of affairs, causal inference is not relevant, so randomized clinical trials (RCTs) are not necessary. When the question is whether an intervention improves outcomes of interest, causal inference is critical, but appropriately designed and analyzed observational studies can yield valid results that better balance internal and external validity than typical RCTs. When the question is one of translation and spread of innovations, a different set of questions comes into play: How and why does the intervention work? How can a model be amended or adapted to work in new settings? In these “delivery system science” settings, causal inference is not the main issue, so a range of quantitative, qualitative, and mixed research designs are needed.

We then describe why RCTs are regarded as the gold standard for assessing cause and effect, how alternative approaches relying on observational data can be used to the same end, and how observational studies of EHD can be effective complements to RCTs. We also describe how RCTs can be a model for designing rigorous observational studies, building an evidence base through iterative studies that build upon each other (i.e., confirmation across multiple investigations).

Analytical Methods for a Learning Health System: 4. Delivery System Science

The last in a series of four papers on how learning health systems can use routinely collected electronic health data (EHD) to advance knowledge and support continuous learning, this review describes how delivery system science provides a systematic means to answer questions that arise in translating complex interventions to other practice settings. When the focus is on translation and spread of innovations, the questions are different than in evaluative research. Causal inference is not the main issue, but rather one must ask: How and why does the intervention work? What works for whom and in what contexts? How can a model be amended to work in new settings? In these settings, organizational factors and design, infrastructure, policies, and payment mechanisms all influence an intervention's success, so a theory-driven formative evaluation approach that considers the full path of the intervention from activities to engage participants and change how they act to the expected changes in clinical processes and outcomes is needed. This requires a scientific approach to quality improvement that is characterized by a basis in theory; iterative testing; clear, measurable process and outcomes goals; appropriate analytic methods; and documented results. To better answer the questions that arise in delivery system science, this paper introduces a number of standard qualitative research approaches that can be applied in a learning health system: Pawson and Tilley's "realist evaluation," theory-based evaluation approaches, mixed-methods and case study research approaches, and the "positive deviance" approach.

Scoping literature review on the Learning Organisation concept as applied to the health system

ᅟ: There is growing interest in the use of the management concept of a 'learning organisation'. The objective of this review is to explore work undertaken towards the application of this concept to the health sector in general and to reach the goal of universal health coverage in particular. Of interest are the exploration of evaluation frameworks and their application in health.

METHOD

We used a scoping literature review based on the York methodology. We conducted an online search using selected keywords on some of the main databases on health science, selected websites and main reference books on learning organisations. We restricted the focus of our search on sources in the English language only. Inclusive and exclusive criteria were applied to arrive at a final list of articles, from which information was extracted and then selected and inserted in a chart.

RESULTS

We identified 263 articles and other documents from our search. From these, 50 articles were selected for a full analysis and 27 articles were used for the summary. The majority of the articles concerned hospital settings (15 articles, 55%). Seven articles (25%) were related to the application of the concept to the health centre setting. Four articles discussed the application of the concept to the health system (14%). Most of the applications involved high-income countries (21 articles, 78%), with only one article being related to a low-income country. We found 13 different frameworks that were applied to different health organisations.

CONCLUSIONS

The scoping review allowed us to assess applications of the learning organisation concept to the health sector to date. Such applications are still rare, but are increasingly being used. There is no uniform framework thus far, but convergence as for the dimensions that matter is increasing. Many methodological questions remain unanswered. We also identified a gap in terms of the use of this concept in low- and middle-income countries and to the health system as a whole.

Multifocal Clinical Performance Improvement Across 21 Hospitals

Improving quality and safety across an entire healthcare system in multiple clinical areas within a short time frame is challenging. We describe our experience with improving inpatient quality and safety at Kaiser Permanente Northern California. The foundations of performance improvement are a “four-wheel drive” approach and a comprehensive driver diagram linking improvement goals to focal areas. By the end of 2011, substantial improvements occurred in hospital-acquired infections (central-line–associated bloodstream infections and Clostridium difficile infections); falls; hospital-acquired pressure ulcers; high-alert medication and surgical safety; sepsis care; critical care; and The Joint Commission core measures.

Depicting the interplay between organisational tiers in the use of a national quality registry to develop quality of care in Sweden

Background

With a pending need to identify potential means to improved quality of care, national quality registries (NQRs) are identified as a promising route. Yet, there is limited evidence with regards to what hinders and facilitates the NQR innovation, what signifies the contexts in which NQRs are applied and drive quality improvement. Supposedly, barriers and facilitators to NQR-driven quality improvement may be found in the healthcare context, in the politico-administrative context, as well as with an NQR itself. In this study, we investigated the potential variation with regards to if and how an NQR was applied by decision-makers and users in regions and clinical settings. The aim was to depict the interplay between the clinical and the politico-administrative tiers in the use of NQRs to develop quality of care, examining an established registry on stroke care as a case study.

Methods

We interviewed 44 individuals representing the clinical and the politico-administrative settings of 4 out of 21 regions strategically chosen for including stroke units representing a variety of outcomes in the NQR on stroke (Riksstroke) and a variety of settings. The transcribed interviews were analysed by applying The Consolidated Framework for Implementation Research (CFIR).

Results

In two regions, decision-makers and/or administrators had initiated healthcare process projects for stroke, engaging the health professionals in the local stroke units who contributed with, for example, local data from Riksstroke. The Riksstroke data was used for identifying improvement issues, for setting goals, and asserting that the stroke units achieved an equivalent standard of care and a certain level of quality of stroke care. Meanwhile, one region had more recently initiated such a project and the fourth region had no similar collaboration across tiers. Apart from these projects, there was limited joint communication across tiers and none that included all individuals and functions engaged in quality improvement with regards to stroke care.

Conclusions

If NQRs are to provide for quality improvement and learning opportunities, advances must be made in the links between the structures and processes across all organisational tiers, including decision-makers, administrators and health professionals engaged in a particular healthcare process.

Electronic supplementary material

The online version of this article (doi:10.1186/s12913-015-1188-2) contains supplementary material, which is available to authorized users.

Implementing an evidence-based breast cancer support and communication tool to newly diagnosed patients as standard care in two institutions

While many women turn to the Internet to obtain health information, it is unlikely that unstructured Internet use provides optimal benefit to women newly diagnosed with breast cancer, due to uneven quality, conflicting claims, redundancy, and search engine idiosyncrasies, which may make finding information and assessing its accuracy and applicability difficult. To answer the need for information and support, the Comprehensive Health Enhancement Support System (CHESS) was developed to provide access to integrated information for decision-making, behavior change, and emotional support, and has been validated in randomized trials. This observational study of real-world implementation focuses on the process of integrating CHESS into standard care in two Denver healthcare systems. Results from this study provide guidance for implementation of other web-based patient information and support programs in large healthcare organizations.

Big data, miniregistries: a rapid-turnaround solution to get quality improvement data into the hands of medical specialists

The cost to build and to maintain traditional registries for many dire, complex, low-frequency conditions is prohibitive. The authors used accessible technology to develop a platform that would generate miniregistries (small, routinely updated datasets) for surveillance, to identify patients who were missing elected utilization and to influence clinicians to change practices to improve care. The platform, tested in 5 medical specialty departments, enabled the specialists to rapidly and effectively communicate clinical questions, knowledge of disease, clinical workflows, and improve opportunities. Each miniregistry required 1 to 2 hours of collaboration by a specialist. Turnaround was 1 to 14 days.

Highly reliable procedural teams: the journey to spread the universal protocol in diagnostic imaging

CONTEXT

The Joint Commission’s Universal Protocol has been widely implemented in surgical settings since publication in 2003. The elements improved patient safety in operating rooms, and the same rigor is being applied to procedures occurring in other health care arenas, in particular, diagnostic imaging.

OBJECTIVE

In 2011, Kaiser Permanente West Los Angeles’s Diagnostic Imaging Department desired to adapt previous work on Universal Protocol implementation to improve patient safety in interventional radiology and mammography procedures.

DESIGN

The teams underwent human factors training and then adapted key interventions used in surgical suites to their workflows. Time-out posters, use of whiteboards, "glitch books," and regular audits provided structure to overcome the risks that human factors present.

MAIN OUTCOME MEASURES

Staff and physician perceptions of the teamwork and safety climates in their modalities were measured using the Safety Attitudes Questionnaire at baseline and at 18 months after training. Unusual Occurrence Reports were also reviewed to identify events and near misses that could be prevented. Implementation of key process changes were identified as process measures.

RESULTS

Perception of the safety climate improved 25% in interventional radiology and 4.5% in mammography. Perception of the teamwork climate decreased 5.4% in interventional radiology and 16.6% in mammography. Unusual occurrences were underreported at baseline, and there is ongoing reluctance to document near misses.

CONCLUSION

This work provides important considerations of the impact of departmental cultures for the implementation of the Universal Protocol in procedural areas. It also reveals unexpected challenges, and requires long-term effort and focus.

Guidance for research-practice partnerships (R-PPs) and collaborative research

PURPOSE

The purpose of this paper is to provide evidence based guidance to researchers and practice personnel about forming and carrying out effective research partnerships.

DESIGN/METHODOLOGY/APPROACH

A review of the literature, interviews and discussions with colleagues in both research and practice roles, and a review of the authors' personal experiences as researchers in partnership research.

FINDINGS

Partnership research is, in some respects, a distinct "approach" to research, but there are many different versions. An analysis of research publications and of their research experience led the authors to develop a framework for planning and assessing the partnership research process, which includes defining expected outcomes for the partners, their roles, and steps in the research process.

PRACTICAL IMPLICATIONS

This review and analysis provides guidance that may reduce commonly-reported misunderstandings and help to plan more successful partnerships and projects. It also identifies future research which is needed to define more precisely the questions and purposes for which partnership research is most appropriate, and methods and designs for specific types of partnership research.

ORIGINALITY/VALUE

As more research moves towards increased participation of practitioners and patients in the research process, more precise and differentiated understanding of the different partnership approaches is required, and when each is most suitable. This article describes research approaches that have the potential to reduce "the research-practice gap". It gives evidence- and experience-based guidance for choosing and establishing a partnership research process, so as to improve partnership relationship-building and more actionable research.

Exploring a "community of practice" methodology as a regional platform for large-scale collaboration in cancer surgery-the Ottawa approach

Pressing challenges have forced health care providers to rethink traditional silos and professional boundaries. Communities of practice (cops) have been identified as a means to share knowledge across silos and boundaries. However, clarity sufficient to enable their easy and uniform reproducibility is lacking, leading to a gap between cop conceptualization and implementation. This paper explores a cop structure and outlines a framework that is adaptable, measurable, and implementable across health disciplines in a regional cancer surgery program.