The science of learning health systems: A scoping review of the empirical research (Preprint)

Implementation of large, multi-site hospital interventions: a realist evaluation of strategies for developing capability

BACKGROUND

This study presents guidelines for implementation distilled from the findings of a realist evaluation. The setting was local health districts in New South Wales, Australia that implemented three clinical improvement initiatives as part of a state-wide program. We focussed on implementation strategies designed to develop health professionals' capability to deliver value-based care initiatives for multisite programs. Capability, which increases implementers' ability to cope with unexpected scenarios is key to managing change.

METHODS

We used a mixed methods realist evaluation which tested and refined program theories elucidating the complex dynamic between context (C), mechanism (M) and outcome (O) to determine what works, for whom, under what circumstances. Data was drawn from program documents, a realist synthesis, informal discussions with implementation designers, and interviews with 10 key informants (out of 37 identified) from seven sites. Data analysis employed a retroductive approach to interrogate the causal factors identified as contributors to outcomes.

RESULTS

CMO statements were refined for four initial program theories: Making it Relevant- where participation in activities was increased when targeted to the needs of the staff; Investment in Quality Improvement- where engagement in capability development was enhanced when it was valued by all levels of the organisation; Turnover and Capability Loss- where the effects of staff turnover were mitigated; and Community-Wide Priority- where there was a strategy of spanning sites. From these data five guiding principles for implementers were distilled: (1) Involve all levels of the health system to effectively implement large-scale capability development, (2) Design capability development activities in a way that supports a learning culture, (3) Plan capability development activities with staff turnover in mind, (4) Increased capability should be distributed across teams to avoid bottlenecks in workflows and the risk of losing key staff, (5) Foster cross-site collaboration to focus effort, reduce variation in practice and promote greater cohesion in patient care.

CONCLUSIONS

A key implementation strategy for interventions to standardise high quality practice is development of clinical capability. We illustrate how leadership support, attention to staff turnover patterns, and making activities relevant to current issues, can lead to an emergent learning culture.

Implementation strategies and outcome measures for advancing learning health systems: a mixed methods systematic review

BACKGROUND

Learning health systems strive to continuously integrate data and evidence into practice to improve patient outcomes and ensure value-based healthcare. While the LHS concept is gaining traction, the operationalization of LHSs is underexplored.

OBJECTIVE

To identify and synthesize the existing evidence on the implementation and evaluation of advancing learning health systems across international health care settings.

METHODS

A mixed methods systematic review was conducted. Six databases (CINAHL, Embase, Medline, PAIS, Scopus and Nursing at Allied Health Database) were searched up to July 2022 for terms related to learning health systems, implementation, and evaluation measures. Any study design, health care setting and population were considered for inclusion. No limitations were placed on language or date of publication. Two reviewers independently screened the titles, abstracts, and full texts of identified articles. Data were extracted and synthesized using a convergent integrated approach. Studies were critically appraised using relevant JBI critical appraisal checklists.

RESULTS

Thirty-five studies were included in the review. Most studies were conducted in the United States (n = 21) and published between 2019 and 2022 (n = 24). Digital data capture was the most common LHS characteristic reported across studies, while patient engagement, aligned governance and a culture of rapid learning and improvement were reported least often. We identified 33 unique strategies for implementing LHSs including: change record systems, conduct local consensus discussions and audit & provide feedback. A triangulation of quantitative and qualitative data revealed three integrated findings related to the implementation of LHSs: (1) The digital infrastructure of LHSs optimizes health service delivery; (2) LHSs have a positive impact on patient care and health outcomes; and (3) LHSs can influence health care providers and the health system.

CONCLUSION

This paper provides a comprehensive overview of the implementation of LHSs in various healthcare settings. While this review identified key implementation strategies, potential outcome measures, and components of functioning LHSs, further research is needed to better understand the impact of LHSs on patient, provider and population outcomes, and health system costs. Health systems researchers should continue to apply the LHS concept in practice, with a stronger focus on evaluation.

Patient-Reported Outcomes Measures in Rhinoplasty: Need for Use and Implementation

Patient-reported outcome metrics (PROMs) are increasingly utilized to capture data about patients' quality of life. PROMs play an important role in the value-based health care movement by providing a patient-centered metric of quality. There are many barriers to the implementation of PROMs, and widespread adoption requires buy-in from numerous stakeholders including patients, clinicians, institutions, and payers. Several validated PROMs have been utilized by facial plastic surgeons to measure both functional and aesthetic outcomes among rhinoplasty patients. These PROMs can help clinicians and rhinoplasty patients participate in shared decision making (SDM), a process via which clinicians and patients arrive at treatment decisions together through a patient-centered approach. However, widespread adoption of PROMs and SDM has not yet been achieved. Further work should focus on overcoming barriers to implementation and engaging key stakeholders to increase the utilization of PROMs in rhinoplasty.

The rise of resilient healthcare research during COVID-19: scoping review of empirical research

BACKGROUND

The COVID-19 pandemic has presented many multi-faceted challenges to the maintenance of service quality and safety, highlighting the need for resilient and responsive healthcare systems more than ever before. This review examined empirical investigations of Resilient Health Care (RHC) in response to the COVID-19 pandemic with the aim to: identify key areas of research; synthesise findings on capacities that develop RHC across system levels (micro, meso, macro); and identify reported adverse consequences of the effort of maintaining system performance on system agents (healthcare workers, patients).

METHODS

Three academic databases were searched (Medline, EMBASE, Scopus) from 1st January 2020 to 30th August 2022 using keywords pertaining to: systems resilience and related concepts; healthcare and healthcare settings; and COVID-19. Capacities that developed and enhanced systems resilience were synthesised using a hybrid inductive-deductive thematic analysis.

RESULTS

Fifty publications were included in this review. Consistent with previous research, studies from high-income countries and the use of qualitative methods within the context of hospitals, dominated the included studies. However, promising developments have been made, with an emergence of studies conducted at the macro-system level, including the development of quantitative tools and indicator-based modelling approaches, and the increased involvement of low- and middle-income countries in research (LMIC). Concordant with previous research, eight key resilience capacities were identified that can support, develop or enhance resilient performance, namely: structure, alignment, coordination, learning, involvement, risk awareness, leadership, and communication. The need for healthcare workers to constantly learn and make adaptations, however, had potentially adverse physical and emotional consequences for healthcare workers, in addition to adverse effects on routine patient care.

CONCLUSIONS

This review identified an upsurge in new empirical studies on health system resilience associated with COVID-19. The pandemic provided a unique opportunity to examine RHC in practice, and uncovered emerging new evidence on RHC theory and system factors that contribute to resilient performance at micro, meso and macro levels. These findings will enable leaders and other stakeholders to strengthen health system resilience when responding to future challenges and unexpected events.

Stroke Learning Health Systems: A Topical Narrative Review With Case Examples

To our knowledge, the adoption of Learning Health System (LHS) concepts or approaches for improving stroke care, patient outcomes, and value have not previously been summarized. This topical review provides a summary of the published evidence about LHSs applied to stroke, and case examples applied to different aspects of stroke care from high and low-to-middle income countries. Our attempt to systematically identify the relevant literature and obtain real-world examples demonstrated the dissemination gaps, the lack of learning and action for many of the related LHS concepts across the continuum of care but also elucidated the opportunity for continued dialogue on how to study and scale LHS advances. In the field of stroke, we found only a few published examples of LHSs and health systems globally implementing some selected LHS concepts, but the term is not common. A major barrier to identifying relevant LHS examples in stroke may be the lack of an agreed taxonomy or terminology for classification. We acknowledge that health service delivery settings that leverage many of the LHS concepts do so operationally and the lessons learned are not shared in peer-reviewed literature. It is likely that this topical review will further stimulate the stroke community to disseminate related activities and use keywords such as learning health system so that the evidence base can be more readily identified.

Learning to make a difference for chILD: Value creation through network collaboration and team science

Addressing the recognized challenges and inequalities in providing high quality healthcare for rare diseases such as children's interstitial lung disease (chILD) requires collaboration across institutional, geographical, discipline, and system boundaries. The Children's Interstitial Lung Disease Respiratory Network of Australia and New Zealand (chILDRANZ) is an example of a clinical network that brings together multidisciplinary health professionals for collaboration, peer learning, and advocacy with the goal of improving the diagnosis and management of this group of rare and ultra-rare conditions. This narrative review explores the multifaceted benefits arising from social learning spaces within rare disease clinical networks by applying the value creation framework. The operation of the chILDRANZ network is used as an example across the framework to highlight how value is generated, realized, and transferred within such collaborative clinical and research networks. The community of practice formed in the chILDRANZ multidisciplinary meetings provides a strong example of social learning that engages with the uncertainty inherent in rare disease diagnosis and management and pays attention to generate new knowledge and best practice to make a difference for children and families living with chILD. This review underscores international calls for further investment in, and support of, collaborative clinical networks and virtual centers of excellence for rare disease.

The journey to a learning health system in primary care: a qualitative case study utilising an embedded research approach

BACKGROUND

Healthcare systems may be resilient and adaptive, but they are not fit for purpose in their current state. Increasing threats to health system sustainability have underscored the need to move towards a learning health system in which research and data are used routinely in clinical practice to facilitate system improvement. This study aimed to establish which elements of the learning health system were being realised within a university-based general practice and determine acceptability from staff to embrace further the transition towards a learning health system.

METHODS

Semi-structured interviews were conducted with practice staff, including clinical and administrative staff, to determine the current state of the learning health system in the practice. An embedded researcher was placed within the general practice on a part-time basis to investigate the learning health system model. Interviews were transcribed and thematically analysed based on the National Academy of Medicine's framework of learning health systems.

RESULTS

In total, 32 (91%) practice staff were interviewed, comprising general practitioners (n = 15), nurses (n = 3), administrative staff (n = 13), and a psychologist (n = 1). Participants indicated that the practice was operating with several characteristics of a learning health system (e.g., emphasising science and informatics; focusing on patient-clinician partnerships; applying incentives; supporting a continuous learning culture; and establishing structures and governance for learning). These measures were supported by the university-based setting, and resultant culture of learning. Nevertheless, there were areas of the practice where the learning health system could be strengthened, specifically relating to the use of patient data and informatics. Staff generally expressed willingness to engage with the process of strengthening the learning health system within their practice.

CONCLUSION

Although the idea of a learning health system has been gaining traction in recent years, there are comparatively few empirical studies presented in the literature. This research presents a case study of a general practice that is operating as a learning health system and highlights the utility of using the learning health system framework.

A conceptual framework to describe and evaluate a socially accountable learning health system: Development and application in a northern, rural, and remote setting

Health care and academic institutions are increasingly committing to social accountability, a strategic shift that requires priorities, activities, and evaluations to be co-determined with all relevant partners. Consequently, governments, accreditors, funders, and communities are calling for these institutions to communicate their progress towards social accountability. The purpose of this study was to develop a conceptual framework around a socially accountable learning health system. This article presents an integrated analysis of two studies: (i) a narrative review of 11 prominent social accountability and health services conceptual frameworks and (ii) a reflexive thematic analysis of 18 key informant interviews. Using a systematic conceptual framework development and integrated theory of change/realist evaluation methodologies, we describe a synthesis of these findings to develop a conceptual framework for describing and evaluating socially accountable health professional education. The resulting framework describes assessment phases of social accountability, transitions between phases, learning cycles, and the actors and systems that collectively mobilise social accountability at multiple levels in health and education systems. The framework can be used to evaluate interventions or characterise progress towards social accountability in different settings, as illustrated in the example at the end of the paper. The framework emphasises the significance of designing, mobilising, and evaluating social accountability as part of a contextualised learning health system.

Evaluating a learning health system initiative: Lessons learned during COVID-19 in Saskatchewan, Canada

Introduction

Evaluating a learning health system (LHS) encourages continuous system improvement and collaboration within the healthcare system. Although LHS is a widely accepted concept, there is little knowledge about evaluating an LHS. To explore the outputs and outcomes of an LHS model, we evaluated the COVID-19 Evidence Support Team (CEST) in Saskatchewan, Canada, an initiative to rapidly review scientific evidence about COVID-19 for decision-making. By evaluating this program during its formation, we explored how and to what extent the CEST initiative was used by stakeholders. An additional study aim was to understand how CEST could be applied as a functional LHS and the value of similar knowledge-to-action cycles.

Methods

Using a formative evaluation design, we conducted qualitative interviews with key informants (KIs) who were involved with COVID-19 response strategies in Saskatchewan. Transcripts were analyzed using reflexive thematic analysis to identify key themes. A program logic model was created to represent the inputs, activities, outputs, and outcomes of the CEST initiative.

Results

Interview data from 11 KIs were collated under three overarching categories: (1) outputs, (2) short-term outcomes, and (3) long-term outcomes from the CEST initiative. Overall, participants found the CEST initiative improved speed and access to reliable information, supported and influenced decision-making and public health strategies, leveraged partnerships, increased confidence and reassurance, and challenged misinformation. Themes relating to the long-term outcomes of the initiative included improving coordination, awareness, and using good judgment and planning to integrate CEST sustainably into the health system.

Conclusion

This formative evaluation demonstrated that CEST was a valued program and a promising LHS model for Saskatchewan. The future direction involves addressing program recommendations to implement this model as a functional LHS in Saskatchewan.

Mapping continuous learning using social network research: a social network study of Australian Genomics as a Learning Health System

OBJECTIVES

To explore a macrolevel Learning Health System (LHS) and examine if an intentionally designed network can foster a collaborative learning community over time. The secondary aim was to demonstrate the application of social network research to the field of LHS.

DESIGN

Two longitudinal online questionnaires of the Australian Genomics learning community considering relationships between network members at three time points: 2016, 2018, 2019. The questionnaire included closed Likert response questions on collaborative learning patterns and open-response questions to capture general perceptions of the community. Social network data were analysed and visually constructed using Gephi V.0.9.2 software, Likert questions were analysed using SPSS, and open responses were analysed thematically using NVivo.

SETTING

Australian Genomic Health Alliance.

PARTICIPANTS

Clinicians, scientists, researchers and community representatives.

RESULTS

Australian Genomics members highlighted the collaborative benefits of the network as a learning community to foster continuous learning in the ever-evolving field of clinical genomics. The learning community grew from 186 members (2016), to 384 (2018), to 439 (2019). Network density increased (2016=0.023, 2018=0.043), then decreased (2019=0.036). Key players remained consistent with potential for new members to achieve focal positions in the network. Informal learning was identified as the most influential learning method for genomic practice.

CONCLUSIONS

This study shows that intentionally building a network provides a platform for continuous learning-a fundamental component for establishing an LHS. The Australian Genomics learning community shows evidence of maturity and sustainability in supporting the continuous learning culture of clinical genomics. The network provides a practical means to spread new knowledge and best practice across the entire field. We show that intentionally designed networks provide the opportunity and means for interdisciplinary learning between diverse agents over time and demonstrate the application of social network research to the LHS field.