A framework for value-creating learning health systems

A systematic review and proposed framework for sustainable learning healthcare systems

BACKGROUND

The healthcare sector is a complex domain that faces challenges in effectively learning from practices and outcome data. The Learning Health System (LHS) has emerged as a potential framework to improve healthcare by promoting continuous learning. However, its adoption remains limited, often involving only a single clinical department or a part of the LHS cycle. There is a need to gain a better understanding of implementing LHS on a larger scale.

AIM

To identify complete implementations of the LHS for providing recommendations into their implementation strategies, success factors, barriers, and outcomes.

METHODS

A systematic review was conducted in accordance with the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) guidelines using PubMed and Scopus databases. Data from the included papers were thematically categorized into four primary areas: (1) Scale of LHS Implementation; (2) Implementation strategies and the factors that facilitated the implementation of LHS; (3) LHS outcomes; and (4) Barriers /challenges related to the LHS implementation.

RESULTS

We identified 1,279 papers, of which 37 were included in the final analysis. Barriers to implementing LHS included interoperability, data integration, electronic health records (EHRs) challenges, organizational culture, leadership, and regulatory hurdles. Most LHS initiatives lacked discussion on long-term economic sustainability models, and only 16 papers provided objective measurements of performance changes. Drawing from the findings of the included studies, this paper offers recommendations for the effective implementation of the LHS.

CONCLUSION

The establishment of sustainable LHS necessitates several key components. First, there is a need to develop long-term economic sustainability models. Secondly, governance at the national level should promote common Application Programming Interfaces (APIs) across LHS implementations, communication channels to share tacit knowledge, efficient Institutional Review Board, ethical approval processes, and connect various initiatives currently operating independently. Lastly, the success of LHS relies not only on technological infrastructure but also on the active participation of multidisciplinary teams in decision-making and sharing of tacit knowledge.

Developing Policy Infrastructure to Guide Genomics-Informed Oncology Nursing in Canada: An Interpretive Descriptive Study

BACKGROUND

Genomic technologies such as genetic testing and precision treatments are rapidly becoming routine in oncology care, and nurses play an increasingly important role in supporting the growing demands for genomics-informed healthcare. Policy infrastructure such as competencies, standards, scope of practice statements, and education and curriculum frameworks are urgently needed to guide these practice and education changes.

PURPOSE

This study is part of a larger three-phase project to develop recommendations and catalyze action for genomics-informed oncology nursing education and practice for the Canadian Association of Nurses in Oncology and the Canadian Association of Schools of Nursing. This phase aimed to enhance understanding of policy needs and action drivers for genomics-informed oncology nursing education and practice through the perspectives of Canadian oncology nurses and patient partners.

METHODS

Interpretive description methodology guided the study. Twenty semi-structured virtual interviews were conducted; 17 with oncology nurses in various domains of practice, and three with patient partner representatives. Data collection and analysis occurred concurrently.

RESULTS

Our analysis identified three themes: 1) nurses and patients recognize that it is time for action, 2) nurses and patients see advantages to executing intentional, strategic, and collaborative policy development, and 3) leadership and advocacy are required to drive action.

CONCLUSION

Nursing policy infrastructure is required to increase genomic literacy, support nurses in providing safe patient care, and establish clear roles, responsibilities, and accountabilities within the interdisciplinary team. Strong leadership and advocacy at the practice, organizational, and systems levels are vital to accelerating action.

Lessons learned from a pay-for-performance scheme for appropriate prescribing using electronic health records from general practices in the Netherlands

INTRODUCTION

A nationwide pay-for-performance (P4P) scheme was introduced in the Netherlands between 2018 and 2023 to incentivize appropriate prescribing in general practice. Appropriate prescribing was operationalised as adherence to prescription formularies and measured based on electronic health records (EHR) data. We evaluated this P4P scheme from a learning health systems perspective.

METHODS

We conducted semi-structured interviews with 15 participants representing stakeholders of the scheme: general practitioners (GPs), health insurers, pharmacists, EHR suppliers and formulary committees. We used a thematic approach for data analysis.

RESULTS

Using EHR data showed several benefits, but lack of uniformity of EHR systems hindered consistent measurements. Specific indicators were favoured over general indicators as they allow GPs to have more control over their performance. Most participants emphasized the need for GPs to jointly reflect on their performance. Communication to GPs appeared to be challenging. Partly because of these challenges, impact of the scheme on prescribing behaviour was perceived as limited. However, several unexpected positive effects of the scheme were mentioned, such as better EHR recording habits.

CONCLUSIONS

This study identified benefits and challenges useful for future P4P schemes in promoting appropriate care with EHR data. Enhancing uniformity in EHR systems is crucial for more consistent quality measurements. Future P4P schemes should focus on high-quality feedback, peer-to-peer learning and establish a single point of communication for healthcare providers.

Addressing unmet need for primary care in Canada

Primary care is the key health system strategy for improving health, enhancing patient and clinician experience, saving money, and promoting equity. Once a pioneer in primary care, Canada now fails to provide access to millions of people. This crisis is widely recognized, but policy responses are varied and mostly incremental and piecemeal. The goal of providing primary care to everyone seems unrealistic and elusive in Canada, yet it has long been attained in many other countries. Without an explicit policy goal of primary care for all, most likely on a geographic basis, Canada will continue to underinvest and underperform in primary care, with ramifications that include rapidly escalating costs, emergency department and hospital overcrowding and a growing and inequitable burden of preventable suffering. A commitment to work towards this goal is needed now to ensure that Canadians have access to high-quality well-organized care for everyone.

Community Provider Perspectives on an Autism Learning Health Network: A Qualitative Study

Although autism is highly prevalent, no single care center has enough patients to produce generalizable knowledge of optimal care; this slows the pace of quality improvement research. The Autism Care Network (ACNet) is a learning health network (LHN) dedicated to developing the most effective approach to care for autistic children and adolescents through integrating clinical and research data. Given that most autistic patients receive care in the community, expanding ACNet to include community providers is essential to improve autism care. Our objectives were to: (1) understand the current data collection practices, learning needs, capacity, and overall interest of community clinicians in participating in an autism LHN; (2) identify their perspectives on participating in a LHN and ways in which their engagement and interest can be cultivated. Participants were purposively sampled from community physicians who participated in ASD-focused educational programming. In-depth semi-structured interviews were conducted. Analysis of 29 participant interviews yielded five primary themes: Navigating Administrative Challenges, Improving Data Collection Practices, Increasing Provider Confidence and Competence, Breaking Down Silos, and System and Societal Barriers to Achieving Best Practices. This study provides a rich and nuanced understanding of the experiences of community providers regarding the challenges of ASD care provision in the community. Overall, these findings suggest that LHNs have the potential to address several of the issues in community autism care highlighted by community providers.

Evaluating allied health primary contact models of care: A mixed methods analysis of current practice

RATIONALE

Allied Health Primary Contact Clinic (AHPCC) models of care are increasingly used to manage growing demands on health service capacity. There is a critical need for new models of care to demonstrate value, however comprehensive evaluation of AHPCCs, including use of metrics frameworks like the Moretto framework, have been slow to uptake, and the reasons for this are unclear.

AIMS AND OBJECTIVES

To understand current evaluation practices as mapped to the Moretto framework, and explore clinician attitudes to the process of service evaluation across a variety of AHPCC models implemented within a metropolitan health service in Queensland, Australia.

METHOD

A convergent mixed methods approach was used. Data were collected in 2022 using a quantitative presurvey, followed by a qualitative descriptive interview with AHPCC lead clinicians. Thirty AHPCCs were eligible, and all potential participants who provided consent were included. Descriptive statistics and thematic analysis were used for quantitative and qualitative data respectively, then merged and reported jointly.

RESULTS

Twenty-three clinicians representing 22 different AHPCCs participated. AHPCC models were found to be complex and varied. Evaluation practices were variable across AHPCCs, although more than half collected most of the Moretto framework measures. Quality of life and resource use measures were least commonly collected. Themes regarding participants' experience of AHPCCs evaluation were that: Evaluation is complex and challenging; Evaluation is important; and Evaluation needs to be enabled.

CONCLUSION

For health services to fully understand the value of their AHPCC services and direct their limited resources appropriately, evaluation activity needs to be better valued and enabled at a local, statewide and national level. Strategies should include protected time, funding, administrative support, leadership support, access to mentorship, development of structures which enable collaborative evaluation at a state-wide (or broader) level, and a shared understanding of value and core areas for measurement across stakeholders.

Trends in Gaps of Care for Patients With Congenital Heart Disease: Implications for Social Determinants of Health and Child Opportunity Index

BACKGROUND

Lifelong continuity of care is essential for patients with congenital heart disease (CHD) to maximize health outcomes; unfortunately, gaps in care (GIC) are common. Trends in GIC and of social determinants of health factors contributing to GIC are poorly understood.

METHODS AND RESULTS

This retrospective cohort study included patients with CHD, aged 0 to 34 years, who underwent surgery between January 2003 and May 2020, followed up at a pediatric subspeciality hospital. Patients were categorized as having simple, moderate, and complex CHD based on 2018 American Heart Association and American College of Cardiology guidelines. Social determinants of health, such as race, ethnicity, language, insurance status, and Child Opportunity Index, based on home address zip code, were analyzed. Of 2012 patients with CHD, a GIC of ≥3 years was identified in 56% (n=1119). The proportion of patients with GIC per year increased by 0.51% (P<0.001). Multivariable longitudinal models showed that the odds of GIC were higher for patients who were ≥10.5 years old, had simple CHD, lived out of state, lived farther from care site, received public insurance, had less protection with additional insurance plans, and with low Child Opportunity Index. A separate model for patients with only moderate/complex CHD showed similar findings. Race and ethnicity were not associated with the odds of experiencing GIC over time.

CONCLUSIONS

GIC have increased over time for patients with CHD. Social determinants of health, like insurance, access, and neighborhood opportunity, are key risk factors for increasing GIC. Efforts to reduce GIC in patients with CHD should focus on addressing the impact of specific social determinants of health.

Academic Health System Framework for Health Services Transformation: A Perspective View from West Java, Indonesia

Although the field of medicine has made significant progress in recent decades, low and middle-income countries continue to face significant difficulties in addressing the unprecedented obstacles to improving health. Medical schools should play a critical role in driving health services reform and take on a new leadership role in strengthening the health system. This paper discusses a conceptual framework and outlines the Academic Health Systems (AHS) agenda in Indonesia. A comprehensive analysis was conducted on the existing literature and frameworks regarding AHS, applying the system thinking method, which engages stakeholders actively. The findings and interpretations from interviews, focus groups, and collaborative workshops were consolidated and a set of proposed frameworks, fundamental principles, and a route for enhancing the health system in Indonesia were put forward. Our recommendations include transformative learning, community engagement, and translational research as pillar principles of AHS program. These recommendations have the potential to be modified and implemented in other low- and middle-income countries.

Learning health systems to implement chronic disease prevention programs: A novel framework and perspectives from an Australian health service

Background

Chronic diseases are a considerable burden to health systems, communities, and patients. Much of this burden, however, could be prevented if interventions effective in reducing chronic disease risks were routinely implemented.

Aims

The aim of this paper is to discuss the role of public health agencies in preventing chronic disease through the application of learning health system (LHS) approaches to improve the implementation of evidence-based interventions.

Materials and Methods

We draw on the literature and our experience operating a local LHS in Australia that has achieved rapid improvements in the implementation of chronic disease prevention interventions.

Results

The proposed LHS framework has been adapted to be both implementation and chronic disease prevention focused. The framework describes both broad improvement processes, and the infrastructure and other support (pillars) recommended to support its core functions.

Conclusion

The framework serves as a basis for further exploration of the potentially transformative role LHS's may have in addressing the chronic disease health crisis.

Implementing Foundry: A cohort study describing the regional and virtual expansion of a youth integrated service in British Columbia, Canada

AIM

Integrated youth services (IYS) have been identified as a national priority in response to the youth mental health and substance use (MHSU) crisis in Canada. In British Columbia (BC), an IYS initiative called Foundry expanded to 11 physical centres and launched a virtual service. The aim of the study was to describe the demographics of Foundry clients and patterns of service utilization during this expansion, along with the impact of the COVID-19 pandemic.

METHODS

Data were analysed for all youth (ages 12-24) accessing both in-person (April 27th, 2018-March 31st, 2021) and virtual (May 1st, 2020-March 31st, 2021) services. Cohorts containing all clients from before (April 27th, 2018-March 16th, 2020) and during (March 17th, 2020-March 31st, 2021) the COVID-19 pandemic were also examined.

RESULTS

A total of 23 749 unique youth accessed Foundry during the study period, with 110 145 services provided. Mean client age was 19.54 years (SD = 3.45) and 62% identified as female. Over 60% of youth scored 'high' or 'very high' for distress and 29% had a self-rated mental health of 'poor', with similar percentages seen for all services and virtual services. These ratings stayed consistent before and during the COVID-19 pandemic.

CONCLUSIONS

Foundry has continued to reach the target age group, with a 65% increase in number of clients during the study period compared with the pilot stage. This study highlights lessons learned and next steps to promote youth-centred data capture practices over time within an integrated youth services context.

Bridging the Telehealth Digital Divide With Collegiate Navigators: Mixed Methods Evaluation Study of a Service-Learning Health Disparities Course

BACKGROUND

Limited digital literacy is a barrier for vulnerable patients accessing health care.

OBJECTIVE

The Stanford Technology Access Resource Team (START), a service-learning course created to bridge the telehealth digital divide, trained undergraduate and graduate students to provide hands-on patient support to improve access to electronic medical records (EMRs) and video visits while learning about social determinants of health.

METHODS

START students reached out to 1185 patients (n=711, 60% from primary care clinics of a large academic medical center and n=474, 40% from a federally qualified health center). Registries consisted of patients without an EMR account (at primary care clinics) or patients with a scheduled telehealth visit (at a federally qualified health center). Patient outcomes were evaluated by successful EMR enrollments and video visit setups. Student outcomes were assessed by reflections coded for thematic content.

RESULTS

Over 6 academic quarters, 57 students reached out to 1185 registry patients. Of the 229 patients contacted, 141 desired technical support. START students successfully established EMR accounts and set up video visits for 78.7% (111/141) of patients. After program completion, we reached out to 13.5% (19/141) of patients to collect perspectives on program utility. The majority (18/19, 94.7%) reported that START students were helpful, and 73.7% (14/19) reported that they had successfully connected with their health care provider in a digital visit. Inability to establish access included a lack of Wi-Fi or device access, the absence of an interpreter, and a disability that precluded the use of video visits. Qualitative analysis of student reflections showed an impact on future career goals and improved awareness of health disparities of technology access.

CONCLUSIONS

Of the patients who desired telehealth access, START improved access for 78.7% (111/141) of patients. Students found that START broadened their understanding of health disparities and social determinants of health and influenced their future career goals.

Mobilizing strategic inflection points for sustainment of an effective intervention in an integrated learning health system: an interpretive description

BACKGROUND

Innovative models of care have the potential to improve the sustainability of health systems by improving patient and provider experiences and population outcomes while simultaneously reducing costs. Yet, it is challenging to recognize the distinctive points during research and quality improvement processes that contribute to sustainment of effective interventions. The business concept of an inflection point-the position on the curve of a trajectory where the progress in implementation of an intervention is accelerated or decelerated-may be useful to understand implementation and improve sustainability and ultimately sustainment of effective interventions. The purpose of this study was to retrospectively identify and describe strategic inflection points that accelerated the sustainability process and led to the sustainment of Alberta Family Integrated Care.

METHODS

This qualitative study was conducted in Alberta, Canada and employed an interpretive description design. Purposively sampled documents (proposals, project management plans, reports to funders and sponsors, meeting minutes, and fidelity audit and feedback checklists) from the Alberta Family Integrated Care cluster randomized controlled trial and quality improvement project constituted data for this study.

RESULTS

To accelerate sustainability in the research context, we identified (1) alignment with strategic priorities, (2) iterative, user-centered co-design, and (3) contextualization of implementation as strategic inflection points. To accelerate sustainability in the health system context, we identified (1) the learning health system, (2) enduring partnerships, (3) responsivity to societal and system change, (4) embedded governance, and (5) intentional integration into the health system as strategic inflection points. Capitalizing on these strategic inflection points led to sustainment of Alberta Family Integrated Care in the provincial health system.

CONCLUSIONS

We identified key inflection points in the research and health system contexts that led to sustainment of Alberta Family Integrated Care. By anticipating, recognizing, and leveraging inflection points in the sustainability process, researchers may be able to accelerate implementation and achieve sustainment of multi-component interventions in complex systems.

TRIAL REGISTRATION

ClinicalTrials.gov: NCT02879799. Registration date: May 27, 2016. Protocol version: June 9, 2016; version 2. Protocol publication: https://doi.org/10.1186/s13063-017-2181-3 .

Healthcare can't stop evolving: innovation as the catalyst for unleashing the managerial potential of value-based healthcare by stimulating intangible assets and enhancing organizational resilience

Background

With increasing healthcare service utilization and the introduction of costly therapies, healthcare organizations are pressured to deliver cost-effective services within constrained budgets. Rising costs and the need for efficient healthcare delivery are major concerns for governments, insurers, and health plans.

Objectives

It aims to understand the impact of these intangible assets on creating value and organizational resilience in healthcare, informing better practices and strategies for VBHC implementation.

Methods

An applied research approach using the Work Breakdown Structure (WBS) methodology was adopted. The research was divided into seven interconnected Work Packages (WPs), each designed to investigate different aspects of the integration between VBHC and intangible assets, with a focus on enhancing organizational resilience through innovative health processes. Key methodologies included literature reviews and qualitative analyses, employing Open Innovation and Design Thinking.

Results

The study revealed a dynamic interplay between VBHC, organizational resilience, and intangible assets. It showed that managerial effectiveness is influenced by direct patient outcomes and elements like intellectual capital and organizational reputation. Data integration from various Work Packages provided new insights into how intangible assets underpin VBHC strategies, proposing novel management approaches. Findings highlight the essential role of intangible assets in enhancing service delivery and fostering sustainable healthcare practices.

Discussion

The study highlights a significant oversight in the integration of intangible assets within healthcare organizations, despite their crucial role in optimizing VBHC. It supports literature emphasizing the importance of intellectual capital and organizational culture in enhancing healthcare management efficiency and resilience. A paradigm shift in VBHC to include these assets is needed for building a more adaptable and sustainable healthcare system. This integration can lead to better clinical outcomes, patient satisfaction, and overall healthcare efficiency, aligning more closely with VBHC goals.

Conclusion

Recognizing and effectively managing intangible assets are paramount for the successful implementation of VBHC and enhanced organizational resilience. Strategic integration of these assets into healthcare management practices can significantly improve patient outcomes and create a more sustainable, patient-centered, and resilient healthcare system. Future studies should develop methodologies for robust measurement and integration of these assets to fully realize the potential of VBHC.

A systems approach to managing the risk of healthcare acquired infection in an acute hospital setting supported by human factors ergonomics, data science, data governance and AI

Innovative approaches are needed for managing risk and system change in healthcare. This paper presents a case study of a project that took place over two years, taking a systems approach to managing the risk of healthcare acquired infection in an acute hospital setting, supported by an Access Risk Knowledge Platform which brings together Human Factors Ergonomics, Data Science, Data Governance and AI expertise. Evidence for change including meeting notes and use of the platform were studied. The work on the project focused on first systematically building a rich picture of the current situation from a transdisciplinary perspective. This allowed for understanding risk in context and developing a better capability to support enterprise risk management and accountability. From there a linking of operational and risk data took place which led to mapping of the risk pattern in the hospital.

Exploring the barriers, facilitators and needs to use patient outcomes in district nursing care: A multi-method qualitative study

AIM AND OBJECTIVES

To provide an in-depth insight into the barriers, facilitators and needs of district nurses and nurse assistants on using patient outcomes in district nursing care.

BACKGROUND

As healthcare demands grow, particularly in district nursing, there is a significant need to understand how to systematically measure and improve patient outcomes in this setting. Further investigation is needed to identify the barriers and facilitators for effective implementation.

DESIGN

A multi-method qualitative study.

METHODS

Open-ended questions of a survey study (N = 132) were supplemented with in-depth online focus group interviews involving district nurses and nurse assistants (N = 26) in the Netherlands. Data were analysed using thematic analysis.

RESULTS

Different barriers, facilitators and needs were identified and compiled into 16 preconditions for using outcomes in district nursing care. These preconditions were summarised into six overarching themes: follow the steps of a learning healthcare system; provide patient-centred care; promote the professional's autonomy, attitude, knowledge and skills; enhance shared responsibility and collaborations within and outside organisational boundaries; prioritise and invest in the use of outcomes; and boost the unity and appreciation for district nursing care.

CONCLUSIONS

The preconditions identified in this study are crucial for nurses, care providers, policymakers and payers in implementing the use of patient outcomes in district nursing practice. Further exploration of appropriate strategies is necessary for a successful implementation.

RELEVANCE TO CLINICAL PRACTICE

This study represents a significant step towards implementing the use of patient outcomes in district nursing care. While most research has focused on hospitals and general practitioner settings, this study focuses on the needs for district nursing care. By identifying 16 key preconditions across themes such as patient-centred care, professional autonomy and unity, the findings offer valuable guidance for integrating a learning healthcare system that prioritises the measurement and continuous improvement of patient outcomes in district nursing.

REPORTING METHOD

Consolidated Criteria for Reporting Qualitative Research (COREQ) guidelines.

PATIENT OF PUBLIC CONTRIBUTION

No patient or public contribution.

Improving the learning capacity of regional health systems for their transformation towards health and well-being systems: a qualitative study of ten Dutch regions

PURPOSE

There is growing recognition that transformation of healthcare systems towards health and well-being systems requires a continuous learning process. This explorative study aims to gain insight into the experiences with and investment in these learning processes within regional partnerships for health and in what they need to enhance their learning capacity to use the learning for transformation.

DESIGN/METHODOLOGY/APPROACH

17 interviews were held with programme managers, data scientists, trusted advisors and a citizen representative, all involved in the learning process on a regional level in ten Dutch regional partnerships. The interviews were inductively and thematically analysed, focusing on the experiences and perceptions underlying the learning processes.

FINDINGS

Regional partnerships invest in learning processes by organizing interactions between different groups of stakeholders and by reflecting on specific themes or on a region-wide level. Difficulty was found in region-wide reflection and in enhancing the learning capacity within the partnerships. Further enhancing the learning capacity required: (1) Investment in (the use of) expertise for translating learning outcomes into concrete action; (2) Leadership for change, underpinned by a shared sense of urgency to learn for transformation and (3) A facilitative environment for change which is both based on facilitative system structures and a basis of trust and commitment to learn and adapt.

ORIGINALITY/VALUE

The study highlighted the difficulty of learning on a region-wide level and the struggle to apply this learning for transformation. It provides insights into how learning processes and learning capacity can be further improved.

Case management in primary healthcare for people with complex needs to improve integrated care: a large-scale implementation study protocol

INTRODUCTION

Case management (CM) is among the most studied effective models of integrated care for people with complex needs. The goal of this study is to scale up and assess CM in primary healthcare for people with complex needs.

METHODS AND ANALYSIS

The research questions are: (1) which mechanisms contribute to the successful scale-up of CM for people with complex needs in primary healthcare?; (2) how do contextual factors within primary healthcare organisations contribute to these mechanisms? and (3) what are the relationships between the actors, contextual factors, mechanisms and outcomes when scaling-up CM for people with complex needs in primary healthcare? We will conduct a mixed methods Canadian interprovincial project in Quebec, New-Brunswick and Nova Scotia. It will include a scale-up phase and an evaluation phase. At inception, a scale-up committee will be formed in each province to oversee the scale-up phase. We will assess scale-up using a realist evaluation guided by the RAMESES checklist to develop an initial programme theory on CM scale-up. Then we will test and refine the programme theory using a mixed-methods multiple case study with 10 cases, each case being the scalable unit of the intervention in a region. Each primary care clinic within the case will recruit 30 adult patients with complex needs who frequently use healthcare services. Qualitative data will be used to identify contexts, mechanisms and certain outcomes for developing context-mechanism-outcome configurations. Quantitative data will be used to describe patient characteristics and measure scale-up outcomes.

ETHICS AND DISSEMINATION

Ethics approval was obtained. Engaging researchers, decision-makers, clinicians and patient partners on the study Steering Committee will foster knowledge mobilisation and impact. The dissemination plan will be developed with the Steering Committee with messages and dissemination methods targeted for each audience.

Tools and frameworks for evaluating the implementation of learning health systems: a scoping review

INTRODUCTION

Despite increased interest in learning health systems (LHS), a paucity of guidance and tools for evaluating LHS implementation exists. To address this, we aim to undertake a scoping review on existing tools and evaluation of exemplars of LHS implementation.

METHODS

We conducted a scoping review of peer-reviewed studies within Scopus, EMBASE, MEDLINE, and MEDLINE in-process that described (1) the evaluation of the implementation of an operating LHS or (2) the development of a framework or tool to facilitate this evaluation. Anima, basic research, abstracts, non-English language articles, and publications before 2018 were excluded. All study designs were considered.

FINDINGS

From 1300 studies initially identified, 4 were eligible, revealing three tools with nine implementation evaluation examples. The identified tools shared constructs which were evaluated, including: Stakeholders, Data, Research Evidence, Implementation, and Sociotechnical Infrastructure. However, there was divergence in evaluation methodology. Tools ranged from a five-point numerical rating system for process maturity with a radar chart called the Network Maturity Grid (NMG); the Kaiser Permanente Washington (KPWA) LHS Logic Model, which provides a broad list of constructs and sample measures relevant to LHS operations; and finally LADDERS, a simple tool or form-based template designed for consistent evaluation over time. The NMG tool was the most mature in terms of adaptation and adoption. Notably, two (NMG and the KPWA LHS Logic Model) out of three tools conceptualized the LHS as a suite of processes and devised tools were processes that linked these constructs.

IMPLICATIONS FOR TOOLKIT DEVELOPMENT

The evaluation of LHS implementation remains an under explored area of investigation, as this scoping review found only three tools for LHS implementation evaluation. Our findings indicate a need for further empirical research in this area and suggest early consensus in constructs that need to be considered during evaluation.

Learning Health System to rapidly improve the implementation of a school physical activity policy

BACKGROUND

Learning Health Systems (LHS) - characterised by cycles of evidence generation and application - are increasingly recognised for their potential to improve public health interventions and optimise health impacts; however there is little evidence of their application in the context of public health practice. Here, we describe how an Australian public health unit applied a LHS approach to successfully improve a model of support for implementation of a school-based physical activity policy.

METHODS

This body of work was undertaken in the context of a strong research-practice partnership. Core LHS capabilities included: i) partnerships and stakeholder engagement; ii) workforce development and learning health communities; iii) multi-disciplinary scientific expertise; iv) practice data collection and management system; v) evidence surveillance and synthesis; and vi) governance and organisational processes of decision making. Three cycles of data generation and application were used. Within each cycle, randomised controlled trials conducted in NSW primary schools were used to generate data on the support model's effectiveness for improving schools' implementation of a government physical activity policy, its delivery costs, and process measures such as adoption and acceptability. Each type of data were analysed independently, synthesised, and then presented to a multi-disciplinary team of researchers and practitioners, in consult with stakeholders, leading to collaborative decisions for incremental improvements to the support model.

RESULTS

Cycle 1 tested the first version of the support model (composed of five implementation strategies targeting identified barriers of policy implementation) and showed the model's feasibility and efficacy for improving schools' policy implementation. Data-informed changes were made to enhance impact, including the addition of three implementation strategies to address outstanding barriers. Cycle 2 (now, testing a package of eight implementation strategies) established the model's effectiveness and cost-effectiveness for improving school's policy implementation. Data-informed changes were made to reduce delivery costs, specifically adapting the costliest strategies to reduce in-person contact from external support personnel. Cycle 3 showed that the adaptations minimised the relative cost of delivery without adversely impacting on the effect.

CONCLUSIONS

Through this process, we identified an effective, cost-effective, acceptable and scalable policy implementation support model for service delivery. This provides important information to inform or support LHS approaches for other agencies seeking to optimise the health impact of evidence-based interventions.

Learning health systems on the front lines to strengthen care against future pandemics and climate change: a rapid review

BACKGROUND

An essential component of future-proofing health systems against future pandemics and climate change is strengthening the front lines of care: principally, emergency departments and primary care settings. To achieve this, these settings can adopt learning health system (LHS) principles, integrating data, evidence, and experience to continuously improve care delivery. This rapid review aimed to understand the ways in which LHS principles have been applied to primary care and emergency departments, the extent to which LHS approaches have been adopted in these key settings, and the factors that affect their adoption.

METHODS

Three academic databases (Embase, Scopus, and PubMed) were searched for full text articles reporting on LHSs in primary care and/or emergency departments published in the last five years. Articles were included if they had a primary focus on LHSs in primary care settings (general practice, allied health, multidisciplinary primary care, and community-based care) and/or emergency care settings. Data from included articles were catalogued and synthesised according to the modified Institute of Medicine's five-component framework for LHSs (science and informatics, patient-clinician partnerships, incentives, continuous learning culture, and structure and governance).

RESULTS

Thirty-seven articles were included, 32 of which reported LHSs in primary care settings and seven of which reported LHSs in emergency departments. Science and informatics was the most commonly reported LHS component, followed closely by continuous learning culture and structure and governance. Most articles (n = 30) reported on LHSs that had been adopted, and many of the included articles (n = 17) were descriptive reports of LHS approaches.

CONCLUSIONS

Developing LHSs at the front lines of care is essential for future-proofing against current and new threats to health system sustainability, such as pandemic- and climate change-induced events. Limited research has examined the application of LHS concepts to emergency care settings. Implementation science should be utilised to better understand the factors influencing adoption of LHS approaches on the front lines of care, so that all five LHS components can be progressed in these settings.

Siloed mentality, health system suboptimization and the healthcare symphony: a Canadian perspective

Measuring and optimizing a health system is challenging when patient care is split between many independent organizations. For example, patients receive care from their primary care provider, outpatient specialist clinics, hospitals, private providers and, in some instances, family members. These silos are maintained through different funding sources (or lack of funding) which incentivize siloed service delivery. A shift towards prioritizing patient outcomes and keeping the patient at the centre of care is emerging. However, competing philosophies on patient needs, how health is defined and how health is produced and funded is creating and engraining silos in the delivery of health services. Healthcare and health outcomes are produced through a series of activities conducted by diverse teams of health professionals working in concert. Health professionals are continually learning from each patient interaction; however, silos are barriers to information exchange, collaborative evidence generation and health system improvement. This paper presents a systems view of healthcare and provides a systems lens to approach current challenges in health systems. The first part of the paper provides a background on the current state and challenges to healthcare in Canada. The second part presents potential reasons for continued health system underperformance. The paper concludes with a system perspective for addressing these challenges.

A protocol for the formative evaluation of the implementation of patient-reported outcome measures in child and adolescent mental health services as part of a learning health system

BACKGROUND

Mental health conditions affect one in seven young people and research suggests that current mental health services are not meeting the needs of most children and youth. Learning health systems are an approach to enhancing services through rapid, routinized cycles of continuous learning and improvement. Patient-reported outcome measures provide a key data source for learning health systems. They have also been shown to improve outcomes for patients when integrated into routine clinical care. However, implementing these measures into health systems is a challenging process. This paper describes a protocol for a formative evaluation of the implementation of patient-reported measures in a newly operational child and adolescent mental health centre in Calgary, Canada. The purpose is to optimize the collection and use of patient-reported outcome measures. Our specific objectives are to assess the implementation progress, identify barriers and facilitators to implementation, and explore patient, caregivers and clinician experiences of using these measures in routine clinical care.

METHODS

This study is a mixed-methods, formative evaluation using the Consolidated Framework for Implementation Research. Participants include patients and caregivers who have used the centre's services, as well as leadership, clinical and support staff at the centre. Focus groups and semi-structured interviews will be conducted to assess barriers and facilitators to the implementation and sustainability of the use of patient-reported outcome measures, as well as individuals' experiences with using these measures within clinical care. The data generated by the patient-reported measures over the first five months of the centre's operation will be analyzed to understand implementation progress, as well as validity of the chosen measures for the centres' population.

DISCUSSION

The findings of this evaluation will help to identify and address the factors that are affecting the successful implementation of patient-reported measures at the centre. They will inform the co-design of strategies to improve implementation with key stakeholders, which include patients, clinical staff, and leadership at the centre. To our knowledge, this is the first study of the implementation of patient-reported outcome measures in child and adolescent mental health services and our findings can be used to enhance future implementation efforts in similar settings.

Development and validity testing of a matrix to evaluate maturity of clinical pathways: a case study in Saskatchewan, Canada

BACKGROUND

Healthcare systems are transforming into learning health systems that use data-driven and research-informed approaches to achieve continuous improvement. One of these approaches is the use of clinical pathways, which are tools to standardize care for a specific population and improve healthcare quality. Evaluating the maturity of clinical pathways is necessary to inform pathway development teams and health system decision makers about required pathway revisions or implementation supports. In an effort to improve the development, implementation, and sustainability of provincial clinical pathways, we developed a clinical pathways maturity evaluation matrix. To explore the initial content and face validity of the matrix, we used it to evaluate a case pathway within a provincial health authority in Saskatchewan, Canada.

METHODS

By using iterative consensus-based processes, we gathered feedback from stakeholders including patient and family partners, policy makers, clinicians, and quality improvement specialists, to rank, retain, or remove enablers and sub-enablers of the draft matrix. We tested the matrix on the Chronic Pain Pathway (CPP) for primary care in a local pilot area and revised the matrix based on feedback from the CPP development team leader.

RESULTS

The final matrix contains five enablers (i.e., Design, Ownership and Performer, Infrastructure, Performance Management, and Culture), 20 sub-enablers, and three trajectory definitions for each sub-enabler. Supplemental documents were created for six sub-enablers. The CPP scored 15 out of 40 possible points of maturity. Although the pathway scored highest in the Design enabler (10/12), it requires more attention in several areas, specifically the Ownership and Performer and the Performance Management enablers, each of which scored zero. Additionally, the Infrastructure and Culture enablers scored 2/4 and 3/8 points, respectively. These areas of the CPP are in need of improvement in order to enhance the overall maturity of the CPP.

CONCLUSIONS

We developed a clinical pathways maturity matrix to evaluate the various dimensions of clinical pathways' development and implementation. The goals of this initial work were to develop and validate a tool to assess the maturity and readiness of new or existing pathways and to track pathways' revisions and improvements.

Patient perceptions of behavioral flags in the emergency department: A qualitative analysis

BACKGROUND

To combat increasing levels of violence in the emergency department (ED), hospitals have implemented several safety measures, including behavioral flags. These electronic health record (EHR)-based notifications alert future clinicians of past incidents of potentially threatening patient behavior, but observed racial disparities in their placement may unintentionally introduce bias in patient care. Little is known about how patients perceive these flags and the disparities that have been found in their placement.

OBJECTIVE

This study aims to investigate patient perceptions and perceived benefits and harms associated with the use of behavioral flags.

METHODS

Twenty-five semistructured qualitative interviews were conducted with a convenience sample of patients in the ED of a large, urban, academic medical center who did not have a behavioral flag in their EHR. Interviews lasted 10-20 min and were recorded then transcribed. Thematic analysis of deidentified transcripts took place in NVivo 20 software (QSR International) using a general inductive approach.

RESULTS

Participant perceptions of behavioral flags varied, with both positive and negative opinions being shared. Five key themes, each with subthemes, were identified: (1) benefits of behavioral flags, (2) concerns and potential harms of flags, (3) transparency with patients, (4) equity, and (5) ideas for improvement.

CONCLUSIONS

Patient perspectives on the use of behavioral flags in the ED vary. While many saw flags as a helpful tool to mitigate violence, concerns around negative impacts on care, transparency, and equity were also shared. Insights from this stakeholder perspective may allow for health systems to make flags more effective without compromising equity or patient ideals.

Patient-oriented research: An essential driver of learning health system capacity development

Canada's health system faces a lag in implementing high-quality evidence and research-driven innovation into service delivery, while demonstrating accountability and benefit to the public. To address these challenges, Patient-Oriented Research (POR) builds teams that engage researchers, healthcare providers, decision-makers, and most importantly, patients (people with lived and living experience) in the process of generating and applying evidence to inform health services and decision-making. A Learning Health System (LHS) systematically integrates external evidence with internal data and experience and puts that knowledge into practice in a continuous cycle. Using a POR/LHS example from a BC health authority, we describe nine enablers required to support LHS capacity development. The LHS case study, Walk With Me, addresses a health system high-priority topic: the toxic drug crisis. Understanding the value of learning health systems, along with the enablers required to support and implement them, will empower health leaders to champion and orchestrate positive change.

How to use communities of practice to support change in learning health systems: A landscape of roles and guidance for management

Background

Communities of practice support evidence-based practice and can be, in and of themselves, applied learning spaces in organizations. However, the variety of ways that communities of practice can support learning health systems are poorly characterized. Furthermore, health system leaders have little guidance on designing and resourcing communities of practice to effectively serve learning health systems.

Methods

We conducted a collective case study, examining a cross-section of Canadian-based communities of practice dedicated to supporting evidence-based practice. We held semi-structured interviews with 21 participants representing 16 communities of practice and 5 community of practice facilitation platforms that provide administration support, tools, and oversight for multiple communities of practice. Using the Conceptual Framework for Value-Creating Learning Health Systems, we characterized the numerous roles that communities of practice can take to support learning health systems. We also pulled insights from the interviews on properly resourcing and managing communities of practice.

Results

Communities of practice can advance learning health systems across learning cycles (ie, identifying learning priorities, generating data and knowledge, and implementing and evaluating change). They also act as important infrastructure required to share and coordinate across learning health systems. Community of practice facilitation platforms reduce staff members' workload, in turn, creating greater efficiency and effectiveness across community of practice lifespans. Furthermore, these platforms can be a mechanism to coordinate critical activities (e.g., priority alignment, knowledge brokerage/sharing across the broader system).

Conclusion

To the authors' knowledge, this is the first study to characterize communities of practice across the learning health system landscape. With these results, learning health system leaders have a catalog that clarifies the potential communities of practice roles in knowledge generation, implementation, and uptake of new evidence. Furthermore, the results provide evidence that organizational investment in overarching community of practice facilitation platforms will strengthen and accelerate community of practice supports in learning health systems.

Experiences and lessons learned from a patient-engagement service established by a national research consortium in the U.S. Veterans Health Administration

Introduction

Meaningful engagement of patients in the research process has increased over the past 20 years. Few accounts are available of engagement infrastructure and processes used by large research organizations. The Pain/Opioid Consortium of Research (Consortium) is a U.S. Department of Veterans Affairs (VA) research network that provides infrastructure to accelerate health research and implementation of evidence-based health care. The Consortium's key activities include facilitating Veteran-engaged research and building community between Veterans and VA researchers. This report sought to describe experiences and lessons learned from the first 3 years of a national research engagement service, featuring a Veteran Engagement (VE) Panel, established by the Consortium.

Methods

We gathered authors' experiences to describe development and operation of the Consortium's VE Panel. Engagement staff collected program evaluation data about partners (Veterans and researchers), projects about which the VE Panel consulted, and meeting attendance during operation of the engagement service.

Results

We created a 12-member VE Panel; all of whom had lived experience with chronic pain, prescription opioid medication use, or opioid use disorder. Engagement staff and VE Panel members implemented an engagement service operational model designed to continuously learn and adapt. The panel consulted on 48 projects spanning the research process. Seventy-eight percent of panel members, on average, attended each monthly meeting. VE Panel members and participating researchers reported high satisfaction with the quality, ease, and outcomes of their engagement service experiences.

Conclusions

This work provides an illustrative example of how a national research consortium facilitated Veteran-engaged research and built community between Veterans and VA researchers by developing and operating an ongoing engagement consulting service, featuring a VE Panel. The service, designed as a learning community, relied on skilled engagement staff to cultivate high quality experiences and outcomes for all partners.

The impact of cognitive biases, mental models, and mindsets on leadership and change in the health system

Understanding how cognitive biases, mental models, and mindsets impact leadership in health systems is essential. This article supports the notion of cognitive biases as flawed thinking or cognitive traps which negatively influence leadership. Mental models that do not fit with current evidence limit our ability to comprehend and respond to system issues. Resulting mindsets affect cognition, behaviour, and decision-making. Metacognition is critical. The wicked problems in today's complex health system require leaders and everyone involved to elevate their personal, organizational, and disciplinary perspectives to a systems level. Three examples of mental models/mindsets are reviewed. They do not change simply because we wish or will them to. The first step is being aware of what they are and how they impact our thinking and decision-making. Some tips for managing these traps are offered as examples of how to challenge our leadership approach in the health system.

Deriving Treatment Decision Support From Dutch Electronic Health Records by Exploring the Applicability of a Precision Cohort-Based Procedure for Patients With Type 2 Diabetes Mellitus: Precision Cohort Study

BACKGROUND

The rapidly increasing availability of medical data in electronic health records (EHRs) may contribute to the concept of learning health systems, allowing for better personalized care. Type 2 diabetes mellitus was chosen as the use case in this study.

OBJECTIVE

This study aims to explore the applicability of a recently developed patient similarity-based analytics approach based on EHRs as a candidate data analytical decision support tool.

METHODS

A previously published precision cohort analytics workflow was adapted for the Dutch primary care setting using EHR data from the Nivel Primary Care Database. The workflow consisted of extracting patient data from the Nivel Primary Care Database to retrospectively generate decision points for treatment change, training a similarity model, generating a precision cohort of the most similar patients, and analyzing treatment options. This analysis showed the treatment options that led to a better outcome for the precision cohort in terms of clinical readouts for glycemic control.

RESULTS

Data from 11,490 registered patients diagnosed with type 2 diabetes mellitus were extracted from the database. Treatment-specific filter cohorts of patient groups were generated, and the effect of past treatment choices in these cohorts was assessed separately for glycated hemoglobin and fasting glucose as clinical outcome variables. Precision cohorts were generated for several individual patients from the filter cohorts. Treatment options and outcome analyses were technically well feasible but in general had a lack of statistical power to demonstrate statistical significance for treatment options with better outcomes.

CONCLUSIONS

The precision cohort analytics workflow was successfully adapted for the Dutch primary care setting, proving its potential for use as a learning health system component. Although the approach proved technically well feasible, data size limitations need to be overcome before application for clinical decision support becomes realistically possible.

Priority setting in times of crises: an analysis of priority setting for the COVID-19 response in the Western Pacific Region

BACKGROUND

While priority setting is recognized as critical for promoting accountability and transparency in health system planning, its role in supporting rational, equitable and fair pandemic planning and responses is less well understood. This study aims to describe how priority setting was used to support planning in the initial stage of the pandemic response in a subset of countries in the Western Pacific Region (WPR).

METHODS

We purposively sampled a subset of countries from WPR and undertook a critical document review of the initial national COVID-19 pandemic response plans. A pre-specified tool guided data extraction and the analysis examined the use of quality parameters of priority setting, and equity considerations.

RESULTS

Nine plans were included in this analysis, from the following countries: Papua New Guinea, Tonga, The Philippines, Fiji, China, Australia, New Zealand, Japan, and Taiwan. Most commonly the plans described strong political will to respond swiftly, resource needs, stakeholder engagement, and defined the roles of institutions that guided COVID-19 response decision-making. The initial plans did not reflect strong evidence of public engagement or considerations of equity informing the early responses to the pandemic.

CONCLUSION

This study advances an understanding of how priority setting and equity considerations were integrated to support the development of the initial COVID-19 responses in nine countries in WPR and contributes to the literature on health system planning during emergencies. This baseline assessment reveals evidence of the common priority setting parameters that were deployed in the initial responses, the prioritized resources and equity considerations and reinforces the importance of strengthening health system capacity for priority setting to support future pandemic preparedness.

Progress with the Learning Health System 2.0: a rapid review of Learning Health Systems' responses to pandemics and climate change

BACKGROUND

Pandemics and climate change each challenge health systems through increasing numbers and new types of patients. To adapt to these challenges, leading health systems have embraced a Learning Health System (LHS) approach, aiming to increase the efficiency with which data is translated into actionable knowledge. This rapid review sought to determine how these health systems have used LHS frameworks to both address the challenges posed by the COVID-19 pandemic and climate change, and to prepare for future disturbances, and thus transition towards the LHS2.0.

METHODS

Three databases (Embase, Scopus, and PubMed) were searched for peer-reviewed literature published in English in the five years to March 2023. Publications were included if they described a real-world LHS's response to one or more of the following: the COVID-19 pandemic, future pandemics, current climate events, future climate change events. Data were extracted and thematically analyzed using the five dimensions of the Institute of Medicine/Zurynski-Braithwaite's LHS framework: Science and Informatics, Patient-Clinician Partnerships, Continuous Learning Culture, Incentives, and Structure and Governance.

RESULTS

The search yielded 182 unique publications, four of which reported on LHSs and climate change. Backward citation tracking yielded 13 additional pandemic-related publications. None of the climate change-related papers met the inclusion criteria. Thirty-two publications were included after full-text review. Most were case studies (n = 12, 38%), narrative descriptions (n = 9, 28%) or empirical studies (n = 9, 28%). Science and Informatics (n = 31, 97%), Continuous Learning Culture (n = 26, 81%), Structure and Governance (n = 23, 72%) were the most frequently discussed LHS dimensions. Incentives (n = 21, 66%) and Patient-Clinician Partnerships (n = 18, 56%) received less attention. Twenty-nine papers (91%) discussed benefits or opportunities created by pandemics to furthering the development of an LHS, compared to 22 papers (69%) that discussed challenges.

CONCLUSIONS

An LHS 2.0 approach appears well-suited to responding to the rapidly changing and uncertain conditions of a pandemic, and, by extension, to preparing health systems for the effects of climate change. LHSs that embrace a continuous learning culture can inform patient care, public policy, and public messaging, and those that wisely use IT systems for decision-making can more readily enact surveillance systems for future pandemics and climate change-related events.

TRIAL REGISTRATION

PROSPERO pre-registration: CRD42023408896.

Using a Rapid Learning Health System for Stratified Care in Emerging Adult Mental Health Services: Protocol for the Implementation of Patient-Reported Outcome Measures

BACKGROUND

Mental illness among emerging adults is often difficult to ameliorate due to fluctuating symptoms and heterogeneity. Recently, innovative approaches have been developed to improve mental health care for emerging adults, including (1) implementing patient-reported outcome measures (PROMs) to assess illness severity and inform stratified care to assign emerging adults to a treatment modality commensurate with their level of impairment and (2) implementing a rapid learning health system in which data are continuously collected and analyzed to generate new insights, which are then translated to clinical practice, including collaboration among clients, health care providers, and researchers to co-design and coevaluate assessment and treatment strategies.

OBJECTIVE

The aim of the study is to determine the feasibility and acceptability of implementing a rapid learning health system to enable a measurement-based, stratified care treatment strategy for emerging adults.

METHODS

This study takes place at a specialty clinic serving emerging adults (age 16-24 years) in Calgary, Canada, and involves extensive collaboration among researchers, providers, and youth. The study design includes six phases: (1) developing a transdiagnostic platform for PROMs, (2) designing an initial stratified care model, (3) combining the implementation of PROMs with stratified care, (4) evaluating outcomes and disseminating results, (5) modification of stratified care based on data derived from PROMs, and (6) spread and scale to new sites. Qualitative and quantitative feedback will be collected from health care providers and youth throughout the implementation process. These data will be analyzed at regular intervals and used to modify the way future services are delivered. The RE-AIM (Reach, Effectiveness, Adoption, Implementation, and Maintenance) framework is used to organize and evaluate implementation according to 3 key objectives: improving treatment selection, reducing average wait time and treatment duration, and increasing the value of services.

RESULTS

This project was funded through a program grant running from 2021 to 2026. Ethics approval for this study was received in February 2023. Presently, we have developed a system of PROMs and organized clinical services into strata of care. We will soon begin using PROMs to assign clients to a stratum of care and using feedback from youth and clinicians to understand how to improve experiences and outcomes.

CONCLUSIONS

This study has key implications for researchers and clinicians looking to understand how to customize emerging adult mental health services to improve the quality of care and satisfaction with care. This study has significant implications for mental health care systems as part of a movement toward value-based health care.

INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID)

PRR1-10.2196/51667.

Assessing Alternative Approaches for Wound Closure in a National Pediatric Learning Health System

INTRODUCTION

Our objective was to perform a feasibility study using real-world data from a learning health system (LHS) to describe current practice patterns of wound closure and explore differences in outcomes associated with the use of tissue adhesives and other methods of wound closure in the pediatric surgical population to inform a potentially large study.

METHODS

A multi-institutional cross-sectional study was performed of a random sample of patients <18 y-old who underwent laparoscopic appendectomy, open or laparoscopic inguinal hernia repair, umbilical hernia repair, or repair of traumatic laceration from January 1, 2019, to December 31, 2019. Sociodemographic and operative characteristics were obtained from 6 PEDSnet (a national pediatric LHS) children's hospitals and OneFlorida Clinical Research Consortium (a PCORnet collaboration across 14 academic health systems). Additional clinical data elements were collected via chart review.

RESULTS

Of the 692 patients included, 182 (26.3%) had appendectomies, 155 (22.4%) inguinal hernia repairs, 163 (23.6%) umbilical hernia repairs, and 192 (27.8%) traumatic lacerations. Of the 500 surgical incisions, sutures with tissue adhesives were the most frequently used (n = 211, 42.2%), followed by sutures with adhesive strips (n = 176, 35.2%), and sutures only (n = 72, 14.4%). Most traumatic lacerations were repaired with sutures only (n = 127, 64.5%). The overall wound-related complication rate was 3.0% and resumption of normal activities was recommended at a median of 14 d (interquartile ranges 14-14).

CONCLUSIONS

The LHS represents an efficient tool to identify cohorts of pediatric surgical patients to perform comparative effectiveness research using real-world data to support medical and surgical products/devices in children.

A call-to-action: integrate a learning health system framework into longitudinal population studies to improve health response in Africa

Longitudinal population studies (LPSs) in Africa have the potential to become powerful engines of change by adopting a learning health system (LHS) framework. This is a call-to-action opinion and highlights the importance of integrating an LHS approach into LPSs, emphasizing their transformative potential to improve population health response, drive evidence-based decision making, and enhance community well-being. Operators of LPS platforms, community members, government officials, and funding agencies have a role to contribute to this transformative journey of driving evidence-based interventions, promoting health equity, and fostering long-term public health solutions for African communities.

Frameworks, guidelines, and tools to develop a learning health system for Indigenous health: An environmental scan for Canada

Introduction

First Nations, Inuit, and Métis (FNIM) peoples experience systemic health disparities within Ontario's healthcare system. Learning health systems (LHS) is a rapidly growing interdisciplinary area with the potential to address these inequitable health outcomes through a comprehensive health system that draws on science, informatics, incentives, and culture for ongoing innovation and improvement. However, global literature is in its infancy with grounding theories and principles still emerging. In addition, there is inadequate information on LHS within Ontario's health care context.

Methods

We conducted an environmental scan between January and April 2021 and again in June 2022 to identify existing frameworks, guidelines, and tools for designing, developing, implementing, and evaluating an LHS.

Results

We found 37 relevant sources. This paper maps the literature and identifies gaps in knowledge based on five key pillars: (a) data and evidence-driven, (b) patient-centeredness, (c) system-supported, (d) cultural competencies enabled, and (e) the learning health system.

Conclusion

We provide recommendations for implementation accordingly. The literature on LHS provides a starting point to address the health disparities of FNIM peoples within the healthcare system but Indigenous community partnerships in LHS development and operation will be key to success.

Implementation strategies and outcome measures for advancing learning health systems: a mixed methods systematic review

BACKGROUND

Learning health systems strive to continuously integrate data and evidence into practice to improve patient outcomes and ensure value-based healthcare. While the LHS concept is gaining traction, the operationalization of LHSs is underexplored.

OBJECTIVE

To identify and synthesize the existing evidence on the implementation and evaluation of advancing learning health systems across international health care settings.

METHODS

A mixed methods systematic review was conducted. Six databases (CINAHL, Embase, Medline, PAIS, Scopus and Nursing at Allied Health Database) were searched up to July 2022 for terms related to learning health systems, implementation, and evaluation measures. Any study design, health care setting and population were considered for inclusion. No limitations were placed on language or date of publication. Two reviewers independently screened the titles, abstracts, and full texts of identified articles. Data were extracted and synthesized using a convergent integrated approach. Studies were critically appraised using relevant JBI critical appraisal checklists.

RESULTS

Thirty-five studies were included in the review. Most studies were conducted in the United States (n = 21) and published between 2019 and 2022 (n = 24). Digital data capture was the most common LHS characteristic reported across studies, while patient engagement, aligned governance and a culture of rapid learning and improvement were reported least often. We identified 33 unique strategies for implementing LHSs including: change record systems, conduct local consensus discussions and audit & provide feedback. A triangulation of quantitative and qualitative data revealed three integrated findings related to the implementation of LHSs: (1) The digital infrastructure of LHSs optimizes health service delivery; (2) LHSs have a positive impact on patient care and health outcomes; and (3) LHSs can influence health care providers and the health system.

CONCLUSION

This paper provides a comprehensive overview of the implementation of LHSs in various healthcare settings. While this review identified key implementation strategies, potential outcome measures, and components of functioning LHSs, further research is needed to better understand the impact of LHSs on patient, provider and population outcomes, and health system costs. Health systems researchers should continue to apply the LHS concept in practice, with a stronger focus on evaluation.

Research protocol of the Laval-ROSA Transilab: a living lab on transitions for people living with dementia

BACKGROUND

The Laval-ROSA Transilab is a living lab that aims to support the Laval Integrated Health and Social Services Centres (Quebec, Canada) in consolidating the Quebec Alzheimer Plan. It aims to improve care transitions between different settings (Family Medicine Groups, home care, and community services) and as such improve the care of people living with dementia and their care partners. Four transition-oriented innovations are targeted. Two are already underway and will be co-evaluated: A) training of primary care professionals on dementia and interprofessional collaboration; B) early referral process to community services. Two will be co-developed and co-evaluated: C) developing a structured communication strategy around the dementia diagnosis disclosure; D) designation of a care navigator from the time of dementia diagnosis. The objectives are to: 1) co-develop a dashboard for monitoring transitions; 2) co-develop and 3) co-evaluate the four targeted innovations on transitions. In addition, we will 4) co-evaluate the impact and implementation process of the entire Laval-ROSA Transilab transformation, 5) support its sustainability, and 6) transfer it to other health organizations.

METHODS

Multi-methods living lab approach based on the principles of a learning health system. Living labs are open innovation systems that integrate research co-creation and knowledge exchange in real-life settings. Learning health systems centers care improvement on developing the organization's capacity to learn from their practices. We will conduct two learning cycles (data to knowledge, knowledge to practice, and practice to data) and involve various partners. We will use multiple data sources, including health administrative databases, electronic health records data, surveys, semi-structured interviews, focus groups, and observations.

DISCUSSION

Through its structuring actions, the Laval-ROSA Transilab will benefit people living with dementia, their care partners, and healthcare professionals. Its strategies will support sustainability and will thus allow for improvements throughout the care continuum so that people can receive the right services, at the right time, in the right place, and from the right staff.

Kailo: a systemic approach to addressing the social determinants of young people's mental health and wellbeing at the local level

The mental health and wellbeing of children and young people is deteriorating. It is increasingly recognised that mental health is a systemic issue, with a wide range of contributing and interacting factors. However, the vast majority of attention and resources are focused on the identification and treatment of mental health disorders, with relatively scant attention on the social determinants of mental health and wellbeing and investment in preventative approaches. Furthermore, there is little attention on how the social determinants manifest or may be influenced at the local level, impeding the design of contextually nuanced preventative approaches. This paper describes a major research and design initiative called Kailo that aims to support the design and implementation of local and contextually nuanced preventative strategies to improve children's and young people's mental health and wellbeing. The Kailo Framework involves structured engagement with a wide range of local partners and stakeholders - including young people, community partners, practitioners and local system leaders - to better understand local systemic influences and support programmes of youth-centred and evidence-informed co-design, prototyping and testing. It is hypothesised that integrating different sources of knowledge, experience, insight and evidence will result in better embedded, more sustainable and more impactful strategies that address the social determinants of young people's mental health and wellbeing at the local level.

A Global Survey of Emergency Care Clinical Networks: Discussion and Implications for Canadian Learning Health Systems

Clinical networks (CNs) can promote innovation and collaboration across providers and stakeholders. However, little is known about the structure and operations of CNs, particularly in emergency care. As Canada advances learning health systems (LHSs), foundational research is essential to enable future comparisons across CNs to identify those that contribute to positive system change. Drawing from the results of our international survey, we provide a description of 32 emergency care CNs worldwide, including their structure, operations and sustainability. Future research should consider the context of such networks, how they may contribute to an LHS and how they impact patient outcomes.

Implementation model for a national learning health system (IMPLEMENT-National LHS): a concept analysis and systematic review protocol

INTRODUCTION

Despite efforts and repeated calls to improve the organisation and quality of healthcare and services, and in view of the many challenges facing health systems, the results and capacity to adapt and integrate innovations and new knowledge remain suboptimal. Learning health systems (LHS) may be an effective model to accelerate the application of research for real quality improvement in healthcare. However, while recognising the enormous potential of LHS, the literature suggests the model remains more of an aspiration than a reality.

METHODS AND ANALYSIS

To reach a fine understanding of the implementation of the concepts involved in LHS, we will use a hybrid method which combines concept analyses with systematic review methodology. We will use a two-step analysis, a content analysis to analyse the definitions, uses and attributes of the concept and a systematic review to analyse the concept's implementation mechanisms. We will search eight databases and grey literature and present a broad synthesis of the available evidence regarding design, implementation and evaluation of LHS in a multilevel perspective. We will follow the latest Preferred Reporting Items for Systematic Review and Meta-Analysis statement for conducting and reporting a systematic review. Two reviewers will independently screen the titles and abstracts against the eligibility criteria followed by full-text screening of potentially relevant articles for final inclusion decision. Conflicts will be resolved with a senior author. We will include published primary studies that use qualitative, quantitative or mixed methods. The assessment of risk of bias will be made using the Mixed-Methods Appraisal Tool.

ETHICS AND DISSEMINATION

This systematic review is exempt from ethics approval. The results formulated will highlight evidence-based interventions that support the implementation of a national LHS. They will be of particular interest to decision makers, researchers, managers, clinicians and patients allowing finally to implement the promising proposal of LHSs at national scale.

PROSPERO REGISTRATION NUMBER

CRD42023393565.

Implementation of Patient-Reported Outcome Measures for Your Practice Needs

Patient-reported outcome (PRO) measures offer a unique opportunity to systematically incorporate patient perspectives of their health and wellbeing into treatment, view progress over time, and develop opportunities related to clinical decision support, shared decision making, and outcomes-driven payment and practice transformation. As healthcare institutions increasingly look toward adopting PRO measures in their practices, more information is needed regarding the practicalities of implementing a successful PRO program outside of the research setting. This article offers a stepwise approach to implementing a PRO program while overcoming barriers and leveraging facilitators that commonly exist in the realms of human contribution or team building, technological infrastructure, and process design. To routinely adopt and implement these instruments, we need a multifaceted approach to operationalizing patient-reported outcome measure (PROMs) for high-value outcomes-driven, patient-centered musculoskeletal care.

Exploring the impact of evaluation on learning and health innovation sustainability: protocol for a realist synthesis

BACKGROUND

Within the Learning Health System (LHS) model, learning routines, including evaluation, allow for continuous incremental change to take place. Within these learning routines, evaluation assists in problem identification, data collection, and data transformation into contextualized information, which is then re-applied to the LHS environment. Evaluation that catalyzes learning and improvement may also contribute to health innovation sustainability. However, there is little consensus as to why certain evaluations seem to support learning and sustainability, while others impede it. This realist synthesis seeks to understand the contextual factors and underlying mechanisms or drivers that best support health systems learning and sustainable innovation.

METHODS

This synthesis will be guided by Pawson and colleagues' 2005 and Emmel and colleagues' 2018 guidelines for conducting realist syntheses. The review process will encompass five steps: (1) scoping the review, (2) building theories, (3) identifying the evidence, (4) evidence selection and appraisal, and (5) data extraction and synthesis. An Expert Committee comprised of leaders in evaluation, innovation, sustainability, and realist methodology will guide this synthesis. Review findings will be reported using the RAMESES guidelines.

DISCUSSION

The use of a realist review will allow for exploration and theorizing about the contextual factors and underlying mechanisms that make evaluations 'work' (or 'not work') to support learning and sustainability. Depending on results, we will attempt to synthesize findings into a series of recommendations for evaluations with the intention to support health systems learning and sustainability. Finalized results will be presented at national and international conferences, as well as disseminated via a peer-reviewed publication.

SYSTEMATIC REVIEW REGISTRATION

This realist synthesis protocol has been registered with PROSPERO ( https://www.crd.york.ac.uk/prospero/  ID 382690).

The use and role of digital technology in learning health systems: A scoping review

OBJECTIVE

The review aimed to identify which digital technologies are proposed or used within learning health systems (LHS) and to analyze the extent to which they support learning processes in LHS.

MATERIALS AND METHODS

Multiple databases and grey literature were searched with terms related to LHS. Manual searches and backward searches of reference lists were also undertaken. The review considered publications from 2007 to 2022. Records focusing on LHS, referring to one or more digital technologies, and describing how at least one digital technology could be used in LHS were included.

RESULTS

2046 records were screened for inclusion and 154 records were included in the analysis. Twenty categories of digital technology were identified. The two most common ones across records were data recording and processing and electronic health records. Digital technology was primarily leveraged to support data access and aggregation and data analysis, two of the seven recognized learning processes within LHS learning cycles.

DISCUSSION

The results of the review show that a wide array of digital technologies is being leveraged to support learning cycles within LHS. Nevertheless, an over-reliance on a narrow set of technologies supporting knowledge discovery, a lack of direct evaluation of digital technologies and ambiguity in technology descriptions are hindering the realization of the LHS vision.

CONCLUSION

Future LHS research and initiatives should aim to integrate digital technology to support practice change and impact evaluation. The use of recognized evaluation methods for health information technology and more detailed descriptions of proposed technologies are also recommended.

Implications of the Use of Artificial Intelligence Predictive Models in Health Care Settings : A Simulation Study

BACKGROUND

Substantial effort has been directed toward demonstrating uses of predictive models in health care. However, implementation of these models into clinical practice may influence patient outcomes, which in turn are captured in electronic health record data. As a result, deployed models may affect the predictive ability of current and future models.

OBJECTIVE

To estimate changes in predictive model performance with use through 3 common scenarios: model retraining, sequentially implementing 1 model after another, and intervening in response to a model when 2 are simultaneously implemented.

DESIGN

Simulation of model implementation and use in critical care settings at various levels of intervention effectiveness and clinician adherence. Models were either trained or retrained after simulated implementation.

SETTING

Admissions to the intensive care unit (ICU) at Mount Sinai Health System (New York, New York) and Beth Israel Deaconess Medical Center (Boston, Massachusetts).

PATIENTS

130 000 critical care admissions across both health systems.

INTERVENTION

Across 3 scenarios, interventions were simulated at varying levels of clinician adherence and effectiveness.

MEASUREMENTS

Statistical measures of performance, including threshold-independent (area under the curve) and threshold-dependent measures.

RESULTS

At fixed 90% sensitivity, in scenario 1 a mortality prediction model lost 9% to 39% specificity after retraining once and in scenario 2 a mortality prediction model lost 8% to 15% specificity when created after the implementation of an acute kidney injury (AKI) prediction model; in scenario 3, models for AKI and mortality prediction implemented simultaneously, each led to reduced effective accuracy of the other by 1% to 28%.

LIMITATIONS

In real-world practice, the effectiveness of and adherence to model-based recommendations are rarely known in advance. Only binary classifiers for tabular ICU admissions data were simulated.

CONCLUSION

In simulated ICU settings, a universally effective model-updating approach for maintaining model performance does not seem to exist. Model use may have to be recorded to maintain viability of predictive modeling.

PRIMARY FUNDING SOURCE

National Center for Advancing Translational Sciences.

Establishing a novel partnership model to deploy health services for children with autism in a low-resource setting - experience from Sri Lanka

Sri Lanka is representative of challenges faced by low-income and middle-income countries, including the rise in the prevalence of autism and the lack of sufficient autism-specific services in the state sectors. The experience in establishing a Center to provide services for children with autism in Northern Sri Lanka is described. Funding and resourcing were accessed through an innovative partnership-based public/non-governmental organisation/charity model, where service-based outcomes were the main objectives. This model, incorporating state institutions, local and international charity organisations, and volunteers, devised a bespoke approach to care provision using the available resources under the clinical supervision of a consultant psychiatrist and the administrative purview of the Regional Director of Health Services. The evolution of this Center into a Learning Health System is described, reflecting how a minimalistic partnership approach focused on the integration of existing organisations and services could be a feasible model for the delivery of high-quality healthcare in low-resource settings.

Internet-Delivered Cognitive Behavioral Therapy Tailored to Spouses and Significant Others of Public Safety Personnel: Formative Evaluation Study

BACKGROUND

Spouses and significant others (SSOs) of public safety personnel (PSP) are affected by the risks and requirements of these occupations. Internet-delivered cognitive behavioral therapy (ICBT) provides a convenient and accessible treatment format that can be tailored to the needs of SSOs of PSP.

OBJECTIVE

This study aimed to assess the initial use and client perceptions (eg, likes, helpfulness, and areas for improvement) of a self-guided, transdiagnostic ICBT course designed for Canadian SSOs of PSP and identify opportunities to further tailor ICBT for this group.

METHODS

SSOs were invited to complete a 5-lesson, self-guided, transdiagnostic ICBT course. Descriptive statistics were used to analyze the demographic and clinical characteristics of participants. Content analysis was used to analyze the data from open-ended survey responses and interviews to understand their experiences with ICBT.

RESULTS

Clients (N=118) endorsed various mental health concerns (eg, depression, anxiety, posttraumatic stress symptoms, and relationship concerns) with a range of severity levels. Most clients identified as White (110/116, 94.8%) and women (108/116, 91.5%), with a mean age of 42.03 (SD 9.36) years. Of the 26 clients who were interviewed, 89% (23/26) reported believing that ICBT is helpful and 92% (24/26) reported finding at least 1 skill helpful. Clients provided suggestions for course improvements. On the basis of this feedback and quantitative data, changes were made to areas such as the delivery of materials, content, case stories, and timelines. Overall, the results indicated that many SSOs of PSP had positive perceptions of ICBT tailored to their needs and found several aspects of the course helpful, supporting the continued delivery of tailored ICBT to this population. However, there remains a need for continued promotion of the course and outreach to diverse groups of SSOs of PSP.

CONCLUSIONS

Findings from this formative evaluation provide insight into the unique experiences and needs of SSOs of PSP and provide preliminary evidence for the use of tailored ICBT to support the mental health of this group in Canada.

Facilitators of and barriers to patient and public involvement in building learning health systems in community health services settings: a scoping review protocol

INTRODUCTION

The development of learning health systems (LHSs) has often focused on optimally leveraging data. More attention should be paid to patient and public involvement or community engagement in forming learning communities that work together to build LHS. This scoping review aims to identify facilitators of and barriers to involving patients and the public in building LHSs in community health services settings.

METHODS AND ANALYSIS

We will use the Joanna Briggs Institute's scoping review methodology. We will review literature in English published from 1 January 2007 to 31 December 2022. The databases that will be searched are MEDLINE, CINAHL, Embase, Web of Science, Scopus, AgeLine, PsycINFO and Web of Science. Key inclusion and exclusion criteria include the following: we will only consider a learning community in a community health services context (eg, home care, long-term care, primary care); we will exclude literature on acute care settings; and we will consider any research designs apart from big data analytics. We will review all sources, including university student theses and dissertations. The review will proceed in three steps: (1) we will identify keywords and index terms from the MEDLINE and CINAHL databases; (2) using the keywords and index terms identified in step (1), we will search other databases and (3) we will handsearch the reference lists of the selected literature and will search for grey literature using Google. Two research assistants will screen the titles and abstracts separately, with reference to the inclusion criteria. Two researchers will then assess the full text of selected studies, also in reference to the inclusion criteria. We will present the findings in a charting table and provide a narrative summary.

ETHICS AND DISSEMINATION

This work does not require ethics approval because the data for this scoping review are publicly available. The findings will be presented in a journal article and at conferences.