Realist evaluation of a complex integrated care programme: protocol for a mixed methods study

Characteristics of the studies using realist evaluation to assess interventions that address psychosocial healthcare issues in older adults: a scoping review

OBJECTIVES

Interventions to address psychosocial healthcare issues in older adults are increasing. Realist evaluation (RE) helps us understand how these interventions work for their issues. It is significant to obtain implications for further developing such research. We aimed to identify the characteristics of studies using RE to assess interventions that address psychosocial healthcare issues in older adults by mapping relevant literature.

DESIGN

Scoping review.

DATA SOURCES

MEDLINE, the Cumulative Index to Nursing and Allied Health Literature, PsycINFO, Web of Science, Cochrane Library, ICHUSHI (a Japanese database) and Google Scholar were used for searches between 5 January 2022 and 4 January 2024.

ELIGIBILITY CRITERIA

(1) Showing that most of the participants were older adults or their stakeholders; (2) stating in the research background or aim sections that the target interventions aimed at addressing older adults' psychosocial healthcare issues and (3) using RE to assess these interventions.

DATA EXTRACTION

Data on country of origin, type of research, study design, qualitative data collection and analysis methods, desirable items for RE and intervention aims and purposes were extracted and summarised using descriptive statistics.

RESULTS

Fifty-four studies were analysed. Most studies were conducted in the UK (54.5%). Mixed methods were used in 28 studies (51.9%), while only qualitative methods were used in 25 studies (46.2 %). Fourteen intervention aims and purposes were identified: improving dementia care, avoiding emergency admissions, preventing social isolation and promoting family involvement in the care of older adults.

CONCLUSION

RE is useful for promoting an understanding of how interventions work for addressing psychosocial healthcare issues in older adults. RE also promotes the updating of plausible theories that lead to improving interventions. Our findings show the implications of managing time and resources to address the challenge of RE's time and resource intensiveness and carefully considering the data collection methods to reduce burdens on older adults.

Understanding the impact of a residential housing program for people living with HIV/AIDS: results from a realist evaluation

BACKGROUND

In Canada, the Public Health Agency surveillance of new HIV cases has demonstrated annual increasing rates since 2020. The rates of new HIV cases are highest in the province of Saskatchewan.

OBJECTIVES

The aim of the project was to conduct a resident-oriented realist evaluation of an innovative supportive housing programme, Sanctum, for people living with HIV/AIDS who also face social care issues, such as homelessness. This project took place in Saskatchewan, a province that is seeking innovative approaches to improve quality of life and HIV/AIDS management for its citizens. Our evaluation addressed how and why participants were successful (or not) within the Sanctum programme.

DESIGN AND SETTING

Sanctum is a housing programme located in an inner-city location within the province of Saskatchewan. A unique component of this evaluation was the inclusion of an individual with lived experience, a resident partner, as a member of the research team.

PARTICIPANTS

11 recent Sanctum graduates, seven men and four women, were recruited for client partner-led in-depth, semistructured interviews.

INTERVENTIONS

Prior to the evaluation, we developed a realist programme theory with potential causal explanations, known as context-mechanism-outcomes (CMO) configurations. Interview data from the evaluation and ongoing discussions with Sanctum board members and our resident partner were used to test, refine and validate the final programme theory and CMO configurations.

RESULTS

CMO configurations at the micro (individual), meso (interpersonal) and macro (community) levels complement the over-arching programme theory. Key findings were the importance of Sanctum's harm reduction philosophy, accompanied by a non-judgmental and patient-oriented approach. Participants were supported to reduce risky behaviour, improve self-care management and develop healthier relationships within a 'safe' home-like setting. Underlying mechanisms that contributed to participants' success in the programme included: intrinsic motivation, self-worth, belongingness, empowerment and self-efficacy. Evidence-informed recommendations are offered to support Sanctum-like programme development for individuals with holistic health needs related to HIV/AIDS diagnoses and lack of access to necessary social determinants of health.

CONCLUSIONS

Stigma associated with HIV/AIDS and living circumstances, such as homelessness, were successfully addressed using harm reduction principles and judgement-free approaches within a family-like environment.

Mechanisms behind gender transformative approaches targeting adolescent pregnancy in low- and middle-income countries: a realist synthesis protocol

INTRODUCTION

Adolescent pregnancy is defined as pregnancy at the age of 19 or below. Pregnancy and childbirth complications are the most significant cause of death among 15-19-year-old girls. Several studies have indicated that inequitable gender norms can increase the vulnerability of adolescent girls, including violence exposure, early marriage, and adolescent pregnancy. To address these disparities, gender transformative approaches aim to challenge and transform restrictive gender norms, roles, and relations through targeted interventions, promoting progressive changes. This realist review aims to synthesise existing evidence from a broad range of data sources to understand how, why, for whom, and in what contexts gender transformative approaches succeed in reducing adolescent pregnancy in low- and middle-income countries.

METHOD AND ANALYSIS

We employ a five-step realist synthesis approach: (1) clarify the scope of review and assessment of published literature, (2) development of initial programme theories, (3) systematic search for evidence, (4) development of refined programme theories, and (5) expert feedback and dissemination of results. This protocol presents the results of the first three steps and provides details of the next steps. We extracted data from 18 studies and outlined eight initial programme theories on how gender transformative approaches targeting adolescent pregnancy work in the first three steps. These steps were guided by experts in the field of sexual and reproductive health, implementation science, and realist methodology. As a next step, we will systematically search evidence from electronic databases and grey literature to identify additional studies eligible to refine the initial programme theories. Finally, we will propose refined programme theories that explain how gender transformative approaches work, why, for whom, and under which circumstances.

ETHICS AND DISSEMINATION

Ethics approval is not required because the included studies are published articles and other policy and intervention reports. Key results will be shared with the broader audience via academic papers in open-access journals, conferences, and policy recommendations. The protocol for this realist review is registered in PROSPERO (CRD42023398293).

Telehealth Diabetes Prevention Program for Adults With Prediabetes in an Academic Medical Center Setting: Protocol for a Hybrid Type III Trial

BACKGROUND

Diabetes is a costly epidemic in the United States associated with both health and economic consequences. These consequences can be mitigated by participation in structured lifestyle change programs such as the National Diabetes Prevention Program (DPP) led by the Centers for Disease Control and Prevention. Mississippi consistently has among the highest rates of diabetes and prediabetes nationally. Implementing the National DPP through large health care systems can increase reach and accessibility for populations at the highest risk for diabetes. Translational research on the National DPP in Mississippi has not been studied.

OBJECTIVE

This study aims to evaluate the implementation and impact of the National DPP delivered using telehealth modalities at the University of Mississippi Medical Center in Jackson, Mississippi.

METHODS

An effectiveness-implementation hybrid type III research design is proposed. The study design is guided by the Reach, Effectiveness, Adoption, Implementation, and Maintenance framework and the Practical, Robust Implementation and Sustainability Model. Participants are being recruited via provider referral, and the DPP is being delivered by trained lifestyle coaches. Study participants include adult (≥18 years) patients eligible for the DPP with at least 1 encounter at 1 of 3 ambulatory clinic specialties (lifestyle medicine, family medicine, and internal medicine) between January 2019 and December 2023. The National DPP eligibility criteria include a BMI ≥25 kg/m2 and hemoglobin A1c between 5.7% and 6.4%. The University of Mississippi Medical Center criteria include Medicare or Medicaid beneficiaries. The University of Mississippi Medical Center's a priori implementation plan was developed using the Consolidated Framework for Implementation Research and includes 23 discrete strategies. The primary aim will use an embedded mixed method process analysis to identify and mitigate challenges to implementation. The secondary aim will use a nonrandomized quasi-experimental design to assess the comparative effectiveness of the DPP on health care expenditures. A propensity score matching method will be implemented to compare case subjects to control subjects. The primary outcomes include patient referrals, participant enrollment, retention, engagement, the incidence of diabetes, and health care resource use and costs.

RESULTS

At baseline, of the 26,151 patients across 3 ambulatory clinic specialties, 1010 (3.9%) had prediabetes and were eligible for the National DPP. Of the 1010 patients, more than half (n=562, 55.6%) were aged 65 years or older, 79.5% (n=803) were Medicare beneficiaries, 65.9% (n=666) were female, and 70.8% (n=715) were obese.

CONCLUSIONS

This is the first translational study of the National DPP in Mississippi. The findings will inform implementation strategies impacting the uptake and sustainability of the National DPP delivered in an academic medical setting using distance learning and telehealth modalities.

TRIAL REGISTRATION

ClinicalTrials.gov NCT04822480; https://clinicaltrials.gov/study/NCT03622580.

INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID)

DERR1-10.2196/50183.

A realist process evaluation of an intervention to promote competencies in interprofessional collaboration among interdisciplinary integrated care teams for older people: Study protocol

Background: International policy is increasingly committed to placing interdisciplinary team-working at the centre of health and social care integration across the lifespan. The National Clinical Programme for Older People in Ireland has a critical role in the design and implementation of the National Older Person's Service Model, which aims to shift the delivery of care away from acute hospitals towards community-based care. Interdisciplinary Community Specialist Teams for older persons (CST-OPs) play an important role in this service model. To support the development of competencies for interprofessional collaboration and an interdisciplinary team-based approach to care integration, a culture shift will be required within care delivery. Design:This study builds upon a collaborative partnership project which co-designed a framework describing core competencies for interprofessional collaboration in CST-OPs. A realist-informed process evaluation of the framework will be undertaken as the competencies described in the framework are being fostered in newly developed CST-OPs under the national scale-up of the service model. Realist evaluation approaches reveal what worked, why it worked (or did not), for whom and under what circumstances. Three iterative and integrated work packages are proposed which combine multiple methods of data collection, analysis and synthesis. Prospective data collection will be undertaken within four CST-OPs, including qualitative exploration of the care experiences of older people and family carers. Discussion: The realist explanatory theory will provide an understanding of how interprofessional collaboration can be fostered and sustained in various contexts of care integration for older people. It will underpin curriculum development for team-based education and training of health and social care professionals, a key priority area in the national Irish health strategy. It will provide healthcare leaders with knowledge of the resources and supports required to harness the benefits of interprofessional collaboration and to realise the goals of integrated care for older people.

Improving Performance in Complex Surroundings: A Mixed Methods Evaluation of Two Hospital Strategies in the Netherlands

BACKGROUND

Hospital strategies aimed at increasing quality of care and simultaneously reducing costs show potential to improve healthcare, but knowledge on real-world effectiveness is limited. In 2014, two Dutch hospitals introduced such quality-driven strategies. Our aim was to evaluate contexts, mechanisms, and outcomes of both strategies using multiple perspectives.

METHODS

We conducted a mixed methods evaluation. Four streams of data were collected and analysed: (1) semi-structured interviewing of 62 stakeholders, such as medical doctors, nurses, managers, general practitioners (GPs), and consultants; (2) financial statements of both organisations and other hospitals in the Netherlands (counterfactual); (3) national database of quality indicators, and patient-reported experiences; and (4) existing material on strategy development and effects.

RESULTS

Both strategies resulted in a relative decrease in volume of care within the hospital, while quality of care has not been affected negatively. One hospital failed to cut operating costs sufficiently, resulting in declining profit margins. We identified six main mechanisms that impacted these outcomes: (1) Quality-improvement projects spur change and commitment; (2) increased coordination between hospital and primary care leads to substitution of care; (3) insufficient use of data and support hinder quality improvement; (4) scaling down hospital facilities is required to convert volume reductions to cost savings; (5) shared savings through global budgets lead to shared efforts between payer and hospital; and (6) financial security for physicians facilitates shift towards quality-driven care.

CONCLUSION

This integrated analysis of mixed data sources demonstrated that the institution-wide nature of the strategies has induced a shift from a focus on production towards quality of care. Longer-term (financial) sustainability of hospital strategies aimed at decelerating production growth requires significant efforts in reducing fixed costs. This strategy poses financial risks for the hospital if operating costs are insufficiently reduced or if payer alignment is compromised.

How health systems facilitate patient-centered care and care coordination: a case series analysis to identify best practices

Large- and small-scale transformation of healthcare delivery toward improved patient experience through promotion of patient-centered and coordinated care continues to be at the forefront of health system efforts in the United States. As part of a Quality Improvement (QI) project at a large, midwestern health system, a case series of high-performing organizations was explored with the goal of identifying best practices in patient-centered care and/or care coordination (PCC/CC). Identification of best practices was done through rapid realist review of peer-reviewed literature supporting three PCC/CC interventions per case. Mechanisms responsible for successful intervention outcomes and associated institutional-level facilitators were evaluated, and cross-case analysis produced high-level focus items for health system leadership, including (1) institutional values surrounding PCC/CC, (2) optimization of IT infrastructure to enhance performance and communication, (3) pay structures and employment models that enhance accountability, and (4) organizing bodies to support implementation efforts. Health systems may use this review to gain insight into how institutional-level factors may facilitate small-scale PCC/CC behaviors, or to conduct similar assessments in their own QI projects. Based on our analysis, we recommend health systems seeking to improve PCC/CC at any level or scale to evaluate how IT infrastructure affects provider-provider and provider-patient communication, and the extent to which institutional prioritization of PCC/CC is manifest and held accountable in performance feedback, incentivization, and values shared among departments and settings. Ideally, this evaluation work should be performed and/or supported by cross-department organizing bodies specifically devoted to PCC/CC implementation work.

A human centred approach to digital technologies in health care delivery among mothers, children and adolescents

BACKGROUND

Healthcare outcomes in child, adolescent and maternal in Tanzania are poor, and mostly characterised by fragmentary service provision. In order to address this weakness, digital technologies are sought to be integrated in the milieu of health as they present vast opportunities especially in the ability to improve health information management and coordination. Prior to the design and implementation of the Afya-Tek digital intervention, formative research was carried out to ensure that the solution meets the needs of the users. The formative research aimed to examine: the burden of disease and related health seeking behaviour; workflow procedures and challenges experiencing healthcare actors; adolescent health and health seeking behaviour; and lastly examine technological literacy and perceptions on the use of digital technologies in healthcare delivery. This paper therefore, presents findings from the formative research.

METHODS

The study employed exploratory design grounded in a qualitative approach. In-depth interview, focus group discussion, participant observation and documentary review methods were used for collecting data at different levels. The analysis was done thematically, whereby meaning was deduced behind the words which the participants used.

RESULTS

Findings suggest that the perceived burden of diseases and health seeking behaviour differ across age and social group. Multiple work-related challenges, such as lack of proper mechanism to track referrals and patient's information were noted across healthcare actors. There was a keen interest in the use of technologies shown by all study participants to improve care coordination and health outcomes among health system actors. Participants shared their views on how they envision the digital system working.

CONCLUSION

The formative research provided insightful background information with regard to the study objectives. The findings are used for informing the subsequent phases of the co-development and implementation of the Afya-Tek digital health intervention; with a view to making it relevant to the needs of those who will use it in the future. As such, the findings have to a large extent met the purpose of the current study by envisaging the best ways to design digital intervention tailored to meet the needs of those who will be using it.

Expanding the Focus of Occupational Safety and Health: Lessons from a Series of Linked Scientific Meetings

There is widespread recognition that the world of work is changing, and agreement is growing that the occupational safety and health (OSH) field must change to contribute to the protection of workers now and in the future. Discourse on the evolution of OSH has been active for many decades, but formalized support of an expanded focus for OSH has greatly increased over the past 20 years. Development of approaches such as the National Institute for Occupational Safety and Health (NIOSH)'s Total Worker Health® concept and the World Health Organization (WHO)'s Healthy Workplace Framework are concrete examples of how OSH can incorporate a new focus with a wider view. In 2019, NIOSH initiated a multi-year effort to explore an expanded focus for OSH. This paper is a report on the outputs of a three-year cooperative agreement between NIOSH and The University of Texas School of Public Health, which led to subject matter expert workshops in 2020 and an international conference of global interest groups in 2021. This article traces the background of these meetings and identifies and assesses the lessons learned. It also reviews ten thematic topics that emerged from the meetings: worker health inequalities; training new OSH professionals; future OSH research and practice; tools to measure well-being of workers; psychosocial hazards and adverse mental health effects; skilling, upskilling and improving job quality; socioeconomic influences; climate change; COVID-19 pandemic influences; and strategic foresight. Cross-cutting these themes is the need for systems and transdisciplinary thinking and operationalization of the concept of well-being to prepare the OSH field for the work of the future.

Supporting direct support professionals in enabling people with intellectual disabilities to engage in meaningful activities: protocol for the Meaningful Activities 4 All (MA4A) study based on the human-centred design process

INTRODUCTION

Meaningful activities (MA) have a positive impact on identity, well-being, participation and inclusion. Although people with intellectual disabilities (PID) depend on their direct support professionals (DSPs) to engage in MA, the DSPs need support which could enable them to offer more qualitative care and support.

METHODS AND ANALYSIS

To identify DSPs' needs, and to develop a tool/service, an innovative and iterative approach is developed, based on the human-centred design (HCD) process, combined with traditional qualitative and quantitative research methods. In the inspiration phase (needs analysis), in-depth interviews will be conducted in two day care centres in Flanders using an interpretative phenomenological analyses, one with a supply-driven approach and the other with a demand-driven approach, followed by a survey sent to all Flemish day care centres. In the ideation phase, the insights of phase 1 will guide a cocreation process (comprising a World Cafe, brainstorm and prototype sessions) with the DSPs, PID and other stakeholders. In the implementation phase, the solution will be tested in the two day care centres from phase 1 by means of living labs and a realist evaluation. By adopting this protocol, the functionality, quality, usability and acceptance are expected to increase. This protocol adopts all phases of the HCD process and shows the complementarity of HCD with traditional research methods. PID and the DSPs will benefit as the end result is truly grounded in their specific needs and wishes.

ETHICS AND DISSEMINATION

Ethical approval by the Ethics Committee of the University Hospital Ghent, Belgium (reference numbers: B670202042983 and PA2021-091). All participants will sign informed consent forms. Results of this study will be submitted for publication in relevant peer-reviewed journals and will be presented at relevant conferences.

Realist evaluation of the impact of the research translation process on health system sustainability: a study protocol

INTRODUCTION

Sustainability at a system level relates to the capacity of the system to be able to service the ongoing health needs of the population. It is a multifaceted concept encompassing both the affordability and efficiency of a system and the system's ability to adapt and change.To address issues that currently threaten health system sustainability, healthcare leaders, policy makers, clinicians and researchers are searching for solutions to ensure the delivery of safe, value-based care into the future. The timely translation of research evidence into sustainable interventions that can be adopted into the health system is one way of bolstering the sustainability of the system as a whole. We present a protocol for the realist evaluation of a research translation funding programme to understand how the research translation process contributes to health system sustainability and value-based healthcare.

METHODS AND ANALYSIS

Underpinned by the realist evaluation framework, we will: (1) Develop the Initial Program Theory (IPT) of the research translation process; (2) Test the program theory through case study analysis; and (3) Refine and consolidate the theory through stakeholder consultation. The evaluation uses a case example of a research translation programme, chosen for its representation of a microcosm of the broader health system and the heterogeneity of service improvement activities taking place within it. Across the three phases, analysis of data from documents about the research translation program and interviews and focus groups with stakeholders and program users will draw on the context (C), mechanism (M), outcome (O) formula that is core to realist evaluation. In addition, system dynamic methods will capture the feedback loops and complex relationships among the IPT and context-mechanism-outcome configurations. This approach to evaluation of a research translation funding programme may be adapted to similar programmes operating in other settings.

ETHICS AND DISSEMINATION

Curtin University Human Research Ethics Committee, Western Australia, approved this study (approval number: HRE2020-0464). Results will be published in scientific journals, and communicated to respondents and relevant partners.

Integrated Child and Family Hub models for detecting and responding to family adversity: protocol for a mixed-methods evaluation in two sites

INTRODUCTION

Integrated community healthcare Hubs may offer a 'one stop shop' for service users with complex health and social needs, and more efficiently use service resources. Various policy imperatives exist to implement Hub models of care, however, there is a dearth of research specifically evaluating Hubs targeted at families experiencing adversity. To contribute to building this evidence, we propose to co-design, test and evaluate integrated Hub models of care in two Australian community health services in low socioeconomic areas that serve families experiencing adversity: Wyndham Vale in Victoria and Marrickville in New South Wales.

METHODS AND ANALYSIS

This multisite convergent mixed-methods study will run over three phases to (1) develop the initial Hub programme theory through formative research; (2) test and, then, (3) refine the Hub theory using empirical data. Phase 1 involves co-design of each Hub with caregivers, community members and practitioners. Phase 2 uses caregiver and Hub practitioner surveys at baseline, and 6 and 12 months after Hub implementation, and in-depth interviews at 12 months. Two stakeholder groups will be recruited: caregivers (n=100-200 per site) and Hub practitioners (n=20-30 per site). The intervention is a co-located Hub providing health, social, legal and community services with no comparator. The primary outcomes are caregiver-reported: (i) identification of, (ii) interventions received and/or (iii) referrals received for adversity from Hub practitioners. The study also assesses child, caregiver, practitioner and system outcomes including mental health, parenting, quality of life, care experience and service linkages. Primary and secondary outcomes will be assessed by examining change in proportions/means from baseline to 6 months, from 6 to 12 months and from baseline to 12 months. Service linkages will be analysed using social network analysis. Costs of Hub implementation and a health economics analysis of unmet need will be conducted. Thematic analysis will be employed to analyse qualitative data.

ETHICS AND DISSEMINATION

Royal Children's Hospital and Sydney Local Health District ethics committees have approved the study (HREC/62866/RCHM-2020). Participants and stakeholders will receive results through meetings, presentations and publications.

TRIAL REGISTRATION NUMBER

ISRCTN55495932.

The evolution and inter-sectoral interaction mechanism of China's national medical alliance: An analysis based on complex systems theory

This work investigates the performance and inter-sectoral interaction mechanism of China's largest vertically integrated care network, the national medical alliance (NMA). The data collected derive from the China Health Statistics Bulletin and the China Health Statistical Yearbook for the period 2009-2018. The data include 64 observation indicators for five medical sectors in the NMA, namely, tertiary hospitals (THS), secondary hospitals (SHS), community health centres (CHCS), township hospitals (TsHS) and professional public health institutions (PPHIS). This research combines complex systems theory with a multilevel structural dynamic factor model, and yields two main results. First, although the trend for the NMA's global factor is increasing, the evolutionary paths for sectoral factors differ substantially. Among the sectoral factors, the sectoral factor of THS continued to decline, and neither the sectoral factor of CHCS nor the sectoral factor of TsHS has significantly improved. Then, the interaction mechanism between the various NMA sectors is investigated. While a close relationship has been formed between THS and CHCS and between SHS and CHCS, there remains no close two-way relationship between either THS and TsHS or THS and SHS. Thus, going forward, to reach the policy expectations, China's NMA implementation must consider the interaction between different constituent sectors.

Capabilities for transdisciplinary research

Problems framed as societal challenges have provided fresh impetus for transdisciplinary research. In response, funders have started programmes aimed at increasing transdisciplinary research capacity. However, current programme evaluations do not adequately measure the skills and characteristics of individuals and collectives doing this research. Addressing this gap, we propose a systematic framework for evaluating transdisciplinary research based on the Capability Approach, a set of concepts designed to assess practices, institutions, and people based on public values. The framework is operationalized through a mixed-method procedure which evaluates capabilities as they are valued and experienced by researchers themselves. The procedure is tested on a portfolio of ‘pump-priming’ research projects in the UK. We find these projects are sites of capability development in three ways: through convening cognitive capabilities required for academic practice; cultivating informal tacit capabilities; and maintaining often unacknowledged backstage capabilities over durations that extend beyond the lifetime of individual projects. Directing greater attention to these different modes of capability development in transdisciplinary research programmes may be useful formatively in identifying areas for ongoing project support, and also in steering research system capacity towards societal needs.

Developing a patient-oriented realist evaluation for COVID-19 vaccine implementation in Saskatchewan: a methodologic framework

BACKGROUND

There is an urgent need to inform decision-making and safe delivery of vaccines in a timely manner. Our objective is to describe the methods we used to perform a patient-oriented realist evaluation of COVID-19 vaccination implementation in Saskatchewan, Canada, in order to understand the underlying mechanisms and contexts of vaccination implementation and vaccine uptake.

METHODS

This methodology paper describes a patient-oriented, realist, mixed-method evaluation to assess COVID-19 vaccination implementation in Regina, Saskatoon and Prince Albert, Saskatchewan. The study comprised 3 iterative phases guided by Realist And Meta-narrative Evidence Synthesis: Evolving Standards II (RAMESES II). In phase 1 (January-February 2021), we developed the initial program theory, in phase 2 (March-May 2021), we tested and refined the initial program theory, and in phase 3 (June-July 2021), we established the final program theory. Three patient and family partners with different backgrounds and experiences were selected purposively from various locations (urban and rural) in Saskatchewan to engage collaboratively in the evaluation. Data analysis and synthesis occurred at all 3 phases of the project. We analysed qualitative data from phases 2 and 3 using a "retroductive" approach. We used quantitative data to compare outcomes from the 3 sites.

INTERPRETATION

This protocol describes how we developed a final program theory for COVID-19 vaccination implementation with patient and family partners to show for whom, under what circumstances, how and why Saskatchewan's COVID-19 vaccination program has led to vaccine uptake. With patient and family partners' engagement, the evaluation findings will be shared with the Saskatchewan Health Authority and provincial government policy-makers and communications departments, published in peer-reviewed journals, presented at provincial or national conferences, and disseminated through any additional media identified by the patient and family partners.

Health and Social Care Reform in Scotland - What Next?

INTRODUCTION

This paper analyses the important enablers, barriers and impacts of country-wide implementation of integrated health and social care in Scotland. It offers insights for other systems seeking to advance similar policy and practice.

DESCRIPTION

Landmark legislation was based on a shared vision and narrative about improving outcomes for people and communities. Implementation has involved coordination of multiple policies and interventions for different life stages, care groups, care settings and local context within a dynamic and complex system.

DISCUSSION

Relational and citizen led approaches are critical for success, but it takes time to build trusting relationships, influence organisational and professional cultures and cede power. Assessing national impacts is challenging and progress at a national level can seem slower than local experience suggests, due in part to the relative immaturity of national datasets for community interventions. Five years on there are many examples of innovation and positive outcomes despite increasing demographic, workforce, and financial challenges. However, inequalities continue to increase.

CONCLUSION

Realising the true value from integration will require a stronger focus on place-based prevention and early intervention to achieve a fairer Scotland where everybody thrives. Solidarity, equity, and human rights must guide the next phase of Scotland's story.

Using Realist Evaluation to Understand Process Outcomes in a COVID-19-Impacted Yoga Intervention Trial: A Worked Example

Realist evaluation offers a valuable way to understand how interventions function and thus how they can be improved and locally adapted. Consequently, realist evaluation is increasingly conducted in parallel with intervention trials. It comprises a clear philosophical foundation and view of causality, pragmatic mixed data collection methods, and a theory-driven approach in which hypothesised program theories are tested and refined. However, detailed methods for data analysis are seldom well-described in realist studies and no clear method for analysing and presenting realist evaluation data has yet emerged. In this methodological paper we use the worked example of our realist process evaluation of the SAGE yoga trial to illustrate an applied process of data analysis and presentation of findings. We show how we drew on other realist studies for ideas, provide examples of six key tasks involved in conducting a realist process evaluation (including coding data and structuring results) and describe strategies that did not work and our rationale for rejecting them. This detailed account of the decisions and methods that worked for us is intended to provide a practical and informed point of departure for researchers conducting a realist evaluation.

A Non-Pharmacologic Approach to Manage Behaviours in Confused Medically Ill Older Adults in Acute Care

Background

Non-pharmacological interventions are recommended to manage challenging behaviours among cognitively impaired older adults, however few studies have enrolled patients in acute care. This study aimed to determine the feasibility of implementing non-pharmacological interventions to manage behaviours in hospitalized older adults.

Method

A self-identity approach was used to identify potentially engaging activities for 13 older medically ill adults admitted to acute hospital; these activities were trialed for a two-week period. Data were collected on frequency of intervention administration and assistance required, as well as frequency of behaviours and neuroleptic use in the seven days prior to and following the trial of activities.

Results

Per participant, 5–11 interventions were prescribed. Most frequently interventions were tried two or more times (46%); 9% were not tried at all. Staff or family assistance was not required for 27% of activities. The mean number of documented behaviours across participants was 4.8 ± 2.3 in the pre-intervention period and 2.1 ± 1.9 in the post-intervention period. Overall the interventions were feasible and did not result in increasing neuroleptic use

Conclusion

Non-pharmacologic interventions may be feasible to implement in acute care. More research in this area is justified.

Understanding the impact of a residential housing programme for people living with HIV/AIDS: a realist evaluation protocol

INTRODUCTION

Housing instability and homelessness are significant barriers to medical treatment for people living with HIV/AIDS. For these individuals, lack of stable housing and stigma is associated with insufficient access to care, poor adherence to medication and higher cost burdens to the healthcare system. This protocol reports on the efforts to evaluate Sanctum V.1.0, a hospice and transitional care home for adults with HIV/AIDS in Saskatoon, Saskatchewan, Canada. The current project was developed out of a need to identify how Sanctum V.1.0 produces varying programme outcomes to assist in endeavours to replicate the programme in other geographic locations.

METHODS AND ANALYSIS

A realist evaluation will be conducted to explore how and why Sanctum V.1.0 is successful or unsuccessful, in which circumstances and for whom. Rather than explore the degree to which a programme is effective, realist evaluations seek to uncover mechanisms that explain processual links between programme inputs and outcomes. The completed first phase of the project involved the development of an initial realist programme theory. Phases 2 and 3 will consist of methods to test, refine and validate the initial theory using various data sources.

ETHICS AND DISSEMINATION

Ethics approval was obtained from the institutional review board at the University of Saskatchewan on 2 July 2020. Results will be disseminated according to stakeholders' desires.

Telehealth Diabetes Prevention Intervention for the Next Generation of African American Youth: Protocol for a Pilot Trial

Background

In 1999, type 2 diabetes mellitus (T2DM) was identified as an emerging epidemic in youth, and racial and ethnic minority youth were identified with high risk. Two decades later, no gold standard T2DM prevention intervention has been established for this population.

Objective

This study tests the efficacy of a telehealth diabetes prevention intervention for African American (AA) families with children with risk for T2DM. Concurrently, investigators aim to evaluate an implementation strategy for the uptake of the intervention by the University of Mississippi Medical Center’s (UMMC) pediatric weight management clinic.

Methods

This single-arm trial will enroll 20 parents with overweight or obesity of children (8-11 years) with overweight or obesity, both of whom are at risk for T2DM. Parents will meet in small groups (5 parents per group) weekly for 11 weeks and then monthly for 4 monthly maintenance sessions via videoconference using Wi-Fi–enabled iPads with cellular connectivity. The intervention will be adapted from the National Diabetes Prevention Program and Power to Prevent, a diabetes prevention program tailored for AA families. The same lifestyle intervention facilitated by a racially concordant lifestyle coach trained in the Diabetes Prevention Program will be delivered to all groups (n=4). Participants will be recruited in-person during patient encounters at the UMMC’s pediatric weight management clinic. Sessions will consist of dietary and physical activity behavior change strategies facilitated using problem-solving and goal-setting skills. The implementation strategy has 2 targets: the pediatric weight management clinic site and clinical team and parents of children at risk for T2DM engaged in intensive obesity treatment to prevent T2DM. The multifaceted implementation protocol includes 4 discrete strategies: creating a new clinical team, changing the service site, intervening with families, and promoting organizational readiness for change.

Results

Recruitment and enrollment began in December 2020, and the intervention is scheduled to be delivered to the first cohort of parents in March 2021. The results are expected to be submitted for publication beginning in November 2021 through 2022. The primary outcome measure for the pilot trial will include changes from baseline to 12 and 30 weeks in the child BMI z score and parent BMI. The implementation evaluation will include multiple measures of feasibility, acceptability, appropriateness, fidelity, and efficacy. This protocol was approved by the UMMC’s Institutional Review Board (#2020V0249).

Conclusions

The proposed intervention approach is supported by the scientific literature and is scalable given the current and future health care subsidies for telehealth. Findings from this pilot trial will begin to address critical barriers to defining a gold standard lifestyle intervention for AA families with children at risk for T2DM. If effective, the intervention could be feasibly disseminated to treat obesity and prevent T2DM in high-risk AA pediatric populations.

International Registered Report Identifier (IRRID)

PRR1-10.2196/25699

Exploring the dimensions of patient experience for community-based care programmes in a multi-ethnic Asian context

Introduction

The aim of this study is to explore patients’ experiences with community-based care programmes (CCPs) and develop dimensions of patient experience salient to community-based care in Singapore. Most countries like Singapore are transforming its healthcare system from a hospital-centric model to a person-centered community-based care model to better manage the increasing chronic disease burden resulting from an ageing population. It is thus critical to understand the impact of hospital to community transitions from the patients’ perspective. The exploration of patient experience will guide the development of an instrument for the evaluation of CCPs for quality improvement purposes.

Methods

A qualitative exploratory study was conducted where face-to-face in-depth interviews were conducted using a purposive sampling method with patients enrolled in CCPs. In total, 64 participants aged between 41 to 94 years were recruited. A deductive framework was developed using the Picker Patient Experience instrument to guide our analysis. Inductive coding was also conducted which resulted in emergence of new themes.

Results

Our findings highlighted eight key themes of patient experience: i) ensuring care continuity, ii) involvement of family, iii) access to emotional support, vi) ensuring physical comfort, v) coordination of services between providers, vi) providing patient education, vii) importance of respect for patients, and viii) healthcare financing.

Conclusion

Our results demonstrated that patient experience is multi-faceted, and dimensions of patient experience vary according to healthcare settings. As most patient experience frameworks were developed based on a single care setting in western populations, our findings can inform the development of a culturally relevant instrument to measure patient experience of community-based care for a multi-ethnic Asian context.

Evaluating Large-Scale Integrated Care Projects: The Development of a Protocol for a Mixed Methods Realist Evaluation Study in Belgium

BACKGROUND

The twelve Integrated Care Program pilot projects (ICPs) created by the government plan 'Integrated Care for Better Health' aim to achieve four outcome types (the Quadruple Aim) for people with chronic diseases in Belgium: improved population health, improved patient and provider experiences and improved cost efficiency. The aim of this article is to present the development of a mixed methods realist evaluation of this large-scale, whole system change programme.

METHODS

A scientific team was commissioned to co-design and implement an evaluation protocol in close collaboration with the government, the ICPs and several other involved stakeholders.

RESULTS

A protocol for a mixed methods realist evaluation was developed to gain insights into the mechanisms that foster successful results in ICPs. The qualitative evaluation proposed will be based on the document analysis of yearly ICP progress reports, selected case studies and focus group interviews with stakeholders. Processes and outcomes of all the projects will be monitored using indicators based on administrative data on population health and the quality and costs of care. A yearly survey will be organized to collect data on patient-reported outcomes and experiences and on provider-reported measures of inter-professional collaboration and proper wellbeing. Using both quantitative and qualitative data, we will develop theories about the mechanisms and the associated contextual factors that lead to integrated care and the Quadruple Aim outcomes.

DISCUSSION

The objective of this study is to deliver policy recommendations on strategies and best practices to improve care integration in Belgium and to implement a sustainable monitoring system that serves both policy makers and the stakeholders within the ICPs. Some challenges due to the large scale of the project and the multiple stakeholders involved may impede the successful implementation of this proposal.

Shifting care from hospital to community, a strategy to integrate care in Singapore: process evaluation of implementation fidelity

Background

Accessibility to efficient and person-centered healthcare delivery drives healthcare transformation in many countries. In Singapore, specialist outpatient clinics (SOCs) are commonly congested due to increasing demands for chronic care. To improve this situation, the National University Health System (NUHS) Regional Health System (RHS) started an integrated care initiative,the Right-Site Care (RSC) program in 2014. Through collaborations between SOCs at the National University Hospital and primary and community care (PCC) clinics in the western region of the county, the program was designed to facilitate timely discharge and appropriate transition of patients, who no longer required specialist care, to the community. The aim of this study was to evaluate the implementation fidelity of the NUHS RHS RSC program using the modified Conceptual Framework for Implementation Fidelity (CFIF), at three distinct levels; providers, organizational, and system levels to explain outcomes of the program and to inform further development of (similar) programs.

Methods

A convergent parallel mixed methods study using the realist evaluation approach was used. Data were collected between 2016 and 2018 through non-participatory observations, reviews of medical records and program database, together with semi-structured interviews with healthcare providers. Triangulation of data streams was applied guided by the modified CFIF.

Results

Our findings showed four out of six program components were implemented with low level of fidelity, and 9112 suitable patients were referred to the program while 3032 (33.3%) declined to be enrolled. Moderating factors found to influence fidelity included: (i) complexity of program, (ii) evolving providers’ responsiveness, (iii) facilitation through synergistic partnership, training of PCC providers by specialists and supportive structures: care coordinators, guiding protocols, shared electronic medical record and shared pharmacy, (iv) lack of organization reinforcement, and (v) mismatch between program goals, healthcare financing and providers’ reimbursement.

Conclusion

Functional integration alone is insufficient for a successful right-site care program implementation. Improvement in relationships between providers, organizations, and patients are also warranted for further development of the program.

Conducting a Cost-Benefit Analysis of Transitional Care Programmes: The Key Challenges and Recommendations

Transitional care encompasses a range of services designed to promote care integration as patients transfer between different locations or different levels of care. Transitional care programmes have been proven to produce positive outcomes in reducing hospital readmissions and improving patients’ health outcomes. However, little is known about the benefits of the programmes on healthcare cost and the published results have been inconsistent. With increasing healthcare expenditures and limited public healthcare resources, cost-benefit analyses become paramount in informing healthcare resource allocation decisions. This perspective paper describes the approaches used in estimating the total costs of a bundle of transitional care services from an academic medical centre, identifies the key methodological challenges encountered in the process of cost-benefit analysis, and recommends potential solutions to tackle these challenges. By providing a comprehensive perspective on the methodological challenges, this paper encourages program evaluators to take these possible challenges into consideration for future cost-benefit analyses.

Challenges in turning a great idea into great health policy: the case of integrated care

BACKGROUND

In the organization of health care and health care systems, there is an increasing trend towards integrated care. Policy-makers from different countries are creating policies intended to promote cooperation and collaboration between health care providers, while facilitating the integration of different health care services. Hopes are high, as such collaboration and integration of care are believed to save resources and improve quality. However, policy-makers are likely to encounter various challenges and limitations when attempting to turn these great ideas into effective policies. In this paper, we look into these challenges.

MAIN BODY

We argue that the organization of health care and integrated care is of public concern, and should thus be of crucial interest to policy-makers. We highlight three challenges or limitations likely to be encountered by policy-makers in integrated care. These are: (1) conceptual challenges; (2) empirical/methodological challenges; and (3) resource challenges. We will argue that it is still unclear what integrated care means and how we should measure it. 'Integrated care' is a single label that can refer to a great number of different processes. It can describe the integration of care for individual patients, the integration of services aimed at particular patient groups or particular conditions, or it can refer to institution-wide collaborations between different health care providers. We subsequently argue that health reform inevitably possesses a political context that should be taken into account. We also show how evidence supporting integrated care may not guarantee success in every context. Finally, we will discuss how promoting collaboration and integration might actually demand more resources. In the final section, we look at three different paradigmatic examples of integrated care policy: Norway, the UK's NHS, and Belgium.

CONCLUSIONS

There seems widespread agreement that collaboration and integration are the way forward for health care and health care systems. Nevertheless, we argue that policy-makers should remain careful; they should carefully consider what they hope to achieve, the amount of resources they are willing to invest, and how they will evaluate the success of their policy.

Mixed methods protocol for a realist evaluation of electronic personal health records design features and use to support medication adherence (ePHRma)

BACKGROUND

National Health Service policy suggests that increasing usage of electronic personal health records (PHR) by patients will result in cost savings and improved public health. Medication adherence means that patients take their prescribed medication as agreed with their doctors. Some of the claimed benefits of PHRs are decreasing healthcare costs and improving medication adherence and patient outcomes.

METHODS

This is a mixed methods convergent study, primarily qualitative. The qualitative and quantitative data collection and analysis will occur in parallel, and then be synthesised. We are interviewing and surveying adults with long-term conditions to identify what are the most important and useful features of their current PHR. The data collection comprises patient demographics, the Medication Adherence Questionnaire, the personality scale Big Five Inventory-2 Extra-Short Form and the WHO Quality of Life-BREF scale. Qualitative data will be analysed using the Framework method.

ETHICS

We have received a favourable ethical opinion from the Health Research Authority/Research Ethics Committee.

Does a Long-Term Quality and Safety Curriculum for Health Care Professionals Improve Clinical Practice? An Evaluation of Quality Improvement Projects

INTRODUCTION

Little is known about the effects on clinical practice of continuing education quality and safety curricula. The aim of this study is to gain insight into learning outcomes on the fourth level of the Kirkpatrick evaluation model for systematically deployed quality improvement projects performed by health care professionals during a Masters in Healthcare Quality and Safety in the Netherlands.

METHODS

The researchers reviewed 35 projects led by health care professionals in 16 different hospitals to determine their scopes and effects. Afterward, professionals took an online survey to determine the extent of their project's sustainability and spread.

RESULTS

Improving health care safety was the most prevalent quality dimension (n = 11, 31%). A positive change was measured by professionals for 64% (n = 35) of the primary outcomes. Statistical significance was measured in 19 (35%) of the outcomes, of which nine (47%) were found to have a statistically significant effect. A minority of professionals (17%) judged their project as sustained by the department, while some stated that the intervention (37%) or the results of the project (11%) had spread.

DISCUSSION

Although most projects indicated an improvement in their primary outcomes, only a few resulted in statistically significant changes. Teaching professionals in using evaluation methods that take into account the complex context where these projects are performed and teaching them leadership skills is needed to reduce the likelihood of unmeasured outcomes. Analyzing learning experiences of professionals in performing the project is important to see what they learned from performing quality improvement projects, providing experiences that may lead to sustainable effects in future projects.

A Qualitative Evaluation of a Clinical Faculty Mentorship Program Using a Realist Evaluation Approach

OBJECTIVE

Clinically focused faculty (full-time clinical faculty and clinician educators) comprise an increasing proportion of academic faculty, yet they underutilize mentorship nationally. The aims of this study were to test and refine a program theory for an institutional mentorship program for junior clinically focused faculty and to understand the facilitators and barriers of sustained participation.

METHODS

We conducted a qualitative study using a realist evaluation approach. Between July and December 2017, we performed in-depth semistructured interviews of 2 participant groups from a junior faculty mentorship program at our institution: 1) those who attended more than two thirds of the program sessions; and 2) those who only attended 1 session. We used inductive thematic analysis to identify key context and program mechanisms that led to meaningful outcomes for faculty mentorship.

RESULTS

We interviewed 23 junior faculty representing 15 pediatric specialties. We identified 4 contextual themes (past personal experience, current competing priorities, institutional culture, and gaps in support and resources), 3 mechanisms (connecting with faculty, sharing ideas and strategies, and self-reflecting), and 3 outcomes (sense of community, acquired tools and skills, and broadened perspectives), which we organized into a programmatic theory representing the program's impact on participants. Themes that emerged were consistent between both groups.

CONCLUSIONS

A mentorship program that provided junior faculty with opportunities to connect, share ideas and strategies, and self-reflect led to improvement in meaningful outcomes for clinically focused faculty. Our program theory provides a basis for institutions seeking to build a mentorship program targeted towards this increasing proportion of junior faculty.

Protocol for the impact of paramedics in NHS primary care: application of realist approaches to improve understanding and support intelligent policy and future workforce planning

Introduction:

In the United Kingdom, changing demands on ambulance services has caused a change in what is expected of a paramedic. As well as advanced life support, paramedics now need to be skilled in managing a range of urgent case presentations, with emphasis on treat-at-scene. The change in the scope of work paramedics can undertake has established their role within primary care. However, as paramedics transition to roles within primary care, their knowledge and skillset will undoubtedly need to change. The current opportunities for paramedics’ employment in primary care require careful evaluation. In order to contribute to patients’ and the NHS’ primary care agenda, evidence must be generated to show how and why these changes would work, for whom, in what context and to what extent.

Methods and analysis:

The purpose of this research is to produce findings that will improve our understanding of the ways in which (i.e. how, why and in what contexts) paramedics impact on the primary care workforce. A theory-driven approach to evidence-synthesis will be conducted in a realist review, to produce a programme theory. This programme theory will be tested using empirical data collected through a realist evaluation. Survey and interview data will be collected from paramedics working in primary care, general practitioners and patients to assess under which contexts, and by which mechanisms, paramedics are working in primary care, and thus test the programme theory. Based on the findings, we will be able to highlight the role of paramedics in primary care, as well as how they operate and under what conditions.

Ethics and dissemination:

Formal ethics review is not required for the review, as it is secondary research, but will be sought for the evaluation. Findings will be disseminated in a peer-reviewed journal, at national and international conferences and to relevant professional associations.

A scaling-up strategy supporting the expansion of integrated care: a study protocol

Purpose

To ensure that more people will benefit from integrated care initiatives, scaling-up of successful initiatives is the way forward. However, new challenges present themselves as knowledge on how to achieve successful large-scale implementation is scarce. The EU-funded project SCIROCCO uses a step-based scaling-up strategy to explore what to scale-up, and how to scale-up integrated care initiatives by matching the complementary strengths and weaknesses of five European regions involved in integrated care. The purpose of this paper is to describe a multi-method evaluation protocol designed to understand what factors influence the implementation of the SCIROCCO strategy to support the scaling-up of integrated care.

Design/methodology/approach

The first part of the protocol focuses on the assessment of the implementation fidelity of the SCIROCCO step-based strategy. The objective is to gain insight in whether the step-based strategy is implemented as it was designed to explore what works and does not work when implementing the scaling-up strategy. The second part concerns a realist evaluation to examine what it is about the SCIROCCO’s strategy that works for whom, why, how and in which circumstances when scaling-up integrated care.

Findings

The intended study will provide valuable information on the implementation of the scaling-up strategy which will help to explain for what specific reasons the implementation succeeds and will facilitate further improvement of project outcomes.

Originality/value

The expected insights could be useful to guide the development, implementation and evaluation of future scaling-up strategies to advance the change towards more sustainable health and care systems.

Retrospective evaluation of healthcare utilisation and mortality of two post-discharge care programmes in Singapore

OBJECTIVE

To evaluate the impact on healthcare utilisation frequencies and charges, and mortality of a programme for frequent hospital utilisers and a programme for patients requiring high acuity post-discharge care as part of an integrated healthcare model.

DESIGN

A retrospective quasi-experimental study without randomisation where patients who received post-discharge care interventions were matched 1:1 with unenrolled patients as controls.

SETTING

The National University Health System (NUHS) Regional Health System (RHS), which was one of six RHS in Singapore, implemented the NUHS RHS Integrated Interventions and Care Extension (NICE) programme for frequent hospital utilisers and the NUHS Transitional Care Programme (NUHS TCP) for high acuity post-discharge care. The programmes were supported by the Ministry of Health in Singapore, which is a city-state nation located in Southeast Asia with a 5.6 million population.

PARTICIPANTS

Linked healthcare administrative data, for the time period of January 2013 to December 2016, were extracted for patients enrolled in NICE (n=554) or NUHS TCP (n=270) from June 2014 to December 2015, and control patients.

INTERVENTIONS

For both programmes, teams conducted follow-up home visits and phone calls to monitor and manage patients' post-discharge.

PRIMARY OUTCOME MEASURES

One-year pre- and post-enrolment healthcare utilisation frequencies and charges of all-cause inpatient admissions, emergency admissions, emergency department attendances, specialist outpatient clinic (SOC) attendances, total inpatient length of stay and mortality rates were compared.

RESULTS

Patients in NICE had lower mortality rate, but higher all-cause inpatient admission, emergency admission and emergency department attendance charges. Patients in NUHS TCP did not have lower mortality rate, but had higher emergency admission and SOC attendance charges.

CONCLUSIONS

Both NICE and NUHS TCP had no improvements in 1 year healthcare utilisation across various setting and metrics. Singular interventions might not be as impactful in effecting utilisation without an overhauling transformation and restructuring of the hospital and healthcare system.

Implementation fidelity of a strategy to integrate service delivery: learnings from a transitional care program for individuals with complex needs in Singapore

BACKGROUND

To cope with rising demand for healthcare services in Singapore, Regional Health Systems (RHS) comprising of health and social care providers across care settings were set up to integrate service delivery. Tasked with providing care for the western region, in 2012, the National University Health System (NUHS) - RHS developed a transitional care program for elderly patients with complex healthcare needs who consumed high levels of hospital resources. Through needs assessment, development of personalized care plans and care coordination, the program aimed to: (i) improve quality of care, (ii) reduce hospital utilization, and (iii) reduce healthcare-related costs. In this study, recognizing the need for process evaluation in conjunction with outcome evaluation, we aim to evaluate the implementation fidelity of the NUHS-RHS transitional care program to explain the outcomes of the program and to inform further development of (similar) programs.

METHODS

Guided by the modified version of the Conceptual Framework for Implementation Fidelity (CFIF), adherence and moderating factors influencing implementation were assessed using non-participatory observations, reviews of medical records and program databases.

RESULTS

Most (10 out of 14) components of the program were found to be implemented with low or moderate level of fidelity. The frequency or duration of the program components were observed to vary based on the needs of users, availability of care coordinators (CC) and their confidence. Variation in fidelity was influenced predominantly by: (1) complexity of the program, (2) extent of facilitation through guiding protocols, (3) facilitation of program implementation through CCs' level of training and confidence, (4) evolving healthcare participant responsiveness, and (5) the context of suboptimal capability among community providers.

CONCLUSION

This is the first study to assess the context-specific implementation process of a transitional care program in the context of Southeast Asia. It provides important insights to facilitate further development and scaling up of transitional care programs within the NUHS-RHS and beyond. Our findings highlight the need for greater focus on engaging both healthcare providers and users, training CCs to equip them with the relevant skills required for their jobs, and building the capability of the community providers to implement such programs.

Establishing a balanced scorecard measurement system for integrated care organizations in China

To address, among other issues, the regional and international challenges of the heavy health care burden caused by an aging population, integrated care organizations (ICOs) were proposed at the end of the 20th century for health care delivery. However, the implementation of ICOs has not progressed smoothly, and the current results have not eliminated the imbalance of medical service capabilities among hospitals of different levels. To make up for the deficiency in the current evaluation system at ICOs and offer suggestions for improved sustainable health planning and management, this study establishes a balanced scorecard based on a comprehensive measurement system valid for a Chinese ICO by surveying the staff at the West China Hospital ICO. This study collected valid responses from 216 professional staff members at the ICO via questionnaires. K-means clustering and the coefficient of variation method were used to evaluate the weights of the first- and second-level indicators. The results show the importance ranking of the core perspectives of the ICO balanced scorecard in the following order: patient, internal process, learning and growth, and financial. The weight-based analysis identified the importance ranking of all indicators and pointed to the areas that require close attention in future ICO planning and management.