Recommendations for evaluation of health care improvement initiatives

Identification of Intervention Characteristics Within Diabetes Prevention Programs Using the Template for Intervention Description and Replication: A Scoping Review

OBJECTIVES

Diabetes prevention programs (DPPs) targeting dietary and physical activity behaviour change have been shown to decrease the incidence of type 2 diabetes; however, a more thorough reporting of intervention characteristics is needed to expedite the translation of such programs into different communities. In this scoping review, we aim to synthesize how DPPs are being reported and implemented.

METHODS

A scoping review using Arkey and O'Malley methods was conducted in accordance with the guidelines of Preferred Reporting Items for Systematic Reviews and Meta-Analysis. MEDLINE, CINAHL, PsycINFO, EMBASE, and SPORTDiscus were searched for studies relating to diabetes prevention and diet/exercise interventions. Only studies delivering a diet/exercise intervention for adults identified as "at risk" for developing type 2 diabetes were included. The Template for Intervention Description and Replication (TIDieR) was used to guide data extraction, and each DPP was scored on a scale from 0 to 2 for how thoroughly it reported each of the items (0 = did not report, 2 = reported in full; total score out of 26).

RESULTS

Of the 25,110 publications screened, 351 (based on 220 programs) met the inclusion criteria and were included for data extraction. No studies comprehensively reported on all TIDieR domains (mean TIDieR score: 15.7 of 26; range 7 to 25). Reporting was particularly poor among domains related to "modifications," "tailoring," and "how well (planned/actual)." "How well (planned)" assesses the intended delivery of an intervention, detailing the initial strategies and components as per the original design, whereas "how well (actual)" evaluates the extent to which the intervention was executed as planned during the study, including any deviations or modifications made in practice.

CONCLUSIONS

Although there is evidence to suggest that DPPs are efficacious, a more thorough reporting of program content and delivery is needed to improve the ability for effective programs to be implemented or translated into different communities.

Modified realist evaluation of a complex, multi-centred, multi-intervention programme

Well North was a complex, multi-intervention health improvement programme spanning 10 sites across the North of England. The aim was to address inequalities by improving the health of the poorest fastest, increasing resilience and reducing levels of worklessness. The intention of the programme was for all sites to have freedom and flexibility to conduct different interventions reflecting local priorities. Evaluation ran concurrently with the programme, and an iterative approach was required to ensure constant feedback, allowing the programme to be adapted and improved as necessary. Realist methodology was chosen for evaluation, as it provides insight into what works, for whom and in what circumstances. Due to the complex nature of the programme and diverse approaches, it was necessary to adapt the methodology to meet the needs of the evaluation. The Evaluation Team utilized a range of qualitative and quantitative techniques within the context of a Rapid Cycle Evaluation framework. For each project, Contexts, Mechanisms and Outcomes (CMOs) were identified at three stages and were incorporated into the CMO configuration, leading to the development of a middle range theory. Validation and testing of theory took place at every stage. Realist methodology was the most appropriate existing method. However, it still necessitated modification.

A randomized controlled trial of the online OurRelationship program with varying levels of coach support

Online programs that reduce relationship distress fill a critical need; however, their scalability is limited by their reliance on coach calls. To determine the effectiveness of the online OurRelationship program with varying levels of coach support, we conducted a comparative effectiveness trial with 740 low-income couples in the United States. Couples were randomly assigned to full-coach (ncouples = 226; program as originally designed), automated-coach (ncouples = 145; as a stand-alone program with tailored automated emails only), contingent-coach (ncouples = 145; as an adaptive program where tailored automated emails are followed by more coaching if couples did not meet progress milestones), or a waitlist control condition (ncouples = 224). All analyses were conducted within a Bayesian framework. Completion rates were comparable across conditions (full-coach: 65 %, automated-coach: 59 %, contingent-coach: 54 %). All intervention couples reported reliable pre-post gains in relationship satisfaction compared to waitlist control couples (dfull = 0.46, dcontingent = 0.47, and dautomated = 0.40) with no reliable differences across intervention conditions. Over four-month follow-up, couples in full- and contingent-coach conditions maintained gains in relationship satisfaction and couples in the automated-coach condition continued to improve. Given the comparable completion rates and minimal differences in effect sizes across intervention conditions, all three coaching models appear viable; therefore, the choice of model can vary depending on available resources as well as couple or stakeholder preferences. This study was preregistered (ClinicalTrials.govNCT03568565).

Advancing healthcare equity through dissemination and implementation science

OBJECTIVE

To provide guiding principles and recommendations for how approaches from the field of dissemination and implementation (D&I) science can advance healthcare equity.

DATA SOURCES AND STUDY SETTING

This article, part of a special issue sponsored by the Agency for Healthcare Research and Quality (AHRQ), is based on an outline drafted to support proceedings of the 2022 AHRQ Health Equity Summit and further revised to reflect input from Summit attendees.

STUDY DESIGN

This is a narrative review of the current and potential applications of D&I approaches for understanding and advancing healthcare equity, followed by discussion and feedback with Summit attendees.

DATA COLLECTION/EXTRACTION METHODS

We identified major themes in narrative and systematic reviews related to D&I science, healthcare equity, and their intersections. Based on our expertise, and supported by synthesis of published studies, we propose recommendations for how D&I science is relevant for advancing healthcare equity. We used iterative discussions internally and at the Summit to refine preliminary findings and recommendations.

PRINCIPAL FINDINGS

We identified four guiding principles and three D&I science domains with strong promise for accelerating progress toward healthcare equity. We present eight recommendations and more than 60 opportunities for action by practitioners, healthcare leaders, policy makers, and researchers.

CONCLUSIONS

Promising areas for D&I science to impact healthcare equity include the following: attention to equity in the development and delivery of evidence-based interventions; the science of adaptation; de-implementation of low-value care; monitoring equity markers; organizational policies for healthcare equity; improving the economic evaluation of implementation; policy and dissemination research; and capacity building.

A digital patient-reported outcome (electronic patient-reported outcome) system for patients with severe psychiatric disorders: User-centered development study and study protocol of a multicenter-controlled trial

Background

The effective treatment of patients with severe psychiatric disorders primarily relies on subjective reporting of symptoms and side-effects. This information is crucial for a clinician's decision regarding medication adjustment. Treatment adjustment usually happens at a low frequency (∼4-8 weeks). In between points of care, patients are left alone with their symptoms and side-effects. This leads to uncertainty regarding the treatment, non-adherence, possible relapse, and rehospitalization.

Objectives

We aim to design a flexible electronic patient-reported outcome (ePRO) system, which allows patients with severe psychiatric disorders to: (a) record their symptoms using an app; (b) share the data with the clinical team at points of care; and (c) utilize the data to support therapy decisions.

Methods

In this article, we describe the development process which included the following steps: (a) formation of a co-design team; (b) stakeholder interviews with patients, practitioners, and digital health experts to access needs, requirements, and barriers; (c) prototype conceptualization and design; (d) user acceptance testing and refinement; and (e) finalization of the system for testing in a pilottrial.

Results

We included input from patients with lived experience of psychiatric disorders, clinical team members, software engineers, and researchers. A prototype system was refined, and iterative changes were made before finalization during a series of operational meetings. The system allows patients to digitally self-report their symptoms and provides longitudinal ePRO symptom data for export into the electronic health record.

Conclusions

Routine ePRO collection has the potential to improve outcomes and hereby also reduce health service costs. We have successfully developed a trial-ready ePRO system for severe psychiatric disorders. The findings were incorporated in the planning of a feasibility pilot trial. Assuming feasibility will be established, the system might be subjected to a certification process evaluation of safety and efficacy including a randomized controlled trial.

Closing the treatment gap for alcohol use disorders in low- and middle-income countries

The alcohol-attributable disease burden is greater in low- and middle-income countries (LMICs) as compared to high-income countries. Despite the effectiveness of interventions such as health promotion and education, brief interventions, psychological treatments, family-focused interventions, and biomedical treatments, access to evidence-based care for alcohol use disorders (AUDs) in LMICs is limited. This can be explained by poor access to general health and mental health care, limited availability of relevant clinical skills among health care providers, lack of political will and/or financial resources, historical stigma and discrimination against people with AUDs, and poor planning and implementation of policies. Access to care for AUDs in LMICs could be improved through evidence-based strategies such as designing innovative, local and culturally acceptable solutions, health system strengthening by adopting a collaborative stepped care approach, horizontal integration of care into existing models of care (e.g., HIV care), task sharing to optimise limited human resources, working with families of individuals with AUD, and leveraging technology-enabled interventions. Moving ahead, research, policy and practice in LMICs need to focus on evidence-based decision-making, responsiveness to context and culture, working collaboratively with a range of stakeholders to design and implement interventions, identifying upstream social determinants of AUDs, developing and evaluating policy interventions such as increased taxation on alcohol, and developing services for special populations (e.g., adolescents) with AUDs.

mHealth prompts within diabetes prevention programs: a scoping review

BACKGROUND

Mobile health (mHealth) prompts (e.g., text messaging, push notifications) are a commonly used technique within behaviour change interventions to prompt or cue a specific behaviour. Such prompts are being increasingly integrated into diabetes prevention programs (DPPs). While mHealth prompts provide a convenient and cost-effective way to reinforce behaviour change, no reviews to date have examined mHealth prompt use within DPPs. This scoping review aims to: (I) understand how mHealth prompts are being used within behaviour change interventions for individuals at risk for developing type 2 diabetes (T2D); and (II) provide recommendations for future mHealth prompt research, design, and application.

METHODS

The scoping review methodology outlined by Arksey and O'Malley were followed. Medline, CINAHL, PsycInfo, Web of Science, and SportDiscus were searched. The search strategy combined keywords relating to T2D risk and mHealth prompts in conjunction with database-controlled vocabulary when available (e.g., MeSH for Medline).

RESULTS

Of the 4,325 publications screened, 44 publications (based on 33 studies) met the inclusion criteria and were included for data extraction. Text messaging was the most widely used mHealth prompt (73%) followed by push notifications (21%). Only 30% of studies discussed the theoretical basis for prompt content and time of day messages were sent, and only 27% provided justification for prompt timing and frequency. Fourteen studies assessed participant satisfaction with mHealth prompts of which only two reported dissatisfaction due to either prompting frequency (hourly) or message content (solely focused on weight). Nine studies assessed behavioural outcomes including weight loss, physical activity, and diabetes incidence, and found mixed effects overall.

CONCLUSIONS

While mHealth prompts were well-received by participants, there are mixed effects on the influence of mHealth prompts on behavioural outcomes and diabetes incidence. More thorough reporting of prompt content development and delivery is needed, and more experimental research is needed to identify optimal content, delivery characteristics, and impact on behavioural and clinical outcomes.

Capability building for large-scale transformational change: learning from an evaluation of a national programme

BACKGROUND

The Long Term Plan presents an ambitious vision for England's National Health Service which will require a sustained programme of transformational change. The Virtual Academy of Large-Scale Change (VALSC) was developed to build capability in health and care system teams involved in transformation or redesign programmes.

METHODS

To evaluate the VALSC, quantitative and qualitative data were collected and reviewed against the Kirkpatrick model. Quantitative data were collected via end-of-session surveys to assess individual knowledge before and after participating in capability-building interventions. Qualitative data were also collected and included post-intervention surveys and interviews. Interviews were transcribed and analysed using an inductive approach to identify themes that were subsequently assessed against the Kirkpatrick model.

RESULTS

Results suggest that the VALSC programme has helped build capability for large-scale change in terms of learning, behaviour change and impact. Participants' ipsative self-assessment of knowledge demonstrated a significant change (p<0.001) and qualitative data suggested three broad themes in which the VALSC made an impact. First, participants were empowered with transformation and change skills which they applied to local health and care challenges. Second, VALSC helped strengthen connections within and between transformational change teams. Third, VALSC helped transformational change teams to engage more effectively with their stakeholders.

CONCLUSIONS

The VALSC developed knowledge, skills, behavioural change and application impact that built capability in individuals and teams. Therefore, continuing to develop capability-building offers that empower and build agency in front-line staff working on service transformation and equip them with approaches, methods and tools to increase their chances of success, is recommended.

Cost of neonatal abstinence syndrome: an economic analysis of English national data held in the National Neonatal Research Database

OBJECTIVE

To determine the incidence of neonatal abstinence syndrome (NAS) across neonatal units, explore healthcare utilisation and estimate the direct cost to the NHS.

DESIGN

Population cohort study.

SETTING

NHS neonatal units, using data held in the National Neonatal Research Database.

PARTICIPANTS

Infants born between 2012 and 2017, admitted to a neonatal unit in England, receiving a diagnosis of NAS (n=6411).

MAIN OUTCOME MEASURES

Incidence, direct annual cost of care (£, 2016-2017 prices), duration of neonatal unit stay (discharge HR), predicted additional cost of care, and odds of receiving pharmacotherapy.

RESULTS

Of 524 334 infants admitted during the study period, 6411 had NAS. The incidence (1.6/1000 live births) increased between 2012 and 2017 (β=0.07, 95% CI (0 to 0.14)) accounting for 12/1000 admissions and 23/1000 cot days nationally. The direct cost of care was £62 646 661 over the study period. Almost half of infants received pharmacotherapy (n=2631; 49%) and their time-to-discharge was significantly longer (median 18.2 vs 5.1 days; adjusted HR (aHR) 0.16, 95% CI (0.15 to 0.17)). Time-to-discharge was longer for formula-fed infants (aHR 0.73 (0.66 to 0.81)) and those discharged to foster care (aHR 0.77 (0.72 to 0.82)). The greatest predictor of additional care costs was receipt of pharmacotherapy (additional mean adjusted cost of £8420 per infant).

CONCLUSIONS

This population study highlights the substantial cot usage and economic costs of caring for infants with NAS on neonatal units. A shift in how healthcare systems provide routine care for NAS could benefit infants and families while alleviating the burden on services.

A breakthrough series collaborative to increase patient participation with hemodialysis tasks: A stepped wedge cluster randomised controlled trial

BACKGROUND

Compared to in-centre, home hemodialysis is associated with superior outcomes. The impact on patient experience and clinical outcomes of consistently providing the choice and training to undertake hemodialysis-related treatment tasks in the in-centre setting is unknown.

METHODS

A stepped-wedge cluster randomised trial in 12 UK renal centres recruited prevalent in-centre hemodialysis patients with sites randomised into early and late participation in a 12-month breakthrough series collaborative that included data collection, learning events, Plan-Study-Do-Act cycles, and teleconferences repeated every 6 weeks, underpinned by a faculty, co-production, materials and a nursing course. The primary outcome was the proportion of patients undertaking five or more hemodialysis-related tasks or home hemodialysis. Secondary outcomes included independent hemodialysis, quality of life, symptoms, patient activation and hospitalisation. ISRCTN Registration Number 93999549.

RESULTS

586 hemodialysis patients were recruited. The proportion performing 5 or more tasks or home hemodialysis increased from 45.6% to 52.3% (205 to 244/449, difference 6.2%, 95% CI 1.4 to 11%), however after analysis by step the adjusted odds ratio for the intervention was 1.63 (95% CI 0.94 to 2.81, P = 0.08). 28.3% of patients doing less than 5 tasks at baseline performed 5 or more at the end of the study (69/244, 95% CI 22.2-34.3%, adjusted odds ratio 3.71, 95% CI 1.66-8.31). Independent or home hemodialysis increased from 7.5% to 11.6% (32 to 49/423, difference 4.0%, 95% CI 1.0-7.0), but the remaining secondary endpoints were unaffected.

CONCLUSIONS

Our intervention did not increase dialysis related tasks being performed by a prevalent population of centre based patients, but there was an increase in home hemodialysis as well as an increase in tasks among patients who were doing fewer than 5 at baseline. Further studies are required that examine interventions to engage people who dialyse at centres in their own care.

Continuous, risk-based, consultation peer review in out-of-hours general practice: a qualitative interview study of the benefits and limitations

Background

Systems to detect and minimise unwarranted variation in clinician practice are crucial to ensure increasingly multidisciplinary healthcare workforces are supported to practise to their full potential. Such systems are limited in English general practice settings, with implications for the efficiency and safety of care.

Aim

To evaluate the benefits and limitations of a continuous, risk-based, consultation peer-review system used for 10 years by an out-of-hours general practice service in Bristol, UK.

Design and setting

A qualitative study in South West England.

Method

Semi-structured interviews with intervention users (clinicians, peer reviewers, and clinical management), analysed by inductive thematic analysis and integrated into a programme theory.

Results

Twenty clinicians were interviewed between September 2018 and January 2019. Interviewees indicated that the intervention supported clinician learning through improved peer feedback, highlighting learning needs and validating practice. It was compared favourably with existing structures of ensuring clinician competence, supporting standardisation of supervision, clinical governance, and learning culture. These benefits were potentially limited by intervention factors such as differential feedback quality between clinician groups, the efficiency of methods to identify learning needs, and limitations of assessments based on written clinical notes. Contextual factors such as clinician experience, motivation, and organisational learning culture influenced the perception of the intervention as a support or a stressor.

Conclusion

The findings demonstrate the potential of continuous, risk-based, consultation peer review to support clinicians in an increasingly multidisciplinary general practice workforce to efficiently and safely practise to their full potential. The programme theory provides a theoretical basis to maximise the benefits and accommodate the potential limitations of this methodology

Formative process evaluation of a guideline-driven process for improving the cultural responsiveness of alcohol and drug treatment services

Background

To improve Australian Aboriginal and Torres Strait Islander people’s access to, and experience of, healthcare services, including Alcohol and other Drug (AoD) treatment services, principles and frameworks have been developed to optimise cultural responsiveness. Implementing those principles in practice, however, can be difficult to achieve. This study has five aims: i) to describe a five-step process developed to operationalise improvements in culturally responsive practice in AoD services; ii) to evaluate the fidelity of implementation for this five-step process; iii) to identify barriers and enablers to implementation; iv) to assess the feasibility and acceptability of this approach; and v) to describe iterative adaptation of implementation processes based on participant feedback.

Methods

Participating services were 15 non-Aboriginal AoD services in New South Wales, Australia. Implementation records were used to assess the implementation fidelity of the project. Structured interviews with chief executive officers or senior management were conducted, and interview data were thematically analysed to identify project acceptability, and the key enablers of, and barriers to, project implementation. Quantitative descriptive analyses were performed on the post-implementation workshop survey data, and responses to the free text questions were thematically analysed.

Results

A high level of implementation fidelity was achieved. Key enablers to improving culturally responsive practice were the timing of the introduction of the five-step process, the active interest of staff across a range of seniority and the availability of resources and staff time to identify and implement activities. Key barriers included addressing the unique needs of a range of treatment sub-groups, difficulty adapting activities to different service delivery models, limited time to implement change in this evaluation (three months) and the varied skill level across staff. The project was rated as being highly acceptable and relevant to service CEOs/managers and direct service staff, with planned changes perceived to be achievable and important. Based on CEO/management feedback after the project was implemented at the initial services, several improvements to processes were made.

Conclusion

The operationalisation of the five-step process developed to improve cultural responsiveness was feasible and acceptable and may be readily applicable to improving the cultural responsiveness of a wide variety of health and human services.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12913-021-06367-7.

General Practice in the Time of COVID-19: A Mixed-Methods Service Evaluation of a Primary Care COVID-19 Service

Primary care coronavirus disease 2019 (COVID-19) clinics were rapidly introduced across the UK to review potentially infectious patients. Evaluation of these services is needed to guide future implementation. This mixed-methods study evaluates patient demographics, clinical presentation, co-morbidities, service usage, and outcomes for the Islington COVID-19 service (London, UK) and from April to May 2020 and thematically analyses survey responses from 29 service clinicians and 41 GP referrers on their service experience. Of the 237 patients booked into the service, a significant number of referrals (n = 91; 38.6%) were made after the presumed infectious period of 14 days. Almost half of all adult referrals (49%) were dealt with remotely (via telephone/video consultation +/− remote oxygen saturation monitoring). The service was perceived to provide a safe way to see patients; it developed local expertise, learning, and empowerment; and it was a positive teamworking experience. These findings suggest that the management of many patients with COVID-19 symptoms is possible in routine general practice with minimal risk through the implementation of remote consultation methods and in patients who present after the post-infectious period. Additionally, the use of remote saturation monitoring and local GP COVID-19 “experts” can support practices to manage COVID-19 patients. Future primary care COVID-19 services should act as empowerment tools to assist GPs to safely manage their own patients and provide support for GPs in this process.

Saving 20 000 Days and Beyond: a realist evaluation of two quality improvement campaigns to manage hospital demand in a New Zealand District Health Board

Background

The current paper reports on a realist evaluation of two consecutive quality improvement campaigns based on the Institute for Healthcare Improvement’s Breakthrough Series. The campaigns were implemented by a District Health Board to manage hospital demand in South Auckland, New Zealand. A realist evaluation design was adopted to investigate what worked in the two campaigns and under what conditions.

Methods

A mixed-methods approach was used, involving three phases of data collection. During the first phase, a review of campaign materials and relevant literature, as well as key informant interviews were undertaken to generate an initial logic model of how the campaign was expected to achieve its objective. In phase II, the model was tested against the experiences of participants in the first campaign via a questionnaire to all campaign participants, interviews with campaign sponsors and collaborative team leaders and a review of collaborative team dashboards. In phase III, the refined model was tested further against the experiences of participants in the second campaign through interviews with collaborative team leaders, case studies of four collaborative teams and a review of the overall system-level dashboard.

Results

The evaluation identified four key mechanisms through which the campaigns’ outcomes were achieved. These were characterised as ‘an organisational preparedness to change’, ‘enlisting the early adopters’, ‘strong collaborative teams’ and ‘learning from measurement’. Contextual factors that both enabled and constrained the operation of these mechanisms were also identified.

Conclusions

By focusing on the explication of a theory of how the campaigns achieved their outcomes and under what circumstances, the realist evaluation reported in this paper provides some instructive lessons for future evaluations of quality improvement initiatives.

Mixed methods protocol for a realist evaluation of electronic personal health records design features and use to support medication adherence (ePHRma)

BACKGROUND

National Health Service policy suggests that increasing usage of electronic personal health records (PHR) by patients will result in cost savings and improved public health. Medication adherence means that patients take their prescribed medication as agreed with their doctors. Some of the claimed benefits of PHRs are decreasing healthcare costs and improving medication adherence and patient outcomes.

METHODS

This is a mixed methods convergent study, primarily qualitative. The qualitative and quantitative data collection and analysis will occur in parallel, and then be synthesised. We are interviewing and surveying adults with long-term conditions to identify what are the most important and useful features of their current PHR. The data collection comprises patient demographics, the Medication Adherence Questionnaire, the personality scale Big Five Inventory-2 Extra-Short Form and the WHO Quality of Life-BREF scale. Qualitative data will be analysed using the Framework method.

ETHICS

We have received a favourable ethical opinion from the Health Research Authority/Research Ethics Committee.

Shaping innovation and coordination of healthcare delivery across boundaries and borders

Purpose

Using a complexity-informed approach, we aim to understand why introduction of a mobile service delivery model for osteoporosis across diverse organisational and country contexts in the UK National Health Service (NHS) met with variable success.

Design/methodology/approach

Six comparative case studies; three prospectively in Scotland using an action research-informed approach; and three retrospectively in England with variable degrees of success. The Non-adoption, Abandonment, Scale-up, Spread and Sustainability framework explored interactions between multi-level contextual factors and their influence on efforts to introduce and sustain services.

Findings

Cross-boundary service development was a continuous process of adaptation and evolution in rapidly shifting healthcare context. Whilst the outer healthcare policy context differed significantly across cases, inner contextual features predominated in shaping the success or otherwise of service innovations. Technical and logistical issues, organisational resources, patient and staff actions combined in unpredictable ways to shape the lifecycle of service change. Patient and staff thoughts about place and access to services actively shaped service development. The use of tacit “soft intelligence” and a sense of “chronic unease” emerged as important in successfully navigating around awkward people and places.

Practical implications

“Chronic unease” and “soft intelligence” can be used to help individuals and organisations “tame” complexity, identify hidden threats and opportunities to achieving change in a particular context, and anticipate how these may change over time. Understanding how patients think and feel about where, when and how care is delivered provides unique insights into previously unseen aspects of context, and can usefully inform development and sustainability of patient-centred healthcare services.

Originality/value

This study has uniquely traced the fortunes of a single service innovation across diverse organisational and country contexts. Novel application of the NASSS framework enabled comparative analysis across real-time service change and historical failures. This study also adds to theories of context and complexity by surfacing the neglected role of patients in shaping healthcare context.

Prototyping for public health in a local context: a streamlined evaluation of a community-based weight management programme (Momenta), Northumberland, UK

OBJECTIVES

Stakeholder co-production in design of public health programmes may reduce the 'implementation gap' but can be time-consuming and costly. Prototyping, iterative refining relevant to delivery context, offers a potential solution. This evaluation explored implementation and lessons learnt for a 12-week referral-based weight-management programme, 'Momenta', along with feasibility of an iterative prototyping evaluation framework.

DESIGN

Mixed methods evaluation: Qualitative implementation exploration with referrers and service users; preliminary analysis of anonymised quantitative service data (12 and 52 weeks).

SETTING

Two leisure centres in Northumberland, North East England.

PARTICIPANTS

Individual interviews with referring professionals (n=5) and focus groups with service users (n=13). Individuals (n=182) referred by healthcare professionals (quantitative data).

INTERVENTIONS

Three 12-week programme iterations: Momenta (n=59), Momenta-Fitness membership (n=58) and Fitness membership only (n=65).

PRIMARY AND SECONDARY OUTCOME MEASURES

Primary outcome: Qualitative themes developed through stakeholder-engagement. Secondary outcomes included preliminary exploration of recruitment, uptake, retention, and changes in weight, body mass index, waist circumference and psychological well-being.

RESULTS

Service users reported positive experiences of Momenta. Implementation gaps were revealed around the referral process and practitioner knowledge. Prototyping enabled iterative refinements such as broadening inclusion criteria. Uptake and 12-week retention were higher for Momenta (84.7%, 45.8%) and Momenta-Fitness (93.1%, 60.3%) versus Fitness only (75.4%, 24.6%). Exploration of other preliminary outcomes (completers only) suggested potential for within-group weight loss and increased psychological well-being for Momenta and Momenta-Fitness at 12 weeks. 52 week follow-up data were limited (32%, 33% and 6% retention for those who started Momenta, Momenta-Fitness and Fitness, respectively) but suggested potential weight loss maintenance for Momenta-Fitness.

CONCLUSIONS

Identification of issues within the referral process enabled real-time iterative refinement, while lessons learnt may be of value for local implementation of 'off-the-shelf' weight management packages more generally. Our preliminary data for completers suggest Momenta may have potential for weight loss, particularly when offered with a fitness membership.

A systematic review and meta-analysis of intervention characteristics in postpartum weight management using the TIDieR framework: A summary of evidence to inform implementation

Postpartum weight retention is a significant contributor to obesity in reproductive-aged women, but the key implementation characteristics of postpartum weight management interventions have not been systematically identified to inform policy and practice. This study aimed to evaluate the intervention characteristics associated with weight loss in postpartum women using the Template for Intervention Description and Replication (TIDieR) framework. We searched MEDLINE, CINAHL, EMBASE, PSYCINFO, and EBM databases to identify lifestyle intervention RCTs in postpartum women (within 2 years after birth) published up to January 2018. From 4512 studies, 33 studies were included in the systematic review and meta-analysis (n = 4960 women). Health professional-delivered interventions had significantly greater weight loss than those delivered by nonhealth professionals (mean difference, 95% confidence interval: (-3.22 kg [-4.83, -1.61] vs -0.99 kg [-1.53, -0.45], P = 0.01 for subgroup differences)). Diet and physical activity combined had significantly greater weight loss compared with physical activity-only interventions (-3.15 kg [-4.34, -1.96] vs -0.78 kg [-1.73, 0.16], P = 0.009 for subgroup differences). The extent of weight loss was not influenced by intervention intensity (duration, number of sessions) and setting (individual or group).

Goals, Recommendations, and the How-To Strategies for Developing and Facilitating Patient Safety and System Integration Simulations

PURPOSE

The aim of this article is to outline overall goals, recommendations, and provide practical How-To strategies for developing and facilitating patient safety and system integration (PSSI) simulations for healthcare team members and organizations.

BACKGROUND

Simulation is increasingly being used as a quality improvement tool to better understand the tasks, environments, and processes that support the delivery of healthcare services. These PSSI simulations paired with system-focused debriefing can occur prior to implementing a new process or workflow to proactively identify system issues. They occur as part of a continuous cycle of quality improvement and have unique considerations for planning, implementation, and delivery of healthcare.

METHOD

The Delphi technique was used to develop the recommendations and How-To strategies to guide those interested in conducting a PSSI simulations. The Delphi technique is a structured communication technique and systematic process of gathering information from a group of identified experts through a series of questionnaires to gain consensus regarding judgments on complex processes, where precise information is not available in the literature. The Delphi technique permitted an iterative and multistaged approach to transform expert opinions into group consensus.

RESULTS

The goals, recommendations, and How-To strategies include a focus on project management, stakeholder engagement, sponsorship, scenario design, prebriefing and debriefing, and evaluation metrics. The intent is to proactively identify system issues and disseminate actionable findings.

CONCLUSIONS

This article highlights salient features to consider when using simulation as a strategy and tool for patient safety and quality improvement.

Interpersonal touch interventions for patients in intensive care: A design-oriented realist review

AIM

To develop a theoretical framework to inform the design of interpersonal touch interventions intended to reduce stress in adult intensive care unit patients.

DESIGN

Realist review with an intervention design-oriented approach.

METHODS

We searched CINAHL, MEDLINE, EMBASE, CENTRAL, Web of Science and grey literature sources without date restrictions. Subject experts suggested additional articles. Evidence synthesis drew on diverse sources of literature and was conducted iteratively with theory testing. We consulted stakeholders to focus the review. We performed systematic searches to corroborate our developing theoretical framework.

RESULTS

We present a theoretical framework based around six intervention construction principles. Theory testing provided some evidence in favour of treatment repetition, dynamic over static touch and lightening sedation. A lack of empirical evidence was identified for construction principles relating to intensity and positive/negative evaluation of emotional experience, moderate pressure touch for sedated patients and intervention delivery by relatives versus healthcare practitioners.

A systematic review of the outcome data supporting the Healthy Living Pharmacy concept and lessons from its implementation

BACKGROUND

The Healthy Living Pharmacy (HLP) project, launched in England, UK in 2009 was a novel approach of introducing public health services within community pharmacy to tackle local health inequalities. A national roll-out followed a reported successful pilot; subsequent local evaluations ensued.

OBJECTIVES

To summarise reported outcomes and investigate contextual factors that indicate the presence, absence and maturity of implementation determinants, thus offering useful lessons to stakeholders in implementing future initiatives to achieve successful outcomes.

METHODS

A systematic review was conducted to identify all publications reporting on the HLP project. All HLP articles and conference abstracts were considered for inclusion and were assessed for methodological quality. The Consolidated Framework for Implementation Research (CFIR) was utilised to identify potential implementation determinants reported. Each article was then analysed to identify reported economic, humanistic or clinical outcomes.

RESULTS

The review included six peer-reviewed journal articles and 12 conference abstracts. Joanna Briggs Institute Qualitative Assessment and Review Instrument indicated deficiencies in methodological quality. Through adoption of the CFIR framework, the implementation determinants relevant to the implementation of HLP into community pharmacy were identified. A resonating issue emerged in that the absence of adopting an evidence-based implementation process limited the ability to capture meaningful outcome data. This resulted in a lack of evidence to support sustainability and the failure to address many of the well cited barriers, e.g. lack of awareness amongst patients, public and other healthcare professionals, and weak support for future investment in resource for training and dissemination.

CONCLUSIONS

Healthcare systems are increasingly called on to adopt evidence-based interventions that improve quality, control costs, and maximize value, thus offering opportunity to accelerate the implementation of clinical pharmacy services and programs aimed at improving patient care. Interventions, such as the HLP project require focused efforts on implementation and evaluation of those implementation efforts to produce effective and lasting changes in complex health care systems.

Using Exploratory Trials to Identify Relevant Contexts and Mechanisms in Complex Electronic Health Interventions: Evaluating the Electronic Patient-Reported Outcome Tool

Background

Designing appropriate studies for evaluating complex interventions, such as electronic health solutions to support integrated care, remains a methodological challenge. With the many moving parts of complex interventions, it is not always clear how program activities are connected to anticipated and unanticipated outcomes. Exploratory trials can be used to uncover determinants (or mechanisms) to inform content theory that underpins complex interventions before designing a full evaluation plan.

Objective

A multimethod exploratory trial of the electronic patient-reported outcome (ePRO) tool was conducted to uncover contexts, processes and outcome variables, and the mechanisms that link these variables before full-scale evaluation. ePRO is a mobile app and portal designed to support goal-oriented care in interdisciplinary primary health care practices (clinical-level integration). This paper offers evaluation findings and methodological insight on how to use exploratory trial data to identify relevant context, process, and outcome variables, as well as central (necessary to achieving outcomes) versus peripheral (less critical and potentially context dependent) mechanisms at play.

Methods

The 4-month trial was conducted in 2 primary health care practices in Toronto, Canada. The patients were randomized into control and intervention groups and compared pre and post on quality of life and activation outcome measures. Semistructured interviews were conducted with providers and patients in the intervention group. Narrative analysis was used to uncover dominant mechanisms that inform the intervention’s content theory (how context and process variables are linked to outcomes).

Results

Overall, 7 providers, 1 administrator, and 16 patients (7-control, 9-intervention) participated in the study. This study uncovered many complex and nuanced context, process, and outcome variables at play in the intervention. Narrative analysis of patient and provider interviews revealed dominant story lines that help to tease apart central and peripheral mechanisms driving the intervention. Provider and patient story lines centered around fitting the new intervention into everyday work and life of patients and providers and meaningfulness of the intervention. These themes were moderated by patient-provider relationships going into and throughout the intervention, their comfort with technology, and the research process.

Conclusions

Identifying dominant story lines using narrative analysis helps to identify the most relevant context and process variables likely to influence study outcomes. Normalization process theory emerges as a useful theory to uncover underlying mechanisms because of its emphasis on the social production and normalization of technological, processual, and social aspects of work; all found to be critical to our intervention. The number of complex, overlapping influencing variables suggests that complex interventions such as ePRO require us to pay careful attention to central versus peripheral mechanisms that will influence study outcomes. The narrative methods presented here are shown to be useful in uncovering these mechanisms and help to guide subsequent larger evaluation studies.

Adapting health interventions for local fit when scaling-up: a realist review protocol

INTRODUCTION

Scaling-up is essential to ensure universal access of effective health interventions. Scaling-up is a complex process, which occurs across diverse systems and contexts with no one-size-fits-all approach. To date, little attention has been paid to the process of scaling-up in how to make adaptations for local fit. The aim of this research is to develop theory on what actions can be used to make adaptations to health interventions for local fit when scaling-up across diverse contexts that will have practical application for implementers involved in scaling-up.

METHODS AND ANALYSIS

Given the complexity of this subject, a realist review methodology was selected. Specifically, realist review emphasises an iterative, non-linear process, whereby the review is refined as it progresses. The identification of how the context may activate mechanisms to achieve outcomes is used to generate theories on what works for whom in what circumstances. This protocol will describe the first completed stage of development of an initial programme theory framework, which identified potential actions, contexts, mechanisms and outcomes that could be used to make adaptations when scaling-up. It will then outline the methods for future stages of the review which will focus on identifying case examples of scale-up and adaptation in practice. This realist review consists of six stages: (i) clarifying scope and development of a theoretical framework, (ii) developing a search strategy, (iii) selection and appraisal, (iv) data extraction, (v) data synthesis and analysis and (vi) further theory refinement with stakeholders.

ETHICS AND DISSEMINATION

This review will develop theory on how adaptations can be made when scaling-up. Findings will be disseminated in a peer-reviewed journal and through stakeholder engagement as part of the research process. Ethical approval has been received through Health Policy and Management/Centre for Global Health Research Ethics Committee of Trinity College Dublin.

A Framework for Rigorous Qualitative Research as a Component of Mixed Method Rapid-Cycle Evaluation

As federal, state, and local governments continue to test innovative approaches to health care delivery, the ability to produce timely and reliable evidence of what works and why it works is crucial. There is limited literature on methodological approaches to rapid-cycle qualitative research. The purpose of this article is to describe the advantages and limitations of a broadly applicable framework for in-depth qualitative analysis placed within a larger rapid-cycle, multisite, mixed-method evaluation. This evaluation included multiple cycles of primary qualitative data collection and quarterly and annual reporting. Several strategies allowed us to be adaptable while remaining rigorous; these included planning for multiple waves of qualitative coding, a hybrid inductive/deductive approach informed by a cross-program evaluation framework, and use of a large team with specific program expertise. Lessons from this evaluation can inform researchers and evaluators functioning in rapid assessment or rapid-cycle evaluation contexts.

Empowering junior doctors: a qualitative study of a QI programme in South West England

AIM

To explore how the South-West Foundation Doctor Quality Improvement programme affected foundation year 1 (F1) doctors' attitudes and ability to implement change in healthcare.

METHODS

Twenty-two qualitative interviews were carried out with two cohorts of doctors. The first F1 group before and after their participation in the QI programme; the second group comprised those who had completed the programme between 1 and 5 years earlier. Qualitative data were analysed using thematic analysis techniques. RESULTS : Prior to taking part in the QI programme, junior doctors' attitudes towards QI were mixed. Although there was agreement on the importance of QI in terms of patient safety, not all shared enthusiasm for engaging in QI, while some were sceptical that they could bring about any change. Following participation in the programme, attitudes towards QI and the ability to effect change were significantly transformed. Whether their projects were considered a success or not, all juniors reported that they valued the skills learnt and the overall experience they gained through carrying out QI projects. Participants reported feeling more empowered in their role as junior doctors, with several describing how they felt 'listened to' and able to 'have a voice', that they were beginning to see things 'at systems level' and learning to 'engage more critically' in their working environment. CONCLUSIONS : Junior doctors are ideally placed to engage in QI. Training in QI at the start of their medical careers may enable a new generation of doctors to acquire the skills necessary to improve patient safety and quality of care.

What Is Known About Preventing, Detecting, and Reversing Prescribing Cascades: A Scoping Review

OBJECTIVES

To systematically describe the resources available on preventing, detecting, and reversing prescribing cascades using a scoping review methodology.

MEASUREMENTS

We searched Medline, EMBASE, PsychINFO, CINAHL, Cochrane Library, and Sociological Abstracts from inception until July 2017. Other searches (Google Scholar, hand searches) and expert consultations were performed for resources examining how to prevent, detect, or reverse prescribing cascades. We used these three categories along the prescribing continuum as an organizing framework to categorize and synthesize resources.

RESULTS

Of 369 resources identified, 58 met inclusion criteria; 29 of these were categorized as preventing, 20 as detecting, and 9 as reversing prescribing cascades. Resources originated from 14 countries and mostly focused on older adults. The goal of preventing resources was to educate and increase general awareness of the concept of prescribing cascades as a way to prevent inappropriate prescribing and to illustrate application of the concept to specific drugs (e.g., anticholinergics) and conditions (e.g., inflammatory bowel disease). Detecting resources included original investigations or case reports that identified prescribing cascades using health administrative data, patient cohorts, and novel sources such as social media. Reversing prescribing cascade resources focused on the medication review process and deprescribing initiatives.

CONCLUSION

Prescribing cascades are a recognized problem internationally. By learning from the range of resources to prevent, detect, and reverse prescribing cascades, this review contributes to improving drug prescribing, especially in older adults. J Am Geriatr Soc 66:2079-2085, 2018.

A framework for learning about improvement: embedded implementation and evaluation design to optimize learning

Improving health care involves many actors, often working in complex adaptive systems. Interventions tend to be multi-factorial, implementation activities diverse, and contexts dynamic and complicated. This makes improvement initiatives challenging to describe and evaluate as matching evaluation and program designs can be difficult, requiring collaboration, trust and transparency. Collaboration is required to address important epidemiological principles of bias and confounding. If this does not take place, results may lack credibility because the association between interventions implemented and outcomes achieved is obscure and attribution uncertain. Moreover, lack of clarity about what was implemented, how it was implemented, and the context in which it was implemented often lead to disappointment or outright failure of spread and scale-up efforts. The input of skilled evaluators into the design and conduct of improvement initiatives can be helpful in mitigating these potential problems. While evaluation must be rigorous, if it is too rigid necessary adaptation and learning may be compromised. This article provides a framework and guidance on how improvers and evaluators can work together to design, implement and learn about improvement interventions more effectively.

Participatory design of an improvement intervention for the primary care management of possible sepsis using the Functional Resonance Analysis Method

BACKGROUND

Ensuring effective identification and management of sepsis is a healthcare priority in many countries. Recommendations for sepsis management in primary care have been produced, but in complex healthcare systems, an in-depth understanding of current system interactions and functioning is often essential before improvement interventions can be successfully designed and implemented. A structured participatory design approach to model a primary care system was employed to hypothesise gaps between work as intended and work delivered to inform improvement and implementation priorities for sepsis management.

METHODS

In a Scottish regional health authority, multiple stakeholders were interviewed and the records of patients admitted from primary care to hospital with possible sepsis analysed. This identified the key work functions required to manage these patients successfully, the influence of system conditions (such as resource availability) and the resulting variability of function output. This information was used to model the system using the Functional Resonance Analysis Method (FRAM). The multiple stakeholder interviews also explored perspectives on system improvement needs which were subsequently themed. The FRAM model directed an expert group to reconcile improvement suggestions with current work systems and design an intervention to improve clinical management of sepsis.

RESULTS

Fourteen key system functions were identified, and a FRAM model was created. Variability was found in the output of all functions. The overall system purpose and improvement priorities were agreed. Improvement interventions were reconciled with the FRAM model of current work to understand how best to implement change, and a multi-component improvement intervention was designed.

CONCLUSIONS

Traditional improvement approaches often focus on individual performance or a specific care process, rather than seeking to understand and improve overall performance in a complex system. The construction of the FRAM model facilitated an understanding of the complexity of interactions within the current system, how system conditions influence everyday sepsis management and how proposed interventions would work within the context of the current system. This directed the design of a multi-component improvement intervention that organisations could locally adapt and implement with the aim of improving overall system functioning and performance to improve sepsis management.

An evaluation of a safety improvement intervention in care homes in England: a participatory qualitative study

OBJECTIVE

A growing proportion of older people live in care homes and are at high risk of preventable harm. This study describes a participatory qualitative evaluation of a complex safety improvement intervention, comprising training, performance measurement and culture-change elements, on the safety of care provided for residents.

DESIGN

A participatory qualitative study.

SETTING

Ninety care homes in one geographical locality in southern England.

PARTICIPANTS

A purposeful sample of care home managers, front-line staff, residents, quality improvement facilitators and trainers, local government and health service commissioners, and an embedded researcher.

MAIN OUTCOME MEASURES

Changes in care home culture and work processes, assessed using documentary analysis, interviews, observations and surveys and analysed using a framework-based thematic approach.

RESULTS

Participation in the programme appears to have led to changes in the value that staff place on resident safety and to changes in their working practices, in particular in relation to their desire to proactively manage resident risk and their willingness to use data to examine established practice. The results suggest that there is a high level of commitment among care home staff to address the problem of preventable harm. Mobilisation of this commitment appears to benefit from external facilitation and the introduction of new methods and tools.

CONCLUSIONS

An evidence-based approach to reducing preventable harm in care homes, comprising an intervention with both technical and social components, can lead to changes in staff priorities and practices which have the potential to improve outcomes for people who live in care homes.

Twelve tips for efficient procedural simulation

Procedural simulation (PS) is increasingly being used worldwide in healthcare for training caregivers in psychomotor competencies. It has been demonstrated to improve learners' confidence and competence in technical procedures, with consequent positive impacts on patient outcomes and safety. Several frameworks can guide healthcare educators in using PS as an educational tool. However, no theory-informed practical framework exists to guide them in including PS in their training programs. We present 12 practical tips for efficient PS training that translates educational concepts from theory to practice, based on the existing literature. In doing this, we aim to help healthcare educators to adequately incorporate and use PS both for optimal learning and for transfer into professional practice.

An intraorganizational model for developing and spreading quality improvement innovations

BACKGROUND

Recent policy reforms encourage quality improvement (QI) innovations in primary care, but practitioners lack clear guidance regarding spread inside organizations.

PURPOSE

We designed this study to identify how large organizations can facilitate intraorganizational spread of QI innovations.

METHODOLOGY/APPROACH

We conducted ethnographic observation and interviews in a large, multispecialty, community-based medical group that implemented three QI innovations across 10 primary care sites using a new method for intraorganizational process development and spread. We compared quantitative outcomes achieved through the group's traditional versus new method, created a process model describing the steps in the new method, and identified barriers and facilitators at each step.

FINDINGS

The medical group achieved substantial improvement using its new method of intraorganizational process development and spread of QI innovations: standard work for rooming and depression screening, vaccine error rates and order compliance, and Pap smear error rates. Our model details nine critical steps for successful intraorganizational process development (set priorities, assess the current state, develop the new process, and measure and refine) and spread (develop support, disseminate information, facilitate peer-to-peer training, reinforce, and learn and adapt). Our results highlight the importance of utilizing preexisting organizational structures such as established communication channels, standardized roles, common workflows, formal authority, and performance measurement and feedback systems when developing and spreading QI processes inside an organization. In particular, we detail how formal process advocate positions in each site for each role can facilitate the spread of new processes.

PRACTICE IMPLICATIONS

Successful intraorganizational spread is possible and sustainable. Developing and spreading new QI processes across sites inside an organization requires creating a shared understanding of the necessary process steps, considering the barriers that may arise at each step, and leveraging preexisting organizational structures to facilitate intraorganizational process development and spread.

Practical recommendations for the evaluation of improvement initiatives

A lack of clear guidance for funders, evaluators and improvers on what to include in evaluation proposals can lead to evaluation designs that do not answer the questions stakeholders want to know. These evaluation designs may not match the iterative nature of improvement and may be imposed onto an initiative in a way that is impractical from the perspective of improvers and the communities with whom they work. Consequently, the results of evaluations are often controversial, and attribution remains poorly understood. Improvement initiatives are iterative, adaptive and context-specific. Evaluation approaches and designs must align with these features, specifically in their ability to consider complexity, to evolve as the initiative adapts over time and to understand the interaction with local context. Improvement initiatives often identify broadly defined change concepts and provide tools for care teams to tailor these in more detail to local conditions. Correspondingly, recommendations for evaluation are best provided as broad guidance, to be tailored to the specifics of the initiative. In this paper, we provide practical guidance and recommendations that funders and evaluators can use when developing an evaluation plan for improvement initiatives that seeks to: identify the questions stakeholders want to address; develop the initial program theory of the initiative; identify high-priority areas to measure progress over time; describe the context the initiative will be applied within; and identify experimental or observational designs that will address attribution.

Research versus practice in quality improvement? Understanding how we can bridge the gap

The gap between implementers and researchers of quality improvement (QI) has hampered the degree and speed of change needed to reduce avoidable suffering and harm in health care. Underlying causes of this gap include differences in goals and incentives, preferred methodologies, level and types of evidence prioritized and targeted audiences. The Salzburg Global Seminar on 'Better Health Care: How do we learn about improvement?' brought together researchers, policy makers, funders, implementers, evaluators from low-, middle- and high-income countries to explore how to increase the impact of QI. In this paper, we describe some of the reasons for this gap and offer suggestions to better bridge the chasm between researchers and implementers. Effectively bridging this gap can increase the generalizability of QI interventions, accelerate the spread of effective approaches while also strengthening the local work of implementers. Increasing the effectiveness of research and work in the field will support the knowledge translation needed to achieve quality Universal Health Coverage and the Sustainable Development Goals.

Realist evaluation of a complex integrated care programme: protocol for a mixed methods study

INTRODUCTION

The lack of understanding of how complex integrated care programmes achieve their outcomes due to the lack of acceptable methods leads to difficulties in the development, implementation, adaptation and scaling up of similar interventions. In this study, we evaluate an integrated care network, the National University Health System (NUHS) Regional Health System (RHS), consisting of acute hospitals, step down care, primary care providers, social services and community partners using a theory-driven realist evaluation approach. This study aims to examine how and for whom the NUHS-RHS works to improve healthcare utilisations, outcomes, care experiences and reduce healthcare costs. By using a realist approach that balances the needs of context-specific evaluation with international comparability, this study carries the potential to address current research gaps.

METHODS AND ANALYSIS

This evaluation will be conducted in three research phases: (1) development of initial programme theory (IPT) underlying the NUHS-RHS; (2) testing of programme theory using empirical data; and (3) refinement of IPT. IPT was elicited and developed through reviews of programme documents, informal discussions and in-depth interviews with relevant stakeholders. Then, a convergent parallel mixed method study will be conducted to assess context (C), mechanisms (M) and outcomes (O) to test the IPT. Findings will then be analysed according to the realist evaluation formula of CMO in which findings on the context, mechanisms will be used to explain the outcomes. Finally, based on findings gathered, IPT will be refined to highlight how to improve the NUHS-RHS by detailing what works (outcome), as well as how (mechanisms) and under what conditions (context).

ETHICS AND DISSEMINATION

The National Healthcare Group, Singapore, Domain Specific Review Board reviewed and approved this study protocol. Study results will be published in international peer-reviewed journals and presented at conferences and internally to NUHS-RHS and Ministry of Health, Singapore.

Regional process redesign of lung cancer care: a learning health system pilot project

Background

The Ottawa Hospital (toh) defined delay to timely lung cancer care as a system design problem. Recognizing the patient need for an integrated journey and the need for dynamic alignment of providers, toh used a learning health system (lhs) vision to redesign regional diagnostic processes. A lhs is driven by feedback utilizing operational and clinical information to drive system optimization and innovation. An essential component of a lhs is a collaborative platform that provides connectivity across silos, organizations, and professions.

Methods

To operationalize a lhs, we developed the Ottawa Health Transformation Model (ohtm) as a consensus approach that addresses process barriers, resistance to change, and conflicting priorities. A regional Community of Practice (cop) was established to engage stakeholders, and a dedicated transformation team supported process improvements and implementation.

Results

The project operationalized the lung cancer diagnostic pathway and optimized patient flow from referral to initiation of treatment. Twelve major processes in referral, review, diagnostics, assessment, triage, and consult were redesigned. The Ottawa Hospital now provides a diagnosis to 80% of referrals within the provincial target of 28 days. The median patient journey from referral to initial treatment decreased by 48% from 92 to 47 days.

Conclusions

The initiative optimized regional integration from referral to initial treatment. Use of a lhs lens enabled the creation of a system that is standardized to best practice and open to ongoing innovation. Continued transformation initiatives across the continuum of care are needed to incorporate best practice and optimize delivery systems for regional populations.

[Deliberations of the expert advisory council on innovation fund applications]

The "Innovation Fund" provides incentives for the development and testing of healthcare innovations in the area of the statutory health insurance with the aim to improve the quality of care in Germany. Over a period of initially four years (2016-19), 300 million Euro will be allocated annually to projects on "innovative forms of healthcare provision" and "health services research". Using a formalized procedure, the ten-member expert advisory board appointed by the German Federal Ministry of Health (BMG) assess all applications on the basis of various criteria for scientific quality, potential of innovation, relevance for health service delivery, and implementability. The present discussion paper sets out important considerations for submission and assessment and puts them up for discussion. (As supplied by the authors).

Bridging the intervention-implementation gap in primary health care delivery: the critical role of integrated implementation research

For national and local leaders to achieve universal health coverage, a new approach or technique to gathering evidence and understanding the contexts that influence the outcome of a study and goes beyond the quantitative results of clinical trials and pilot projects is important. The Doris Duke Charitable Foundation's African Health Initiative (AHI) was designed to produce this type of knowledge through embedding implementation research into Population Health Implementation and Training (PHIT) partnership projects in five countries (Ghana, Mozambique, Rwanda, Tanzania, and Zambia) with the goal of improving primary health care and population health. In Ghana, this integration of research into implementation has contributed to the successful testing, adaptation and implementation of the Community-based Health Planning and Services (CHPS) model (The Navrongo Pilot Project), with results from the AHI-funded work informing national scale-up of effective practices. Further application of implementation science methods and frameworks to study cross-project lessons also produced the evidence needed by national and local decision makers on how and why different intervention components were successful and where and how local context drove implementation and adaptation. Cross-project research also identified effective approaches across diverse settings for building capacity for data-driven improvement, coaching and mentoring clinicians and researchers, developing locally appropriate interventions to reduce neonatal mortality, and integrating implementation research to inform local implementers and researchers in more effective strategies to strengthen health systems and improve health services and population health. Evidence has already shown the potential for this type of work to accelerate regional learning and spread of successful interventions to achieve targeted health goals more efficiently, better enabling countries to achieve the ambitious, but important, U.N. Sustainable Development Goals.

CE: Oral Care for Head and Neck Cancer Symptom Management

: An evidence-based practice change at a radiation oncology center in a large academic medical center was designed to reduce the severity of oral mucositis in adults receiving radiation treatment for head and neck cancer. In the intervention described, patients were given newly created oral care kits and educational materials to improve their oral hygiene. Evaluations were conducted at three points during the project (before radiation treatment, during week 4 to 5 of treatment, and one month after treatment). At week 4 to 5-when the severity of oral mucositis is expected to peak-patients reported improved oral hygiene practices and reduced oral mucositis severity. The authors conclude that the use of these oral care kits and educational materials lessened the effects of oral mucositis during and after radiation treatment.

Analytical Methods for a Learning Health System: 1. Framing the Research Question

Learning health systems use routinely collected electronic health data (EHD) to advance knowledge and support continuous learning. Even without randomization, observational studies can play a central role as the nation’s health care system embraces comparative effectiveness research and patient-centered outcomes research. However, neither the breadth, timeliness, volume of the available information, nor sophisticated analytics, allow analysts to confidently infer causal relationships from observational data. However, depending on the research question, careful study design and appropriate analytical methods can improve the utility of EHD.

The introduction to a series of four papers, this review begins with a discussion of the kind of research questions that EHD can help address, noting how different evidence and assumptions are needed for each. We argue that when the question involves describing the current (and likely future) state of affairs, causal inference is not relevant, so randomized clinical trials (RCTs) are not necessary. When the question is whether an intervention improves outcomes of interest, causal inference is critical, but appropriately designed and analyzed observational studies can yield valid results that better balance internal and external validity than typical RCTs. When the question is one of translation and spread of innovations, a different set of questions comes into play: How and why does the intervention work? How can a model be amended or adapted to work in new settings? In these “delivery system science” settings, causal inference is not the main issue, so a range of quantitative, qualitative, and mixed research designs are needed.

We then describe why RCTs are regarded as the gold standard for assessing cause and effect, how alternative approaches relying on observational data can be used to the same end, and how observational studies of EHD can be effective complements to RCTs. We also describe how RCTs can be a model for designing rigorous observational studies, building an evidence base through iterative studies that build upon each other (i.e., confirmation across multiple investigations).

Analytical Methods for a Learning Health System: 4. Delivery System Science

The last in a series of four papers on how learning health systems can use routinely collected electronic health data (EHD) to advance knowledge and support continuous learning, this review describes how delivery system science provides a systematic means to answer questions that arise in translating complex interventions to other practice settings. When the focus is on translation and spread of innovations, the questions are different than in evaluative research. Causal inference is not the main issue, but rather one must ask: How and why does the intervention work? What works for whom and in what contexts? How can a model be amended to work in new settings? In these settings, organizational factors and design, infrastructure, policies, and payment mechanisms all influence an intervention's success, so a theory-driven formative evaluation approach that considers the full path of the intervention from activities to engage participants and change how they act to the expected changes in clinical processes and outcomes is needed. This requires a scientific approach to quality improvement that is characterized by a basis in theory; iterative testing; clear, measurable process and outcomes goals; appropriate analytic methods; and documented results. To better answer the questions that arise in delivery system science, this paper introduces a number of standard qualitative research approaches that can be applied in a learning health system: Pawson and Tilley's "realist evaluation," theory-based evaluation approaches, mixed-methods and case study research approaches, and the "positive deviance" approach.

Interventions and approaches to integrating HIV and mental health services: a systematic review

BACKGROUND

The frequency in which HIV and AIDS and mental health problems co-exist, and the complex bi-directional relationship between them, highlights the need for effective care models combining services for HIV and mental health. Here, we present a systematic review that synthesizes the literature on interventions and approaches integrating these services.

METHODS

This review was part of a larger systematic review on integration of services for HIV and non-communicable diseases. Eligible studies included those that described or evaluated an intervention or approach aimed at integrating HIV and mental health care. We searched multiple databases from inception until October 2015, independently screened articles identified for inclusion, conducted data extraction, and assessed evaluative papers for risk of bias.

RESULTS

Forty-five articles were eligible for this review. We identified three models of integration at the meso and micro levels: single-facility integration, multi-facility integration, and integrated care coordinated by a non-physician case manager. Single-site integration enhances multidisciplinary coordination and reduces access barriers for patients. However, the practicality and cost-effectiveness of providing a full continuum of specialized care on-site for patients with complex needs is arguable. Integration based on a collaborative network of specialized agencies may serve those with multiple co-morbidities but fragmented and poorly coordinated care can pose barriers. Integrated care coordinated by a single case manager can enable continuity of care for patients but requires appropriate training and support for case managers. Involving patients as key actors in facilitating integration within their own treatment plan is a promising approach.

CONCLUSION

This review identified much diversity in integration models combining HIV and mental health services, which are shown to have potential in yielding positive patient and service delivery outcomes when implemented within appropriate contexts. Our review revealed a lack of research in low- and middle- income countries, and was limited to most studies being descriptive. Overall, studies that seek to evaluate and compare integration models in terms of long-term outcomes and cost-effectiveness are needed, particularly at the health system level and in regions with high HIV and AIDS burden.

Association between e-alert implementation for detection of acute kidney injury and outcomes: a systematic review

Background

Electronic alerts (e-alerts) for acute kidney injury (AKI) in hospitalized patients are increasingly being implemented; however, their impact on outcomes remains uncertain.

Methods

We performed a systematic review. Electronic databases and grey literature were searched for original studies published between 1990 and 2016. Randomized, quasi-randomized, observational and before-and-after studies that included hospitalized patients, implemented e-alerts for AKI and described their impact on one of care processes, patient-centred outcomes or resource utilization measures were included.

Results

Our search yielded six studies ( n  = 10 165 patients). E-alerts were generally automated, triggered through electronic health records and not linked to clinical decision support. In pooled analysis, e-alerts did not improve mortality [odds ratio (OR) 1.05; 95% confidence intervals (CI), 0.84-1.31; n  = 3 studies; n  = 3425 patients; I 2  =   0%] or reduce renal replacement therapy (RRT) use (OR 1.20; 95% CI, 0.91-1.57; n  = 2 studies; n  = 3236 patients; I 2  =   0%). Isolated studies reported improvements in selected care processes. Pooled analysis found no significant differences in prescribed fluid therapy.

Conclusions

In the available studies, e-alerts for AKI do not improve survival or reduce RRT utilization. The impact of e-alerts on processes of care was variable. Additional research is needed to understand those aspects of e-alerts that are most likely to improve care processes and outcomes.

Evaluating Integrated Mental Health Care Programs for Children and Youth

Evaluations of integrated care programs share many characteristics of evaluations of other complex health system interventions. However, evaluating integrated care for child and adolescent mental health poses special challenges that stem from the broad range of social, emotional, and developmental problems that need to be addressed; the need to integrate care for other family members; and the lack of evidence-based interventions already adapted for primary care settings. Integrated care programs for children's mental health need to adapt and learn on the fly, so that evaluations may best be viewed through the lens of continuous quality improvement rather than evaluations of fixed programs.