Challenges, solutions and future directions in the evaluation of service innovations in health care and public health

Development of the ECLIPSE model of meaningful outcome domains following lower limb amputation and prosthetic rehabilitation, through systematic review and best fit framework synthesis

BACKGROUND

Little is known about which outcome domains characterise meaningful recovery following prosthetic rehabilitation and should be measured. Our previous qualitative work developed a conceptual model of outcome domains which are meaningful to patients. This qualitative synthesis aims to develop that model by exploring views and experiences of recovery captured in the limb loss literature, and use these to produce a second iteration of the model describing outcome domains of importance following prosthetic rehabilitation from the patient's perspective.

METHODS

Systematic searches were conducted using CINAHL, Psychinfo and Web of Science from 2011 to early 2023. Studies with a qualitative design focusing on views and experiences of lower limb prosthetic users were eligible for inclusion. Quality was assessed using the CASP tool. 'Best Fit' framework synthesis was used to synthesise the evidence and develop the conceptual model.

RESULTS

40 studies were included, describing the experiences of 539 participants. Data supported the pre-existing conceptual model and led to development of four of the five domains. The newly named ECLIPSE model describes meaningful outcome domains as 1) Being able to participate in important activities and roles, 2) Participating in the way I want to, 3) My prosthesis works for me, 4) If I am in pain, I can manage it, and 5) I am able to accept my new normal. Studies came from 15 countries showing good coverage of high-income settings. Few participants from low-and-middle-income countries were included, it is unclear if the ECLIPSE model describes outcome domains of importance in these settings.

CONCLUSIONS

This synthesis provides a rigorous foundation for understanding outcome domains of importance following lower limb prosthetic rehabilitation from the patient's perspective. The ECLIPSE model is an accessible representation of recovery which could direct rehabilitation programmes, as well as inform the evaluation of prosthetic care through the selection of outcome measures.

Performance measurement and user-centeredness in the healthcare sector: Opening the black box adapting the framework of Donabedian

The framework of Donabedian is widely applied to performance assessment at the healthcare system level. Donabedian categorised the care quality measurement around three dimensions, namely structure, process, and outcomes. The first dimension concerns the inputs; the second one, the combinations of factors and inputs; the last one, the effectiveness in terms of patients' health status. Donabedian early included in the last dimension the patient satisfaction. Nevertheless, nowadays, outcomes are generally measured through hard endpoints, such as re-admissions and mortality indicators. Recently, the Patient-Reported Outcome Measures (PROMs) have been included among the outcome measures within the Donabedian framework. How to move the concept of patient-centeredness to a macro level, including the patient point of view in care quality measurement, evaluation, and improvement? This paper integrates the Donabedian structure-process-outcome framework, by incorporating in the proper dimension the patient-indicators, namely the abovementioned PROMs and Patient-Reported Experience Measures (PREMs). While PROMs are clearly measures of outcome, PREMs can be collocated in the process dimension, since they can be useful for mapping processes and care pathways, in a lean perspective, as well as in the outcome dimension, because inherently linked to outcome, and enablers of patient-centeredness.

Socioeconomic Patterns in the Frequency of Doctor Visits in Germany and Spain in Subjects With and Without Chronic Diseases

The objective of universal health care systems is to achieve equality in the use of health services at the same level of care need. This study evaluates the relationship of socioeconomic position with the frequency of doctor visits in subjects with and without chronic diseases in Germany and Spain. The dependent variables included number of consultations and if a medical consultation occurred. The socioeconomic factors were income and education. The magnitude of the relationship between socioeconomic position and medical consultation frequency was estimated by calculating the percentage ratio using binomial regression and by calculating the difference in consultations by analysis of the covariance, in the case of number of visits. Statistically significant findings according to education were not observed. The percentage ratio in the medical consultations among those with lower and higher income was 1.03 (95% confidence interval [CI] 1.01-2.88) in Germany and 1.11 (95% CI 1.03-1.20) in Spain among subjects with any of the studied chronic conditions. Also, in Germany the difference in the average number of consultations comparing lower income subjects with higher was 3.98 (95% CI 2.40-5.57) in those with chronic conditions. In both countries, there were no differences in the frequency of doctor visits according to education. However, a pro-inequality trend exists in favor of subjects with lower income.

Modeling resource allocation strategies for insecticide-treated bed nets to achieve malaria eradication

Large reductions in the global malaria burden have been achieved, but plateauing funding poses a challenge for progressing towards the ultimate goal of malaria eradication. Using previously published mathematical models of Plasmodium falciparum and Plasmodium vivax transmission incorporating insecticide-treated nets (ITNs) as an illustrative intervention, we sought to identify the global funding allocation that maximized impact under defined objectives and across a range of global funding budgets. The optimal strategy for case reduction mirrored an allocation framework that prioritizes funding for high-transmission settings, resulting in total case reductions of 76% and 66% at intermediate budget levels, respectively. Allocation strategies that had the greatest impact on case reductions were associated with lesser near-term impacts on the global population at risk. The optimal funding distribution prioritized high ITN coverage in high-transmission settings endemic for P. falciparum only, while maintaining lower levels in low-transmission settings. However, at high budgets, 62% of funding was targeted to low-transmission settings co-endemic for P. falciparum and P. vivax. These results support current global strategies to prioritize funding to high-burden P. falciparum-endemic settings in sub-Saharan Africa to minimize clinical malaria burden and progress towards elimination, but highlight a trade-off with 'shrinking the map' through a focus on near-elimination settings and addressing the burden of P. vivax.

The patients' lived experiences with equitable nursing care

BACKGROUND

Equitable care is a fundamental value in the nursing profession. Healthcare workers have both a moral and professional duty to ensure that they do not discriminate.

AIM

This study aimed to explore how patients perceive equitable nursing care.

RESEARCH DESIGN, PARTICIPANTS, AND RESEARCH CONTEXT

This descriptive phenomenological qualitative research study used purposeful sampling to select 17 patients from various departments of a general hospital in southern Iran. The participants were then interviewed using a semi-structured in-depth interview format, which aimed to delve into their experiences with equitable nursing care. The collected data were analyzed using Colaizzi's seven-step method and MAXQDA20 software.

ETHICAL CONSIDERATIONS

Oral and written information about the study was provided before the participants gave their written consent. The transcribed interviews were de-identified. The study was approved by the Ethics Committee of Jiroft University of Medical Sciences.

FINDINGS

The data analysis of the study identified three main themes and six subthemes that were related to the experiences of patients with equitable nursing care. The first theme, equitable care, encompassed subthemes such as nurses' dedicated efforts to facilitate patient recovery and adherence to ethical behavior. The second theme, unconscious causes of inequitable nursing care, included subthemes such as unintentional discrimination stemming from organizational constraints and unconscious biases resulting from a lack of knowledge and skills. The third theme, discriminatory care, comprised subthemes such as deliberate discrimination based on personal traits and selective discrimination.

CONCLUSION

The study findings indicate that achieving equitable nursing care requires a multifaceted approach. This includes effective hospital management, organizational reforms, and regulatory enhancements. Additionally, it is crucial to pay close attention to the needs of patients, enhance nurses' theoretical and practical skills in providing equitable care, fostering a culture of equality within healthcare settings, and consider the personality dimensions and moral characteristics of nurses.

What do we want to get out of this? a critical interpretive synthesis of the value of process evaluations, with a practical planning framework

BACKGROUND

Process evaluations aim to understand how complex interventions bring about outcomes by examining intervention mechanisms, implementation, and context. While much attention has been paid to the methodology of process evaluations in health research, the value of process evaluations has received less critical attention. We aimed to unpack how value is conceptualised in process evaluations by identifying and critically analysing 1) how process evaluations may create value and 2) what kind of value they may create.

METHODS

We systematically searched for and identified published literature on process evaluation, including guidance, opinion pieces, primary research, reviews, and discussion of methodological and practical issues. We conducted a critical interpretive synthesis and developed a practical planning framework.

RESULTS

We identified and included 147 literature items. From these we determined three ways in which process evaluations may create value or negative consequences: 1) through the socio-technical processes of 'doing' the process evaluation, 2) through the features/qualities of process evaluation knowledge, and 3) through using process evaluation knowledge. We identified 15 value themes. We also found that value varies according to the characteristics of individual process evaluations, and is subjective and context dependent.

CONCLUSION

The concept of value in process evaluations is complex and multi-faceted. Stakeholders in different contexts may have very different expectations of process evaluations and the value that can and should be obtained from them. We propose a planning framework to support an open and transparent process to plan and create value from process evaluations and negotiate trade-offs. This will support the development of joint solutions and, ultimately, generate more value from process evaluations to all.

Program adaptation by health departments

Introduction

The dissemination of evidence-based interventions (i.e., programs, practices, and policies) is a core function of US state health departments (SHDs). However, interventions are originally designed and tested with a specific population and context. Hence, adapting the intervention to meet the real-world circumstances and population's needs can increase the likelihood of achieving the expected health outcomes for the target population from the implemented intervention. This study identified how SHD employees decide to adapt public health programs and what influences decisions on how to adapt them.

Materials and methods

SHD employees (n = 45) were interviewed using a qualitative semi-structured interview guide. Telephone interviews were audio-recorded and transcribed verbatim. The transcripts were consensus-coded and themes were identified using thematic analysis. Several themes aligned with the Model for Adaptation Design and Impact.

Results

Data, outcomes, and health department evaluations influenced decisions to adapt a program (pre-adaptation), and reasons to adapt a program included organizational and sociopolitical contextual factors. SHD middle-level managers, program managers and staff, and local agencies were involved in the decisions to adapt the programs. Finally, the goals for adapting a program included enhancing effectiveness/outcomes, reach and satisfaction with the program; funding; and partner engagement. After SHD employees decided to adapt a program, data and evidence guided the changes. Program staff and evaluators were engaged in the adaptation process. Program managers consulted partners to gather ideas on how best to adapt a program based on partners' experiences implementing the program and obtaining community input. Lastly, program managers also received input on adapting content and context from coalition meetings and periodic technical assistance calls.

Discussion

The findings related to decisions to adapt public health programs provide practitioners with considerations for adapting them. Findings reaffirm the importance of promoting public health competencies in program evaluation and adaptation, as well as systematically documenting and evaluating the adaptation processes. In addition, the themes could be studied in future research as mechanisms, mediators, and moderators to implementation outcomes.

Magnitude of terminological bias in international health services research: a disambiguation analysis in mental health

AIMS

Health services research (HSR) is affected by a widespread problem related to service terminology including non-commensurability (using different units of analysis for comparisons) and terminological unclarity due to ambiguity and vagueness of terms. The aim of this study was to identify the magnitude of the terminological bias in health and social services research and health economics by applying an international classification system.

METHODS

This study, that was part of the PECUNIA project, followed an ontoterminology approach (disambiguation of technical and scientific terms using a taxonomy and a glossary of terms). A listing of 56 types of health and social services relevant for mental health was compiled from a systematic review of the literature and feedback provided by 29 experts in six European countries. The disambiguation of terms was performed using an ontology-based classification of services (Description and Evaluation of Services and DirectoriEs - DESDE), and its glossary of terms. The analysis focused on the commensurability and the clarity of definitions according to the reference classification system. Interrater reliability was analysed using κ.

RESULTS

The disambiguation revealed that only 13 terms (23%) of the 56 services selected were accurate. Six terms (11%) were confusing as they did not correspond to services as defined in the reference classification system (non-commensurability bias), 27 (48%) did not include a clear definition of the target population for which the service was intended, and the definition of types of services was unclear in 59% of the terms: 15 were ambiguous and 11 vague. The κ analyses were significant for agreements in unit of analysis and assignment of DESDE codes and very high in definition of target population.

CONCLUSIONS

Service terminology is a source of systematic bias in health service research, and certainly in mental healthcare. The magnitude of the problem is substantial. This finding has major implications for the international comparability of resource use in health economics, quality and equality research. The approach presented in this paper contributes to minimise differentiation between services by taking into account key features such as target population, care setting, main activities and type and number of professionals among others. This approach also contributes to support financial incentives for effective health promotion and disease prevention. A detailed analysis of services in terms of cost measurement for economic evaluations reveals the necessity and usefulness of defining services using a coding system and taxonomical criteria rather than by 'text-based descriptions'.

Diabetes and Cardiovascular Diseases Risk Assessment in Community Pharmacies: An Implementation Study

The implementation of a new service is often challenging when translating research findings into routine clinical practices. This paper presents the results of the implementation study of a pilot project for a diabetes and cardiovascular diseases risk-assessment service in Belgian community pharmacies. To evaluate the implementation of the service, a mixed method was used that follows the RE-AIM framework. During the testing stage, 37 pharmacies participated, including five that dropped out due to a lack of time or COVID-19-related temporary obligations. Overall, 502 patients participated, of which 376 (74.9%) were eligible for according-to-protocol analysis. Of these, 80 patients (21.3%) were identified as being at high risk for the targeted diseases, and 100 (26.6%) were referred to general practice for further investigation. We presented the limited effectiveness and the key elements influencing optimal implementation. Additional strategies, such as interprofessional workshops, a data-sharing platform, and communication campaigns, should be considered to spread awareness of the new role of pharmacists. Such strategies could also promote collaboration with general practitioners to ensure the follow-up of patients at high risk. Overall, this service was considered easy to perform and feasible in practice but would require financial and external support to ensure its effectiveness, sustainability, and larger-scale implementation.

Scale-up of ABC care bundle for intracerebral haemorrhage across two hyperacute stroke units in one region in England: a mixed methods evaluation of a quality improvement project

Background

Intracerebral haemorrhage (ICH) accounts for 10%–15% of strokes in the UK, but is responsible for half of all annual global stroke deaths. The ABC bundle for ICH was developed and implemented at Salford Royal Hospital, and was associated with a 44% reduction in 30-day case fatality. Implementation of the bundle was scaled out to the other hyperacute stroke units (HASUs) in the region from April 2017. A mixed methods evaluation was conducted alongside to investigate factors influencing implementation of the bundle across new settings, in order to provide lessons for future spread.

Methods

A harmonised quality improvement registry at each HASU captured consecutive patients with spontaneous ICH from October 2016 to March 2018 to capture process and outcome measures for preimplementation (October 2016 to March 2017) and implementation (April 2017 to March 2018) time periods. Statistical analyses were performed to determine differences in process measures and outcomes before and during implementation. Multiple qualitative methods (interviews, non-participant observation and project document analysis) captured how the bundle was implemented across the HASUs.

Results

HASU1 significantly reduced median anticoagulant reversal door-to-needle time from 132 min (IQR: 117–342) preimplementation to 76 min (64–113.5) after implementation and intensive blood pressure lowering door to target time from 345 min (204–866) preimplementation to 84 min (60–117) after implementation. No statistically significant improvements in process targets were observed at HASU2. No significant change was seen in 30-day mortality at either HASU. Qualitative evaluation identified the importance of facilitation during implementation and identified how contextual changes over time impacted on implementation. This identified the need for continued implementation support.

Conclusion

The findings show how the ABC bundle can be successfully implemented into new settings and how challenges can impede implementation. Findings have been used to develop an implementation strategy to support future roll out of the bundle outside the region.

Cross-cultural adaptation and measurement property analysis of the Brazilian Portuguese version of the Three Incontinence Questionnaire

INTRODUCTION AND HYPOTHESIS

The differential diagnosis of urinary symptoms may allow health professionals to establish a therapeutic objective and to choose the appropriate treatment for the patient's complaint. The aim of this study was to cross-culturally adapt the Three Incontinence Questionnaire (3IQ) into Brazilian Portuguese (3IQ-Br) and to analyze test-retest reliability, construct, and criterion validity in women.

METHODS

The cross-cultural adaptation of the 3IQ-Br included forward-translation, back-translation, and consensus among an expert committee. Participants with and without urinary incontinence (UI) completed the 3IQ-Br, King's Health Questionnaire (KHQ), and Questionnaire for Female Urinary Incontinence Diagnosis (QUID-Br). Only women with UI answered 3IQ-Br after 7-10 days. Test-retest reliability and construct validity were analyzed using the Cohen linear kappa (k). The 3IQ-Br accuracy was analyzed using the area under the curve (AUC) of the receiver-operating characteristic (ROC) curve, considering the sensitivity and specificity to correctly classify women with and without UI.

RESULTS

The reliability of each question from the 3IQ-Br was considered substantial in the test-retest. The agreement among 3IQ-Br, QUID-Br, and KHQ was almost perfect for UI diagnosis (k > 0.8). The 3IQ-Br was considered to have good accuracy in distinguishing women with UI considering the KHQ (AUC 0.83, 95% confidence interval [CI] 0.78 to 0.87, p < 0.001), and fair to the QUID-Br (AUC 0.73, 95% CI 0.68 to 0.78; p < 0.001).

CONCLUSIONS

The results of this study showed that this version of the 3IQ-Br has acceptable measurement properties for identifying and differentiating UI symptoms in Brazilian women.

Values of Retinoblastoma Survivors and Parents Regarding Treatment Outcomes: A Qualitative Study

PURPOSE

Retinoblastoma is an aggressive pediatric eye cancer. Patient-reported outcome measures reveal important insights into how patients perceive their own health. Currently, there is no widely used or validated measure for assessment of retinoblastoma outcomes. The purpose of this research was to uncover which treatment outcomes that retinoblastoma survivors and their parents value, to inform the development of a future measure.

METHODS

This qualitative, cross-sectional study included retinoblastoma survivors age 6 years and older and parents of retinoblastoma survivors. Participants who did not demonstrate fluency in English were excluded. Study participants participated in semistructured interviews or focus groups, either in person at The Hospital for Sick Children, Toronto, Canada, or through secure videoconference, between March 3, 2019, and January 25, 2020. Iterative rounds of opening coding, codebook development, and coresearcher analysis were used to identify key emergent themes and subthemes.

RESULTS

Seventeen adults participated in six focus groups. Nine pediatric survivors participated in individual interviews. Four common themes emerged from all participant groups: (1) definition of treatment success, (2) enucleation-acceptance and challenges, (3) treatment outcomes to measure, and (4) need for outcome reporting. An additional, unique theme was identified in all pediatric discussions: worries and coping mechanisms. Treatment outcomes deemed valuable were related to the following domains: psychosocial outcomes, daily functioning, functional vision, retinoblastoma education, cosmetic outcomes, and secondary eye conditions.

CONCLUSION

This study represents the first stage in the development of a retinoblastoma-specific patient-reported outcome measure. The findings reveal insight into what outcomes are valued by survivors after treatment and offer promise to improve outcomes assessment for retinoblastoma.

Context-specific adaptation of an eHealth-facilitated, integrated care model and tailoring its implementation strategies-A mixed-methods study as a part of the SMILe implementation science project

Background

Contextually adapting complex interventions and tailoring their implementation strategies is key to a successful and sustainable implementation. While reporting guidelines for adaptations and tailoring exist, less is known about how to conduct context-specific adaptations of complex health care interventions.

Aims

To describe in methodological terms how the merging of contextual analysis results (step 1) with stakeholder involvement, and considering overarching regulations (step 2) informed our adaptation of an Integrated Care Model (ICM) for SteM cell transplantatIon faciLitated by eHealth (SMILe) and the tailoring of its implementation strategies (step 3).

Methods

Step 1: We used a mixed-methods design at University Hospital Basel, guided by the Basel Approach for coNtextual ANAlysis (BANANA). Step 2: Adaptations of the SMILe-ICM and tailoring of implementation strategies were discussed with an interdisciplinary team (n = 28) by considering setting specific and higher-level regulatory scenarios. Usability tests were conducted with patients (n = 5) and clinicians (n = 4). Step 3: Adaptations were conducted by merging our results from steps 1 and 2 using the Framework for Reporting Adaptations and Modifications-Enhanced (FRAME). We tailored implementation strategies according to the Expert Recommendations for Implementing Change (ERIC) compilation.

Results

Step 1: Current clinical practice was mostly acute-care-driven. Patients and clinicians valued eHealth-facilitated ICMs to support trustful patient-clinician relationships and the fitting of eHealth components to context-specific needs. Step 2: Based on information from project group meetings, adaptations were necessary on the organizational level (e.g., delivery of self-management information). Regulations informed the tailoring of SMILe-ICM`s visit timepoints and content; data protection management was adapted following Swiss regulations; and steering group meetings supported infrastructure access. The usability tests informed further adaptation of technology components. Step 3: Following FRAME and ERIC, SMILe-ICM and its implementation strategies were contextually adapted and tailored to setting-specific needs.

Discussion

This study provides a context-driven methodological approach on how to conduct intervention adaptation including the tailoring of its implementation strategies. The revealed meso-, and macro-level differences of the contextual analysis suggest a more targeted approach to enable an in-depth adaptation process. A theory-guided adaptation phase is an important first step and should be sufficiently incorporated and budgeted in implementation science projects.

Disablement in the Physically Active Scale Short Form-8: psychometric evaluation

Background

Patient-centered care and evidence-based practice (EBP) are core competencies for health care professionals. The importance of EBP has led to an increase in research involving clinical outcomes; current recommendations emphasize collecting patient focused measures, thus increasing the need for psychometrically sound patient reported outcome measures (PROMs) of health. Disablement has been identified as a valuable multi-dimensional construct for patient care. The Disablement in the Physically Active Scale Short Form-8 (DPA SF-8) has been proposed as a tool to be used in the physically active population that assesses a physical summary component of health and a quality of life component however, further analysis is necessary to ensure the instrument is psychometrically sound.

Methods

Confirmatory factor analyses (CFAs) were conducted on the DPA SF-8 at each time point to ensure factor structure. Reliability of the scale and internal consistency of the subscales were assessed, and a minimal detectable change (MDC) calculated. Additionally, a minimal clinically important difference (MCID) was also established, and invariance testing across three time points and groups was conducted.

Results

The CFAs at all three visits exceeded recommended model fit indices. The interclass correlation coefficient value (.924) calculated indicated excellent scale reliability and Cronbach’s alpha for subscales PHY and QOL were within recommend values. The MDC value calculated was 5.83 and the MCID for persistent injuries were 2 points and for acute injuries, 3 points. The DPA SF-8 was invariant across time and across subgroups.

Conclusions

The DPA SF-8 met CFA recommendations and criteria for multi-group and longitudinal invariance testing, which indicates the scale may be used to assess for differences between the groups or across time. Our overall analysis indicates the DPA SF-8 is a valid, reliable, and responsive instrument to assess patient improvement in the physically active population.

Supplementary Information

The online version contains supplementary material available at 10.1186/s13102-021-00380-3.

Large-scale implementation of stroke early supported discharge: the WISE realist mixed-methods study

Background

In England, the provision of early supported discharge is recommended as part of an evidence-based stroke care pathway.

Objectives

To investigate the effectiveness of early supported discharge services when implemented at scale in practice and to understand how the context within which these services operate influences their implementation and effectiveness.

Design

A mixed-methods study using a realist evaluation approach and two interlinking work packages was undertaken. Three programme theories were tested to investigate the adoption of evidence-based core components, differences in urban and rural settings, and communication processes.

Setting and interventions

Early supported discharge services across a large geographical area of England, covering the West and East Midlands, the East of England and the North of England.

Participants

Work package 1: historical prospective patient data from the Sentinel Stroke National Audit Programme collected by early supported discharge and hospital teams. Work package 2: NHS staff (n = 117) and patients (n = 30) from six purposely selected early supported discharge services.

Data and main outcome

Work package 1: a 17-item early supported discharge consensus score measured the adherence to evidence-based core components defined in an international consensus document. The effectiveness of early supported discharge was measured with process and patient outcomes and costs. Work package 2: semistructured interviews and focus groups with NHS staff and patients were undertaken to investigate the contextual determinants of early supported discharge effectiveness.

Results

A variety of early supported discharge service models had been adopted, as reflected by the variability in the early supported discharge consensus score. A one-unit increase in early supported discharge consensus score was significantly associated with a more responsive early supported discharge service and increased treatment intensity. There was no association with stroke survivor outcome. Patients who received early supported discharge in their stroke care pathway spent, on average, 1 day longer in hospital than those who did not receive early supported discharge. The most rural services had the highest service costs per patient. NHS staff identified core evidence-based components (e.g. eligibility criteria, co-ordinated multidisciplinary team and regular weekly multidisciplinary team meetings) as central to the effectiveness of early supported discharge. Mechanisms thought to streamline discharge and help teams to meet their responsiveness targets included having access to a social worker and the quality of communications and transitions across services. The role of rehabilitation assistants and an interdisciplinary approach were facilitators of delivering an intensive service. The rurality of early supported discharge services, especially when coupled with capacity issues and increased travel times to visit patients, could influence the intensity of rehabilitation provision and teams’ flexibility to adjust to patients’ needs. This required organising multidisciplinary teams and meetings around the local geography. Findings also highlighted the importance of good leadership and communication. Early supported discharge staff highlighted the need for collaborative and trusting relationships with patients and carers and stroke unit staff, as well as across the wider stroke care pathway.

Limitations

Work package 1: possible influence of unobserved variables and we were unable to determine the effect of early supported discharge on patient outcomes. Work package 2: the pragmatic approach led to ‘theoretical nuggets’ rather than an overarching higher-level theory.

Conclusions

The realist evaluation methodology allowed us to address the complexity of early supported discharge delivery in real-world settings. The findings highlighted the importance of context and contextual features and mechanisms that need to be either addressed or capitalised on to improve effectiveness.

Trial registration

Current Controlled Trials ISRCTN15568163.

Funding

This project was funded by the National Institute for Health Research (NIHR) Health Services and Delivery Research programme and will be published in full in Health Services and Delivery Research; Vol. 9, No. 22. See the NIHR Journals Library website for further project information.

Evaluating complex interventions in context: systematic, meta-narrative review of case study approaches

BACKGROUND

There is a growing need for methods that acknowledge and successfully capture the dynamic interaction between context and implementation of complex interventions. Case study research has the potential to provide such understanding, enabling in-depth investigation of the particularities of phenomena. However, there is limited guidance on how and when to best use different case study research approaches when evaluating complex interventions. This study aimed to review and synthesise the literature on case study research across relevant disciplines, and determine relevance to the study of contextual influences on complex interventions in health systems and public health research.

METHODS

Systematic meta-narrative review of the literature comprising (i) a scoping review of seminal texts (n = 60) on case study methodology and on context, complexity and interventions, (ii) detailed review of empirical literature on case study, context and complex interventions (n = 71), and (iii) identifying and reviewing 'hybrid papers' (n = 8) focused on the merits and challenges of case study in the evaluation of complex interventions.

RESULTS

We identified four broad (and to some extent overlapping) research traditions, all using case study in a slightly different way and with different goals: 1) developing and testing complex interventions in healthcare; 2) analysing change in organisations; 3) undertaking realist evaluations; 4) studying complex change naturalistically. Each tradition conceptualised context differently-respectively as the backdrop to, or factors impacting on, the intervention; sets of interacting conditions and relationships; circumstances triggering intervention mechanisms; and socially structured practices. Overall, these traditions drew on a small number of case study methodologists and disciplines. Few studies problematised the nature and boundaries of 'the case' and 'context' or considered the implications of such conceptualisations for methods and knowledge production.

CONCLUSIONS

Case study research on complex interventions in healthcare draws on a number of different research traditions, each with different epistemological and methodological preferences. The approach used and consequences for knowledge produced often remains implicit. This has implications for how researchers, practitioners and decision makers understand, implement and evaluate complex interventions in different settings. Deeper engagement with case study research as a methodology is strongly recommended.

Developing a patient-oriented realist evaluation for COVID-19 vaccine implementation in Saskatchewan: a methodologic framework

BACKGROUND

There is an urgent need to inform decision-making and safe delivery of vaccines in a timely manner. Our objective is to describe the methods we used to perform a patient-oriented realist evaluation of COVID-19 vaccination implementation in Saskatchewan, Canada, in order to understand the underlying mechanisms and contexts of vaccination implementation and vaccine uptake.

METHODS

This methodology paper describes a patient-oriented, realist, mixed-method evaluation to assess COVID-19 vaccination implementation in Regina, Saskatoon and Prince Albert, Saskatchewan. The study comprised 3 iterative phases guided by Realist And Meta-narrative Evidence Synthesis: Evolving Standards II (RAMESES II). In phase 1 (January-February 2021), we developed the initial program theory, in phase 2 (March-May 2021), we tested and refined the initial program theory, and in phase 3 (June-July 2021), we established the final program theory. Three patient and family partners with different backgrounds and experiences were selected purposively from various locations (urban and rural) in Saskatchewan to engage collaboratively in the evaluation. Data analysis and synthesis occurred at all 3 phases of the project. We analysed qualitative data from phases 2 and 3 using a "retroductive" approach. We used quantitative data to compare outcomes from the 3 sites.

INTERPRETATION

This protocol describes how we developed a final program theory for COVID-19 vaccination implementation with patient and family partners to show for whom, under what circumstances, how and why Saskatchewan's COVID-19 vaccination program has led to vaccine uptake. With patient and family partners' engagement, the evaluation findings will be shared with the Saskatchewan Health Authority and provincial government policy-makers and communications departments, published in peer-reviewed journals, presented at provincial or national conferences, and disseminated through any additional media identified by the patient and family partners.

Use of Patient-reported Outcome Measures and Patient-reported Experience Measures within Evaluation Studies of Telemedicine Applications: Systematic Review

Background

With the rise of digital health technologies and telemedicine, the need for evidence-based evaluation is growing. Patient-reported outcome measures (PROMs) and patient-reported experience measures (PREMs) are recommended as an essential part of the evaluation of telemedicine. For the first time, a systematic review has been conducted to investigate the use of PROMs and PREMs in the evaluation studies of telemedicine covering all application types and medical purposes.

Objective

This study investigates the following research questions: in which scenarios are PROMs and PREMs collected for evaluation purposes, which PROM and PREM outcome domains have been covered and how often, which outcome measurement instruments have been used and how often, does the selection and quantity of PROMs and PREMs differ between study types and application types, and has the use of PROMs and PREMs changed over time.

Methods

We conducted a systematic literature search of the MEDLINE and Embase databases and included studies published from inception until April 2, 2020. We included studies evaluating telemedicine with patients as the main users; these studies reported PROMs and PREMs within randomized controlled trials, controlled trials, noncontrolled trials, and feasibility trials in English and German.

Results

Of the identified 2671 studies, 303 (11.34%) were included; of the 303 studies, 67 (22.1%) were feasibility studies, 70 (23.1%) were noncontrolled trials, 20 (6.6%) were controlled trials, and 146 (48.2%) were randomized controlled trials. Health-related quality of life (n=310; mean 1.02, SD 1.05), emotional function (n=244; mean 0.81, SD 1.18), and adherence (n=103; mean 0.34, SD 0.53) were the most frequently assessed outcome domains. Self-developed PROMs were used in 21.4% (65/303) of the studies, and self-developed PREMs were used in 22.3% (68/303). PROMs (n=884) were assessed more frequently than PREMs (n=234). As the evidence level of the studies increased, the number of PROMs also increased (τ=−0.45), and the number of PREMs decreased (τ=0.35). Since 2000, not only has the number of studies using PROMs and PREMs increased, but the level of evidence and the number of outcome measurement instruments used have also increased, with the number of PREMs permanently remaining at a lower level.

Conclusions

There have been increasingly more studies, particularly high-evidence studies, which use PROMs and PREMs to evaluate telemedicine. PROMs have been used more frequently than PREMs. With the increasing maturity stage of telemedicine applications and higher evidence level, the use of PROMs increased in line with the recommendations of evaluation guidelines. Health-related quality of life and emotional function were measured in almost all the studies. Simultaneously, health literacy as a precondition for using the application adequately, alongside proper training and guidance, has rarely been reported. Further efforts should be pursued to standardize PROM and PREM collection in evaluation studies of telemedicine.

Framework for the development and evaluation of complex interventions: gap analysis, workshop and consultation-informed update

BACKGROUND

The Medical Research Council published the second edition of its framework in 2006 on developing and evaluating complex interventions. Since then, there have been considerable developments in the field of complex intervention research. The objective of this project was to update the framework in the light of these developments. The framework aims to help research teams prioritise research questions and design, and conduct research with an appropriate choice of methods, rather than to provide detailed guidance on the use of specific methods.

METHODS

There were four stages to the update: (1) gap analysis to identify developments in the methods and practice since the previous framework was published; (2) an expert workshop of 36 participants to discuss the topics identified in the gap analysis; (3) an open consultation process to seek comments on a first draft of the new framework; and (4) findings from the previous stages were used to redraft the framework, and final expert review was obtained. The process was overseen by a Scientific Advisory Group representing the range of relevant National Institute for Health Research and Medical Research Council research investments.

RESULTS

Key changes to the previous framework include (1) an updated definition of complex interventions, highlighting the dynamic relationship between the intervention and its context; (2) an emphasis on the use of diverse research perspectives: efficacy, effectiveness, theory-based and systems perspectives; (3) a focus on the usefulness of evidence as the basis for determining research perspective and questions; (4) an increased focus on interventions developed outside research teams, for example changes in policy or health services delivery; and (5) the identification of six 'core elements' that should guide all phases of complex intervention research: consider context; develop, refine and test programme theory; engage stakeholders; identify key uncertainties; refine the intervention; and economic considerations. We divide the research process into four phases: development, feasibility, evaluation and implementation. For each phase we provide a concise summary of recent developments, key points to address and signposts to further reading. We also present case studies to illustrate the points being made throughout.

LIMITATIONS

The framework aims to help research teams prioritise research questions and design and conduct research with an appropriate choice of methods, rather than to provide detailed guidance on the use of specific methods. In many of the areas of innovation that we highlight, such as the use of systems approaches, there are still only a few practical examples. We refer to more specific and detailed guidance where available and note where promising approaches require further development.

CONCLUSIONS

This new framework incorporates developments in complex intervention research published since the previous edition was written in 2006. As well as taking account of established practice and recent refinements, we draw attention to new approaches and place greater emphasis on economic considerations in complex intervention research. We have introduced a new emphasis on the importance of context and the value of understanding interventions as 'events in systems' that produce effects through interactions with features of the contexts in which they are implemented. The framework adopts a pluralist approach, encouraging researchers and research funders to adopt diverse research perspectives and to select research questions and methods pragmatically, with the aim of providing evidence that is useful to decision-makers.

FUTURE WORK

We call for further work to develop relevant methods and provide examples in practice. The use of this framework should be monitored and the move should be made to a more fluid resource in the future, for example a web-based format that can be frequently updated to incorporate new material and links to emerging resources.

FUNDING

This project was jointly funded by the Medical Research Council (MRC) and the National Institute for Health Research (Department of Health and Social Care 73514).

Impact of Ed-LinQ: A Public Policy Strategy to Facilitate Engagement between Schools and the Mental Health Care System in Queensland, Australia

Ed-LinQ is a mental health policy initiative to enhance the early detection and treatment of children with mental illness by improving the liaison between schools and health services in Queensland, Australia. We measured its impact from policy to practice to inform further program developments and public strategies. We followed a mixed quantitative/qualitative approach. The Adoption Impact Ladder (AIL) was used to analyse the adoption of this initiative by end-users (decision makers both in the health and education sectors) and the penetration of the initiative in the school sector. Survey respondents included representatives of schools (n = 186) and mental health providers (n = 78). In total, 63% of the school representative respondents were at least aware of the existence of the Ed-LinQ initiative, 74% were satisfied with the initiative and 28% of the respondent schools adopted the initiative to a significant extent. Adoption was higher in urban districts and in the health sector. The overall level of penetration in the school sector of Queensland was low (3%). The qualitative analysis indicated an improvement in the referral and communication processes between schools and the health sectors and the importance of funding in the implementation of the initiative. Mapping of existing programs is needed to assess the implementation of a new one as well as the design of different implementation strategies for urban and rural areas. Assessing the adoption of health policy strategies and their penetration in a target audience is critical to understand their proportional impacts across a defined ecosystem and constitutes a necessary preliminary step for the evaluation of their quality and efficiency.

Adapted fairness benchmarks for the evaluation of Iran health system reform

INTRODUCTION

This study has been designed to localize fairness benchmarks for the evaluation of health system reform with an ethical approach.

METHODS

A descriptive-analytic study was applied in which a questionnaire with selected indicators and based on main fairness benchmarks was validated. Besides, the final indicators were chosen for surveys of 255 health equity experts from 31 provinces of Iran to localize fairness benchmarks using the exploratory factor analysis through the SPSS software.

RESULTS

Seven fairness benchmarks were adopted: (1) Efficacy, Efficiency, and Quality of Health Care; (2) Impact of Information Infrastructure on Health Reform; (3) Responsiveness; (4) Equitable Access, and Universal Health Coverage; (5) Equitable Financing; (6) Social Determinant of Health; and (7) Financial Barriers to Equitable Access.

CONCLUSION

The priorities of fairness benchmarking through localization included focusing on the quantity and quality of primary health care, allocating resources based on the need; for equitable efficiency, and paying attention to community-based information infrastructure, and social determinants for fair reform. Moreover, respecting patient rights as a part of democratic accountability was more close to equity. Elimination of financial and nonfinancial barriers for health access and coverage based on vulnerability, through fair financing, was also confirmed.

Program Evaluation of Population- and System-Level Policies: Evidence for Decision Making

BACKGROUND

Policy evaluations often focus on ex post estimation of causal effects on short-term surrogate outcomes. The value of such information is limited for decision making, as the failure to reflect policy-relevant outcomes and disregard for opportunity costs prohibits the assessment of value for money. Further, these evaluations do not always consider all relevant evidence, other courses of action, or decision uncertainty.

METHODS

In this article, we explore how policy evaluation could better meet the needs of decision making. We begin by defining the evidence required to inform decision making. We then conduct a literature review of challenges in evaluating policies. Finally, we highlight potential methods available to help address these challenges.

RESULTS

The evidence required to inform decision making includes the impacts on the policy-relevant outcomes, the costs and associated opportunity costs, and the consequences of uncertainty. Challenges in evaluating health policies are described using 8 categories: 1) valuation space; 2) comparators; 3) time of evaluation; 4) mechanisms of action; 5) effects; 6) resources, constraints, and opportunity costs; 7) fidelity, adaptation, and level of implementation; and 8) generalizability and external validity. Methods from a broad set of disciplines are available to improve policy evaluation, relating to causal inference, decision-analytic modeling, theory of change, realist evaluation, and structured expert elicitation.

LIMITATIONS

The targeted review may not identify all possible challenges, and the methods covered are not exhaustive.

CONCLUSIONS

Evaluations should provide appropriate evidence to inform decision making. There are challenges in evaluating policies, but methods from multiple disciplines are available to address these challenges.

IMPLICATIONS

Evaluators need to carefully consider the decision being informed, the necessary evidence to inform it, and the appropriate methods.[Box: see text].

Trends and equity in the use of health services in Spain and Germany around austerity in Europe

Background

Following the 2008 economic crisis many countries implemented austerity policies, including reducing public spending on health services. This paper evaluates the trends and equity in the use of health services during and after that period in Spain – a country with austerity policies – and in Germany – a country without restriction on healthcare spending.

Methods

Data from several National Surveys in Spain and several waves of the Socio-Economic Panel in Germany, carried out between 2009 and 2017, were used. The dependent variables were number of doctor’s consultations and whether or not a hospital admission occurred. The measure of socioeconomic position was education. In each year, the estimates were made for people with and without pre-existing health problems. First, the average number of doctor’s consultations and the percentage of respondents who had had been hospitalized were calculated. Second, the relationship between education and use of those health services was estimated by calculating the difference in consultations using covariance analysis – in the case of number of consultations – and by calculating the percentage ratio using binomial regression – in the case of hospitalization.

Results

The annual mean number of consultations went down in both countries. In Spain the average was 14.2 in 2009 and 10.4 in 2017 for patients with chronic conditions; 16.6 and 13.5 for those with a mental illness; and 6.4 and 5.9 for those without a defined illness. In Germany, the averages were 13.8 (2009) and 12.9 (2017) for the chronic group; 21.1 and 17.0 for mental illness; and 8.7 and 7.5 with no defined illness. The hospitalization frequency also decreased in both countries. The majority of the analyses presented no significant differences in relation to education.

Conclusion

In both Spain and Germany, service use decreased between 2009 and 2017. In the first few years, this reduction coincided with a period of austerity in Spain. In general, we did not find socioeconomic differences in health service use.

How does reorganisation in child and adolescent mental health services affect access to services? An observational study of two services in England

BACKGROUND

Child and Adolescent Mental Health Services (CAMHS) in England are making significant changes to improve access and effectiveness. This 'transformation' variously involves easier access to services through a Single Point of Access (SPA), more integrated services within CAMHS and enhanced co-provision across education and third sector or non-profit organisations.

METHODS

A mixed-methods observational study was conducted to explore the process and impact of transformation over four years in two services. Ethnographic observations and in-depth interviews were conducted and Electronic Patient Records with over one million contacts analysed. Difference-in-differences analysis with propensity score matching to estimate the causal impact of the transformation on patient access was utilised.

OUTCOMES

Spend and staffing increased across both CAMHS. The SPA had growing rates of self-referral and new care pathways were seeing patients according to expected degree of psychopathology. Third sector partners were providing increasing numbers of low-intensity interventions. Although the majority of staff were supportive of the changes, the process of transformation led to service tensions. In the first year after transformation there was no change in the rate of new patients accessing services or new spells (episodes of care) in the services. However, by year three, the number of new patients accessing CAMHS was 19% higher (Incidence Rate Ratio: 1·19, CI: 1·16, 1·21) and the rate of new spells was 12% higher (Incidence Rate Ratio: 1·12, CI: 1·05, 1·20).

INTERPRETATION

Transformation investment, both financial and intellectual, can help to increase access to CAMHS in England, but time is needed to realise the benefits of reorganisation.

Evidence-based stroke rehabilitation: do priorities for practice change and feasibility of implementation vary across high income, upper and lower-middle income countries?

PURPOSE

The context of implementation plays an important role in the delivery of optimal treatments in stroke recovery and rehabilitation. Considering that stroke systems of care vary widely across the globe, the goal of the present paper is to compare healthcare providers' priority of key areas in translating stroke research to clinical practice among High Income Countries, Upper Middle- and Lower Middle-Income Countries (HICs, UMICs, LMICs). We also aimed to compare perceptions regarding the key areas' feasibility of implementation, and formulate recommendations specific to each socioeconomic region.

METHODS

Data related to recommendations for knowledge translation in stroke, from a primary survey from the second Stroke Recovery and Rehabilitation Roundtable were segregated based on socioeconomic region. Frequency distribution was used to compare the key areas for practice change and examine the perceived feasibility of implementation of the same across HIC, UMIC and LMICs.

RESULTS

A total of 632 responses from healthcare providers across 28 countries were received. Interdisciplinary care and access to services were high priorities across the three groups. Transitions in Care and Intensity of Practice were high priority areas in HICs, whereas Clinical Practice Guidelines were a high priority in LMICs. Interventions specific to clinical discipline, screening and assessment were among the most feasible areas in HICs, whereas Intensity of practice and Clinical Practice Guidelines were perceived as most feasible to implement in LMICs.

CONCLUSION

We have identified healthcare providers' priorities for addressing international practice change across socioeconomic regions. By focusing on the most feasible key areas, we can aid the channeling of appropriate resources to bridge the disparities in stroke outcomes across HICs, UMICs and LMICs.IMPLICATIONS FOR REHABILITATIONIt is pertinent to examine the differences in priorities of stroke rehabilitation professionals and the feasibility of implementing evidence-based practice across socioeconomic regions.There is an urgent necessity for the development of clinical practice guidelines for stroke rehabilitation in Low-Middle Income Countries, taking into consideration the cultural, economic and geographical constraints.In upper-middle income countries, encouraging family support and timely screening and assessment for aphasia, cognition and depression appear to be the low hanging fruits to enhance quality of life after stroke.Innovative ways to increase intensity of practice and channelling of resources to improve transitions in care may prove to be the most beneficial in advancing stroke rehabilitation in high income countries.

Self-management education among women with pre-existing diabetes in pregnancy: A scoping review

BACKGROUND

Education is a cornerstone of self-management for adults with diabetes. Self-management is particularly important during pregnancy for women with type 1 and type 2 diabetes, as perinatal outcomes are affected by maternal glycemic control. To our knowledge, literature describing the provision of diabetes education and support during pregnancy for women with type 1 and type 2 diabetes has not been synthesized, nor examined within its context as a complex intervention.

OBJECTIVES AND DESIGN

This scoping review aims to synthesize the evidence regarding prenatal diabetes education and support for women with type 1 and type 2 diabetes and to apply the Medical Research Council framework for complex interventions where appropriate.

DATA SOURCES AND METHODS

We searched EMBASE, CINAHL, and MEDLINE from inception to February 2019 for primary studies focused on prenatal diabetes education among women with type 1 and type 2 diabetes. Two independent reviewers screened eligible studies against inclusion criteria. A narrative synthesis of the included studies was conducted.

RESULTS

Of 511 identified citations, 30 studies were included in the final review. Approximately 44% of the pooled sample were women with type 1 diabetes, 46% had gestational diabetes mellitus, and 10% had type 2 diabetes. Education focused on self-monitoring of blood glucose, attaining glycemic targets, and following a healthy diet. Many studies included educational elements that went beyond traditional didactic teaching and promoted self-management skills and self-management support. The majority of education was delivered via one-on-one outpatient appointments every one to three weeks. About half of the reviewed studies used a multidisciplinary team approach, with most including a combination of physicians, nurses, dietitians, and midwives. Application of the Medical Research Council framework revealed that most studies were limited in methods (i.e., randomization) and few examined process evaluation or intervention cost-effectiveness.

CONCLUSION

We identified a lack of studies centred on educational interventions for women with type 2 diabetes in pregnancy. As pregnancy for women with type 2 diabetes involves significant changes, including the transition from oral hypoglycemics to insulin therapy, often without exposure to diabetes-specific preconception care and counselling, future research may focus on optimizing preconception and prenatal education and support for this high-risk group. This is particularly relevant as the prevalence of type 2 diabetes is increasing worldwide. Future research ought to also design, implement and evaluate interventions in accordance with the Medical Research Council framework for complex interventions.

Effect of stroke early supported discharge on length of hospital stay: analysis from a national stroke registry

OBJECTIVE

The first observational study to investigate the impact of early supported discharge (ESD) on length of hospital stay in real-world conditions.

DESIGN

Using historical prospective Sentinel Stroke National Audit Programme (SSNAP) data (1 January 2013-31 December 2016) and multilevel modelling, cross-sectional (2015-2016; 30 791 patients nested within 55 hospitals) and repeated cross-sectional (2013-2014 vs 2015-2016; 49 266 patients nested within 41 hospitals) analyses were undertaken.

SETTING

Hospitals were sampled across a large geographical area of England covering the West and East Midlands, the East of England and the North of England.

PARTICIPANTS

Stroke patients whose data were entered into the SSNAP database by hospital teams.

INTERVENTIONS

Receiving ESD along the patient care pathway.

PRIMARY AND SECONDARY OUTCOME MEASURES

Length of hospital stay.

RESULTS

When adjusted for important case-mix variables, patients who received ESD on their stroke care pathway spent longer in hospital, compared with those who did not receive ESD. The percentage increase was 15.8% (95% CI 12.3% to 19.4%) for the 2015-2016 cross-sectional analysis and 18.8% (95% CI 13.9% to 24.0%) for the 2013-2014 versus 2015-2016 repeated cross-sectional analysis. On average, the increased length of hospital stay was approximately 1 day.

CONCLUSIONS

This study has shown that by comparing ESD and non-ESD patient groups matched for important patient characteristics, receiving ESD resulted in a 1-day increase in length of hospital stay. The large reduction in length of hospital stay overall, since original trials were conducted, may explain why a reduction was not observed. The longer term benefits of accessing ESD need to be investigated further.

TRIAL REGISTRATION NUMBER

http://www.isrctn.com/ISRCTN15568163.

Examining Equity Effects of Health Interventions in Cost-Effectiveness Analysis: A Systematic Review

OBJECTIVE

This systematic review aims to catalogue and describe published applications of equity-informative cost-effectiveness analysis (CEAs).

METHODS

Following PRISMA guidelines, we searched Medline for English-language, peer-reviewed CEAs published on or before August 2019. We included CEAs that evaluated 2 or more alternatives; explicitly mentioned equity as a consideration or decision-making principle; and applied an equity-informative CEA method to analyze or examine at least 1 equity criterion in an applied CEA. We extracted data on selected characteristics and analyzed reporting quality using the CHEERS checklist.

RESULTS

Fifty-four articles identified through a search and bibliography reviews met the inclusion criteria. All articles were published on or after 2010, with 80% published after 2015. Most studies evaluated primary prevention interventions in disease areas such as cancer, infectious diseases, and cardiovascular disease. Equity impact analysis alone was the most common equity-informative CEA (56%), followed by equity impact analysis with financial protection effects (30%). At least 11 different equity criteria have been used in equity-informative CEAs; socioeconomic status and race/ethnicity were used most frequently. Seventy-eight percent of studies reported finding "greater value" in an intervention after examining its distributional effects.

CONCLUSION

The number of equity-informative CEAs is increasing, and the wide range of equity criteria, diseases, interventions, settings, and populations represented suggests that broad application of these methods is feasible but will require further refinement. Inclusion of equity into CEAs may shift the value of evaluated interventions and can provide crucial additional information for decision makers.

The terminology conflict on efficacy and effectiveness in healthcare

Designers and architects created the rule 'form follows function (FFF)' for their own profession. Our paper demonstrates that this FFF rule applies equally well to the designers of clinical studies. Four examples present are as follows: disregarding this FFF rule causes an inconsistent terminology to differentiate between efficacy and effectiveness, inconsistent differentiation of efficacy and effectiveness interferes with the consistent interpretation of the results of clinical studies, inconsistent interpretation of clinical studies results in an unexpectedly variance of recommendations in clinical guidelines and the fusion of the FFF designer rule and of the demands of Cochrane and Bradford Hill ('can it work?', 'does it work?' and 'is it worth it?') avoids the terminology problem and its misleading consequences. This strategy is presented.

Rasch analysis of the long-term conditions questionnaire (LTCQ) and development of a short-form (LTCQ-8)

Background

The aim of the current study was to evaluate the structural validity of the 20-item long-term conditions questionnaire (LTCQ) and to explore a potential short-form version of the scale using Rasch analysis.

Methods

Data were collected through postal surveys (February 2016–January 2017) from a sample of 1,211 participants diagnosed with at least one long-term condition (LTC). Identified participants were invited through either local authorities for a social care cohort (n = 294) or primary care practices for a health care cohort (n = 917). Participants were mailed a survey, including the LTCQ, demographic questions, a comorbidities measure, and other validated outcome measures. Respondents were invited to complete a follow-up survey including the LTCQ for assessment of reproducibility.

Results

The main assumptions of the Rasch model from the LTCQ were fulfilled, although infit and outfit indices indicated some items showed misfit. Misfitted items, items that did not have a preceding set or showed some local dependence were removed one at a time, with the remaining candidate items to form an 8-item short version, the LTCQ-8. The Rasch model for the LTCQ-8 explained 64% variance and had a reliability estimate greater than 0.80. Several items in the LTCQ showed uniform differential item function (DIF) in relation to the number of reported LTCs, age, cohort and type of LTCs, but fewer items exhibited DIF in the LTCQ-8. Spearman’s rho correlations between the LTCQ and the LTCQ-8 were strong across the total sample and various subgroups. Correlations between the LTCQ-8 and all reference measures were moderate to strong, and comparable to correlations found between the LTCQ and these measures.

Conclusions

The LTCQ measures a unidimensional construct, and it is therefore acceptable to use a summed total score. The LTCQ-8 also met the assumption of unidimensionality and had comparable construct validity with the LTCQ. Additional validation is required in an independent sample.

Case study research for better evaluations of complex interventions: rationale and challenges

BACKGROUND

The need for better methods for evaluation in health research has been widely recognised. The 'complexity turn' has drawn attention to the limitations of relying on causal inference from randomised controlled trials alone for understanding whether, and under which conditions, interventions in complex systems improve health services or the public health, and what mechanisms might link interventions and outcomes. We argue that case study research-currently denigrated as poor evidence-is an under-utilised resource for not only providing evidence about context and transferability, but also for helping strengthen causal inferences when pathways between intervention and effects are likely to be non-linear.

MAIN BODY

Case study research, as an overall approach, is based on in-depth explorations of complex phenomena in their natural, or real-life, settings. Empirical case studies typically enable dynamic understanding of complex challenges and provide evidence about causal mechanisms and the necessary and sufficient conditions (contexts) for intervention implementation and effects. This is essential evidence not just for researchers concerned about internal and external validity, but also research users in policy and practice who need to know what the likely effects of complex programmes or interventions will be in their settings. The health sciences have much to learn from scholarship on case study methodology in the social sciences. However, there are multiple challenges in fully exploiting the potential learning from case study research. First are misconceptions that case study research can only provide exploratory or descriptive evidence. Second, there is little consensus about what a case study is, and considerable diversity in how empirical case studies are conducted and reported. Finally, as case study researchers typically (and appropriately) focus on thick description (that captures contextual detail), it can be challenging to identify the key messages related to intervention evaluation from case study reports.

CONCLUSION

Whilst the diversity of published case studies in health services and public health research is rich and productive, we recommend further clarity and specific methodological guidance for those reporting case study research for evaluation audiences.

Process evaluation within pragmatic randomised controlled trials: what is it, why is it done, and can we find it?-a systematic review

BACKGROUND

Process evaluations are increasingly conducted within pragmatic randomised controlled trials (RCTs) of health services interventions and provide vital information to enhance understanding of RCT findings. However, issues pertaining to process evaluation in this specific context have been little discussed. We aimed to describe the frequency, characteristics, labelling, value, practical conduct issues, and accessibility of published process evaluations within pragmatic RCTs in health services research.

METHODS

We used a 2-phase systematic search process to (1) identify an index sample of journal articles reporting primary outcome results of pragmatic RCTs published in 2015 and then (2) identify all associated publications. We used an operational definition of process evaluation based on the Medical Research Council's process evaluation framework to identify both process evaluations reported separately and process data reported in the trial results papers. We extracted and analysed quantitative and qualitative data to answer review objectives.

RESULTS

From an index sample of 31 pragmatic RCTs, we identified 17 separate process evaluation studies. These had varied characteristics and only three were labelled 'process evaluation'. Each of the 31 trial results papers also reported process data, with a median of five different process evaluation components per trial. Reported barriers and facilitators related to real-world collection of process data, recruitment of participants to process evaluations, and health services research regulations. We synthesised a wide range of reported benefits of process evaluations to interventions, trials, and wider knowledge. Visibility was often poor, with 13/17 process evaluations not mentioned in the trial results paper and 12/16 process evaluation journal articles not appearing in the trial registry.

CONCLUSIONS

In our sample of reviewed pragmatic RCTs, the meaning of the label 'process evaluation' appears uncertain, and the scope and significance of the term warrant further research and clarification. Although there were many ways in which the process evaluations added value, they often had poor visibility. Our findings suggest approaches that could enhance the planning and utility of process evaluations in the context of pragmatic RCTs.

TRIAL REGISTRATION

Not applicable for PROSPERO registration.

User experiences on implementation of patient reported outcome measures (PROMs)in a Haematological outpatient clinic

Background

PROMs can help healthcare professionals gain an improved understanding of patients’ physical burdens, functional levels, and (health-related) quality of life throughout disease and medical treatment. The aim of this study was to investigate the barriers and potential opportunities PROMs may present in a haematological outpatient clinic from three different perspectives: patients, nurses and haematologists.

Methods

The present study synthesizes three previously published studies that separately explored the experiences of patients, nurses and haematologists when implementing PROMs. The studies were all guided by the qualitative methodology Interpretive Description, including a focused ethnographic approach, to develop implications for future practice.

Results

The overall themes that emerged from the analysis were “Structural similarities influence the adoption of PROMs” and “Different perspectives on the potential of PROMs.”

Conclusion

Across the different user groups in the haematological outpatient clinic, the use of PROMs was thwarted due to an unquestioned commitment to biomedical knowledge and the system’s rationality and norms: PROM data was not used in patient consultations. Nurses and haematologists expressed different preferences related to potential future PROMs and different objectives for PROMs in clinical practice. From the different perspectives of the patients, nurses and haematologists, PROMs were not compatible with clinical practice. Further research is recommended to develop PROMs validated for use in haematological outpatient clinics. Moreover, implementation strategies adjusted to the structural barriers of the system are crucial.

Supplementary information

Supplementary information accompanies this paper at 10.1186/s41687-020-00256-z.

Patient-reported outcome measures for retinoblastoma: a scoping review

BACKGROUND

Retinoblastoma is a childhood retinal cancer with lifelong consequences such as vision loss and increased risk of second cancer. Patient-reported outcome measures (PROMs) are instruments that measure outcomes related to health directly reported by patients. The purpose of this study was to determine the scope, characteristics and quality of PROMs used in retinoblastoma and related fields of pediatric ophthalmology and pediatric oncology.

METHODS

Databases MEDLINE and Embase were searched for studies in the English language that reported on PROMs used in retinoblastoma, pediatric oncology, or pediatric ophthalmology; grey literature and studies reporting on developmental PROM phases were excluded. PROMs were grouped by the construct measured and domains assessed, and classified as condition-specific or generic. A subsequent search was then conducted in MEDLINE and Embase for studies assessing measurement properties of the identified PROMs. PROMs with associated studies were assessed for their methodologic quality using the COnsensus-based standard for the Selection of health Measurement INstruments (COSMIN) strategy.

RESULTS

Among 110 eligible studies uncovered by the database searches, 143 PROMs were identified: one retinoblastoma-specific, 56 ophthalmology- and 86 oncology-related. The most common construct measured was 'health-related quality of life' and the most common domain assessed was emotional well-being. Of the 143 PROMs, 100 had associated validation studies; the one retinoblastoma-specific PROM was not validated. Quality assessment revealed 34/100 PROMs received a score of sufficient quality in both subcategories of 'overall content validity'; 3/100 received a score of sufficient quality in both subcategories of 'internal structure'; 0/100 received a score of sufficient quality in all three subcategories of 'remaining measurement properties'. The Patient-Reported Outcome Measure Information System (PROMIS) Pediatric Profile-25 was the highest-scoring PROM identified, meeting COSMIN standards for 2/3 measurement property categories (and 5/7 subcategories). Eleven additional PROMs were identified which had sufficient scores in 1/3 measurement property categories (and 5/7 subcategories).

CONCLUSION

The study identified several PROMs from the pediatric ophthalmology and pediatric oncology literature that could be relevant to the retinoblastoma population, but many have limits to their validation. Future development of a retinoblastoma-specific PROM, performed in partnership with retinoblastoma patients to support optimal content validity, could first focus on the selection and definition of the optimal construct to measure, followed potentially by adaptation and further validation of the relevant PROMs with strong methodologic quality identified in this study.

Interventions to reduce inequalities in avoidable hospital admissions: explanatory framework and systematic review protocol

INTRODUCTION

Internationally there is pressure to contain costs due to rising numbers of hospital admissions. Alongside age, socioeconomic disadvantage is the strongest risk factor for avoidable hospital admission. This equity-focussed systematic review is required for policymakers to understand what has been shown to work to reduce inequalities in hospital admissions, what does not work and where the current gaps in the evidence-base are.

METHODS AND ANALYSIS

An initial framework shows how interventions are hypothesised to reduce socioeconomic inequalities in avoidable hospital admissions. Studies will be included if the intervention focusses exclusively on socioeconomically disadvantaged populations or if the study reports differential effects by socioeconomic status (education, income, occupation, social class, deprivation, poverty or an area-based proxy for deprivation derived from place of residence) with respect to hospital admission or readmission (overall or condition-specific for those classified as ambulatory care sensitive). Studies involving individuals of any age, undertaken in OECD (Organisation for Economic Co-operation and Development) countries, published from 2000 to 29^th February 2020 in any language will be included. Electronic searches will include MEDLINE, Embase, CINAHL, Cochrane CENTRAL and the Web of Knowledge platform. Electronic searches will be supplemented with full citation searches of included studies, website searches and retrieval of relevant unpublished information. Study inclusion, data extraction and quality appraisal will be conducted by two reviewers. Narrative synthesis will be conducted and also meta-analysis where possible. The main analysis will examine the effectiveness of interventions at reducing socioeconomic inequalities in hospital admissions. Interventions will be characterised by their domain of action and approach to addressing inequalities. For included studies, contextual information on where, for whom and how these interventions are organised, implemented and delivered will be examined where possible.

ETHICS AND DISSEMINATION

Ethical approval was not required for this protocol. The research will be disseminated via peer-reviewed publication, conferences and an open-access policy-orientated paper.

PROSPERO REGISTRATION NUMBER

CRD42019153666.

Effectiveness of Stroke Early Supported Discharge: Analysis From a National Stroke Registry

Supplemental Digital Content is available in the text.

Background:

Implementation of stroke early supported discharge (ESD) services has been recommended in many countries’ clinical guidelines, based on clinical trial evidence. This is the first observational study to investigate the effectiveness of ESD service models operating in real-world conditions, at scale.

Methods and Results:

Using historical prospective data from the United Kingdom Sentinel Stroke National Audit Programme (January 1, 2016–December 31, 2016), measures of ESD effectiveness were “days to ESD” (number of days from hospital discharge to first ESD contact; n=6222), “rehabilitation intensity” (total number of treatment days/total days with ESD; n=5891), and stroke survivor outcome (modified Rankin scale at ESD discharge; n=6222). ESD service models (derived from Sentinel Stroke National Audit Programme postacute organizational audit data) were categorized with a 17-item score, reflecting adoption of ESD consensus core components (evidence-based criteria). Multilevel modeling analysis was undertaken as patients were clustered within ESD teams across the Midlands, East, and North of England (n=31). A variety of ESD service models had been adopted, as reflected by variability in the ESD consensus score. Controlling for patient characteristics and Sentinel Stroke National Audit Programme hospital score, a 1-unit increase in ESD consensus score was significantly associated with a more responsive ESD service (reduced odds of patient being seen after ≥1 day of 29% [95% CI, 1%–49%] and increased treatment intensity by 2% [95% CI, 0.3%–4%]). There was no association with stroke survivor outcome measured by the modified Rankin Scale.

Conclusions:

This study has shown that adopting defined core components of ESD is associated with providing a more responsive and intensive ESD service. This shows that adherence to evidence-based criteria is likely to result in a more effective ESD service as defined by process measures.

Registration:

URL: http://www.isrctn.com/; Unique identifier: ISRCTN15568163.

An ecosystems approach to mental health services research

Mental health ecosystems research is an emerging discipline which takes a whole-systems approach to mental healthcare, facilitating analysis of the complex environment and context of mental health systems, and translation of this knowledge into policy and practice. Evidence from the local context is needed in the analysis of complex interventions and of geographic variations in the outcomes of care. Technical tools and support have been developed to gather and interpret evidence from the local context and translate it in a meaningful and relevant manner for planners and policy makers to guide their decision-making.

The rise of rapid implementation: a worked example of solving an existing problem with a new method by combining concept analysis with a systematic integrative review

BACKGROUND

The concept of rapid implementation has emerged in the literature recently, but without a precise definition. Further exploration is required to distinguish the concept's unique meanings and significance from the perspective of implementation science. The study clarifies the concept of rapid implementation and identifies its attributes, antecedents, and consequences. We present a theoretical definition of rapid implementation to clarify its unique meaning and characteristics.

METHODS

Rodgers evolutionary concept analysis method, combined with a systematic integrative review, were used to clarify the concept of rapid implementation. A comprehensive search of four databases, including EMBASE, MEDLINE, SCOPUS, and WEB OF SCIENCE was conducted, as well as relevant journals and reference lists of retrieved studies. After searching databases, 2442 papers were identified from 1963 to 2019; 24 articles were found to fit the inclusion criteria to capture data on rapid implementation from across healthcare settings in four countries. Data analysis was carried out using descriptive thematic analysis.

RESULTS

The results locate the introduction of rapid implementation, informed by implementation science. Guidance for further conceptualisation to bridge the gap between research and practice and redefine rigour, adapting methods used (current approaches, procedures and frameworks), and challenging clinical trial design (efficacy-effectiveness-implementation pipeline) is provided.

CONCLUSIONS

It is possible that we are on the cusp of a paradigm shift within implementation brought about by the need for faster results into practice and policy. Researchers can benefit from a deeper understanding of the rapid implementation concept to guide future implementation of rapid actionable results in clinical practice.

Supporting Innovative Person-Centred Care in Financially Constrained Environments: The WE CARE Exploratory Health Laboratory Evaluation Strategy

The COST CARES project aims to support healthcare cost containment and improve healthcare quality across Europe by developing the research and development necessary for person-centred care (PCC) and health promotion. This paper presents an overview evaluation strategy for testing 'Exploratory Health Laboratories' to deliver these aims. Our strategy is theory driven and evidence based, and developed through a multi-disciplinary and European-wide team. Specifically, we define the key approach and essential criteria necessary to evaluate initial testing, and on-going large-scale implementation with a core set of accompanying methods (metrics, models, and measurements). This paper also outlines the enabling mechanisms that support the development of the "Health Labs" towards innovative models of ethically grounded and evidenced-based PCC.

Mapping the evidence on health equity considerations in economic evaluations of health interventions: a scoping review protocol

BACKGROUND

Equity in health has become an important policy agenda around the world, prompting health economists to advance methods to enable the inclusion of equity in economic evaluations. Among the methods that have been proposed to explicitly include equity are the weighting analysis, equity impact analysis, and equity trade-off analysis. This is a new development and a comprehensive overview of trends and concepts of health equity in economic evaluations is lacking. Thus, our objective is to map the current state of the literature with respect to how health equity is considered in economic evaluations of health interventions reported in the academic and gray literature.

METHODS

We will conduct a scoping review to identify and map evidence on how health equity is considered in economic evaluations of health interventions. We will search relevant electronic, gray literature and key journals. We developed a search strategy using text words and Medical Subject Headings terms related to health equity and economic evaluations of health interventions. Articles retrieved will be uploaded to reference manager software for screening and data extraction. Two reviewers will independently screen the articles based on their titles and abstracts for inclusion, and then will independently screen a full text to ascertain final inclusion. A simple numerical count will be used to quantify the data and a content analysis will be conducted to present the narrative; that is, a thematic summary of the data collected.

DISCUSSION

The results of this scoping review will provide a comprehensive overview of the current evidence on how health equity is considered in economic evaluations of health interventions and its research gaps. It will also provide key information to decision-makers and policy-makers to understand ways to include health equity into the prioritization of health interventions when aiming for a more equitable distribution of health resources.

SYSTEMATIC REVIEW REGISTRATION

This protocol was registered with Open Science Framework (OSF) Registry on August 14, 2019 (https://osf.io/9my2z/registrations).

The Lived Experience and Patient-reported Benefits of Yoga Participation in an Inpatient Brain Injury Rehabilitation Setting

CONTEXT

The multifactorial benefits of yoga have been well documented in the literature, with the integration of yoga therapy into healthcare being an emerging field. In general, yoga therapy programs are utilized in the community as an adjunct to other therapy. At present, limited rehabilitation units routinely incorporate integrative therapy options within a hospital environment.

AIMS

The aim of this study is to explore the lived experience and patient-reported benefits of yoga in an inpatient brain injury rehabilitation setting.

SETTINGS AND DESIGN

Thirty-one participants were recruited to the study after voluntarily participating in a yoga class within an inpatient brain injury rehabilitation unit of a major metropolitan hospital. Yoga sessions were held weekly for 60 min and consisted of a modified Hatha yoga style. This was a mixed-methods, quasi-experimental one-group pretest-posttest study.

METHODOLOGY

Quantitative data were collected to measure perceptions of relaxation and well-being before and after yoga classes, along with the satisfaction of the class. Semi-structured interviews were utilized to collect qualitative data of experiences and perceptions associated with yoga participation.

STATISTICAL ANALYSIS USED

Thematic analysis was completed for qualitative data. Quantitative data were analyzed using nonparametric statistical methods, and descriptive statistics were also provided.

RESULTS

The benefits described by participants are reported in this paper. These include improved relaxation, physical well-being, emotional well-being, being present, and self-awareness.

CONCLUSIONS

This study describes the personal benefits experienced from regular yoga participation within an inpatient rehabilitation setting.

Equality as an ethical concept within the context of nursing care rationing

The concept of equality is subject to many different interpretations, and it is closely connected to similar concepts such as equity, justice, fairness, and human rights. As an ideal, equality entails many aspects that are untenable. For instance, genetic and social inequalities may never be extinct, but they can both be ameliorated by proper distribution of society's resources. Likewise, within the context of health care, equality can be promoted by proper rationing of health resources, amongst which nursing care stands out. In the field of nursing, the principle of equality presents itself in various forms of ethical and deontological mandates. However, beyond good intentions and abstract notions, there is a need to examine the ways in which nurses enforce this principle in practice, within the reality of modern health systems. Although there is scarcity of qualitative evidence in the nursing care rationing literature, existing studies suggest that fair treatment pertains to a largely intuitive sense of equality which involves subjective perceptions and judgements about rationing. Nurses' initial predisposition is to view all patients as equal and treat them in an equal manner; yet, on an individual basis, each patient has a different starting point, different needs and different prospects that render rationing decisions complex and uncertain. Equality should be accepted with its unavoidable limitations in practice and be further examined within the context of nursing care rationing, in the hope that it can be advanced in a consistent way, despite the idealistic nature in many of its aspects.

Role of public involvement in the Royal College of Physicians' Future Hospitals healthcare improvement programme: an evaluation

OBJECTIVES

The Royal College of Physician's (RCP) Future Hospital Programme (FHP) set out a blueprint for a radical new model of care that put patient experience centre stage. This paper reports on the results of an independent evaluation of the FHP and focuses on the role public patient involvement (PPI) played in these projects. The paper explores the perceptions and experiences of those involved in the FHP of how PPI was operationalised in this context, and develops an 'ex-post' programme theory of PPI in the FHP. We conclude by assessing the benefits and challenges of this work.

SETTING

Secondary care. The FHP consisted of eight clinician-led healthcare improvement hospital development sites with two phases.

PARTICIPANTS

Development site clinical teams, patient representatives, the RCP's Patient and Carer Network, members of the FHP team, and fellows and members of the RCP.

DESIGN/METHODS

We conducted an independent evaluation of the FHP using FHP documentation and data collected specifically for the evaluation: qualitative interviews, focus groups and a web-based survey.

RESULTS

The PPI initiatives set out to develop more patient-centred care and improve the patient experience. The mechanisms designed to meet these goals were (1) a programme of PPI in the development site's projects, (2) a better understanding of patient experience and (3) evaluation of patient experience.

CONCLUSION

This evaluation of the FHP identifies some key elements that need to be considered when attempting to more closely integrate PPI and co-production in service re-design. The structure of FHP over two phases enabled learning from phase I to be incorporated into phase II. Having the PPI representatives closely involved, developing communities of practice, and the oversight and measuring activities acted as 'disciplinary structures' that contributed to embedding PPI in the FHP and kept the patient experience at the forefront of the improvement initiatives.

Standardised description of health and social care: A systematic review of use of the ESMS/DESDE (European Service Mapping Schedule/Description and Evaluation of Services and DirectoriEs)

BACKGROUND

Evidence-informed planning and interpretation of research results both require standardised description of local care delivery context. Such context analysis descriptions should be comparable across regions and countries to allow benchmarking and organizational learning, and for research findings to be interpreted in context. The European Service Mapping Schedule (ESMS) is a classification of adult mental health services that was later adapted for the assessment of health and social systems research (Description and Evaluation of Services and DirectoriEs - DESDE). The aim of the study was to review the diffusion and use of the ESMS/DESDE system in health and social care and its impact in health policy and decision-making.

METHOD

We conducted a systematic review following the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines (1997-2018).

RESULTS

Out of 155 papers mentioning ESMS/DESDE, 71 have used it for service research and planning. The classification has been translated into eight languages and has been used by seven international research networks. Since 2000, it has originated 11 instruments for health system research with extensive analysis of their metric properties. The ESMS/DESDE coding system has been used in 585 catchment areas in 34 countries for description of services delivery at local, regional and national levels.

CONCLUSIONS

The ESMS/DESDE system provides a common terminology, a classification of care services, and a set of tools allowing a variety of aims to be addressed in healthcare and health systems research. It facilitates comparisons across and within countries for evidence-informed planning.

Evaluability assessments as an approach to examining social prescribing

We report on two evaluability assessments (EAs) of social prescribing (SP) services in South East England conducted in 2016/7. We aimed to demonstrate how EAs can be used to assess whether a programme is ready to be evaluated for outcomes, what changes would be needed to do so and whether the evaluation would contribute to improved programme performance. We also aimed to draw out the lessons learned through the EA process and consider how these can inform the design and evaluation of SP schemes. EAs followed the steps described by Wholey, New Dir Eval 33:77, (1987) and Leviton et al., Annu Rev Public Health 31:213, (2010), including collaboration with stakeholders, elaboration, testing and refinement of an agreed programme theory, understanding the programme reality, identification and review of existing data sources and assessment against key criteria. As a result, evaluation of the services was not recommended. Necessary changes to allow for future evaluation include gaining access to electronic patient records, establishing procedures for collection of baseline and outcome data and linking to data on use of other healthcare services. Lessons learned include ensuring that: (a) SP schemes are developed with involvement (and buy in) of relevant stakeholders; (b) information governance and data sharing agreements are in place from the start; (c) staffing levels are sufficient to cover the range of activities involved in service delivery, data monitoring, reporting, evaluation and communication with stakeholders; (d) SP schemes are co-located with primary care services; and (e) referral pathways and linkages to health service data systems are established as part of the programme design. We conclude that EA provides a valuable tool for informing the design and evaluation of SP schemes. EA can help commissioners to make best use of limited evaluation resources and prioritise which programmes need to be evaluated, as well as how, why and when.

What is the impact of large-scale implementation of stroke Early Supported Discharge? A mixed methods realist evaluation study protocol

BACKGROUND

Stroke Early Supported Discharge (ESD) is a service innovation that facilitates discharge from hospital and delivery of specialist rehabilitation in patients' homes. There is currently widespread implementation of ESD services in many countries, driven by robust clinical trial evidence. In England, the type of ESD service patients receive on the ground is variable, and in some regions, ESD is still not offered at all. This protocol presents a study designed to investigate the mechanisms and outcomes of implementing ESD at scale in real-world conditions. This will help to establish which models of ESD are most effective and in what context.

METHODS

A realist evaluation approach composed of two interlinking work packages will be adopted to investigate how and why ESD works, for whom and in what circumstances. Work package 1 (WP1) will begin with a rapid evidence synthesis to formulate preliminary realist hypotheses. Quantitative analyses of historical prospective Sentinel Stroke National Audit Programme (SSNAP) data will be performed to evaluate service outcomes based on the degree to which evidence-based ESD has been implemented. Work package 2 (WP2) will involve the qualitative investigation of purposively selected case study sites featuring in WP1 and covering different regions in England. The perspectives of clinicians, managers, commissioners, and service users will be explored qualitatively. Cost implications of ESD models will be examined using a cost-consequence analysis. Cross-case comparisons and triangulation of the data sources from both work packages will be performed to test, revise, and refine initial programme theories and address research aims.

DISCUSSION

This study will investigate whether and how current large-scale implementation of ESD is achieving the outcomes suggested by the evidence base. The theory-driven evaluation approach will highlight key mechanisms and contextual conditions necessary to optimise outcomes and allow us to draw transferable lessons to inform the effective implementation and sustainability of ESD in clinical practice. In addition, the methodological framework will progress the theoretical understanding of implementation and evaluation of complex rehabilitation interventions in stroke care.

TRIAL REGISTRATION

ISRCTN: 15568163, registration date: 26 October 2018.

Increased demand for amphetamine treatment in rural Australia

Background

A substantial increase in substance treatment episodes for methamphetamine problems suggests characteristics of the treatment population could have changed and that targeted treatment programs are required. To determine who methamphetamine treatment should be designed for this study has two aims. First, to empirically describe changes in amphetamine treatment presentations to a rural NSW drug and alcohol treatment agency over time. Second, to examine how these characteristics may affect the likelihood of being treated for amphetamines compared to other drugs.

Method

The Australian Alcohol and Other Drug Treatment Services National Minimum Data Set (AODTS-NMDS) containing closed treatment episodes from a single agency from three time periods was used. Characteristics of people receiving amphetamine treatments in these three periods were compared and the effects of these characteristics on the odds of being treated for amphetamine were estimated using a logistic regression model. The characteristics utilised in the analysis include age, sex, Indigenous status, usual accommodation, living arrangement, source of referral and source of income.

Results

The proportion of amphetamine treatment episodes doubled from 2006/2007 to 2015/2016 and overtook alcohol as the most commonly treated principal drug of concern. The estimated proportion of amphetamine treatments showed an increment across all ages and for men and women. It was found that younger people, women, people in temporary accommodation or homeless, people who were self-referred and people whose main source of income was not through employment are more likely to be treated for amphetamine use.

Conclusion

Significant changes over time in the age, sex and Indigenous status of people receiving treatment for amphetamine as the principal drug of concern requires service delivery to match demand from younger people, particularly women; and Indigenous people. The needs and preferences for treatment of younger women who use amphetamine will be important factors in treatment planning service providers who are more used to providing treatment for young men who use cannabis and older men who use alcohol. Further research on women’s experiences in treatment and outcomes would be useful for informing treatment practices.

Electronic supplementary material

The online version of this article (10.1186/s13722-019-0144-6) contains supplementary material, which is available to authorized users.

Standard comparison of local mental health care systems in eight European countries

AIMS

There is a need of more quantitative standardised data to compare local Mental Health Systems (MHSs) across international jurisdictions. Problems related to terminological variability and commensurability in the evaluation of services hamper like-with-like comparisons and hinder the development of work in this area. This study was aimed to provide standard assessment and comparison of MHS in selected local areas in Europe, contributing to a better understanding of MHS and related allocation of resources at local level and to lessen the scarcity in standard service comparison in Europe. This study is part of the Seventh Framework programme REFINEMENT (Research on Financing Systems' Effect on the Quality of Mental Health Care in Europe) project.

METHODS

A total of eight study areas from European countries with different systems of care (Austria, England, Finland, France, Italy, Norway, Romania, Spain) were analysed using a standard open-access classification system (Description and Evaluation of Services for Long Term Care in Europe, DESDE-LTC). All publicly funded services universally accessible to adults (≥18 years) with a psychiatric disorder were coded. Care availability, diversity and capacity were compared across these eight local MHS.

RESULTS

The comparison of MHS revealed more community-oriented delivery systems in the areas of England (Hampshire) and Southern European countries (Verona - Italy and Girona - Spain). Community-oriented systems with a higher proportion of hospital care were identified in Austria (Industrieviertel) and Scandinavian countries (Sør-Trøndelag in Norway and Helsinki-Uusimaa in Finland), while Loiret (France) was considered as a predominantly hospital-based system. The MHS in Suceava (Romania) was still in transition to community care.

CONCLUSIONS

There is a significant variation in care availability and capacity across MHS of local areas in Europe. This information is relevant for understanding the process of implementation of community-oriented mental health care in local areas. Standard comparison of care provision in local areas is important for context analysis and policy planning.

Healthcare ecosystems research in mental health: a scoping review of methods to describe the context of local care delivery

BACKGROUND

Evidence from the context of local health ecosystems is highly relevant for research and policymaking to understand geographical variations in outcomes of health care delivery. In mental health systems, the analysis of context presents particular challenges related to their complexity and to methodological difficulties. Method guidelines and standard recommendations for conducting context analysis of local mental health care are urgently needed. This scoping study reviews current methods of context analysis in mental health systems to establish the parameters of research activity examining availability and capacity of care at the local level, and to identify any gaps in the literature.

METHODS

A scoping review based on a systematic search of key databases was conducted for the period 2005-2016. A systems dynamics/complexity approach was adopted, using a modified version of Tansella and Thornicroft's matrix model of mental health care as the conceptual framework for our analysis.

RESULTS

The lack of a specific terminology in the area meant that from 10,911 titles identified at the initial search, only 46 papers met inclusion criteria. Of these, 21 had serious methodological limitations. Fifteen papers did not use any kind of formal framework, and five of those did not describe their method. Units of analysis varied widely and across different levels of the system. Six instruments to describe service availability and capacity were identified, of which three had been psychometrically validated. A limitation was the exclusion of grey literature from the review. However, the imprecise nature of the terminology, and high number of initial results, makes the inclusion of grey literature not feasible.

CONCLUSION

We identified that, in spite of its relevance, context studies in mental health services is a very limited research area. Few validated instruments are available. Methodological limitations in many papers mean that the particular challenges of mental health systems research such as system complexity, data availability and terminological variability are generally poorly addressed, presenting a barrier to valid system comparison. The modified Thornicroft and Tansella matrix and related ecological production of care model provide the main model for research within the area of health care ecosystems.