Improvement and evaluation

Factors influencing measurement for improvement skills in healthcare staff: trainee, and trainer perspectives

BACKGROUND

Measurement for improvement is an integral component of quality improvement (QI) trainings and demonstrates whether a change resulted in an improvement. Despite its critical role, the development of measurement for improvement skills for QI is relatively under-explored.

PURPOSE

To explore the training, curricular and contextual factors that influence the development of measurement for improvement skills in healthcare professionals.

METHODS

This is a retrospective, qualitative, multiple case study design, based on two QI collaboratives. Trainees and trainers from these programmes participated in semi-structured interviews. A framework drawing on the Kirkpatrick's evaluation model and the Model for Understanding Success in Quality (MUSIQ) model was developed. The interviews were analysed based on a three-step qualitative thematic analysis method.

RESULTS

A total of 21 participants were interviewed (15 trainees and 6 trainers). Six themes emerged in the analysis of trainee interviewees: impact of differences in job role and hierarchical levels, narrow conception of QI, knowledge disparity between trained and untrained staff, balancing the benefits and burdens of measurement, early adopters of QI driving change and supportive and engaged leadership. Themes in trainer perspectives were knowledge and understanding of measurement, application of PDSA approach to programme design, balancing consistency with adaptation to context, and attributes of sites receptive to change as predictors of development of measurement for improvement skills in staff.

CONCLUSION

Training alone does not determine the development, sustainability and spread of measurement and QI skills. Instead, it is influenced by a combination of curricular, training, and contextual support structures. Training programmes should be aware of the impact of job role and hierarchy, increased knowledge disparity between trained and untrained staff and trainees equating QI to bundle implementation while designing programmes. Similarly, organisational support through leaders, encouraging staff who have an interest in measurement and a culture receptive to QI also supports development of measurement skills. The study highlights the need for trainees, trainers, and organisations to work together in balancing the benefits and burdens of measurement, leading to sustainable skill development in line with international best practices.

The biopsychosociotechnical model: a systems-based framework for human-centered health improvement

The biopsychosocial model is among the most influential frameworks for human-centered health improvement but has faced significant criticism- both conceptual and pragmatic. This paper extends and fundamentally re-structures the biopsychosocial model by combining it with sociotechnical systems theory. The resulting biopsychosociotechnical model addresses key critiques of the biopsychosocial model, providing a more "practical theory" for human-centered health improvement. It depicts the determinants of health as complex adaptive system of systems; includes the the artificial world (technology); and provides a roadmap for systems improvement by: differentiating between "health status" and "health and needs assessment", [promoting problem framing]; explaining health as an emergent property of the biopsychosociotechnical context [imposing a systems orientation]; focusing on "interventions" vs. "treatments" to modify the biopsychosociotechnical determinants of health, [expanding the solution space]; calling for a participatory design process [supporting systems awareness and goal-orientation]; and including intervention management to support the full lifecycle of health improvement.

Effects of work conditions on provider mental well-being and quality of care: a mixed-methods intervention study in the emergency department

BACKGROUND

Emergency departments (EDs) are highly dynamic and stressful care environments that affect provider and patient outcomes. Yet, effective interventions are missing. This study evaluated prospective effects of a multi-professional organizational-level intervention on changes in ED providers' work conditions and well-being (primary outcomes) and patient-perceived quality of ED care (secondary outcome).

METHODS

A before and after study including an interrupted time-series (ITS) design over 1 year was established in the multidisciplinary ED of a tertiary referral hospital in Southern Germany. Our mixed-methods approach included standardized provider surveys, expert work observations, patient surveys, and register data. Stakeholder interviews were conducted for qualitative process evaluation. ITS data was available for 20 days pre- and post-intervention (Dec15/Jan16; Dec16/Jan17). The intervention comprised ten multi-professional meetings in which ED physicians and nurses developed solutions to work stressors in a systematic moderated process. Most solutions were consecutively implemented. Changes in study outcomes were assessed with paired t-tests and segmented regression analyses controlling for daily ED workload.

RESULTS

One hundred forty-nine surveys were returned at baseline and follow-up (response at baseline: 76 out of 170; follow-up: 73 out of 157). Forty-one ED providers participated in both waves. One hundred sixty expert work observations comprising 240 observation hours were conducted with 156 subsequent work stress reports. One thousand four hundred eighteen ED patients were surveyed. Considering primary outcomes, respondents reported more job control and less overtime hours at follow-up. Social support, job satisfaction, and depersonalization deteriorated while respondents' turnover intentions and inter-professional interruptions increased. Considering the secondary outcome, patient reports indicated improvements in ED organization and waiting times. Interviews revealed facilitators (e.g., comprehensive approach, employee participation) and barriers (e.g., understaffing, organizational constraints) for intervention implementation.

CONCLUSIONS

To the best of our knowledge, this is the first study to report prospective effects of an ED work system intervention on provider well-being and patient-perceived quality of ED care. We found inconsistent results with partial improvements in work conditions and patient perceptions of care. However, aspects of provider mental well-being deteriorated. Given the lack of organizational-level intervention research in EDs, our findings provide valuable insights into the feasibility and effects of participatory interventions in this highly dynamic hospital setting.

Empowering junior doctors: a qualitative study of a QI programme in South West England

AIM

To explore how the South-West Foundation Doctor Quality Improvement programme affected foundation year 1 (F1) doctors' attitudes and ability to implement change in healthcare.

METHODS

Twenty-two qualitative interviews were carried out with two cohorts of doctors. The first F1 group before and after their participation in the QI programme; the second group comprised those who had completed the programme between 1 and 5 years earlier. Qualitative data were analysed using thematic analysis techniques. RESULTS : Prior to taking part in the QI programme, junior doctors' attitudes towards QI were mixed. Although there was agreement on the importance of QI in terms of patient safety, not all shared enthusiasm for engaging in QI, while some were sceptical that they could bring about any change. Following participation in the programme, attitudes towards QI and the ability to effect change were significantly transformed. Whether their projects were considered a success or not, all juniors reported that they valued the skills learnt and the overall experience they gained through carrying out QI projects. Participants reported feeling more empowered in their role as junior doctors, with several describing how they felt 'listened to' and able to 'have a voice', that they were beginning to see things 'at systems level' and learning to 'engage more critically' in their working environment. CONCLUSIONS : Junior doctors are ideally placed to engage in QI. Training in QI at the start of their medical careers may enable a new generation of doctors to acquire the skills necessary to improve patient safety and quality of care.

Measuring harm and informing quality improvement in the Welsh NHS: the longitudinal Welsh national adverse events study

Background, objectives and setting

Despite global activity over the past 15 years to improve patient safety, the measurement of adverse events (AEs) remains challenging.

Objectives

We aimed to obtain definitive longitudinal data on harm across NHS Wales and to compare the performance of the Global Trigger Tool (GTT) with the two-stage retrospective review process, using our findings to consolidate an approach to the ongoing surveillance of harm in Wales.

Data sources

Eleven of the 13 major Welsh NHS hospitals.

Review methods

The two-stage retrospective review methodology was used to quantify harm across NHS Wales. In total, 4536 inpatient episodes were screened for AEs by research nurses. Records that were highly suggestive of AEs were further assessed by physicians. NHS-led teams undertook GTT reviews on the same case notes.

Results

At least one AE was determined in 10.3% of episodes of care [95% confidence interval (CI) 9.4% to 11.2%] and 51.5% were preventable (95% CI 46.9% to 56.1%). The percentage of patients identified with AEs using the GTT methodology was lower, at 9.0% (95% CI 8.82% to 9.18%). Differences in AEs were evident across study sites. Methods were developed to profile the risk of AEs in individual organisations by producing signatures of harm for each NHS site. Analysis indicated that neither the GTT nor the two-stage process was a candidate tool for routine surveillance, and a hybrid one-stage tool (Harm2), based on phase 1 findings, was developed for ongoing AE monitoring. Using the Harm2 tool, AEs were identified in 371 out of 3352 randomly selected discharge reviews (11.3%, 95% CI 10.2% to 12.4%), and 59.6% (95% CI 55.3 to 63.9) of these were preventable. In a cohort of randomly selected deceased patient reviews, at least one AE was determined in 315 out of 1018 admissions (30.1%, 95% CI 28.1% to 33.8%), and 61.7% (95% CI 57.5% to 65.9%) of these were preventable. Factors associated with AEs in the randomly selected discharge reviews included having peripheral vascular disease [odds ratio (OR) 2.52], hemiplegia (OR 2.27) or dementia (OR 2.27). No association with chronic disease was identified in the deceased episodes of care.

Limitations

The dependence on our health service partners in identifying notes to be reviewed, along with the small sample examined each month, limits the generalisability of these findings and rates were not standardised for hospital and size and level of services provided. We cannot rule out the possibility that the rates we report may be underestimated.

Conclusion

The extent of harm detected across NHS Wales using both the two-stage retrospective review process and the new Harm2 tool conforms to the findings in the literature, but this is the first longitudinal study using these methods. With training and using a structured review process, non-physician reviewers can undertake case note review efficiently and effectively, and the rates of AEs and of the preventability and the breakdown of problems in care conform to those reported in studies in which physicians undertake these classifications. Whether the patient died or was discharged alive significantly influences the rate and composition of AEs. The Harm2 tool performed with moderate reliability in the determination of AEs.

Future work

Future large-scale studies should attempt to specify types of AEs, such as hospital-acquired infections and surgical complications, to enable the surveillance of the specific types of harm as well as the overall level of AEs. In the longer term, we need to automate harm surveillance and set measures of harm alongside measures of the beneficial effects of health care.

Funding

The National Institute for Health Research Health Services and Delivery Research programme.

Improvement Science: conceptual and theoretical foundations for its application to healthcare quality improvement

The development and study of healthcare quality improvement interventions have been reshaped, moving from more intuitive approaches, dominated by biomedical vision and premised on easy transferability, to gradually acknowledge the need for more planning and systematization, with greater incorporation of the social sciences and enhancement of the role of context. Improvement Science has been established, with a conceptual and methodological framework for such studies. Considering the incipient of the debate and scientific production on Improvement Science in Brazil, this article aims to expound its principal conceptual and theoretical fundamentals, focusing on three central themes: the linkage of different disciplines; recognition of the role of context; and the theoretical basis for the design, implementation, and evaluation of interventions. Resumo: O desenvolvimento e estudo de intervenções para a melhoria do cuidado de saúde tem ganhado novo contorno, movendo-se das abordagens mais intuitivas, com domínio da visão biomédica e assentadas no pressuposto de fácil transferibilidade, para gradativamente reconhecer a necessidade de mais planejamento e sistematização, com maior incorporação das ciências sociais e valorização do papel do contexto. A Ciência da Melhoria do Cuidado de Saúde vem se estabelecendo, propiciando referencial conceitual e metodológico para tais estudos. Considerando a incipiência do debate e produção sobre Ciência da Melhoria do Cuidado de Saúde no Brasil, este artigo objetiva discorrer sobre as principais bases conceituais e teóricas que a sustentam, com foco em três temas centrais: a articulação de diferentes disciplinas; o reconhecimento do papel do contexto; e o embasamento teórico para o desenho, implementação e avaliação das intervenções.

Myths and realities of training in obstetric emergencies

Training for intrapartum emergencies is a promising strategy to reduce preventable harm during birth; however, not all training is clinically effective. Many myths have developed around such training. These principally derive from misinformed beliefs that all training must be effective, cheap, independent of context and sustainable. The current evidence base for effective training supports local, unit-based and multi-professional training, with appropriate mannequins, and practice-based tools to support the best care. Training programmes based on these principles are associated with improved clinical outcomes, but we need to understand how and why that is, and also why some training is associated with no improvements, or even deterioration in outcomes. Effective training is not cheap, but it can be cost-effective. Insurers have the fiscal power to incentivise training, but they should demand the evidence of clinical effect; aspiration and proxies alone should no longer be sufficient for funding, in any resource setting.