Measurement and improvement of emergency department performance through inspection and rating: an observational study of emergency departments in acute hospitals in England

Association between regulator inspection and ratings on primary care prescribing: an observational study in England 2014 to 2019

BACKGROUND

Healthcare regulators in many countries undertake inspections of healthcare providers and publish inspection outcomes with the intention of improving quality of care. Comprehensive inspections of general practices in England by the Care Quality Commission began for the first time in 2014. It is assumed that inspection and rating will raise standards and improve care, but the presence and extent of any improvements is unknown. We aim to determine if practice inspection ratings are associated with past performance on prescribing indicators and if prescribing behaviour changes following inspection.

METHODS

Longitudinal study using a dataset of 6771 general practices in England. Practice inspection date and score was linked with monthly practice-level data on prescribing indicators relating to antibiotics, hypnotics and non-steroidal anti-inflammatory drugs. The sample covers practices receiving their first inspection between September 2014 and December 2018. Regression analysis and the differential timing of inspections is used to identify the impact on prescribing.

RESULTS

Better-rated practices had better prescribing in the period before inspections began. In the six months following inspections, no overall change in prescribing was observed. However, the differences between the best and worse rated practices were reduced but not fully. The same is also true when taking a longer-term view. There is little evidence that practices responded in anticipation of inspection or reacted differently once the ratings were made public.

CONCLUSION

While some of the observed historic variation in prescribing behaviour has been lessened by the process of inspection and ratings, we find this change is small and appears to come from both improvements among lower-rated practices and deteriorations among higher-rated practices. While inspection and rating no doubt had other impacts, these prescribing indicators were largely unchanged.

External inspection approaches and involvement of stakeholders' views in inspection following serious incidents - a qualitative mixed methods study from the perspectives of regulatory inspectors

OBJECTIVE

The objective was to gain knowledge about how external inspections following serious incidents are played out in a Norwegian hospital context from the perspective of the inspectors, and whether stakeholders' views are involved in the inspection.

METHODS

Based on a qualitative mixed methods design, 10 government bureaucrats and inspectors situated at the National Board of Health Supervision and three County Governors in Norway, were strategically recruited, and individual semi-structured interviews were conducted. Key official government documents were selected, collected, and thematically analyzed along with the interview data.

RESULTS

Our findings overall demonstrate two overarching themes: Theme (1) Perspectives on different external inspection approaches of responding and involving stakeholders in external inspection following serious incidents, Theme (2) Inspectors' internal work practices versus external expectations. Documents and all participants reported a development towards new approaches in external inspection, with more policies and regulatory attention to sensible involvement of stakeholders. Involvement and interaction with patients and informal caregivers could potentially inform the case complexity and the inspector's decision-making process. However, stakeholder involvement was sometimes complex and challenging due to e.g., difficult communication and interaction with patients and/or informal caregivers, due to resource demands and/or the inspector's lack of experience and/or relevant competence, different perceptions of the principle of sound professional practice, quality, and safety. The inspectors considered balancing the formal objectives and expectations, with the expectations of the public and different stakeholders (i.e. hospitals, patients and/or informal caregivers) a challenging part of their job. This balance was seen as an important part of the continuous development of ensuring public trust and legitimacy in external inspection processes.

CONCLUSIONS AND IMPLICATIONS

Our study suggests that the regulatory system of external inspection and its available approaches of responding to a serious incident in the Norwegian setting is currently not designed to accommodate the complexity of needs from stakeholders at the levels of hospital organizations, patients, and informal caregivers altogether. Further studies should direct attention to how the wider system of accountability structures may support the internal work practices in the regulatory system, to better algin its formal objectives with expectations of the public.

Strategies to Measure and Improve Emergency Department Performance: A Review

Emergency Departments (EDs) globally face escalating challenges such as overcrowding, resource limitations, and increased patient demand. This study aims to identify and analyze strategies to enhance the structural performance of EDs, with a focus on reducing overcrowding, optimizing resource allocation, and improving patient outcomes. Through a comprehensive review of the literature and observational studies, the research highlights the effectiveness of various approaches, including triage optimization, dynamic staffing, technological integration, and strategic resource management. Key findings indicate that tailored strategies, such as implementing advanced triage protocols and leveraging telemedicine, can significantly reduce wait times and enhance patient throughput. Furthermore, evidence suggests that dynamic staffing models and the integration of cutting-edge diagnostic tools contribute to operational efficiency and improved quality of care. These strategies, when combined, offer a multifaceted solution to the complex challenges faced by EDs, promising better patient care and satisfaction. The study underscores the need for a comprehensive approach, incorporating both organizational and technological innovations, to address the evolving needs of emergency healthcare.

Association between cardiopulmonary resuscitation audit results with in-situ simulation and in-hospital cardiac arrest outcomes and key performance indicators

INTRODUCTION

In-situ simulation (ISS) is a method to evaluate the performance of hospital units in performing cardiopulmonary resuscitation (CPR). It is conducted by placing a high-fidelity mannequin at hospital units with simulated scenarios and having each unit's performance evaluated. However, little is known about its impact on actual patient outcomes. Therefore, we aimed to evaluate the association between the ISS results and actual outcomes of patients with in-hospital cardiac arrest (IHCA).

METHODS

This retrospective study was conducted by reviewing Siriraj Hospital's CPR ISS results in association with the data of IHCA patients between January 2012 and January 2019. Actual outcomes were determined by patients' outcomes (sustained return of spontaneous circulation (ROSC) and survival to hospital discharge) and arrest performance indicators (time-to-first-epinephrine and time-to-defibrillation). These outcomes were investigated for association with the ISS scores in multilevel regression models with hospital units as clusters.

RESULTS

There were 2146 cardiac arrests included with sustained ROSC rate of 65.3% and survival to hospital discharge rate of 12.9%. Higher ISS scores were significantly associated with improved sustained ROSC rate (adjusted odds ratio 1.32 (95%CI 1.04, 1.67); p = 0.01) and a decrease in time-to-defibrillation (-0.42 (95%CI -0.73, -0.11); p = 0.009). Although higher scores were also associated with better survival to hospital discharge and a decrease in time-to-first-epinephrine, most models for these outcomes failed to reach statistical significance.

CONCLUSION

CPR ISS results were associated with some important patient outcomes and arrest performance indicators. Therefore, it may be an appropriate performance evaluation method that can guide the direction of improvement.

The Special Measures for Quality and Challenged Provider Regimes in the English NHS: A Rapid Evaluation of a National Improvement Initiative for Failing Healthcare Organisations

BACKGROUND

There is limited knowledge about interventions used for the improvement of low-performing healthcare organisations and their unintended consequences. Our evaluation sought to understand how healthcare organisations in the National Health Service (NHS) in England responded to a national improvement initiative (the Special Measures for Quality [SMQ] and challenged provider [CP] regimes) and its perceived impact on achieving quality improvements (QIs).

METHODS

Our evaluation included national-level interviews with key stakeholders involved in the delivery of SMQ (n=6); documentary analysis (n=20); and a qualitative study based on interviews (n=60), observations (n=8) and documentary analysis (n=291) in eight NHS case study sites. The analysis was informed by literature on failure, turnaround and QI in organisations in the public sector.

RESULTS

At the policy level, SMQ/CP regimes were intended to be "support" programmes, but perceptions of the interventions at hospital level were mixed. The SMQ/CP regimes tended to consider failure at an organisational level and turnaround was visualised as a linear process. There was a negative emotional impact reported by staff, especially in the short-term. Key drivers of change included: engaged senior leadership teams, strong clinical input and supportive external partnerships within local health systems. Trusts focused efforts to improve across multiple domains with particular investment in improving overall staff engagement, developing an open, listening organisational culture and better governance to ensure clinical safety and reporting.

CONCLUSION

Organisational improvement in healthcare requires substantial time to embed and requires investment in staff to drive change and cultivate QI capabilities at different tiers. The time this takes may be underestimated by external 'turn-around' interventions and performance regimes designed to improve quality in the short-term and which come at an emotional cost for staff. Shifting an improvement focus to the health system or regional level may promote sustainable improvement across multiple organisations over the long-term.

Impact Sound Insulation Performance Testing of Nano-Inorganic Composite Floor Slabs for Green Buildings

To explore the detection of impact sound insulation performance of nano-inorganic composite floor slabs for green buildings, for 4 types of floor slab practices in a certain area, we carry out the detection of the sound insulation performance of the floor impact sound and analyze and summarize the test results. Finally, several common technical measures to effectively improve the sound insulation performance of floor impact sound are summarized. Experimental results show that with floor slabs using nano-inorganic composite material FBP, the impact sound insulation performance of the floor slab can be greatly improved. FRP materials have the advantages of high strength, lightweight, and good corrosion resistance. They are more and more widely used in construction engineering. At this stage, for environmental friendly green building materials and nano-inorganic composite materials with certain effects, their main practice and application play an important role. It is proved that adding FBP material can greatly improve the impact sound insulation performance of floor slabs.

The Implementation of Improvement Interventions for "Low Performing" and "High Performing" Organisations in Health, Education and Local Government: A Phased Literature Review

BACKGROUND

There is limited understanding about whether and how improvement interventions are effective in supporting failing healthcare organisations and improving the quality of care in high-performing organisations. The aim of this review was to examine the underlying concepts guiding the design of interventions aimed at low and high performing healthcare organisations, processes of implementation, unintended consequences, and their impact on costs and quality of care. The review includes articles in the healthcare sector and other sectors such as education and local government.

METHODS

We carried out a phased rapid systematic review of the literature. Phase one was used to develop a theoretical framework of organisational failure and turnaround, and the types of interventions implemented to improve quality. The framework was used to inform phase 2, which was targeted and focused on organisational failure and turnaround in healthcare, education and local government settings. We used the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) statement to guide the reporting of the methods and findings and the Mixed Methods Appraisal Tool (MMAT) as a quality assessment tool. The review protocol was registered with PROSPERO (CRD: 42019131024).

RESULTS

Failure is frequently defined as the inability of organisations to meet pre-established performance standards and turnaround as a linear process. Improvement interventions are designed accordingly and are focused on the organisation, with limited system-level thinking. Successful interventions included restructuring senior leadership teams, inspections, and organisational restructuring by external organisations. Limited attention was paid to the potential negative consequences of the interventions and their costs.

CONCLUSION

Dominant definitions of success/failure and turnaround have led to the reduced scope of improvement interventions, the linear perception of turnaround, and lack of consideration of organisations within the wider system in which they operate. Future areas of research include an analysis of the costs of delivering these interventions in relation to their impact on quality of care.

The new Care Quality Commission inspection procedures

Emeritus Professor Alan Glasper, from the University of Southampton, discusses recent changes to the way in which the Care Quality Commission (CQC) conducts its health and social care inspections.

Is a hospital quality policy based on a triad of accreditation, public reporting and inspection evidence-based? A narrative review

BACKGROUND

Since 2009, hospital quality policy in Flanders, Belgium, is built around a quality-of-care triad, which encompasses accreditation, public reporting (PR) and inspection. Policy makers are currently reflecting on the added value of this triad.

METHODS

We performed a narrative review of the literature published between 2009 and 2020 to examine the evidence base of the impact accreditation, PR and inspection, both individually and combined, has on patient processes and outcomes. The following patient outcomes were examined: mortality, length of stay, readmissions, patient satisfaction, adverse outcomes, failure to rescue, adherence to process measures and risk aversion. The impact of accreditation, PR and inspection on these outcomes was evaluated as either positive, neutral (i.e. no impact observed or mixed results reported) or negative.

OBJECTIVES

To assess the current evidence base on the impact of accreditation, PR and inspection on patient processes and outcomes.

RESULTS

We identified 69 studies, of which 40 were on accreditation, 24 on PR, three on inspection and two on accreditation and PR concomitantly. Identified studies reported primarily low-level evidence (level IV, n = 53) and were heterogeneous in terms of implemented programmes and patient populations (often narrow in PR research). Overall, a neutral categorization was determined in 30 articles for accreditation, 23 for PR and four for inspection. Ten of these recounted mixed results. For accreditation, a high number (n = 12) of positive research on adherence to process measures was discovered.

CONCLUSION

The individual impact of accreditation, PR and inspection, the core of Flemish hospital quality, was found to be limited on patient outcomes. Future studies should investigate the combined effect of multiple quality improvement strategies.

Do performance indicators predict regulator ratings of healthcare providers? Cross-sectional study of acute hospitals in England

OBJECTIVE

To determine whether a large set of care performance indicators ('Intelligent Monitoring (IM)') can be used to predict the Care Quality Commission's (CQC) acute hospital trust provider ratings.

DESIGN

The IM dataset and first-inspection ratings were used to build linear and ordered logistic regression models for the whole dataset (all trusts). This was repeated for subsets of the trusts, with these models then applied to predict the inspection ratings of the remaining trusts.

SETTING

The United Kingdom Department of Health and Social Care's Care Quality Commission is the regulator for all health and social care services in England. We consider their first-inspection cycle of acute hospital trusts (2013-2016).

PARTICIPANTS

All 156 English NHS acute hospital trusts.

INTERVENTION(S)

None.

MAIN OUTCOME MEASURE(S)

Percentage of correct predictions and weighted kappa.

RESULTS

Only 24% of the predicted overall ratings for the test sample were correct and the weighted kappa of 0.01 indicates very poor agreement between predicted and actual ratings. This lack of predictive power is also found for each of the rating domains.

CONCLUSION

While hospital inspections draw on a much wider set of information, the poor power of performance indicators to predict subsequent inspection ratings may call into question the validity of indicators, ratings or both. We conclude that a number of changes to the way performance indicators are collected and used could improve their predictive value, and suggest that assessing predictive power should be undertaken prospectively when the sets of indicators are being designed and selected by regulators.

Promoting leadership and quality improvement through external inspections of management of sepsis in Norwegian hospitals: a focus group study

OBJECTIVE

Inspections and other forms of external assessment may contribute to positive changes in the health services, but the mechanisms of such change remain unclear. We did a study to explore how external inspections may foster clinical improvement in hospitals.

DESIGN

Focus group study.

SETTING

Statutory inspections of sepsis treatment in hospital emergency departments in Norway.

PARTICIPANTS

Clinicians, managers and inspection teams involved with the inspections of sepsis treatment in emergency departments at four different hospitals. Twelve focus group interviews were carried out, with a total of 47 participants.

RESULTS

Three themes emerged as central for understanding how the inspections could contribute to clinical improvement in the emergency departments: (1) increasing awareness about the need to improve the quality of care by providing data on clinical performance, (2) building acceptance for improvement through professional credibility and focus on clinical practice, and (3) fostering leadership commitment.

CONCLUSION

Our findings suggest that the inspections have the potential to enhance hospital management and staff's understanding of complicated care processes and help strengthen the organisational commitment to bring about systemic quality improvements.

Effects of external inspections on sepsis detection and treatment: a stepped-wedge study with cluster-level randomisation

OBJECTIVE

To evaluate the effects of external inspections on (1) hospital emergency departments' clinical processes for detecting and treating sepsis and (2) length of hospital stay and 30-day mortality.

DESIGN

Incomplete cluster-randomised stepped-wedge design using data from patient records and patient registries. We compared care processes and patient outcomes before and after the intervention using regression analysis.

SETTING

Nationwide inspections of sepsis care in emergency departments in Norwegian hospitals.

PARTICIPANTS

7407 patients presenting to hospital emergency departments with sepsis.

INTERVENTION

External inspections of sepsis detection and treatment led by a public supervisory institution.

MAIN OUTCOME MEASURES

Process measures for sepsis diagnostics and treatment, length of hospital stay and 30-day all-cause mortality.

RESULTS

After the inspections, there were significant improvements in the proportions of patients examined by a physician within the time frame set in triage (OR 1.28, 95% CI 1.07 to 1.53), undergoing a complete set of vital measurements within 1 hour (OR 1.78, 95% CI 1.10 to 2.87), having lactate measured within 1 hour (OR 2.75, 95% CI 1.83 to 4.15), having an adequate observation regimen (OR 2.20, 95% CI 1.51 to 3.20) and receiving antibiotics within 1 hour (OR 2.16, 95% CI 1.83 to 2.55). There was also significant reduction in mortality and length of stay, but these findings were no longer significant when controlling for time.

CONCLUSIONS

External inspections were associated with improvement of sepsis detection and treatment. These findings suggest that policy-makers and regulatory agencies should prioritise assessing the effects of their inspections and pay attention to the mechanisms by which the inspections might contribute to improve care for patients.

TRIAL REGISTRATION

NCT02747121.

Mediators of change in healthcare organisations subject to external assessment: a systematic review with narrative synthesis

OBJECTIVES

External inspections are widely used to improve the quality of care. The effects of inspections remain unclear and little is known about how they may work. We conducted a narrative synthesis of research literature to identify mediators of change in healthcare organisations subject to external inspections.

METHODS

We performed a literature search (1980-January 2020) to identify empirical studies addressing change in healthcare organisations subject to external inspection. Guided by the Consolidated Framework for Implementation Research, we performed a narrative synthesis to identify mediators of change.

RESULTS

We included 95 studies. Accreditation was the most frequent type of inspection (n=68), followed by statutory inspections (n=19), and external peer review (n=9). Our findings suggest that the regulatory context in which the inspections take place affect how they are acted on by those being inspected. The way inspections are conducted seem to be critical for how the inspection findings are perceived and followed up. Inspections can engage and involve staff, facilitate leader engagement, improve communication and enable the creation of new networks for reflection on clinical practice. Inspections can contribute to creating an awareness of the inspected organisation's current practice and performance gaps, and a commitment to change. Moreover, they can contribute to facilitating the planning and implementation of change, as well as self-evaluation and the use of data to evaluate performance.

CONCLUSIONS

External inspections can affect different mediators of organisational change. The way and to what extent they do depend on a range of factors related to the outer setting, the way inspections are conducted and how they are perceived and acted on by the inspected organisation. To improve the quality of care, the organisational change processes need to involve and impact the way care is delivered to the patients.

The measurement and improvement of maternity service performance through inspection and rating: An observational study of maternity services in acute hospitals in England

•

The performance of maternity departments varies and is a policy concern.

•

Inspection and ratings is a possible means to increase performance.

•

Routinely collected data may help measure the performance of maternity departments.

•

Using routine data, post-inspection performance was not associated with inspection scores.

•

Inspection and rating do not stimulate improvement on the performance measures studied.

Objectives

To determine whether the prior performance of maternity services, as measured by Royal College of Obstetricians and Gynaecologists performance indicators, is associated with ratings by the Care Quality Commission at subsequent inspection, and whether performance changes occur after inspection.

Methods

We used hospital activity data from 176 maternity sites inspected between October 2013 and March 2016 to generate a set of performance indicators developed by the Royal College of Obstetricians and Gynaecologists. We linked these data to Care Quality Commission data on inspection dates and rating scores and used regression models, controlling for site level effects, to estimate the relationships between inspection ratings and performance indicators before and after inspections.

Results

Coefficients measuring the relationship between indicator performance and subsequent inspection rating score had wide confidence intervals which crossed zero suggesting no statistically significant relationship prior to inspection. The same absence of statistical significance was observed for changes in indicator performance after inspection.

Conclusions

The use of routine data for performance monitoring is becoming increasingly important as regular inspection is costly and regulators require accurate and timely intelligence. However, we found no statistically significant relationships between inspection ratings and performance indicators before or after inspections in maternity services. This calls into question the validity and reliability of the performance indicators, the inspection process and ratings, or both, as measures of performance.

Using quality indicators to predict inspection ratings: cross-sectional study of general practices in England

Background

The Care Quality Commission regulates, inspects, and rates general practice providers in England. Inspections are costly and infrequent, and are supplemented by a system of routine quality indicators, measuring patient satisfaction and the management of chronic conditions. These indicators can be used to prioritise or target inspections.

Aim

To determine whether this set of indicators can be used to predict the ratings awarded in subsequent inspections.

Design and setting

This cross-sectional study was conducted using a dataset of 6860 general practice providers in England.

Method

The indicators and first-inspection ratings were used to build ordered logistic regression models to predict inspection outcomes on the four-level rating system (‘outstanding’, ‘good’, ‘requires improvement’, and ‘inadequate’) for domain ratings and the ‘overall’ rating. Predictive accuracy was assessed using the percentage of correct predictions and a measure of agreement (weighted κ).

Results

The model correctly predicted 79.7% of the ‘overall’ practice ratings. However, 78.8% of all practices were rated ‘good’ on ‘overall’, and the weighted κ measure of agreement was very low (0.097); as such, predictions were little more than chance. This lack of predictive power was also found for each of the individual domain ratings.

Conclusion

The poor power of performance of these indicators to predict subsequent inspection ratings may call into question the validity and reliability of the indicators, inspection ratings, or both. A number of changes to the way data relating to performance indicators are collected and used are suggested to improve the predictive value of indicators. It is also recommended that assessments of predictive power be undertaken prospectively when sets of indicators are being designed and selected by regulators.