Effects of patient safety auditing in hospital care: results of a mixed-method evaluation (part 1)

OBJECTIVE

To evaluate the effectiveness of internal auditing in hospital care focussed on improving patient safety.

DESIGN, SETTING AND PARTICIPANTS

A before-and-after mixed-method evaluation study was carried out in eight departments of a university medical center in the Netherlands.

INTERVENTION(S)

Internal auditing and feedback focussed on improving patient safety.

MAIN OUTCOME MEASURE(S)

The effect of internal auditing was assessed 15 months after the audit, using linear mixed models, on the patient, professional, team and departmental levels. The measurement methods were patient record review on adverse events (AEs), surveys regarding patient experiences, safety culture and team climate, analysis of administrative hospital data (standardized mortality rate, SMR) and safety walk rounds (SWRs) to observe frontline care processes on safety.

RESULTS

The AE rate decreased from 36.1% to 31.3% and the preventable AE rate from 5.5% to 3.6%; however, the differences before and after auditing were not statistically significant. The patient-reported experience measures regarding patient safety improved slightly over time (P < 0.001). The SMR, patient safety culture and team climate remained unchanged after the internal audit. The SWRs showed that medication safety and information security were improved (P < 0.05).

CONCLUSIONS

Internal auditing was associated with improved patient experiences and observed safety on wards. No effects were found on adverse outcomes, safety culture and team climate 15 months after the internal audit.

Process evaluation of the effects of patient safety auditing in hospital care (part 2)

Objective

To identify factors that explain the observed effects of internal auditing on improving patient safety.

Design setting and participants

A process evaluation study within eight departments of a university medical centre in the Netherlands.

Intervention(s)

Internal auditing and feedback for improving patient safety in hospital care.

Main outcome measure(s)

Experiences with patient safety auditing, percentage implemented improvement actions tailored to the audit results and perceived factors that hindered or facilitated the implementation of improvement actions.

Results

The respondents had positive audit experiences, with the exception of the amount of preparatory work by departments. Fifteen months after the audit visit, 21% of the intended improvement actions based on the audit results were completely implemented. Factors that hindered implementation were short implementation time: 9 months (range 5–11 months) instead of the 15 months’ planned implementation time; time-consuming and labour-intensive implementation of improvement actions; and limited organizational support for quality improvement (e.g. insufficient staff capacity and time, no available quality improvement data and information and communication technological (ICT) support).

Conclusions

A well-constructed analysis and feedback of patient safety problems is insufficient to reduce the occurrence of poor patient safety outcomes. Without focus and support in the implementation of audit-based improvement actions, quality improvement by patient safety auditing will remain limited.

Important factors for effective patient safety governance auditing: a questionnaire survey

Background

Audits are increasingly used for patient safety governance purposes. However, there is little insight into the factors that hinder or stimulate effective governance based on auditing. The aim of this study is to quantify the factors that influence effective auditing for hospital boards and executives.

Methods

A questionnaire of 32 factors was developed using influencing factors found in a qualitative study on effective auditing. Factors were divided into four categories. The questionnaire was sent to the board of directors, chief of medical staff, nursing officer, medical department head and director of the quality and safety department of 89 acute care hospitals in the Netherlands.

Results

We approached 522 people, of whom 211 responded. Of the 32 factors in the questionnaire, 30 factors had an agreement percentage higher than 50%. Important factors per category were ‘audit as an improvement tool as well as a control tool’, ‘department is aware of audit purpose’, ‘quality of auditors’ and ‘learning culture at department’. We found 14 factors with a significant difference in agreement between stakeholders of at least 20%. Amongst these were ‘medical specialist on the audit team’, ‘soft signals in the audit report’, ‘patients as auditors’ and ‘post-audit support’.

Conclusion

We found 30 factors for effective auditing, which we synthesised into eight recommendations to optimise audits. Hospitals can use these recommendations as a framework for audits that enable boards to become more in control of patient safety in their hospital.

Electronic supplementary material

The online version of this article (10.1186/s12913-018-3577-9) contains supplementary material, which is available to authorized users.

Evaluation of the organisation and effectiveness of internal audits to govern patient safety in hospitals: a mixed-methods study

OBJECTIVES

Hospital boards are legally responsible for safe healthcare. They need tools to assist them in their task of governing patient safety. Almost every Dutch hospital performs internal audits, but the effectiveness of these audits for hospital governance has never been evaluated. The aim of this study is to evaluate the organisation of internal audits and their effectiveness for hospitals boards to govern patient safety.

DESIGN AND SETTING

A mixed-methods study consisting of a questionnaire regarding the organisation of internal audits among all Dutch hospitals (n=89) and interviews with stakeholders regarding the audit process and experienced effectiveness of audits within six hospitals.

RESULTS

Response rate of the questionnaire was 76% and 43 interviews were held. In every responding hospital, the internal audits followed the plan-do-check-act cycle. Every hospital used interviews, document analysis and site visits as input for the internal audit. Boards stated that effective aspects of internal audits were their multidisciplinary scope, their structured and in-depth approach, the usability to monitor improvement activities and to change hospital policy and the fact that results were used in meetings with staff and boards of supervisors. The qualitative methods (interviews and site visits) used in internal audits enable the identification of soft signals such as unsafe culture or communication and collaboration problems. Reported disadvantages were the low frequency of internal audits and the absence of soft signals in the actual audit reports.

CONCLUSION

This study shows that internal audits are regarded as effective for patient safety governance, as they help boards to identify patient safety problems, proactively steer patient safety and inform boards of supervisors on the status of patient safety. The description of the Dutch internal audits makes these audits replicable to other healthcare organisations in different settings, enabling hospital boards to complement their systems to govern patient safety.

Effects of auditing patient safety in hospital care: design of a mixed-method evaluation

BACKGROUND

Auditing of patient safety aims at early detection of risks of adverse events and is intended to encourage the continuous improvement of patient safety. The auditing should be an independent, objective assurance and consulting system. Auditing helps an organisation accomplish its objectives by bringing a systematic, disciplined approach to evaluating and improving the effectiveness of risk management, control, and governance. Audits are broadly conducted in hospitals, but little is known about their effects on the behaviour of healthcare professionals and patient safety outcomes. This study was initiated to evaluate the effects of patient safety auditing in hospital care and to explore the processes and mechanisms underlying these effects.

METHODS AND DESIGN

Our study aims to evaluate an audit system to monitor and improve patient safety in a hospital setting. We are using a mixed-method evaluation with a before-and-after study design in eight departments of one university hospital in the period October 2011-July 2014. We measure several outcomes 3 months before the audit and 15 months after the audit. The primary outcomes are adverse events and complications. The secondary outcomes are experiences of patients, the standardised mortality ratio, prolonged hospital stay, patient safety culture, and team climate. We use medical record reviews, questionnaires, hospital administrative data, and observations to assess the outcomes. A process evaluation will be used to find out which components of internal auditing determine the effects.

DISCUSSION

We report a study protocol of an effect and process evaluation to determine whether auditing improves patient safety in hospital care. Because auditing is a complex intervention targeted on several levels, we are using a combination of methods to collect qualitative and quantitative data about patient safety at the patient, professional, and department levels. This study is relevant for hospitals that want to early detect unsafe care and improve patient safety continuously.

TRIAL REGISTRATION

Netherlands Trial Register (NTR): NTR3343.