Design and Validation of a Questionnaire to Measure Interprofessional Collaborative Practice for Auditing Integrated Hospital Care: Empirical Research

INTRODUCTION

Interprofessional teamwork is the key issue of delivering integrated hospital care; however, measuring interprofessional collaboration for auditing is fragmented. In this study, a questionnaire to measure InterProfessional collaborative Practice for Integrated Hospital care (IPPIH) has been developed and validated.

METHODS

A four-step iterative process was conducted: (1) literature search to find suitable questionnaires; (2) semistructured stakeholder interviews (individual and in focus groups) to discuss the topics and questions (face validity), (3) pretesting the prototype of the questionnaire in two different integrated care pathways for feasibility, usability, and internal consistency, and (4) testing (content and construct validity and responsiveness) of the revised questionnaire in eight integrated care pathways; the validation and responsiveness was tested by means of exploratory factor analysis, calculation of Cronbach alpha, item analysis, and linear mixed model analysis.

RESULTS

Based on six questionnaires and the opinion of direct stakeholders, the questionnaire IPPIH comprised 27 items. Five different domains could be distinguished: own skills, culture, coordination and collaboration, practical support, and appreciation with the Cronbach alpha varied from 0.91 to 0.48. The self-reported intensity of the collaboration within a specific care pathway significantly influenced the outcome (P = .000).

CONCLUSION

The product is a questionnaire, IPPIH, which can measure the degree of interprofessional collaborative practice in integrated hospital care pathways. The IPPIH was initially developed for quality assurance. However, the IPPIH also seems to be suitable as a self-assessment tool for directors to monitor and improve the interprofessional collaboration and the quality of their integrated care pathway.

Effects of patient safety auditing in hospital care: results of a mixed-method evaluation (part 1)

OBJECTIVE

To evaluate the effectiveness of internal auditing in hospital care focussed on improving patient safety.

DESIGN, SETTING AND PARTICIPANTS

A before-and-after mixed-method evaluation study was carried out in eight departments of a university medical center in the Netherlands.

INTERVENTION(S)

Internal auditing and feedback focussed on improving patient safety.

MAIN OUTCOME MEASURE(S)

The effect of internal auditing was assessed 15 months after the audit, using linear mixed models, on the patient, professional, team and departmental levels. The measurement methods were patient record review on adverse events (AEs), surveys regarding patient experiences, safety culture and team climate, analysis of administrative hospital data (standardized mortality rate, SMR) and safety walk rounds (SWRs) to observe frontline care processes on safety.

RESULTS

The AE rate decreased from 36.1% to 31.3% and the preventable AE rate from 5.5% to 3.6%; however, the differences before and after auditing were not statistically significant. The patient-reported experience measures regarding patient safety improved slightly over time (P < 0.001). The SMR, patient safety culture and team climate remained unchanged after the internal audit. The SWRs showed that medication safety and information security were improved (P < 0.05).

CONCLUSIONS

Internal auditing was associated with improved patient experiences and observed safety on wards. No effects were found on adverse outcomes, safety culture and team climate 15 months after the internal audit.

Process evaluation of the effects of patient safety auditing in hospital care (part 2)

Objective

To identify factors that explain the observed effects of internal auditing on improving patient safety.

Design setting and participants

A process evaluation study within eight departments of a university medical centre in the Netherlands.

Intervention(s)

Internal auditing and feedback for improving patient safety in hospital care.

Main outcome measure(s)

Experiences with patient safety auditing, percentage implemented improvement actions tailored to the audit results and perceived factors that hindered or facilitated the implementation of improvement actions.

Results

The respondents had positive audit experiences, with the exception of the amount of preparatory work by departments. Fifteen months after the audit visit, 21% of the intended improvement actions based on the audit results were completely implemented. Factors that hindered implementation were short implementation time: 9 months (range 5–11 months) instead of the 15 months’ planned implementation time; time-consuming and labour-intensive implementation of improvement actions; and limited organizational support for quality improvement (e.g. insufficient staff capacity and time, no available quality improvement data and information and communication technological (ICT) support).

Conclusions

A well-constructed analysis and feedback of patient safety problems is insufficient to reduce the occurrence of poor patient safety outcomes. Without focus and support in the implementation of audit-based improvement actions, quality improvement by patient safety auditing will remain limited.

[Sharing internal audit results with the Inspectorate; interviews on the possibility and preconditions]

OBJECTIVE

To study to what extent internal audit results of hospitals can be shared with external supervisors and the necessary preconditions for this.

DESIGN

Qualitative interview research.

METHOD

In 2013-2015, we interviewed 36 individuals from six hospitals: 12 department heads (all medical specialists), 10 department managers; five members of the Board of Directors; five members of the Supervisory Board and the four account-holding hospital inspectors. We also performed a focus group interview with six other hospital inspectors of the Health and Youth Care Inspectorate. The interview data were analysed thematically.

RESULTS

The interviewees pointed out that there is no coordination between internal and external supervision. They were in favour of sharing internal audit results with external supervisors to reduce the supervisory burden. They stated that internal audits give insight into quality improvements, how hospital directors govern quality and safety and the culture of improvement within healthcare provider teams. With this information, the Inspectorate can assess to what extent hospitals are learning organisations. The interviewees mentioned the following preconditions for sharing audit results: reliable and risk-based information about quality and safety, collected by expert, trained auditors and careful use of this information by the Inspectorate in order to maintain openness among audited healthcare providers.

CONCLUSION

Internal audit results can be shared conditionally with external supervisors like the Health and Youth Care Inspectorate. When internal audit results show that hospitals are open, learning and self-cleansing organisations, the Inspectorate can supervise the hospitals remotely and supervisory burden will probably be reduced.

Effects of perioperative briefing and debriefing on patient safety: a prospective intervention study

OBJECTIVES

This study was carried out to improve patient safety in the operating theatre by the introduction of perioperative briefing and debriefing, which focused on an optimal collaboration between surgical team members.

DESIGN

A prospective intervention study with one pretest and two post-test measurements: 1 month before and 4 months and 2.5 years after the implementation of perioperative briefing and debriefing, respectively.

SETTING

Operating theatres of a tertiary care hospital with 875 beds in the Netherlands.

PARTICIPANTS

All members of five surgical teams participated in the perioperative briefing and debriefing.

INTERVENTION

The implementation of perioperative briefing and debriefing from July 2012 to January 2014.

PRIMARY AND SECONDARY OUTCOMES

The primary outcome was changes in the team climate, measured by the Team Climate Inventory. Secondary outcomes were the experiences of surgical teams with perioperative briefing and debriefing, measured with a structured questionnaire, and the duration of the briefings, measured by an independent observer.

RESULTS

Two and a half years after the introduction of perioperative briefing and debriefing, the team climate increased statistically significant (p≤0.05). Members of the five surgical teams strongly agreed with the positive influence of perioperative briefing and debriefing on clear agreements and reminding one another of the agreements of the day. They perceived a higher efficiency of the surgical programme with more operations starting on time and less unexpectedly long operation time. The perioperative briefing took less than 4 min to conduct.

CONCLUSIONS

Perioperative briefing and debriefing improved the team climate of surgical teams and the efficiency of their work within the operating theatre with acceptable duration per briefing. Surgical teams with alternating team compositions have the most benefit of briefing and debriefing.

[Measurement of patient safety: a systematic review of the reliability and validity of adverse event detection with record review]

OBJECTIVES

Record review is the most used method to quantify patient safety. We systematically reviewed the reliability and validity of adverse event detection with record review.

DESIGN

A systematic review of the literature.

METHODS

We searched PubMed, EMBASE, CINAHL, PsycINFO and the Cochrane Library and from their inception through February 2015. We included all studies that aimed to describe the reliability and/or validity of record review. Two reviewers conducted data extraction. We pooled κ values (κ) and analysed the differences in subgroups according to number of reviewers, reviewer experience and training level, adjusted for the prevalence of adverse events.

RESULTS

In 25 studies, the psychometric data of the Global Trigger Tool (GTT) and the Harvard Medical Practice Study (HMPS) were reported and 24 studies were included for statistical pooling. The inter-raterreliability of the GTT and HMPS showed a pooled κ of 0.65 and 0.55, respectively. The inter-rater agreement was statistically significantly higher when the group of reviewers within a study consisted of a maximum five reviewers. We found no studies reporting on the validity of the GTT and HMPS.

CONCLUSIONS

The reliability of record review is moderate to substantial and improved when a small group of reviewers carried out record review. The validity of the record review method has never been evaluated, while clinical data registries, autopsy or direct observations of patient care are methods that can be used to test concurrent validity.

Measurement of patient safety: a systematic review of the reliability and validity of adverse event detection with record review

OBJECTIVES

Record review is the most used method to quantify patient safety. We systematically reviewed the reliability and validity of adverse event detection with record review.

DESIGN

A systematic review of the literature.

METHODS

We searched PubMed, EMBASE, CINAHL, PsycINFO and the Cochrane Library and from their inception through February 2015. We included all studies that aimed to describe the reliability and/or validity of record review. Two reviewers conducted data extraction. We pooled κ values (κ) and analysed the differences in subgroups according to number of reviewers, reviewer experience and training level, adjusted for the prevalence of adverse events.

RESULTS

In 25 studies, the psychometric data of the Global Trigger Tool (GTT) and the Harvard Medical Practice Study (HMPS) were reported and 24 studies were included for statistical pooling. The inter-rater reliability of the GTT and HMPS showed a pooled κ of 0.65 and 0.55, respectively. The inter-rater agreement was statistically significantly higher when the group of reviewers within a study consisted of a maximum five reviewers. We found no studies reporting on the validity of the GTT and HMPS.

CONCLUSIONS

The reliability of record review is moderate to substantial and improved when a small group of reviewers carried out record review. The validity of the record review method has never been evaluated, while clinical data registries, autopsy or direct observations of patient care are potential reference methods that can be used to test concurrent validity.

Effects of auditing patient safety in hospital care: design of a mixed-method evaluation

BACKGROUND

Auditing of patient safety aims at early detection of risks of adverse events and is intended to encourage the continuous improvement of patient safety. The auditing should be an independent, objective assurance and consulting system. Auditing helps an organisation accomplish its objectives by bringing a systematic, disciplined approach to evaluating and improving the effectiveness of risk management, control, and governance. Audits are broadly conducted in hospitals, but little is known about their effects on the behaviour of healthcare professionals and patient safety outcomes. This study was initiated to evaluate the effects of patient safety auditing in hospital care and to explore the processes and mechanisms underlying these effects.

METHODS AND DESIGN

Our study aims to evaluate an audit system to monitor and improve patient safety in a hospital setting. We are using a mixed-method evaluation with a before-and-after study design in eight departments of one university hospital in the period October 2011-July 2014. We measure several outcomes 3 months before the audit and 15 months after the audit. The primary outcomes are adverse events and complications. The secondary outcomes are experiences of patients, the standardised mortality ratio, prolonged hospital stay, patient safety culture, and team climate. We use medical record reviews, questionnaires, hospital administrative data, and observations to assess the outcomes. A process evaluation will be used to find out which components of internal auditing determine the effects.

DISCUSSION

We report a study protocol of an effect and process evaluation to determine whether auditing improves patient safety in hospital care. Because auditing is a complex intervention targeted on several levels, we are using a combination of methods to collect qualitative and quantitative data about patient safety at the patient, professional, and department levels. This study is relevant for hospitals that want to early detect unsafe care and improve patient safety continuously.

TRIAL REGISTRATION

Netherlands Trial Register (NTR): NTR3343.