Audit and feedback to reduce unwarranted clinical variation at scale: a realist study of implementation strategy mechanisms

BACKGROUND

Unwarranted clinical variation in hospital care includes the underuse, overuse, or misuse of services. Audit and feedback is a common strategy to reduce unwarranted variation, but its effectiveness varies widely across contexts. We aimed to identify implementation strategies, mechanisms, and contextual circumstances contributing to the impact of audit and feedback on unwarranted clinical variation.

METHODS

Realist study examining a state-wide value-based healthcare program implemented between 2017 and 2021 in New South Wales, Australia. Three initiatives within the program included audit and feedback to reduce unwarranted variation in inpatient care for different conditions. Multiple data sources were used to formulate the initial audit and feedback program theory: a systematic review, realist review, program document review, and informal discussions with key program stakeholders. Semi-structured interviews were then conducted with 56 participants to refute, refine, or confirm the initial program theories. Data were analysed retroductively using a context-mechanism-outcome framework for 11 transcripts which were coded into the audit and feedback program theory. The program theory was validated with three expert panels: senior health leaders (n = 19), Agency for Clinical Innovation (n = 11), and Ministry of Health (n = 21) staff.

RESULTS

The program's audit and feedback implementation strategy operated through eight mechanistic processes. The strategy worked well when clinicians (1) felt ownership and buy-in, (2) could make sense of the information provided, (3) were motivated by social influence, and (4) accepted responsibility and accountability for proposed changes. The success of the strategy was constrained when the audit process led to (5) rationalising current practice instead of creating a learning opportunity, (6) perceptions of unfairness and concerns about data integrity, 7) development of improvement plans that were not followed, and (8) perceived intrusions on professional autonomy.

CONCLUSIONS

Audit and feedback strategies may help reduce unwarranted clinical variation in care where there is engagement between auditors and local clinicians, meaningful audit indicators, clear improvement plans, and respect for clinical expertise. We contribute theoretical development for audit and feedback by proposing a Model for Audit and Feedback Implementation at Scale. Recommendations include limiting the number of audit indicators, involving clinical staff and local leaders in feedback, and providing opportunities for reflection.

The impact of a multilevel interactive nursing quality control and audit application on nursing quality management

Background

This study aims to investigate the effects of a nursing quality control and audit application (app) on the autonomous learning of nursing staff and nursing quality management by nursing supervisors. A multilevel interactive app is developed to assist nursing staff in conducting online autonomous learning and nursing supervisors in identifying problems and creating nursing quality improvement plans. The app could also present the different evaluation results of wards in visual charts for supervisors to review.

Methods

A single-group pre- and post-test design was applied. Data were collected from 131 nurses between October 2019 and October 2020 to analyze the differences between nursing staffs’ willingness to engage in autonomous learning and the integrity of nursing quality improvement plan writing before and after the intervention. The structured questionnaires included open-ended questions that cover aspects of nursing quality control, the audit app, and the information acceptance intention of nurses.

Results

The participants’ age and job title are negatively correlated with the app’s usability, while the ability to use 3C (Computer, Communication, and Consumer Electronics products including mobile phones and laptops) equipment is positively correlated with the willingness to use the app. Nurses’ satisfaction with the convenience of the online autonomous learning method is 92%, which indicates that the app could improve their willingness to learn. Following the intervention of the app, nursing supervisors’ satisfaction with the integrity of nursing quality improvement plan writing increased from 41 to 88%.

Conclusions

Using information technology products to assist in nursing quality management in clinical practice has a significant effect on nurses’ load reduction and head nurses’ satisfaction. Multilevel interactive nursing quality control and audit apps can improve nursing staff’s willingness to learn independently, nursing quality, and the integrity of plan writing. Thus, nursing quality control and audit apps can be considered as suitable nursing quality control tools.

Measured and perceived effects of audit and feedback on nursing performance: a mixed methods systematic review protocol

BACKGROUND

The use of audit and feedback (A&F) interventions in health care has been demonstrated to generally be effective on medical teams. However, literature suggests that the response of nurses to this type of intervention may differ from that of other types of health professionals, in relation to their roles, power, and to the configuration of nursing care activities. To our knowledge, no review has been conducted on A&F interventions with nurses. The objective of this systematic review is to examine the evidence of measured and perceived effects of A&F interventions on nurses' performance.

METHODS

A mixed methods systematic review design with thematic and narrative synthesis is used. Studies reporting quantitative and qualitative data on the effects of A&F interventions specific to nursing care are considered for inclusion. Studies will be appraised for quality using the Mixed Methods Appraisal Tool. Quantitative and qualitative data will be summarized in narrative and tabular form and will be synthetized using a segregated methodologies approach.

DISCUSSION

Results will describe the characteristics of A&F with nurses, as well as the measured and perceived effects specific to nursing care. The associations between the characteristics and the effects as well as the concordance between measured and perceived effects will be presented. We anticipate that combining the evidence from qualitative and quantitative studies will allow us to provide relevant insight that can inform the design of better suited A&F interventions for nurses. Audit and feedback interventions demonstrate potential for improving the performance of nursing care. As their effectiveness varies greatly depending on the context and the professionals involved, a better understanding of the associations between its characteristics and the measured and perceived effects is valuable for improving the effectiveness of A&F.

SYSTEMATIC REVIEW REGISTRATION

PROSPERO CRD42018104973.

[Influence of an observer in the haemolysis produced during the extraction of blood samples in primary care]

OBJECTIVE

To check whether an intervention based on direct observation and complementary information to nurses helps reduce haemolysis when drawing blood specimens.

MATERIAL AND METHODS

Random sampling study in primary care centres in the serrania de Málaga health management area, using a cross-sectional, longitudinal pre- and post-intervention design. The study period was from August 2012 to January 2015. The level of free haemoglobin was measured by direct spectrophotometry in the specimens extracted. It was then checked whether the intervention influenced the level of haemolysis, and if this was maintained over time.

RESULTS

The mean haemolysis measured pre-intervention was 17%, and after intervention it was 6.1%. A year later and under the same conditions, the frequency of haemolysis was measured again the samples analysed, and the percentage was 9% These results are low when compared to the level obtained pre-intervention, but are higher when compared to the levels obtained immediately after the intervention. The transport and analysis conditions were the same.

CONCLUSIONS

An intervention based on a direct and informative observation in the process of collecting blood samples contributes significantly to reduce the level of haemolysis. This effect is maintained in time. This intervention needs to be repeated to maintain its effectiveness. Audits and continuing education programs are useful for quality assurance procedures, and maintain the level of care needed for a good quality of care.