Practitioner based quality improvement: a review of the Royal College of Nursing's dynamic standard setting system

Audit and feedback to reduce unwarranted clinical variation at scale: a realist study of implementation strategy mechanisms

BACKGROUND

Unwarranted clinical variation in hospital care includes the underuse, overuse, or misuse of services. Audit and feedback is a common strategy to reduce unwarranted variation, but its effectiveness varies widely across contexts. We aimed to identify implementation strategies, mechanisms, and contextual circumstances contributing to the impact of audit and feedback on unwarranted clinical variation.

METHODS

Realist study examining a state-wide value-based healthcare program implemented between 2017 and 2021 in New South Wales, Australia. Three initiatives within the program included audit and feedback to reduce unwarranted variation in inpatient care for different conditions. Multiple data sources were used to formulate the initial audit and feedback program theory: a systematic review, realist review, program document review, and informal discussions with key program stakeholders. Semi-structured interviews were then conducted with 56 participants to refute, refine, or confirm the initial program theories. Data were analysed retroductively using a context-mechanism-outcome framework for 11 transcripts which were coded into the audit and feedback program theory. The program theory was validated with three expert panels: senior health leaders (n = 19), Agency for Clinical Innovation (n = 11), and Ministry of Health (n = 21) staff.

RESULTS

The program's audit and feedback implementation strategy operated through eight mechanistic processes. The strategy worked well when clinicians (1) felt ownership and buy-in, (2) could make sense of the information provided, (3) were motivated by social influence, and (4) accepted responsibility and accountability for proposed changes. The success of the strategy was constrained when the audit process led to (5) rationalising current practice instead of creating a learning opportunity, (6) perceptions of unfairness and concerns about data integrity, 7) development of improvement plans that were not followed, and (8) perceived intrusions on professional autonomy.

CONCLUSIONS

Audit and feedback strategies may help reduce unwarranted clinical variation in care where there is engagement between auditors and local clinicians, meaningful audit indicators, clear improvement plans, and respect for clinical expertise. We contribute theoretical development for audit and feedback by proposing a Model for Audit and Feedback Implementation at Scale. Recommendations include limiting the number of audit indicators, involving clinical staff and local leaders in feedback, and providing opportunities for reflection.

Facilitating action planning within audit and feedback interventions: a mixed-methods process evaluation of an action implementation toolbox in intensive care

Background

Audit and feedback (A&F) is more effective if it facilitates action planning, but little is known about how best to do this. We developed an electronic A&F intervention with an action implementation toolbox to improve pain management in intensive care units (ICUs); the toolbox contained suggested actions for improvement. A head-to-head randomised trial demonstrated that the toolbox moderately increased the intervention’s effectiveness when compared with A&F only.

Objective

To understand the mechanisms through which A&F with action implementation toolbox facilitates action planning by ICUs to increase A&F effectiveness.

Methods

We extracted all individual actions from action plans developed by ICUs that received A&F with (n = 10) and without (n = 11) toolbox for 6 months and classified them using Clinical Performance Feedback Intervention Theory. We held semi-structured interviews with participants during the trial. We compared the number and type of planned and completed actions between study groups and explored barriers and facilitators to effective action planning.

Results

ICUs with toolbox planned more actions directly aimed at improving practice (p = 0.037) and targeted a wider range of practice determinants compared to ICUs without toolbox. ICUs with toolbox also completed more actions during the study period, but not significantly (p = 0.142). ICUs without toolbox reported more difficulties in identifying what actions they could take. Regardless of the toolbox, all ICUs still experienced barriers relating to the feedback (low controllability, accuracy) and organisational context (competing priorities, resources, cost).

Conclusions

The toolbox helped health professionals to broaden their mindset about actions they could take to change clinical practice. Without the toolbox, professionals tended to focus more on feedback verification and exploring solutions without developing intentions for actual change. All feedback recipients experienced organisational barriers that inhibited eventual completion of actions.

Trial registration

ClinicalTrials.gov, NCT02922101. Registered on 26 September 2016.

Electronic supplementary material

The online version of this article (10.1186/s13012-019-0937-8) contains supplementary material, which is available to authorized users.

Clinical Performance Feedback Intervention Theory (CP-FIT): a new theory for designing, implementing, and evaluating feedback in health care based on a systematic review and meta-synthesis of qualitative research

BACKGROUND

Providing health professionals with quantitative summaries of their clinical performance when treating specific groups of patients ("feedback") is a widely used quality improvement strategy, yet systematic reviews show it has varying success. Theory could help explain what factors influence feedback success, and guide approaches to enhance effectiveness. However, existing theories lack comprehensiveness and specificity to health care. To address this problem, we conducted the first systematic review and synthesis of qualitative evaluations of feedback interventions, using findings to develop a comprehensive new health care-specific feedback theory.

METHODS

We searched MEDLINE, EMBASE, CINAHL, Web of Science, and Google Scholar from inception until 2016 inclusive. Data were synthesised by coding individual papers, building on pre-existing theories to formulate hypotheses, iteratively testing and improving hypotheses, assessing confidence in hypotheses using the GRADE-CERQual method, and summarising high-confidence hypotheses into a set of propositions.

RESULTS

We synthesised 65 papers evaluating 73 feedback interventions from countries spanning five continents. From our synthesis we developed Clinical Performance Feedback Intervention Theory (CP-FIT), which builds on 30 pre-existing theories and has 42 high-confidence hypotheses. CP-FIT states that effective feedback works in a cycle of sequential processes; it becomes less effective if any individual process fails, thus halting progress round the cycle. Feedback's success is influenced by several factors operating via a set of common explanatory mechanisms: the feedback method used, health professional receiving feedback, and context in which feedback takes place. CP-FIT summarises these effects in three propositions: (1) health care professionals and organisations have a finite capacity to engage with feedback, (2) these parties have strong beliefs regarding how patient care should be provided that influence their interactions with feedback, and (3) feedback that directly supports clinical behaviours is most effective.

CONCLUSIONS

This is the first qualitative meta-synthesis of feedback interventions, and the first comprehensive theory of feedback designed specifically for health care. Our findings contribute new knowledge about how feedback works and factors that influence its effectiveness. Internationally, practitioners, researchers, and policy-makers can use CP-FIT to design, implement, and evaluate feedback. Doing so could improve care for large numbers of patients, reduce opportunity costs, and improve returns on financial investments.

TRIAL REGISTRATION

PROSPERO, CRD42015017541.

Clinical performance comparators in audit and feedback: a review of theory and evidence

BACKGROUND

Audit and feedback (A&F) is a common quality improvement strategy with highly variable effects on patient care. It is unclear how A&F effectiveness can be maximised. Since the core mechanism of action of A&F depends on drawing attention to a discrepancy between actual and desired performance, we aimed to understand current and best practices in the choice of performance comparator.

METHODS

We described current choices for performance comparators by conducting a secondary review of randomised trials of A&F interventions and identifying the associated mechanisms that might have implications for effective A&F by reviewing theories and empirical studies from a recent qualitative evidence synthesis.

RESULTS

We found across 146 trials that feedback recipients' performance was most frequently compared against the performance of others (benchmarks; 60.3%). Other comparators included recipients' own performance over time (trends; 9.6%) and target standards (explicit targets; 11.0%), and 13% of trials used a combination of these options. In studies featuring benchmarks, 42% compared against mean performance. Eight (5.5%) trials provided a rationale for using a specific comparator. We distilled mechanisms of each comparator from 12 behavioural theories, 5 randomised trials, and 42 qualitative A&F studies.

CONCLUSION

Clinical performance comparators in published literature were poorly informed by theory and did not explicitly account for mechanisms reported in qualitative studies. Based on our review, we argue that there is considerable opportunity to improve the design of performance comparators by (1) providing tailored comparisons rather than benchmarking everyone against the mean, (2) limiting the amount of comparators being displayed while providing more comparative information upon request to balance the feedback's credibility and actionability, (3) providing performance trends but not trends alone, and (4) encouraging feedback recipients to set personal, explicit targets guided by relevant information.

Enabling Continuous Quality Improvement in Practice: The Role and Contribution of Facilitation

Facilitating the implementation of continuous quality improvement (CQI) is a complex undertaking. Numerous contextual factors at a local, organizational, and health system level can influence the trajectory and ultimate success of an improvement program. Some of these contextual factors are amenable to modification, others less so. As part of planning and implementing healthcare improvement, it is important to assess and build an understanding of contextual factors that might present barriers to or enablers of implementation. On the basis of this initial diagnosis, it should then be possible to design and implement the improvement intervention in a way that is responsive to contextual barriers and enablers, often described as “tailoring” the implementation approach. Having individuals in the active role of facilitators is proposed as an effective way of delivering a context-sensitive, tailored approach to implementing CQI. This paper presents an overview of the facilitator role in implementing CQI. Drawing on empirical evidence from the use of facilitator roles in healthcare, the type of skills and knowledge required will be considered, along with the type of facilitation strategies that can be employed in the implementation process. Evidence from both case studies and systematic reviews of facilitation will be reviewed and key lessons for developing and studying the role in the future identified.

Establishment of a regional Danish database for patients with a stoma

AIM

To present the Danish Stoma Database Capital Region with clinical variables related to stoma creation including colostomy, ileostomy and urostomy.

METHOD

The stomatherapists in the Capital Region of Denmark developed a database covering patient identifiers, interventions, conditions, short-term outcome, long-term outcome and known major confounders. The completeness of data was validated against the Danish National Patient Register.

RESULTS

In 2013, five hospitals included data from 1123 patients who were registered during the year. The types of stomas formed from 2007 to 2013 showed a variation reflecting the subspecialization and surgical techniques in the centres. Between 92 and 94% of patients agreed to participate in the standard programme aimed at handling of the stoma and more than 88% of patients having planned surgery had the stoma site marked pre-operatively.

CONCLUSION

The database is fully operational with high data completeness and with data about patients with a stoma from before surgery up to 12 months after surgery. The database provides a solid basis for professional learning, clinical research and benchmarking.

Audit: how to do it in practice

Junior doctors can find the process of doing an audit helpful in gaining an understanding of the healthcare process—here’s how to do one

Gesundheitsökonomische Evaluation in der Pflege

By developing evidence-based, national Expert Standards, agreed-upon by an association of nursing professionals, the German Care Science participates in the international discussion. Up to now, five National Expert Standards on relevant care-related topics have been developed and have been widely implemented in Care Practice. However, sufficient evaluations of these Expert Standards are still required, especially from an economic perspective. The following paper addresses this topic by performing a cost-benefit analysis for the National Expert Standard Prophylaxis of Pressure Ulcers. The authors demonstrate which costs are caused by the implementation of this National Expert Standard for a residential care agency providing services. The benefit of the implementation of the Expert Standard is then being compared to its cost for a period of three years. The evaluation concludes that, in consideration of opportunity costs, the introduction of the National Expert Standard Prophylaxis of Pressure Ulcers appears economically viable for the residential care agency only if the rate of pressure ulcers in the reference agency can be lowered at least by 26.48%. In this case, when exclusively considering direct benefits and direct costs, a positive impact of the implementation will be achieved.

Sustainability in changing clinical practice promotes evidence-based nursing care

AIM

To examine the relationship between sustained work with quality improvement (QI) and factors related to research utilization in a group of nurses.

DESIGN

The study was designed as a comparative survey that included 220 nurses from various health care organizations in Sweden. These nurses had participated in uniformly designed 4-day basic training courses to manage a method for QI.

METHODS

A validated questionnaire covering different aspects of research utilization was employed. The response rate was 70% (154 of 220). Nurses in managerial positions at the departmental level were excluded. Therefore, the final sample consisted of 119 respondents. Four years after the training courses, 39% were still involved in audit-related activities, while 61% reported that they had discontinued the QI work (missing = 1).

RESULTS

Most nurses (80-90%) had a positive attitude to research. Those who had continued the QI work over a 4-year period reported more activity in searching research literature compared with those who had discontinued the QI work (P = 0.005). The QI-sustainable nurses also reported more frequent participation in research-related activities, particularly in implementing specific research findings in practice (P = 0.001). Some contextual differences were reported: the QI-sustainable nurses were more likely to obtain support from their chief executive (P = 0.001), consultation from a skilled researcher (P = 0.005) and statistical support (P = 0.001). Within the broader health care organization, the existence of a research committee and a research and development strategy, as well as access to research assistant staff, had a tendency to be more common for nurses who had continued the QI work.

CONCLUSION

Sustainability in QI work was significantly related to supportive leadership, facilitative human resources, increased activity in seeking new research and enhanced implementation of research findings in clinical practice. It appears that these factors constitute a necessary prerequisite for professional development and the establishment of evidence-based practice.

Progress of unit based quality improvement: an evaluation of a support strategy

OBJECTIVE

To evaluate a strategy for supporting nurses to work with quality improvement (QI).

DESIGN

Post-intervention evaluation. Study participants and intervention: 240 nurses participated in a uniformly designed 4 day basic training course in applying a model for QI. Of these, 156 nurses from over 50 healthcare institutions constituted the generic education (GE) group while 84 nurses from 42 neonatal units took part in a project to develop national guidelines, constituting the targeted intervention (TI) group.

METHOD

Postal questionnaire 4 years after the training courses.

RESULTS

The response rate was 80% in the TI group and 64% in the GE group. Nurses in the TI group had a significantly higher rate in completing all phases of the QI cycle (p=0.0002). With no differences between the groups, 39% of all nurses were still involved in QI work 4 years after the training courses. Three factors were significantly related to nurses continuing their involvement in QI projects: remaining employed on the same unit (OR 11.3), taking courses in nursing science (OR 4.1), and maintenance of the same QI model (OR 3.1). Reported motives for remaining active in QI work were the enhancement of knowledge, influence over clinical practice, and development as a nurse. Reasons for discontinuation were organisational restructuring, a lack of facilitation and knowledge, and change of workplace.

CONCLUSIONS

Participation in a national guideline project, including a common focus for improvement, facilitation and opportunities for networking, seems to have enhanced the ability to carry out the process of QI, but not to sustain the QI work over a longer period.

Nurses' participation in audit: a regional study

OBJECTIVES

To find out to what extent nurses were perceived to be participating in audit, to identify factors thought to impede their involvement, and to assess progress towards multidisciplinary audit.

RESEARCH DESIGN

Qualitative.

METHODS

Focus groups and interviews.

PARTICIPANTS

Chairs of audit groups and audit support staff in hospital, community and primary health care and audit leads in health authorities in the North West Region.

RESULTS

In total 99 audit leads/support staff in the region participated representing 89% of the primary health care audit groups, 80% of acute hospitals, 73% of community health services, and 59% of purchasers. Many audit groups remain medically dominated despite recent changes to their structure and organisation. The quality of interprofessional relations, the leadership style of the audit chair, and nurses' level of seniority, audit knowledge, and experience influenced whether groups reflected a multidisciplinary, rather than a doctor centred approach. Nurses were perceived to be enthusiastic supporters of audit, although their active participation in the process was considered substantially less than for doctors in acute and community health services. Practice nurses were increasingly being seen as the local audit enthusiasts in primary health care. Reported obstacles to nurses' participation in audit included hierarchical nurse and doctor relationships, lack of commitment from senior doctors and managers, poor organisational links between departments of quality and audit, work load pressures and lack of protected time, availability of practical support, and lack of knowledge and skills. Progress towards multidisciplinary audit was highly variable. The undisciplinary approach to audit was still common, particularly in acute services. Multidisciplinary audit was more successfully established in areas already predisposed towards teamworking or where nurses had high involvement in decision making. Audit support staff were viewed as having a key role in helping teams to adopt a collaborative approach to audit.

CONCLUSION

Although nurses were undertaking audit, and some were leading developments in their settings, a range of structural and organisational, interprofessional and intraprofessional factors was still impeding progress. If the ultimate goal of audit is to improve patient care, the obstacles that make it difficult for nurses to contribute actively to the process must be acknowledged and considered.