How Well Is Quality Improvement Described in the Perioperative Care Literature? A Systematic Review

Quality and reporting of large-scale improvement programmes: a review of maternity initiatives in the English NHS, 2010-2023

BACKGROUND

Large-scale improvement programmes are a frequent response to quality and safety problems in health systems globally, but have mixed impact. The extent to which they meet criteria for programme quality, particularly in relation to transparency of reporting and evaluation, is unclear.

AIM

To identify large-scale improvement programmes focused on intrapartum care implemented in English National Health Service maternity services in the period 2010-2023, and to conduct a structured quality assessment.

METHODS

We drew on the Preferred Reporting Items for Systematic Reviews and Meta-Analyses extension for Scoping Reviews guidance to inform the design and reporting of our study. We identified relevant programmes using multiple search strategies of grey literature, research databases and other sources. Programmes that met a prespecified definition of improvement programme, that focused on intrapartum care and that had a retrievable evaluation report were subject to structured assessment using selected features of programme quality.

RESULTS

We identified 1434 records via databases and other sources. 14 major initiatives in English maternity services could not be quality assessed due to lack of a retrievable evaluation report. Quality assessment of the 15 improvement programmes meeting our criteria for assessment found highly variable quality and reporting. Programme specification was variable and mostly low quality. Only eight reported the evidence base for their interventions. Description of implementation support was poor and none reported customisation for challenged services. None reported reduction of inequalities as an explicit goal. Only seven made use of explicit patient and public involvement practices, and only six explicitly used published theories/models/frameworks to guide implementation. Programmes varied in their reporting of the planning, scope and design of evaluation, with weak designs evident.

CONCLUSIONS

Poor transparency of reporting and weak or absent evaluation undermine large-scale improvement programmes by limiting learning and accountability. This review indicates important targets for improving quality in large-scale programmes.

A rapid realist review of clinical neuropsychology rehabilitation programmes to improve psychological wellbeing and quality of life for people with acquired brain injuries

Approximately 20% of acquired brain injury (ABI) survivors experience reduced psychological wellbeing (PWB). Neuropsychological rehabilitation (NPR) is one approach supporting people with ABI to participate meaningfully in activities despite challenges. Although literature supports NPR effectiveness, little is known about change mechanisms. This systematic realist review identifies what NPR programmes have been designed, delivered, and evaluated for people with ABI to improve PWB and/or quality of life (QOL), as well as providing a context-relevant understanding of what NPR includes and how NPR might lead to positive outcomes. A rapid realist review was conducted in three phases: (1) structured retrieval and evidence extraction; (2) stakeholder consultation; (3) analysis and synthesis. Searches were completed, and findings from 35 publications and one stakeholder consultation were synthesized into a refined logic model. Six context-mechanism-outcome chains (CMOCs) were identified. Participants' relationships to internal experiences, and feelings of self-worth, mastery, and connection appeared to be mechanisms that led to improved PWB and QOL. Adaptation and individualized programmes were also key mechanisms to explain successful NPR. Embedding CMOCs into NPR could improve PWB and/or QOL for people with ABI. The logic model will inform ongoing development of a new online, group-based, NPR programme.

Characteristics of transformational leadership development programs: A scoping review

The effectiveness of transformational leadership (TFL) on various outcomes is well known. Accordingly, researchers have developed training programs to enhance TFL behaviours of leaders. Yet, no reviews summarizing the characteristics of TFL training programs exist. The purpose of this review was to examine the characteristics, reporting, and application of TFL-informed programs. A search of six databases yielded 4032 articles, 31 of which met the inclusion criteria. Program characteristics were analyzed using the Template for Intervention Description and Replication (TIDieR) checklist tool, while outcomes were analyzed according to the Kirkpatrick model of evaluation. The most common context for TFL program implementation was healthcare (n = 9). Programs were tailored and often delivered using group workshops and individual feedback. Studies reported variation in the dose of programs, rarely evaluated outcomes multiple times post-baseline, and typically employed Level 3a (subjective ratings of behaviour) evaluation measures. Reporting on program location, modifications, and fidelity was poor. Varying conceptualizations of TFL in different contexts lead to disparities in programs and protocols. Evaluation specialists should consider using tools like the TIDiER checklist to ensure that program characteristics are reported appropriately. Program planners should develop common approaches for planning and evaluating TFL programs to improve transparency and replicability of programs.

Publishing quality improvement studies: learning to share and sharing to learn

This editorial welcomes the decision of BJA Open to publish quality improvement (QI) studies. It summarises the current problems with conducting, evaluating, and publishing QI studies. It highlights existing guidance for prospective authors to follow regarding the reporting of QI interventions, their context(s), underlying theories, and evaluation. In so doing, we hope to encourage the publication of more QI studies of sufficient quality to facilitate learning or replication elsewhere.

Methodologic attributes of quality improvement studies in neonatology: a systematic survey

Introduction

Quality improvement (QI) is a growing field of inquiry in healthcare, including neonatology. However, there is limited information on the study setting, and the methodologic approaches used to develop, implement and evaluate QI interventions in neonatology studies. In this study, we describe these intervention characteristics and approaches.

Methods

Articles were taken from a previous publication. There, we searched MEDLINE for publications of QI studies from 2016 to 16 April 2020. We retrieved all relevant full-text publications and sampled 100 of these articles for data abstraction, stratified by the year of publication. For each QI study, we described several methodological characteristics that included: the clinical topic of QI, setting, whether the study was multicentre, stakeholder engagement, root cause analysis and related problem identification methods, implementation techniques for QI interventions, types of outcomes and statistical analysis methods used.

Results

We assessed 100 studies; most were conducted in the USA (56%). Academic settings and multicentre settings comprised 44% and 24% of studies, respectively. Most studies reported stakeholder engagement (81%), but infrequently reported engagement with leadership (32%) and caregivers (10%). Frequently used techniques for implementing interventions include provider education (82%), formal QI methods (42%) and audit, feedback and benchmarking (40%). Both patient-important clinical outcomes (78%) and process outcomes (89%) were frequently reported. P values were frequently reported (80%), but other statistical techniques were infrequently used.

Conclusion

QI studies in neonatology use diverse multicomponent interventions. Reporting of these methodologic details can be useful in designing, implementing and evaluating QI studies in clinical practice.

Expect the Unexpected: A Qualitative Study of the Ripple Effects of Children's Mental Health Services Implementation Efforts

Background

Strategies to implement evidence-based interventions (EBIs) in children's mental health services have complex direct and indirect causal impacts on multiple outcomes. Ripple effects are outcomes caused by EBI implementation efforts that are unplanned, unanticipated, and/or more salient to stakeholders other than researchers and implementers. The purpose of the current paper is to provide a compilation of possible ripple effects associated with EBI implementation strategies in children's mental health services, to be used for implementation planning, research, and quality improvement.

Methods

Participants were identified via expert nomination and snowball sampling. Online surveys were completed by 81 participants, each representing one of five roles: providers of mental health services to children or youth, researchers, policy makers, caregivers, and youth. A partially directed conventional content analysis with consensus decision making was used to code ripple effects.

Results

Four hundred and four unique responses were coded into 66 ripple effects and 14 categories. Categories include general knowledge, skills, attitudes, and confidence about using EBIs; general job-related ripple effects; EBI treatment adherence, fidelity, and alignment; gaming the system; equity and stigma; shifting roles, role clarity, and task shifting; economic costs and benefits; EBI treatment availability, access, participation, attendance, barriers, and facilitators; clinical process and treatment quality; client engagement, therapeutic alliance, and client satisfaction; clinical organization structure, relationships in the organization, process, and functioning; youth client and caregiver outcomes; and use of EBI strategies and insights in one's own life.

Conclusions

This research advances the field by providing children's mental health implementers, researchers, funders, policy makers, and consumers with a menu of potential ripple effects. It can be a practical tool to ensure compliance with guidance from Quality Improvement/Quality Assurance, Complexity Science, and Diffusion of Innovation Theory. Future phases will match potential ripple effects with salient children's mental health implementation strategies for each participant role.

Are quality improvement plans perceived to improve the quality of primary care in Ontario? Qualitative study

OBJECTIVE

To explore primary care administrators' perceptions of provincially mandated quality improvement plans, and barriers to and facilitators of using quality improvement plans as tools for improving the quality of primary care.

DESIGN

Qualitative descriptive study using semistructured interviews.

SETTING

Ontario.

PARTICIPANTS

Eleven primary care administrators (ie, executive directors, director of clinical services, office administrators) at 7 family health teams and 4 community health centres.

METHODS

All interviews were audiotaped and transcribed verbatim. Data were analyzed deductively to generate a framework based on a conceptual model of structural, organizational, individual, and innovation-related factors that influence the success of improvement initiatives and, inductively, to generate additional themes.

MAIN FINDINGS

Provincially mandated quality improvement plans seem to have raised awareness of and provided an overall focus on quality improvement, and have contributed to primary care organizations implementing initiatives to address quality gaps. Four factors that have contributed to the success of quality improvement plans relate to attributes of the quality improvement plans (adaptability and compatibility) and contextual factors (leadership and organizational culture). However, participants expressed that the use of quality improvement plans have not yet led to substantial improvements in the quality of primary care in Ontario, which may be owing to several challenges: poor data quality, lack of staff and physician engagement and buy-in, and lack of resources to support measurement and quality improvement.

CONCLUSION

Awareness of and focused attention on the need for high-quality patient care may have increased, but participants expressed that substantial improvements in quality care have yet to be achieved in Ontario. The lack of perceived improvements is likely the result of multifaceted and complex challenges primary care organizations face when trying to improve patient care. To effect positive change, organization- and health system-level efforts are needed to improve measurement capabilities, improve staff and physician engagement, and increase capacity for quality improvement among organizations.

A hospital-based mixed-methods observational study to evaluate a hip and knee replacement quality improvement project

Purpose

Health services continue to face economic and capacity challenges. Quality improvement (QI) methods that can improve clinical care processes are therefore needed. However, the successful use of current QI methods within hospital settings remains a challenge. There is considerable scope for improvement of elective clinical pathways, such as hip and knee replacement, and so the use and study of QI methods in such settings is warranted.

Design/methodology/approach

A model to manage variability was adapted for use as a QI method and deployed to improve a hip and knee replacement surgical pathway. A prospective observational study, with a mixed-methods sequential explanatory design (quantitative emphasised) that consisted of two distinct phases, was used to assess its effectiveness.

Findings

Following the use of the novel QI method and the subsequent changes to care processes, the length of hospital stay was reduced by 18%. However, the interventions to improve care process highlighted by the QI method were not fully implemented. The qualitative data revealed that staff thought the new QI method (the model to manage variability) was simple, effective, offered advantages over other QI methods and had highlighted the correct changes to make. However, they felt that contextual factors around leadership, staffing and organisational issues had prevented changes being implemented and a greater improvement being made.

Originality/value

The quality of QI reporting in surgery has previously been highlighted as poor and lacking in prospective and comprehensively reported mixed-methods evaluations. This study therefore not only describes and presents the results of using a novel QI method but also provides new insights in regard to important contextual factors that may influence the success of QI methods and efforts.

A systematic review of the quality of reporting of interventions in the surgical treatment of Crohn's anal fistula: an assessment using the TIDiER and Blencowe frameworks

BACKGROUND

Crohn's anal fistula is a challenging condition, and may require multiple surgical procedures. To replicate successful procedures, these must be adequately reported in the literature. The aim of this study was to review the quality of reporting of components of surgical interventions for Crohn's anal fistula.

METHODS

A systematic review was conducted. It was registered with PROSPERO (CRD:42019135157). The Medline and EMBASE databases were searched for studies reporting interventions intended to close fistula in patients with Crohn's disease, published between 1999 and August 2019. Abstracts and full texts were screened for inclusion by two reviewers. Dual extraction of data was performed to compare reporting to the TIDiER and Blencowe frameworks for reporting of interventions.

RESULTS

Initial searches identified 207 unique studies; 38 full texts were screened for inclusion and 33 were included. The most common study design was retrospective cohort (17/33), and the most frequently reported interventions were anal fistula plug (n = 8) and fibrin glue (n = 6). No studies showed coverage of all domains of TIDieR. Reporting was poor among domains related to who provided an intervention, where it was provided, and how it was tailored. Reporting of domains in the Blencowe framework was poor; the majority of studies did not report the component steps of procedures or efforts to standardise them.

CONCLUSIONS

This study demonstrates that reporting on technical aspects of interventions for Crohn's anal fistula is poor. Surgeons should aim to improve reporting to allow accurate reproduction of techniques both in clinical practice and in clinical trials.

Preventing harm: A balance measure for improving the detection of fetal growth restriction

BACKGROUND

Increasing the detection of fetal growth restriction (FGR), while reducing stillbirth, also leads to unnecessary early intervention, and associated morbidity, for normally grown babies who are incorrectly suspected of FGR.

AIMS

We sought to design a balance measure that addresses the specificity of FGR detection.

METHODS

A retrospective cohort study on all singleton births ≥32 weeks gestation in 2016 and 2017 in Victoria. We compared two balance measures for the detection of FGR, defined as the proportion of all babies iatrogenically delivered before 39 weeks gestation for suspected FGR that had a birthweight ≥10th centile (balance measure 1) or ≥25th centile (balance measure 2). Hospital level performance on each balance measure was derived and compared to an existing performance measure for severe FGR detection in Victoria.

RESULTS

Of the 38 hospitals analysed, 12 (32%) had a favourable performance on an existing indicator of FGR detection, seven (18%) hospitals had a favourable performance on balance measure 1, and 15 (39%) had a favourable performance on balance measure 2. There was a moderate correlation between hospital performance on the existing indicator and on balance measure 1 (r = 0.447, P = 0.005) but not balance measure 2 (r = -0.063, P = 0.71). There was no difference in perinatal mortality between high performing hospitals and low performing hospitals.

CONCLUSION

Introducing a balance measure into routine reporting may bring greater awareness to the unintended harm associated with increased detection of FGR.

The current status of daycase hip and knee arthroplasty within the English National Health Service: a retrospective analysis of hospital episode statistics data

INTRODUCTION

A high volume of total hip, total knee and unicompartmental knee arthroplasty procedures, an ageing population and continuing financial pressures within the English NHS mean that strategies to reduce length of hospital stay are attractive. Enhanced recovery after surgery protocols are one such intervention, and have advanced so that daycase arthroplasty is now possible. This study examines the current rate of daycase arthroplasty within the English NHS.

MATERIALS AND METHODS

Hospital episode statistics data from all English NHS providers of arthroplasty procedures were analysed. Activity, daycase rate, length of stay and readmission rates were recorded. All total hip, knee and unicompartmental knee operations undertaken within the time period 1 July 2018 to 30 June 2019 were examined.

RESULTS

Length of stay was analysed for 162,966 patients; 74,665 (46%) were total hip arthroplasty, 79,252 (49%) were total knee arthroplasty and 9,049 were unicompartmental knee arthroplasty (5%). Mean length of stay was 4.08 days for total hip, 4.11 days for total knee and 2.64 days for unicompartmental knee arthroplasties. Daycase rate for total hip arthroplasty was 0.55%, 0.52% for total knee arthroplasty and 5.44% for unicompartmental knee arthroplasty. The percentage of patients staying in hospital for longer than four days (a length of stay of five days or more) was 18.61% of total hip arthroplasty spells, 20.54% of total knee arthroplasty spells and 5.48% of unicompartmental knee arthroplasty spells.

DISCUSSION

This large observational study of unselected patients undergoing hip and knee arthroplasty demonstrates that the national daycase rate for arthroplasty across providers in the NHS is low. Mean length of stay remains higher than selected international comparators.

The Quality Improvement Challenge-How Nurses and Allied Health Professionals Can Solve the Knowing-Doing Gap in Enhanced Recovery after Surgery (ERAS)

The English National Health Service (NHS), and all health services around the world, will continue to face economic and capacity challenges. Quality improvement (QI) interventions, such as Enhanced Recovery after Surgery (ERAS), that are proven to improve patient care and deliver operational benefits are therefore needed. However, widespread implementation remains a challenge. Implementation of ERAS within the NHS over the last 10 years is reviewed, with a focus on total hip arthroplasty (THA) and total knee arthroplasty (TKA). Difficulties with implementation are highlighted, and a recommendation for the future is presented. This perspective is novel in the ERAS literature, and centres around increasing the understanding of perioperative care teams on the need for utilising a recognised QI method (e.g., plan–do–study–act cycles, Lean, and Six Sigma) to implement ERAS protocols (which are a QI intervention) successfully. The importance of differentiating between a QI method and a QI intervention has value across all other ERAS surgical procedures.

Do pharmacy intervention reports adequately describe their interventions? A template for intervention description and replication analysis of reports included in a systematic review

INTRODUCTION

Scientific progress and translation of evidence into practice is impeded by poorly described interventions. The Template for Intervention Description and Replication (TIDieR) was developed to specify the minimal intervention elements that should be reported.

OBJECTIVES

(1) To assess the extent to which outpatient pharmacy interventions were adequately reported. (2) To examine the dimension(s) across which reporting quality varies. (3) To examine trial characteristics that predict better reporting.

METHODS

The sample comprised 86 randomised controlled trials identified in a Cochrane review of the effectiveness of pharmacist interventions on patient health outcomes. Duplicate, independent application of a modified 15-item TIDieR checklist was undertaken to assess the intervention reporting. The reporting/non-reporting of TIDieR items was analysed with principal component analysis to evaluate the dimensionality of reporting quality and regression analyses to assess predictors of reporting quality RESULTS: In total, 422 (40%) TIDieR items were fully reported, 395 (38%) were partially reported and 231 (22%) were not reported. A further 242 items were deemed not applicable to the specific trials. Reporting quality loaded on one component which accounted for 26% of the variance in TIDieR scores. More recent trials reported a slightly greater number of TIDieR items (0.07 (95% CI 0.02 to 0.13) additional TIDieR items per year of publication). Trials reported an 0.09 (95% CI 0.04 to 0.14) additional TIDieR items per unit increase in impact factor (IF) of the journal in which the main report was published.

CONCLUSIONS

Most trials lacked adequate intervention reporting. This diminished the applied and scientific value of their research. The standard of intervention reporting is, however, gradually increasing and appears somewhat better in journals with higher IFs. The use of the TIDieR checklist to improve reporting could enhance the utility and replicability of trials, and reduce research waste.

How to improve healthcare improvement-an essay by Mary Dixon-Woods

As improvement practice and research begin to come of age, Mary Dixon-Woods considers the key areas that need attention if we are to reap their benefits

Publication Bias Among Conference Abstracts Reporting on Pediatric Quality Improvement Projects

This study evaluated progress to publication of pediatric quality improvement (QI) projects initially presented as national conference abstracts, according to project findings and other characteristics. QI abstracts were identified among presentations at the 2010-2015 American Academy of Pediatrics National Conference & Exhibition, and publications were tracked through June 2018. Positive findings (improvement on at least 1 quantitative project outcome), interventions, and analyses were correlated with journal publication. Of 142 abstracts, 128 (90%) reported positive findings. Forty-nine positive abstracts and 3 abstracts reporting negative results resulted in publication (38% vs 21%, respectively; P = .256). Median time to publication was 1.2 years for projects with positive findings, compared to >3 years for abstracts with negative findings ( P = .029). Ninety percent of abstracts reported positive findings, and these abstracts progressed to publication more quickly. Overcoming publication bias for pediatric QI projects may enhance selection of promising interventions as new projects are designed.

A national quality improvement programme to improve survival after emergency abdominal surgery: the EPOCH stepped-wedge cluster RCT

Background

Emergency abdominal surgery is associated with poor patient outcomes. We studied the effectiveness of a national quality improvement (QI) programme to implement a care pathway to improve survival for these patients.

Objectives

The objectives were to assess whether or not the QI programme improves 90-day survival after emergency abdominal surgery; to assess effects on 180-day survival, hospital stay and hospital readmission; and to better understand these findings through an integrated process evaluation, ethnographic study and cost-effectiveness analysis.

Design

This was a stepped-wedge cluster randomised trial. Hospitals were organised into 15 geographical clusters, and commenced the QI programme in random order over 85 weeks. Analyses were performed on an intention-to-treat basis. The primary outcome was analysed using a mixed-effects parametric survival model, adjusting for time-related effects. Ethnographic and economics data were collected in six hospitals. The process evaluation included all hospitals.

Setting

The trial was set in acute surgical services of 93 NHS hospitals.

Participants

Patients aged ≥ 40 years who were undergoing emergency abdominal surgery were eligible.

Intervention

The intervention was a QI programme to implement an evidence-based care pathway.

Main outcome measures

The primary outcome measure was mortality within 90 days of surgery. Secondary outcomes were mortality within 180 days, length of hospital stay and hospital readmission within 180 days. The main economic measure was the quality-adjusted life-years.

Data sources

Data were obtained from the National Emergency Laparotomy Audit database; qualitative interviews and ethnographic observations; quality-of-life and NHS resource use data were collected via questionnaires.

Results

Of 15,873 eligible patients from 93 NHS hospitals, primary outcome data were analysed for 8482 participants in the usual care group and 7374 in the QI group. The primary outcome occurred in 1393 participants in the usual care group (16%), compared with 1210 patients in the QI group (16%) [QI vs. usual care hazard ratio (HR) 1.11, 95% confidence interval (CI) 0.96 to 1.28]. No differences were found in mortality at 180 days or hospital readmission; there was a small increase in hospital stay in the QI group (HR for discharge 0.90, 95% CI 0.83 to 0.97). There were only modest improvements in care processes following QI implementation. The ethnographic study revealed good QI engagement, but limited time and resources to implement change, affecting which processes teams addressed, the rate of change and eventual success. In some sites, there were challenges around prioritising the intervention in busy environments and in obtaining senior engagement. The intervention is unlikely to be cost-effective at standard cost-effectiveness thresholds, but may be cost-effective over the lifetime horizon.

Limitations

Substantial delays were encountered in securing data access to national registries. Fewer patients than expected underwent surgery and the mortality rate was lower than anticipated.

Conclusions

There was no survival benefit from a QI programme to implement a care pathway for patients undergoing emergency abdominal surgery. The modest impact of the intervention on process measures, despite good clinician engagement, may have been limited by the time and resources needed to improve patient care.

Future work

Future QI programmes must balance intervention complexity with the practical realities of NHS services to ensure that such programmes can be delivered with the resources available.

Trial registration

Current Controlled Trials ISRCTN80682973 and The Lancet protocol 13PRT/7655.

Funding

This project was funded by the National Institute for Health Research (NIHR) Health Services and Delivery Research programme and will be published in full in Health Services and Delivery Research; Vol. 7, No. 32. See the NIHR Journals Library website for further project information.

Why is reporting quality improvement so hard? A qualitative study in perioperative care

OBJECTIVES

Quality improvement (QI) may help to avert or mitigate the risks of suboptimal care, but it is often poorly reported in the healthcare literature. We aimed to identify the influences on reporting QI in the area of perioperative care, with a view to informing improvements in reporting QI across healthcare.

DESIGN

Qualitative interview study.

SETTING

Healthcare and academic organisations in Australia, Europe and North America.

PARTICIPANTS

Stakeholders involved in or influencing the publication, writing or consumption of reports of QI studies in perioperative care.

RESULTS

Forty-two participants from six countries took part in the study. Participants included 15 authors (those who write QI reports), 12 consumers of QI reports (practitioners who apply QI research in practice), 11 journal editors and 4 authors of reporting guidelines. Participants identified three principal challenges in achieving high-quality QI reporting. First, the broad scope of QI reporting-ranging from small local projects to multisite research across different disciplines-causes uncertainty about where QI work should be published. Second, context is fundamental to the success of a QI intervention but is difficult to report in ways that support replication and development. Third, reporting is adversely affected by both proximal influences (such as lack of time to write up QI) and more distal, structural influences (such as norms about the format and content of biomedical research reporting), leading to incomplete reporting of QI findings.

CONCLUSIONS

Divergent terminology and understandings of QI, along with existing reporting norms and the challenges of capturing context adequately yet succinctly, make for challenges in reporting QI. We offer suggestions for improvement.

Identifying and resolving the frustrations of reviewing the improvement literature: The experiences of two improvement researchers

Background and aims

Summarising quality improvement (QI) research through systematic literature review has great potential to improve patient care. However, heterogeneous terminology, poor definition of QI concepts and overlap with other scientific fields can make it hard to identify and extract data from relevant literature. This report examines the compromises and pragmatic decisions that undertaking literature review in the field of QI requires and the authors propose recommendations for literature review authors in similar fields.

Methods

Two authors (EJ and JF) provide a reflective account of their experiences of conducting a systematic literature review in the field of QI. They draw on wider literature to justify the decisions they made and propose recommendations to improve the literature review process. A third collaborator, (WC) co-created the paper challenging author’s EJ and JF views and perceptions of the problems and solutions of conducting a review of literature in QI.

Results

Two main challenges were identified when conducting a review in QI. These were defining QI and selecting QI studies. Strategies to overcome these problems include: select a multi-disciplinary authorship team; review the literature to identify published QI search strategies, QI definitions and QI taxonomies; Contact experts in related fields to clarify whether a paper meets inclusion criteria; keep a reflective account of decision making; submit the protocol to a peer reviewed journal for publication.

Conclusions

The QI community should work together as a whole to create a scientific field with a shared vision of QI to enable accurate identification of QI literature. Our recommendations could be helpful for systematic reviewers wishing to evaluate complex interventions in both QI and related fields.

A balanced approach to identifying, prioritising and evaluating all potential consequences of quality improvement: modified Delphi study

OBJECTIVES

Healthcare is a complex system, so quality improvement will commonly lead to unintended consequences which are rarely evaluated. In previous qualitative work, we proposed a framework for considering the range of these potential consequences, in terms of their desirability and the extent to which they were predictable or expected during planning. This paper elaborates on the previous findings, using consensus methods to examine what consequences should be identified, why and how to prioritise, evaluate and interpret all identified consequences, and what stakeholders should be involved throughout this process.

DESIGN

Two-round modified Delphi consensus study.

SETTING AND PARTICIPANTS

Both rounds were completed by 60 panellists from an academic, clinical or management background and experience in designing, implementing or evaluating quality improvement programmes.

RESULTS

Panellists agreed that trade-offs (expected undesirable consequences) and unpleasant surprises (unexpected undesirable consequences) should be actively considered. Measurement of harmful consequences for patients, and those with high workload or financial impact was prioritised, and their evaluation could also involve the use of qualitative methods. Clinical teams were agreed as important to involve at all stages, from identifying potential consequences, prioritising which of those to systematically evaluate, undertaking appropriate evaluation and interpreting the findings. Patients were necessary in identifying consequences, managers in identifying and prioritising, and improvement advisors in interpreting the data.

CONCLUSION

There was consensus that a balanced approach to considering all the consequences of improvement can be achieved by carefully considering predictable trade-offs from the outset and deliberately pausing after implementation to identify any unexpected surprises and make an informed decision as to whether quantitative or qualitative evaluation is needed and feasible. Stakeholders' roles in in the process of identifying, prioritising, evaluating and interpreting potential consequences should be explicitly addressed within planning and revisited during and after implementation.

Monitoring of Sugammadex Dosing at a Large Tertiary Care Pediatric Hospital

Introduction:

Anesthesiologists use sugammadex to reverse neuromuscular blockade (NMB) produced by rocuronium and vecuronium. Its mechanism involves encapsulation of the neuromuscular blocking agent. Sugammadex dosing is based on the depth of NMB, assessed by measuring the train-of-four (TOF).

Methods:

We retrospectively reviewed procedures under general anesthesia in patients older than 1 year of age if they included sugammadex reversal of rocuronium-induced NMB. Documentation of TOF monitoring before and after reversal was noted, along with the dose of sugammadex administered. TOF was considered correctly documented if the anesthesia provider recorded the number of twitches before and after NMB reversal, or if they recorded 4 twitches before NMB reversal. We defined appropriate sugammadex dosing if it was within 10% of the recommended dose for the depth of NMB. We repeated this review after staff education and creating a reminder in the electronic health record system.

Results:

We included 100 patients in the preintervention analysis, of whom 30% had correct TOF documentation. Among patients with TOF assessment before sugammadex administration, the dose was appropriate in 34 of 40 cases. In the postintervention analysis, we reviewed 75 cases and found that correct documentation improved to 45% (P = 0.024). Among postintervention cases with TOF documented before sugammadex administration, sugammadex dosing was appropriate in 62 patients.

Conclusion:

Documentation of TOF was low (30%) before intervention and improved to only 45% after the interventions, suggesting that additional interventions are needed. Even before the intervention, with or without TOF documentation, the dose of sugammadex was generally consistent with recommendations.

Implementing Frontline Worker-Led Quality Improvement in Nursing Homes: Getting to "How"

BACKGROUND

Despite emerging frameworks for quality improvement (QI) implementation, little is known about how the implementation process works, particularly in nursing home settings. A study was conducted to describe "how"' a complex frontline worker-led QI program was implemented in nursing homes.

METHODS

Six focus groups were conducted in February 2017 with participants of a year-long, multicomponent, unit-level QI intervention in seven nursing homes in the Canadian province of Manitoba. Constant comparative analysis was used to examine perspectives of different groups of QI program participants-35 health care aides, health professionals, and managers.

RESULTS

Five themes important to the implementation process were identified: (1) "supportive elements of the QI program structure," (2) "navigating the workplace," (3) "negotiating relationships," (4) "developing individual skills," and (5) "observable program impact." Data on theme integration suggest that "supportive elements of the QI program structure" (Theme 1), "developing individual skills" (Theme 4), and "observable program impact" (on residents, health care aides, and leaders; Theme 5) operated as part of a reinforcing feedback loop that boosted team members' ability to navigate the workplace, negotiate relationships, and implement the QI program.

CONCLUSION

Health care aide-led QI teams are feasible. However, a leadership paradox exists whereby worker-led QI programs also must incorporate concrete mechanisms to promote strong leadership and sponsor support to teams. The findings also point to the underexplored impact of interpersonal relationships between health care aides and professional staff on QI implementation.

Getting messier with TIDieR: embracing context and complexity in intervention reporting

Background

The Template for Intervention Description and Replication (TIDieR) checklist and guide was developed by an international team of experts to promote full and accurate description of trial interventions. It is now widely used in health research. The aim of this paper is to describe the experience of using TIDieR outside of trials, in a range of applied health research contexts, and make recommendations on its usefulness in such settings.

Main body

We used the TIDieR template for intervention description in six applied health research projects. The six cases comprise a diverse sample in terms of clinical problems, population, settings, stage of intervention development and whether the intervention was led by researchers or the service deliverers. There was also variation in how the TIDieR description was produced in terms of contributors and time point in the project. Researchers involved in the six cases met in two workshops to identify issues and themes arising from their experience of using TIDieR.

We identified four themes which capture the difficulties or complexities of using TIDieR in applied health research: (i) fidelity and adaptation: all aspects of an intervention can change over time; (ii) voice: the importance of clarity on whose voice the TIDieR description represents; (iii) communication beyond the immediate context: the usefulness of TIDieR for wider dissemination and sharing; (iv) the use of TIDieR as a research tool.

Conclusion

We found TIDieR to be a useful tool for applied research outside the context of clinical trials and we suggest four revisions or additions to the original TIDieR which would enable it to better capture these complexities in applied health research:

An additional item, ‘voice’ conveys who was involved in preparing the TIDieR template, such as researchers, service users or service deliverers.

An additional item, ‘stage of implementation’ conveys what stage the intervention has reached, using a continuum of implementation research suggested by the World Health Organisation.

A new column, ‘modification’ reminds authors to describe modifications to any item in the checklist.

An extension of the ‘how well’ item encourages researchers to describe how contextual factors affected intervention delivery.

Electronic supplementary material

The online version of this article (10.1186/s12874-017-0461-y) contains supplementary material, which is available to authorized users.

[Transparency and replicability of nursing intervention studies in long-term care: A selective literature review]

BACKGROUND AND OBJECTIVE

The development and evaluation of interventions in long-term care is time-consuming and expensive due to their complexity. To ensure reproducibility and successful implementation, these interventions must be described and published in a comprehensible and qualitative manner. The aim of this study is to analyze intervention studies from the inpatient long-term care setting with regard to their completeness, reporting quality, transparency and thus reproducibility.

METHOD

The completeness and the reporting quality of the interventions described in the publications were examined in the context of a selective literature review by means of intervention studies from the long-term care setting (n=22). To this end, the Template for Intervention Description and Replication (TIDieR) checklist and the Criteria for Reporting the Development and Evaluation of Complex Interventions in Healthcare 2 (CReDECI2-DE) list were used. Transparency criteria included study registration and access to study protocols.

RESULTS

The TIDieR checklist examination revealed that only three studies contained all the information necessary; the CReDECI2 test provided a complete description for only one study. Frequent shortcomings were observed concerning the information on modifications and titrations for the study participants and the location. Protocols were available for eight studies, 14 studies were registered.

CONCLUSIONS

In terms of science, this means that the reproducibility of scientific findings is limited, which is why they cannot provide secure knowledge. As a result, the practical benefit to be derived from published studies that are accessible to decision-makers is limited as well. As far as publishers are concerned they should pay more attention to the completeness, registration and availability of materials.

Does quality improvement improve quality?

Although quality improvement (QI) is frequently advocated as a way of addressing the problems with healthcare, evidence of its effectiveness has remained very mixed. The reasons for this are varied but the growing literature highlights particular challenges. Fidelity in the application of QI methods is often variable. QI work is often pursued through time-limited, small-scale projects, led by professionals who may lack the expertise, power or resources to instigate the changes required. There is insufficient attention to rigorous evaluation of improvement and to sharing the lessons of successes and failures. Too many QI interventions are seen as 'magic bullets' that will produce improvement in any situation, regardless of context. Too much improvement work is undertaken in isolation at a local level, failing to pool resources and develop collective solutions, and introducing new hazards in the process. This article considers these challenges and proposes four key ways in which QI might itself be improved.