Effects of a quality improvement collaborative on the outcome of care of patients with HIV infection: the EQHIV study

What is quality in long covid care? Lessons from a national quality improvement collaborative and multi-site ethnography

BACKGROUND

Long covid (post covid-19 condition) is a complex condition with diverse manifestations, uncertain prognosis and wide variation in current approaches to management. There have been calls for formal quality standards to reduce a so-called "postcode lottery" of care. The original aim of this study-to examine the nature of quality in long covid care and reduce unwarranted variation in services-evolved to focus on examining the reasons why standardizing care was so challenging in this condition.

METHODS

In 2021-2023, we ran a quality improvement collaborative across 10 UK sites. The dataset reported here was mostly but not entirely qualitative. It included data on the origins and current context of each clinic, interviews with staff and patients, and ethnographic observations at 13 clinics (50 consultations) and 45 multidisciplinary team (MDT) meetings (244 patient cases). Data collection and analysis were informed by relevant lenses from clinical care (e.g. evidence-based guidelines), improvement science (e.g. quality improvement cycles) and philosophy of knowledge.

RESULTS

Participating clinics made progress towards standardizing assessment and management in some topics; some variation remained but this could usually be explained. Clinics had different histories and path dependencies, occupied a different place in their healthcare ecosystem and served a varied caseload including a high proportion of patients with comorbidities. A key mechanism for achieving high-quality long covid care was when local MDTs deliberated on unusual, complex or challenging cases for which evidence-based guidelines provided no easy answers. In such cases, collective learning occurred through idiographic (case-based) reasoning, in which practitioners build lessons from the particular to the general. This contrasts with the nomothetic reasoning implicit in evidence-based guidelines, in which reasoning is assumed to go from the general (e.g. findings of clinical trials) to the particular (management of individual patients).

CONCLUSION

Not all variation in long covid services is unwarranted. Largely because long covid's manifestations are so varied and comorbidities common, generic "evidence-based" standards require much individual adaptation. In this complex condition, quality improvement resources may be productively spent supporting MDTs to optimise their case-based learning through interdisciplinary discussion. Quality assessment of a long covid service should include review of a sample of individual cases to assess how guidelines have been interpreted and personalized to meet patients' unique needs.

STUDY REGISTRATION

NCT05057260, ISRCTN15022307.

Development of evidence-based guidelines for the treatment and management of periprosthetic hip infection

Aims

Periprosthetic hip-joint infection is a multifaceted and highly detrimental outcome for patients and clinicians. The incidence of prosthetic joint infection reported within two years of primary hip arthroplasty ranges from 0.8% to 2.1%. Costs of treatment are over five-times greater in people with periprosthetic hip joint infection than in those with no infection. Currently, there are no national evidence-based guidelines for treatment and management of this condition to guide clinical practice or to inform clinical study design. The aim of this study is to develop guidelines based on evidence from the six-year INFection and ORthopaedic Management (INFORM) research programme.

Methods

We used a consensus process consisting of an evidence review to generate items for the guidelines and online consensus questionnaire and virtual face-to-face consensus meeting to draft the guidelines.

Results

The consensus panel comprised 21 clinical experts in orthopaedics, primary care, rehabilitation, and healthcare commissioning. The final output from the consensus process was a 14-item guideline. The guidelines make recommendations regarding increased vigilance and monitoring of those at increased risk of infection; diagnosis including strategies to ensure the early recognition of prosthetic infection and referral to orthopaedic teams; treatment, including early use of DAIR and revision strategies; and postoperative management including appropriate physical and psychological support and antibiotic strategies.

Conclusion

We believe the implementation of the INFORM guidelines will inform treatment protocols and clinical pathways to improve the treatment and management of periprosthetic hip infection.

Cite this article: Bone Jt Open 2023;4(4):226–233.

Development of the Illinois Surgical Quality Improvement Collaborative (ISQIC): Implementing 21 Components to Catalyze Statewide Improvement in Surgical Care

INTRODUCTION

In 2014, 56 Illinois hospitals came together to form a unique learning collaborative, the Illinois Surgical Quality Improvement Collaborative (ISQIC). Our objectives are to provide an overview of the first three years of ISQIC focused on (1) how the collaborative was formed and funded, (2) the 21 strategies implemented to support quality improvement (QI), (3) collaborative sustainment, and (4) how the collaborative acts as a platform for innovative QI research.

METHODS

ISQIC includes 21 components to facilitate QI that target the hospital, the surgical QI team, and the peri-operative microsystem. The components were developed from available evidence, a detailed needs assessment of the hospitals, reviewing experiences from prior surgical and non-surgical QI Collaboratives, and interviews with QI experts. The components comprise 5 domains: guided implementation (e.g., mentors, coaches, statewide QI projects), education (e.g., process improvement (PI) curriculum), hospital- and surgeon-level comparative performance reports (e.g., process, outcomes, costs), networking (e.g., forums to share QI experiences and best practices), and funding (e.g., for the overall program, pilot grants, and bonus payments for improvement).

RESULTS

Through implementation of the 21 novel ISQIC components, hospitals were equipped to use their data to successfully implement QI initiatives and improve care. Formal (QI/PI) training, mentoring, and coaching were undertaken by the hospitals as they worked to implement solutions. Hospitals received funding for the program and were able to work together on statewide quality initiatives. Lessons learned at one hospital were shared with all participating hospitals through conferences, webinars, and toolkits to facilitate learning from each other with a common goal of making care better and safer for the surgical patient in Illinois. Over the first three years, surgical outcomes improved in Illinois.

DISCUSSION

The first three years of ISQIC improved care for surgical patients across Illinois and allowed hospitals to see the value of participating in a surgical QI learning collaborative without having to make the initial financial investment themselves. Given the strong support and buy-in from the hospitals, ISQIC has continued beyond the initial three years and continues to support QI across Illinois hospitals.

Strengths and Challenges of Implementing a Learning Collaborative in the Ryan White HIV/AIDS Program

Learning collaboratives (LCs) are a popular tool for supporting collaboration and shared learning among health programs. Many variations of LCs have been reported in the literature. However, descriptions of key LC components and implementation lack standardization, making it hard to compare and contrast different LC approaches. To advance the field's understanding of how primary elements of LCs are implemented, we describe the implementation of an LC in the Ryan White HIV/AIDS Program using a recently established taxonomy of four primary elements of LCs-innovation, social systems, communication, and time. Additionally, we explain the strengths and challenges we encountered with regard to each of these elements when implementing this LC. We then offer recommendations to others on how to leverage LC facilitators and mitigate challenges in future projects. This information can guide other programs to replicate beneficial practices and avoid pitfalls in future LC projects.

Context in Implementation Science

Changing health care delivery systems and processes of care to improve health care quality is complex. What is done (intervention) is equally important as how it is done (implementation) and where it is done (context). Furthermore, it has been consistently observed that among groups participating in multisite quality improvement (QI) efforts and implementation studies, significant heterogeneity in the improvements is seen. Our objective is to provide a step-by-step guide to assist both researchers and groups practicing QI on the frontlines in addressing context in planning, implementing, and disseminating their QI and implementation interventions. We discuss in depth a sample of the >60 available dissemination and implementation frameworks that consider context. We then provide an approach to addressing context in QI and implementation initiatives and discuss an application of this approach, using a published study as an example. Finally, we discuss next steps for the field of context and implementation science. Data from networks of health systems working together on QI are needed on both network-wide rates of process and outcome measures. Also needed are segmented/stratified data that inform our understanding of the influence of context on successful implementation in subgroups. Finally, multisite prospective studies are needed to develop an in-depth understanding of how specific context and implementation factors affect the successful spread of proven interventions.

Opening the 'black box' of collaborative improvement: a qualitative evaluation of a pilot intervention to improve quality of malaria surveillance data in public health centres in Uganda

BACKGROUND

Demand for high-quality surveillance data for malaria, and other diseases, is greater than ever before. In Uganda, the primary source of malaria surveillance data is the Health Management Information System (HMIS). However, HMIS data may be incomplete, inaccurate or delayed. Collaborative improvement (CI) is a quality improvement intervention developed in high-income countries, which has been advocated for low-resource settings. In Kayunga, Uganda, a pilot study of CI was conducted in five public health centres, documenting a positive effect on the quality of HMIS and malaria surveillance data. A qualitative evaluation was conducted concurrently to investigate the mechanisms of effect and unintended consequences of the intervention, aiming to inform future implementation of CI.

METHODS

The study intervention targeted health workers, including brief in-service training, plus CI with 'plan-do-study-act' (PDSA) cycles emphasizing self-reflection and group action, periodic learning sessions, and coaching from a CI mentor. Health workers collected data on standard HMIS out-patient registers. The qualitative evaluation (July 2015 to September 2016) included ethnographic observations at each health centre (over 12-14 weeks), in-depth interviews with health workers and stakeholders (n = 20), and focus group discussions with health workers (n = 6).

RESULTS

The results suggest that the intervention did facilitate improvement in data quality, but through unexpected mechanisms. The CI intervention was implemented as planned, but the PDSA cycles were driven largely by the CI mentor, not the health workers. In this context, characterized by a rigid hierarchy within the health system of limited culture of self-reflection and inadequate training and supervision, CI became an effective form of high-quality training with frequent supervisory visits. Health workers appeared motivated to improve data collection habits by their loyalty to the CI mentor and the potential for economic benefits, rather than a desire for self-improvement.

CONCLUSIONS

CI is a promising method of quality improvement and could have a positive impact on malaria surveillance data. However, successful scale-up of CI in similar settings may require deployment of highly skilled mentors. Further research, focusing on the effectiveness of 'real world' mentors using robust study designs, will be required to determine whether CI can be translated effectively and sustainably to low-resource settings.

Effectiveness of in-service training plus the collaborative improvement strategy on the quality of routine malaria surveillance data: results of a pilot study in Kayunga District, Uganda

BACKGROUND

Surveillance data are essential for malaria control, but quality is often poor. The aim of the study was to evaluate the effectiveness of the novel combination of training plus an innovative quality improvement method-collaborative improvement (CI)-on the quality of malaria surveillance data in Uganda.

METHODS

The intervention (training plus CI, or TCI), including brief in-service training and CI, was delivered in 5 health facilities (HFs) in Kayunga District from November 2015 to August 2016. HF teams monitored data quality, conducted plan-do-study-act cycles to test changes, attended periodic learning sessions, and received CI coaching. An independent evaluation was conducted to assess data completeness, accuracy, and timeliness. Using an interrupted time series design without a separate control group, data were abstracted from 156,707 outpatient department (OPD) records, laboratory registers, and aggregated monthly reports (MR) for 4 time periods: baseline-12 months, TCI scale-up-5 months; CI implementation-9 months; post-intervention-4 months. Monthly OPD register completeness was measured as the proportion of patient records with a malaria diagnosis with: (1) all data fields completed, and (2) all clinically-relevant fields completed. Accuracy was the relative difference between: (1) number of monthly malaria patients reported in OPD register versus MR, and (2) proportion of positive malaria tests reported in the laboratory register versus MR. Data were analysed with segmented linear regression modelling.

RESULTS

Data completeness increased substantially following TCI. Compared to baseline, all-field completeness increased by 60.1%-points (95% confidence interval [CI]: 46.9-73.2%) at mid-point, and clinically-relevant completeness increased by 61.6%-points (95% CI: 56.6-66.7%). A relative - 57.4%-point (95% confidence interval: - 105.5, - 9.3%) change, indicating an improvement in accuracy of malaria test positivity reporting, but no effect on data accuracy for monthly malaria patients, were observed. Cost per additional malaria patient, for whom complete clinically-relevant data were recorded in the OPD register, was $3.53 (95% confidence interval: $3.03, $4.15).

CONCLUSIONS

TCI improved malaria surveillance completeness considerably, with limited impact on accuracy. Although these results are promising, the intervention's effectiveness should be evaluated in more HFs, with longer follow-up, ideally in a randomized trial, before recommending CI for wide-scale use.

Effect of collaborative quality improvement on stillbirths, neonatal mortality and newborn care practices in hospitals of Telangana and Andhra Pradesh, India: evidence from a quasi-experimental mixed-methods study

Background

Improving quality of care is a key priority to reduce neonatal mortality and stillbirths. The Safe Care, Saving Lives programme aimed to improve care in newborn care units and labour wards of 60 public and private hospitals in Telangana and Andhra Pradesh, India, using a collaborative quality improvement approach. Our external evaluation of this programme aimed to evaluate programme effects on implementation of maternal and newborn care practices, and impact on stillbirths, 7- and 28-day neonatal mortality rate in labour wards and neonatal care units. We also aimed to evaluate programme implementation and mechanisms of change.

Methods

We used a quasi-experimental plausibility design with a nested process evaluation. We evaluated effects on stillbirths, mortality and secondary outcomes relating to adherence to 20 evidence-based intrapartum and newborn care practices, comparing survey data from 29 hospitals receiving the intervention to 31 hospitals expected to receive the intervention later, using a difference-in-difference analysis. We analysed programme implementation data and conducted 42 semi-structured interviews in four case studies to describe implementation and address four theory-driven questions to explain the quantitative results.

Results

Only 7 of the 29 intervention hospitals were engaged in the intervention for its entire duration. There was no evidence of an effect of the intervention on stillbirths [DiD − 1.3 percentage points, 95% CI − 2.6–0.1], on neonatal mortality at age 7 days [DiD − 1.6, 95% CI − 9–6.2] or 28 days [DiD − 3.0, 95% CI − 12.9—6.9] or on adherence to target evidence-based intrapartum and newborn care practices. The process evaluation identified challenges in engaging leaders; challenges in developing capacity for quality improvement; and challenges in activating mechanisms of change at the unit level, rather than for a few individuals, and in sustaining these through the creation of new social norms.

Conclusion

Despite careful planning and substantial resources, the intervention was not feasible for implementation on a large scale. Greater focus is required on strategies to engage leadership. Quality improvement may need to be accompanied by clinical training. Further research is also needed on quality improvement using a health systems perspective.

Supplementary Information

The online version contains supplementary material available at 10.1186/s13012-020-01058-z.

Evaluation of Changes in Quality Improvement Knowledge Following a Formal Educational Curriculum Within a Statewide Learning Collaborative

OBJECTIVE

Our objectives were to (1) develop a curriculum based upon participants' needs, (2) evaluate baseline QI knowledge of the Illinois Surgical Quality Improvement Collaborative (ISQIC) members, and (3) evaluate the effectiveness of the educational curriculum.

DESIGN

The Surgeon Champion (SC), Surgical Clinical Reviewer (SCR), and QI Designee at each ISQIC hospital completed a QI curriculum containing online modules and in-person trainings. A surgical adaptation of QI-KAT, a validated QI knowledge assessment with multiple-choice and free-response sections, was administered pre- and postcurriculum. Three blinded educators scored each exam using a rubric-based scoring tool (54 total points).

SETTING

The ISQIC is a 52-hospital learning collaborative. Generally, ISQIC participants had little prior formal training or experience with quality improvement.

RESULTS

Among 52 hospitals, 144 pretests and 112 post-tests were collected. Mean scores increased from 66% (35.6 points) to 77% (41.6 points; p < 0.001). Across all hospitals, all participant groups scored higher on the post-test (SCs 15%, SCRs 21%, QI Designees 17%). There was no significant difference in post-test mean scores among different team members: SCs 44 points, SCRs 42 points, QI Designees 44 points, (p = 0.76). When the post-test scores were aggregated at the hospital level, hospitals with new surgical QI programs improved more than hospitals with established programs (new 18%, established 11%, p < 0.05).

CONCLUSIONS

QI knowledge significantly improved after completion of the ISQIC curriculum. These data support the value of formalized curricula to rapidly advance QI knowledge and application skills as a foundation for implementing QI initiatives.

Enacting quality improvement in ten European hospitals: a dualities approach

Background

Hospitals undertake numerous initiatives searching to improve the quality of care they provide, but these efforts are often disappointing. Current models guiding improvement tend to undervalue the tensional nature of hospitals. Applying a dualities approach that is sensitive to tensions inherent to hospitals’ quest for improved quality, this article aims to identify which organizational dualities managers should particularly pay attention to.

Methods

A set of cross-national, multi-level case studies was conducted involving 383 semi-structured interviews and 803 h of non-participant observation of key meetings and shadowing of staff in ten purposively sampled hospitals in five European countries (England, the Netherlands, Portugal, Sweden, and Norway).

Results

Six dualities that describe the quest for improved quality, each embracing a seemingly contradictory feature were identified: plural consensus, distributed connectedness, orchestrated emergence, formalized fluidity, patient coreness, and cautious generativeness.

Conclusions

We advocate for a move from the usual sequential and project-based and systemic thinking about quality improvement to the development of meta-capabilities to balance the simultaneous operation of opposing ideas or concepts. Doing so will help hospital managers to deal with major challenges of change inherent to quality improvement initiatives.

Benefits and Sustainability of a Learning Collaborative for Implementation of Treat-to-Target in Rheumatoid Arthritis: Results of a Cluster-Randomized Controlled Phase II Clinical Trial

OBJECTIVE

We conducted a 2-phase randomized controlled trial of a learning collaborative to facilitate implementation of treat-to-target (T2T) to manage rheumatoid arthritis (RA). We found substantial improvement in implementation of T2T in phase I. Here, we report on a second 9 months (phase II), where we examined the maintenance of response in phase I and predictors of greater improvement in T2T adherence.

METHODS

We recruited patients from 11 rheumatology sites and randomized them to either receive the learning collaborative during phase I or to a wait-list control group that received the learning collaborative intervention during phase II. The outcome was change in T2T implementation score (0-100, where 100 = best) from pre- to postintervention. The T2T implementation score was defined as a percent of components documented in visit notes. Analyses examined the extent to which the phase-I intervention teams sustained improvement in T2T, as well as predictors of T2T improvement.

RESULTS

The analysis included 636 RA patients. At baseline, the mean T2T implementation score was 11% in phase I intervention sites and 13% in phase II sites. After the intervention, T2T implementation score improved to 57% in the phase I intervention sites and to 58% in the phase II sites. Intervention sites from phase I sustained the improvement during the phase II (52%). Predictors of greater T2T improvement included having only rheumatologist providers at the site, academic affiliation of the site, having fewer providers per site, and the rheumatologist provider being a trainee.

CONCLUSION

Improvement in T2T remained relatively stable over a postintervention period.

The role of service readiness and health care facility factors in attrition from Option B+ in Haiti: a joint examination of electronic medical records and service provision assessment survey data

Background

Option B+ is a strategy wherein pregnant or breastfeeding women with HIV are enrolled in lifelong antiretroviral therapy (ART) for prevention of mother-to-child transmission (PMTCT) of HIV. In Haiti, attrition from Option B+ is problematic and variable across health care facilities. This study explores service readiness and other facility factors as predictors of Option B+ attrition in Haiti.

Methods

This analysis used longitudinal data from 2012 to 2014 from the iSanté electronic medical record system and cross-sectional data from Haiti's 2013 Service Provision Assessment. Predictors included Service Availability and Readiness Assessment (SARA) measures for antenatal care (ANC), PMTCT, HIV care services and ART services; general facility characteristics and patient-level factors. Multivariable Cox proportional hazards models modelled the time to first attrition.

Results

Analysis of data from 3147 women at 63 health care facilities showed no significant relationships between SARA measures and attrition. Having integrated ANC/PMTCT care and HIV-related training were significant protective factors. Being a public-sector facility, having a greater number of quality improvement activities and training in ANC were significant risk factors.

Conclusion

Several facility-level factors were associated with Option B+ attrition. Future research is needed to explore unmeasured facility factors, clarify causal relationships, and incorporate community-level factors into the analysis of Option B+ attrition.

The effects of the introduction of a chronic care model-based program on utilization of healthcare resources: the results of the Puglia care program

Background

Ageing is continuously increasing the prevalence of patients with chronic conditions, putting pressure on the sustainability of Healthcare Systems. Chronic Care Models (CCM) have been used to address the needs of frail people in the continuum of care, testifying to an improvement in health outcomes and more efficient access to healthcare services. The impact of CCM deployment has already been experienced in a selected cohort of patients affected by specific chronic illnesses. We have investigated its effects in a heterogeneous frail cohort included in a regional CCM-based program.

Methods

a retrospective population-based cohort study was carried out involving a non-oncological cohort of adult subjects with chronic diseases included in the CCM-oriented program (Puglia Care). Individuals in usual care with comparable demographic and clinical characteristics were selected for matched pair analysis. Study cohorts were defined by using a record linkage analysis of administrative databases and electronic medical records, including data on the adult population in the 6 local area health authorities of Puglia in Italy (approximately 2 million people). The effects of Puglia Care on the utilizations of healthcare resources were evaluated both in a before-after and in a case-control analysis.

Results

There were 1074 subjects included in Puglia Care and 2126 matched controls. In before-after analysis of the Puglia Care cohort, 240 unplanned hospitalizations occurred in the pre-inclusion period, while 239 were registered during follow-up. The incidence of unplanned hospitalization was 10.3 per 100 person/year (95% CI, 9.1–11.7) during follow-up and 12.1 per 100 person/year (95% CI, 10.7–13.8) in the pre-inclusion period (IRR, 0.84; 95% CI, 0.80–0.99). During follow-up a significant reduction in costs related to unplanned hospitalizations (IRR, 0.92; 95% CI, 0.91–0.92) was registered, while costs related to drugs (IRR, 1.14; p < 0.01), out-patient specialist visits (IRR, 1.19; p < 0.01), and planned hospitalization (IRR 1.03; p < 0.01) increased significantly. These modifications can be related to the aging of the population and modifications to healthcare delivery; for this reason, a case-control analysis was performed. The results testify to a significantly lower number (IRR, 0.79; 95% CI, 0.68–0.91), length of hospital stay (IRR, 0.80; 95% CI, 0.76–0.84), and costs related to unplanned hospitalizations (IRR, 0.80; 95% CI, 0.80–0.80) during follow-up in the intervention group. However, there was a higher increase in costs of hospitalizations, drugs and out-patients specialist visits during follow-up in Puglia Care when compared with patients in usual care.

Conclusion

In a population-based cohort, inclusion of chronic patients in a CCM-based program was significantly associated with a lower recourse to unplanned hospital admissions when compared with patients in usual care with comparable clinical and demographic characteristics.

Electronic supplementary material

The online version of this article (10.1186/s12913-018-3075-0) contains supplementary material, which is available to authorized users.

Structural Interventions in HIV Prevention: A Taxonomy and Descriptive Systematic Review

One of the four national HIV prevention goals is to incorporate combinations of effective, evidence-based approaches to prevent HIV infection. In fields of public health, techniques that alter environment and affect choice options are effective. Structural approaches may be effective in preventing HIV infection. Existing frameworks for structural interventions were lacking in breadth and/or depth. We conducted a systematic review and searched CDC's HIV/AIDS Prevention Research Synthesis Project's database for relevant interventions during 1988-2013. We used an iterative process to develop the taxonomy. We identified 213 structural interventions: Access (65%), Policy/Procedure (32%), Mass Media (29%), Physical Structure (27%), Capacity Building (24%), Community Mobilization (9%), and Social Determinants of Health (8%). Forty percent targeted high-risk populations (e.g., people who inject drugs [12%]). This paper describes a comprehensive, well-defined taxonomy of structural interventions with 7 categories and 20 subcategories. The taxonomy accommodated all interventions identified.

Are quality improvement collaboratives effective? A systematic review

Background

Quality improvement collaboratives (QIC) have proliferated internationally, but there is little empirical evidence for their effectiveness.

Method

We searched Medline, Embase, CINAHL, PsycINFO and the Cochrane Library databases from January 1995 to December 2014. Studies were included if they met the criteria for a QIC intervention and the Cochrane Effective Practice and Organisation of Care (EPOC) minimum study design characteristics for inclusion in a review. We assessed study bias using the EPOC checklist and the quality of the reported intervention using a subset of SQUIRE 1.0 standards.

Results

Of the 220 studies meeting QIC criteria, 64 met EPOC study design standards for inclusion. There were 10 cluster randomised controlled trials, 24 controlled before-after studies and 30 interrupted time series studies. QICs encompassed a broad range of clinical settings, topics and populations ranging from neonates to the elderly. Few reports fully described QIC implementation and methods, intensity of activities, degree of site engagement and important contextual factors. By care setting, an improvement was reported for one or more of the study’s primary effect measures in 83% of the studies (32/39 (82%) hospital based, 17/20 (85%) ambulatory care, 3/4 nursing home and a sole ambulance QIC). Eight studies described persistence of the intervention effect 6 months to 2 years after the end of the collaborative. Collaboratives reporting success generally addressed relatively straightforward aspects of care, had a strong evidence base and noted a clear evidence-practice gap in an accepted clinical pathway or guideline.

Conclusions

QICs have been adopted widely as an approach to shared learning and improvement in healthcare. Overall, the QICs included in this review reported significant improvements in targeted clinical processes and patient outcomes. These reports are encouraging, but most be interpreted cautiously since fewer than a third met established quality and reporting criteria, and publication bias is likely.

Implementation of Treat-to-Target in Rheumatoid Arthritis Through a Learning Collaborative: Results of a Randomized Controlled Trial

OBJECTIVE

Treat-to-target (TTT) is an accepted paradigm for the management of rheumatoid arthritis (RA), but some evidence suggests poor adherence. The purpose of this study was to test the effects of a group-based multisite improvement learning collaborative on adherence to TTT.

METHODS

We conducted a cluster-randomized quality-improvement trial with waitlist control across 11 rheumatology sites in the US. The intervention entailed a 9-month group-based learning collaborative that incorporated rapid-cycle improvement methods. A composite TTT implementation score was calculated as the percentage of 4 required items documented in the visit notes for each patient at 2 time points, as evaluated by trained staff. The mean change in the implementation score for TTT across all patients for the intervention sites was compared with that for the control sites after accounting for intracluster correlation using linear mixed models.

RESULTS

Five sites with a total of 23 participating rheumatology providers were randomized to intervention and 6 sites with 23 participating rheumatology providers were randomized to the waitlist control. The intervention included 320 patients, and the control included 321 patients. At baseline, the mean TTT implementation score was 11% in both arms; after the 9-month intervention, the mean TTT implementation score was 57% in the intervention group and 25% in the control group (change in score of 46% for intervention and 14% for control; P = 0.004). We did not observe excessive use of resources or excessive occurrence of adverse events in the intervention arm.

CONCLUSION

A learning collaborative resulted in substantial improvements in adherence to TTT for the management of RA. This study supports the use of an educational collaborative to improve quality.

Continuous quality improvement interventions to improve long-term outcomes of antiretroviral therapy in women who initiated therapy during pregnancy or breastfeeding in the Democratic Republic of Congo: design of an open-label, parallel, group randomized trial

Background

Despite the rapid adoption of the World Health Organization’s 2013 guidelines, children continue to be infected with HIV perinatally because of sub-optimal adherence to the continuum of HIV care in maternal and child health (MCH) clinics. To achieve the UNAIDS goal of eliminating mother-to-child HIV transmission, multiple, adaptive interventions need to be implemented to improve adherence to the HIV continuum.

Methods

The aim of this open label, parallel, group randomized trial is to evaluate the effectiveness of Continuous Quality Improvement (CQI) interventions implemented at facility and health district levels to improve retention in care and virological suppression through 24 months postpartum among pregnant and breastfeeding women receiving ART in MCH clinics in Kinshasa, Democratic Republic of Congo. Prior to randomization, the current monitoring and evaluation system will be strengthened to enable collection of high quality individual patient-level data necessary for timely indicators production and program outcomes monitoring to inform CQI interventions. Following randomization, in health districts randomized to CQI, quality improvement (QI) teams will be established at the district level and at MCH clinics level. For 18 months, QI teams will be brought together quarterly to identify key bottlenecks in the care delivery system using data from the monitoring system, develop an action plan to address those bottlenecks, and implement the action plan at the level of their district or clinics.

Discussion

If proven to be effective, CQI as designed here, could be scaled up rapidly in resource-scarce settings to accelerate progress towards the goal of an AIDS free generation.

Trial registration

The protocol was retrospectively registered on February 7, 2017. ClinicalTrials.gov Identifier: NCT03048669.

Antiretroviral Stewardship in a Pediatric HIV Clinic: Development, Implementation and Improved Clinical Outcomes

BACKGROUND

Antiretroviral (ARV) management in pediatrics is a challenging process in which multiple barriers to optimal therapy can lead to poor clinical outcomes. In a pediatric HIV clinic, we implemented a systematic ARV stewardship program to evaluate ARV regimens and make recommendations for optimization when indicated.

METHODS

A comprehensive assessment tool was used to screen for issues related to genotypic resistance, virologic/immunologic response, drug-drug interactions, side effects and potential for regimen simplification. The ARV stewardship team (AST) made recommendations to the HIV clinic provider, and followed patients prospectively to assess clinical outcomes at 6 and 12 months.

RESULTS

The most common interventions made by the AST included regimen optimization in patients on suboptimal regimens based on resistance mutations (35.4%), switching to safer ARVs (33.3%) and averting significant drug-drug interactions (10.4%). In patients anticipated to have a change in viral load (VL) as a result of the AST recommendations, we identified a significant benefit in virologic outcomes at 6 and 12 months when recommendations were implemented within 6 months of ARV review. Patients who had recommendations implemented within 6 months had a 7-fold higher probability of achieving a 0.7 log10 reduction in VL by 6 months, and this benefit remained significant after controlling for adherence [adjusted odds ratio: 6.8 (95% confidence interval: 1.03-44.9; P <0.05)].

CONCLUSIONS

A systematic ARV stewardship program implemented at a pediatric HIV clinic significantly improved clinical outcomes. ARV stewardship programs can be considered a core strategy for continuous quality improvement in the management of HIV-infected children and adolescents.

How to Sustain Change and Support Continuous Quality Improvement

To achieve sustainable change, quality improvement initiatives must become the new way of working rather than something added on to routine clinical care. However, most organizational change is not maintained. In this next article in this Moving Points in Nephrology feature on quality improvement, we provide health care professionals with strategies to sustain and support quality improvement. Threats to sustainability may be identified both at the beginning of a project and when it is ready for implementation. The National Health Service Sustainability Model is reviewed as one example to help identify issues that affect long-term success of quality improvement projects. Tools to help sustain improvement include process control boards, performance boards, standard work, and improvement huddles. Process control and performance boards are methods to communicate improvement results to staff and leadership. Standard work is a written or visual outline of current best practices for a task and provides a framework to ensure that changes that have improved patient care are consistently and reliably applied to every patient encounter. Improvement huddles are short, regular meetings among staff to anticipate problems, review performance, and support a culture of improvement. Many of these tools rely on principles of visual management, which are systems transparent and simple so that every staff member can rapidly distinguish normal from abnormal working conditions. Even when quality improvement methods are properly applied, the success of a project still depends on contextual factors. Context refers to aspects of the local setting in which the project operates. Context affects resources, leadership support, data infrastructure, team motivation, and team performance. For these reasons, the same project may thrive in a supportive context and fail in a different context. To demonstrate the practical applications of these quality improvement principles, these principles are applied to a hypothetical quality improvement initiative that aims to promote home dialysis (home hemodialysis and peritoneal dialysis).

Implementation of the Standards-Based Management and Recognition approach to quality improvement in maternal, newborn, and child health programs in low-resource countries

The Standards-Based Management and Recognition (SBM-R; Jhpiego, Baltimore, MD, USA) approach to quality improvement was developed by Jhpiego to respond to common challenges faced by health systems in low-resource settings, including poor pre-service education, lack of resources for conventional supervisory models, and weak health information systems. Since its introduction in Brazil in 1997, SBM-R has been implemented in approximately 30 countries and continues expanding to new places and service delivery areas. The present article: (1) describes key steps in the SBM-R methodology focusing on provider performance assessment using evidence-based standards; and (2) presents examples of improvements in provider performance in maternal, newborn, and child health care following SBM-R implementation derived from routine program data, quasi-experimental evaluations, and in-depth case studies. SBM-R incorporates evidence-based methods that are known to have positive effects on healthcare quality, including audit and feedback, educational outreach visits, and checklist usage; however, further rigorous research is needed to document the population-level impacts of the SBM-R approach.

Secular trends and evaluation of complex interventions: the rising tide phenomenon

Evaluations of service delivery interventions with contemporaneous controls often yield null results, even when the intervention appeared promising in advance. There can be many reasons for null results. In this paper we introduce the concept of a ‘rising tide’ phenomenon being a possible explanation of null results. We note that evaluations of service delivery interventions often occur when awareness of the problems they intend to address is already heightened, and pressure to tackle them is mounting throughout a health system. An evaluation may therefore take place in a setting where the system as a whole is improving – where there is a pronounced temporal trend or a ‘rising tide causing all vessels to rise’. As a consequence, control sites in an intervention study will improve. This reduces the difference between intervention and control sites and predisposes the study to a null result, leading to the conclusion that the intervention has no effect. We discuss how a rising tide may be distinguished from other causes of improvement in both control and intervention groups, and give examples where the rising tide provides a convincing explanation of such a finding. We offer recommendations for interpretation of research findings where improvements in the intervention group are matched by improvements in the control group. Understanding the rising tide phenomenon is important for a more nuanced interpretation of null results arising in the context of system-wide improvement. Recognition that a rising tide may have predisposed to a null result in one health system cautions against generalising the result to another health system where strong secular trends are absent.

Protocol for a statewide randomized controlled trial to compare three training models for implementing an evidence-based treatment

BACKGROUND

Evidence-based treatments (EBTs) are available for treating childhood behavioral health challenges. Despite EBTs' potential to help children and families, they have primarily remained in university settings. Little empirical evidence exists regarding how specific, commonly used training and quality control models are effective in changing practice, achieving full implementation, and supporting positive client outcomes.

METHODS/DESIGN

This study (NIMH RO1 MH095750; ClinicalTrials.gov Identifier: NCT02543359), which is currently in progress, will evaluate the effectiveness of three training models (Learning Collaborative (LC), Cascading Model (CM), and Distance Education (DE)) to implement a well-established EBT , Parent-Child Interaction Therapy, in real-world, community settings. The three models differ in their costs, skill training, quality control methods, and capacity to address broader implementation challenges. The project is guided by three specific aims: (1) to build knowledge about training outcomes, (2) to build knowledge about implementation outcomes, and (3) to test the differential impact of training clinicians using LC, CM, and DE models on key client outcomes. Fifty (50) licensed psychiatric clinics across Pennsylvania were randomized to one of the three training conditions: (1) LC, (2) CM, or (3) DE. The impact of training on practice skills (clinician level) and implementation/sustainment outcomes (clinic level) are being evaluated at four timepoints coinciding with the training schedule: baseline, 6 (mid), 12 (post), and 24 months (1 year follow-up). Immediately after training begins, parent-child dyads (client level) are recruited from the caseloads of participating clinicians. Client outcomes are being assessed at four timepoints (pre-treatment, 1, 6, and 12 months after the pre-treatment).

DISCUSSION

This proposal builds on an ongoing initiative to implement an EBT statewide. A team of diverse stakeholders including state policy makers, payers, consumers, service providers, and academics from different, but complementary areas (e.g., public health, social work, psychiatry), has been assembled to guide the research plan by incorporating input from multidimensional perspective.

TRIAL REGISTRATION

ClinicalTrials.gov: NCT02543359.

Implementation of HIV treatment as prevention strategy in 17 Canadian sites: immediate and sustained outcomes from a 35-month Quality Improvement Collaborative

BACKGROUND

Rapid scale-up of effective antiretroviral therapy (ART) is required to meet global targets to eliminate new HIV infections and AIDS-related deaths. Yet, gaps persist in all nations striving for these targets. In the intervention setting of British Columbia (BC), Canada, where ART is publicly funded, 73% of HIV-diagnosed were on ART in 2011, and only 49% were achieving viral suppression.

METHODS

An observational case descriptive study of HIV care sites in BC recruited to participate in a 35-month Breakthrough Series Quality Improvement Collaborative and sustainability network. Sites collected four quality indicators, qualitative change descriptions and implemented the chronic care model (CCM) and HIV care and treatment guidelines. Two reviewers assigned monthly implementation scores to evaluate site progress (January 2011-2012). All quality indicators were pooled and analysed using probability-based run chart rules.

RESULTS

Seventeen teams with a pooled median population of 2296 HIV patients joined the initiative. Comprehensive CCM implementation and evidence of improvement was achieved by 29% of sites (implementation score of 4.0 or higher on 5.0 scale). Evidence of sustained improvement was observed for patient engagement (88.8-90.4%), ART uptake among patients unequivocally in need (92.9-94.8%), and ART uptake (≥6 months) and achieving viral suppression (57.3-78.4%) (all p<0.05).

CONCLUSIONS

This study shows evidence of sustained improvements in HIV care processes and treatment outcomes for an estimated population of 2296 HIV patients in 17 BC sites. Overall success points to opportunities for other high-income countries seeking to improve HIV health outcomes.

Development and Validation of the CPO Scale, a New Instrument for Evaluation of Health Care Improvement Efforts

OBJECTIVE

To develop and validate an instrument for guidance and evaluation of quality and safety improvement efforts in health care.

CONTEXT

The instrument is based on the Plan-Do-Study-Act cycle and the 3 fundamental improvement questions regarding aims, measurement, and change-making.

METHODS

An interdisciplinary team of improvement experts developed the Change Process and Outcome (CPO) scale. After studying the improvement literature, the scale was tested and refined on a sample of 5 projects. The CPO evaluation process and classification system was developed when evaluating 189 of the quality improvement projects of the Norwegian Medical Association by their final reports. The scale was validated by applying statistical testing to the evaluation results.

RESULTS

The final CPO scale consists of 13 process items and 7 outcome items. Interrater reliability ranged from 0.53 to 0.79, and test-retest reliability was 0.82. Factor analyses with Varimax rotation identified 2 significant process domains: Aims/change-making and Measurement/reporting, with Cronbach α values 0.88 and 0.95, respectively. The classification system produced 3 performance levels: successful, promising, and uncertain.

CONCLUSION

The CPO scale shows good internal consistency, reliability, and validity for evaluating the success of quality improvement initiatives.

Strange Bedfellows: A Local Insurer/Physician Practice Partnership to Fund Innovation

Despite an unprecedented urgency to control healthcare costs while simultaneously improving quality, there are many barriers to investing in quality improvement. Traditional fee-for-service reimbursement models fail to reward providers whose improved processes lead to decreases in billable clinical activity. In addition, providers may lack the necessary skills for improvement, or the organizational infrastructure to conduct these activities. Insurance firms lack incentives to invest in healthcare delivery system improvements that lead to benefits for all patients, even those covered by competitors. In this article, we describe a novel program in its sixth year of existence that funds ambulatory care improvements through a collaborative partnership between a local academic healthcare delivery system and an insurance firm. The program is designed as a competitive grant program and the payer and healthcare organization jointly benefit from completed improvement projects. Factors contributing to the ongoing success of the program and lessons learned are discussed in order to inform the potential development of similar programs in other markets.

Implementing a user-driven online quality improvement toolkit for cancer care

PURPOSE

Peer-to-peer collaboration within integrated health systems requires a mechanism for sharing quality improvement lessons. The Veterans Health Administration (VA) developed online compendia of tools linked to specific cancer quality indicators. We evaluated awareness and use of the toolkits, variation across facilities, impact of social marketing, and factors influencing toolkit use.

METHODS

A diffusion of innovations conceptual framework guided the collection of user activity data from the Toolkit Series SharePoint site and an online survey of potential Lung Cancer Care Toolkit users.

RESULTS

The VA Toolkit Series site had 5,088 unique visitors in its first 22 months; 5% of users accounted for 40% of page views. Social marketing communications were correlated with site usage. Of survey respondents (n = 355), 54% had visited the site, of whom 24% downloaded at least one tool. Respondents' awareness of the lung cancer quality performance of their facility, and facility participation in quality improvement collaboratives, were positively associated with Toolkit Series site use. Facility-level lung cancer tool implementation varied widely across tool types.

CONCLUSION

The VA Toolkit Series achieved widespread use and a high degree of user engagement, although use varied widely across facilities. The most active users were aware of and active in cancer care quality improvement. Toolkit use seemed to be reinforced by other quality improvement activities. A combination of user-driven tool creation and centralized toolkit development seemed to be effective for leveraging health information technology to spread disease-specific quality improvement tools within an integrated health care system.

Barriers and supportive conditions to improve quality of care for critically ill patients: A team approach to quality improvement

PURPOSE

Despite the fact that Quality Improvement (QI) teams are widespread tools for improving performance in medical settings, little is known about what makes teams effective and successful. The goal of this study was to identify barriers and supportive conditions for QI teams to implement an effective and successful QI project to improve quality of care.

MATERIALS AND METHODS

Multicenter expert interviews with 17 team leaders were conducted in a cluster randomized trial. Interviews were based on a semistructured interview guide and were recorded and transcribed. Qualitative analysis was performed according to the principles of grounded theory.

RESULTS

The major findings of our study can be summarized in a framework of conditions that support the implementation of changes by QI teams. This framework can be divided into 5 core categories: the availability of external support, an interdisciplinary QI team, staff characteristics such as dedicated employees who are aware and experienced, and generally supportive structural circumstances. Furthermore, the interviewees reported that changes should be disseminated through, for example, repeating key elements or addressing employees directly.

CONCLUSIONS

Using a grounded theory-based qualitative approach, we identified a framework of conditions supportive of QI-related change, which can help project initiators to create environments that are supportive of change.

Beyond the collaborative: spreading effective improvement in hand hygiene compliance

BACKGROUND

Data assessing the effectiveness of quality improvement (QI) collaboratives are mixed; spreading improvement beyond the original collaborative group has proved difficult. Little is known about whether organizations that did not participate in the collaborative are able to effectively employ interventions developed or implemented by those organizations that did participate.

METHODS

The Joint Commission Center for Transforming Healthcare conducted a collaborative QI project with eight hospitals, using Lean, Six Sigma, and change management methods to improve hand hygiene compliance. Participating hospitals achieved a 70.5% relative improvement (47.5% to 81.0%; p < .001). Following this project, working with an additional 19 hospitals, the Center created Web-based tools to enable health care organizations to use the same methods employed by the original eight hospitals without needing any knowledge or experience with Lean, Six Sigma, or change management. This Targeted Solutions Tool® (TST)® allowed organizations to discover the most important, specific causes of hand hygiene noncompliance in their facilities and to target interventions at those causes.

RESULTS

In the first three years, 289 health care organizations used the TST to initiate 1,495 projects to improve hand hygiene compliance. Of the 769 projects at 174 organizations for which baseline and improvement data were available, average compliance improved from 57.9% to 83.5% (p < .0001). Similar improvement was observed in many clinical care settings, including ambulatory, long term care, inpatient pediatrics, critical care, and adult medical/surgical units.

CONCLUSION

Hospitals and other health care organizations using the TST achieved levels of hand hygiene compliance comparable to those experienced by the participants in the original collaborative.

Opportunities for tuberculosis diagnosis and prevention among persons living with HIV: a cross-sectional study of policies and practices at four large Ryan White Program-Funded HIV clinics

Objective

We describe the frequency and attributes of tuberculosis testing and treatment at four publicly-funded HIV clinics.

Methods

We abstracted medical records from a random sample of 600 HIV-infected patients having at least one clinic visit in 2009 at four clinics in New York and Los Angeles Metropolitan Statistical areas. We described testing and treatment for tuberculosis infection (TBI), 2008–2010, and estimated adjusted odds ratios (aORs). We interviewed key informants and described clinic policies and practices.

Results

Of 600 patients, 500 were eligible for testing, and 393 (79%) were tested 2008–2010; 107 (21%) did not receive at least one tuberculin skin test or interferon gamma release assay. Results were positive in 20 (5%) patients, negative in 357 (91%), and unknown in 16 (4%). Fourteen (70%) of 20 patients with TBI initiated treatment at the clinics; only three were documented to have completed treatment. Three hundred twenty three (54%) patients had chest radiography, 346 (58%) had tuberculosis symptom screening, and three had tuberculosis disease (117 per 100,000 person-years, 95% confidence interval (CI) = 101–165). Adjusting for site, non-Hispanic ethnicity (aOR = 4.9, 95% CI = 2.6–9.5), and employment (aOR = 1.9, 95% CI = 1.0–3.4) were associated with TBI testing; female gender (aOR = 2.0, 95% CI = 1.4–3.3), non-black race (aOR = 1.7, 95% CI = 1.3–2.5), and unemployment (aOR = 1.5, 95% CI = 1.1–2.1) were associated with chest radiography. Clinics evaluated TBI testing performance annually and identified challenges to TB prevention.

Conclusions

Study clinics routinely tested patients for TBI, but did not always document treatment. In a population with a high TB rate, ensuring treatment of TBI may enhance TB prevention.

Systems analysis and improvement to optimize pMTCT (SAIA): a cluster randomized trial

Background

Despite significant increases in global health investment and the availability of low-cost, efficacious interventions to prevent mother-to-child HIV transmission (pMTCT) in low- and middle-income countries with high HIV burden, the translation of scientific advances into effective delivery strategies has been slow, uneven and incomplete. As a result, pediatric HIV infection remains largely uncontrolled. A five-step, facility-level systems analysis and improvement intervention (SAIA) was designed to maximize effectiveness of pMTCT service provision by improving understanding of inefficiencies (step one: cascade analysis), guiding identification and prioritization of low-cost workflow modifications (step two: value stream mapping), and iteratively testing and redesigning these modifications (steps three through five). This protocol describes the SAIA intervention and methods to evaluate the intervention’s impact on reducing drop-offs along the pMTCT cascade.

Methods

This study employs a two-arm, longitudinal cluster randomized trial design. The unit of randomization is the health facility. A total of 90 facilities were identified in Côte d’Ivoire, Kenya and Mozambique (30 per country). A subset was randomly selected and assigned to intervention and comparison arms, stratified by country and service volume, resulting in 18 intervention and 18 comparison facilities across all three countries, with six intervention and six comparison facilities per country. The SAIA intervention will be implemented for six months in the 18 intervention facilities. Primary trial outcomes are designed to assess improvements in the pMTCT service cascade, and include the percentage of pregnant women being tested for HIV at the first antenatal care visit, the percentage of HIV-infected pregnant women receiving adequate prophylaxis or combination antiretroviral therapy in pregnancy, and the percentage of newborns exposed to HIV in pregnancy receiving an HIV diagnosis eight weeks postpartum. The Consolidated Framework for Implementation Research (CFIR) will guide collection and analysis of qualitative data on implementation process.

Discussion

This study is a pragmatic trial that has the potential benefit of improving maternal and infant outcomes by reducing drop-offs along the pMTCT cascade. The SAIA intervention is designed to provide simple tools to guide decision-making for pMTCT program staff at the facility level, and to identify low cost, contextually appropriate pMTCT improvement strategies.

Trial registration

ClinicalTrials.gov NCT02023658

Did a quality improvement collaborative make stroke care better? A cluster randomized trial

Background

Stroke can result in death and long-term disability. Fast and high-quality care can reduce the impact of stroke, but UK national audit data has demonstrated variability in compliance with recommended processes of care. Though quality improvement collaboratives (QICs) are widely used, whether a QIC could improve reliability of stroke care was unknown.

Methods

Twenty-four NHS hospitals in the Northwest of England were randomly allocated to participate either in Stroke 90:10, a QIC based on the Breakthrough Series (BTS) model, or to a control group giving normal care. The QIC focused on nine processes of quality care for stroke already used in the national stroke audit. The nine processes were grouped into two distinct care bundles: one relating to early hours care and one relating to rehabilitation following stroke. Using an interrupted time series design and difference-in-difference analysis, we aimed to determine whether hospitals participating in the QIC improved more than the control group on bundle compliance.

Results

Data were available from nine interventions (3,533 patients) and nine control hospitals (3,059 patients). Hospitals in the QIC showed a modest improvement from baseline in the odds of average compliance equivalent to a relative improvement of 10.9% (95% CI 1.3%, 20.6%) in the Early Hours Bundle and 11.2% (95% CI 1.4%, 21.5%) in the Rehabilitation Bundle. Secondary analysis suggested that some specific processes were more sensitive to an intervention effect.

Conclusions

Some aspects of stroke care improved during the QIC, but the effects of the QIC were modest and further improvement is needed. The extent to which a BTS QIC can improve quality of stroke care remains uncertain. Some aspects of care may respond better to collaboratives than others.

Trial registration

ISRCTN13893902.

The use of modern quality improvement approaches to strengthen African health systems: a 5-year agenda

There is a growing international consensus that African health systems need to improve, but no agreement on how to accomplish this. From the perspective of modern quality improvement (QI), a central issue for low performance in these health systems is the relative neglect of health-care processes. Both health system leaders and international donors have focused their efforts elsewhere, producing noteworthy health gains. But these gains are at risk if health systems do not develop the capacity to study and improve care processes. Substantial experience with QI in Africa shows impressive potential for broad-based process improvement. But this experience also highlights the need for modifying these growing programs to incorporate a more rigorous learning component to address challenges that have emerged recently. The addition of a region-wide knowledge management program could increase the efficiency of each country's QI program by learning from the experiences of other programs. With a coordinated donor initiative, it is reasonable to project that within 5 years, evidence-based improvement will become a norm in health services, and African health systems will approach the model of a learning organization.

Organizational Factors Influencing the Use of Clinical Decision Support for Improving Cancer Screening Within Community Health Centers

Adoption of clinical decision support (CDS) systems leads to improved clinical performance through improved clinician decision making, adherence to evidence-based guidelines, medical error reduction, and more efficient information transfer and to reduction in health care disparities in under-resourced settings. However, little information on CDS use in the community health care (CHC) setting exists. This study examines if organizational, provider, or patient level factors can successfully predict the level of CDS use in the CHC setting with regard to breast, cervical, and colorectal cancer screening. This study relied upon 37 summary measures obtained from the 2005 Cancer Health Disparities Collaborative (HDCC) national survey of 44 randomly selected community health centers. A multi-level framework was designed that employed an all-subsets linear regression to discover relationships between organizational/practice setting, provider, and patient characteristics and the outcome variable, a composite measure of community health center CDS intensity-of-use. Several organizational and provider level factors from our conceptual model were identified to be positively associated with CDS level of use in community health centers. The level of CDS use (e.g., computerized reminders, provider prompts at point-of-care) in support of breast, cervical, and colorectal cancer screening rate improvement in vulnerable populations is determined by both organizational/practice setting and provider factors. Such insights can better facilitate the increased uptake of CDS in CHCs that allows for improved patient tracking, disease management, and early detection in cancer prevention and control within vulnerable populations.

Recommendations for evaluation of health care improvement initiatives

Intensive efforts are underway across the world to improve the quality of health care. It is important to use evaluation methods to identify improvement efforts that work well before they are replicated across a broad range of contexts. Evaluation methods need to provide an understanding of why an improvement initiative has or has not worked and how it can be improved in the future. However, improvement initiatives are complex, and evaluation is not always well aligned with the intent and maturity of the intervention, thus limiting the applicability of the results. We describe how initiatives can be grouped into 1 of 3 improvement phases-innovation, testing, and scale-up and spread-depending on the degree of belief in the associated interventions. We describe how many evaluation approaches often lead to a finding of no effect, consistent with what has been termed Rossi's Iron Law of Evaluation. Alternatively, we recommend that the guiding question of evaluation in health care improvement be, "How and in what contexts does a new model work or can be amended to work?" To answer this, we argue for the adoption of formative, theory-driven evaluation. Specifically, evaluations start by identifying a program theory that comprises execution and content theories. These theories should be revised as the initiative develops by applying a rapid-cycle evaluation approach, in which evaluation findings are fed back to the initiative leaders on a regular basis. We describe such evaluation strategies, accounting for the phase of improvement as well as the context and setting in which the improvement concept is being deployed. Finally, we challenge the improvement and evaluation communities to come together to refine the specific methods required so as to avoid the trap of Rossi's Iron Law.

Reducing racial/ethnic disparities in diabetes: the Coached Care (R2D2C2) project

BACKGROUND

Despite numerous efforts to change healthcare delivery, the profile of disparities in diabetes care and outcomes has not changed substantially over the past decade.

OBJECTIVE

To understand potential contributors to disparities in diabetes care and glycemic control.

DESIGN

Cross sectional analysis.

SETTING

Seven outpatient clinics affiliated with an academic medical center.

PATIENTS

Adult patients with type 2 diabetes who were Mexican American, Vietnamese American or non-Hispanic white (n = 1,484).

MEASUREMENTS

Glycemic control was measured as hemoglobin A1c (HbA1c) level. Patient, provider and system characteristics included demographic characteristics; access to care; quality of process of care including clinical inertia; quality of interpersonal care; illness burden; mastery (diabetes management confidence, passivity); and adherence to treatment.

RESULTS

Unadjusted HbA1c values were significantly higher for Mexican American patients (n = 782) (mean = 8.3 % [SD:2.1]) compared with non-Hispanic whites (n = 389) (mean = 7.1 % [SD:1.4]). There were no significant differences in HbA1c values between Vietnamese American and non-Hispanic white patients. There were no statistically significant group differences in glycemic control after adjustment for multiple measures of access, and quality of process and interpersonal care. Disease management mastery and adherence to treatment were related to glycemic control for all patients, independent of race/ethnicity.

LIMITATIONS

Generalizability to other minorities or to patients with poorer access to care may be limited.

CONCLUSIONS

The complex interplay among patient, physician and system characteristics contributed to disparities in HbA1c between Mexican American and non-Hispanic white patients. In contrast, Vietnamese American patients achieved HbA1c levels comparable to non-Hispanic whites and adjustment for numerous characteristics failed to identify confounders that could have masked disparities in this subgroup. Disease management mastery appeared to be an important contributor to glycemic control for all patient subgroups.

Systematic review of the application of the plan-do-study-act method to improve quality in healthcare

BACKGROUND

Plan-do-study-act (PDSA) cycles provide a structure for iterative testing of changes to improve quality of systems. The method is widely accepted in healthcare improvement; however there is little overarching evaluation of how the method is applied. This paper proposes a theoretical framework for assessing the quality of application of PDSA cycles and explores the consistency with which the method has been applied in peer-reviewed literature against this framework.

METHODS

NHS Evidence and Cochrane databases were searched by three independent reviewers. Empirical studies were included that reported application of the PDSA method in healthcare. Application of PDSA cycles was assessed against key features of the method, including documentation characteristics, use of iterative cycles, prediction-based testing of change, initial small-scale testing and use of data over time.

RESULTS

73 of 409 individual articles identified met the inclusion criteria. Of the 73 articles, 47 documented PDSA cycles in sufficient detail for full analysis against the whole framework. Many of these studies reported application of the PDSA method that failed to accord with primary features of the method. Less than 20% (14/73) fully documented the application of a sequence of iterative cycles. Furthermore, a lack of adherence to the notion of small-scale change is apparent and only 15% (7/47) reported the use of quantitative data at monthly or more frequent data intervals to inform progression of cycles.

DISCUSSION

To progress the development of the science of improvement, a greater understanding of the use of improvement methods, including PDSA, is essential to draw reliable conclusions about their effectiveness. This would be supported by the development of systematic and rigorous standards for the application and reporting of PDSAs.

Systematic review of the application of the plan-do-study-act method to improve quality in healthcare

Background

Plan–do–study–act (PDSA) cycles provide a structure for iterative testing of changes to improve quality of systems. The method is widely accepted in healthcare improvement; however there is little overarching evaluation of how the method is applied. This paper proposes a theoretical framework for assessing the quality of application of PDSA cycles and explores the consistency with which the method has been applied in peer-reviewed literature against this framework.

Methods

NHS Evidence and Cochrane databases were searched by three independent reviewers. Empirical studies were included that reported application of the PDSA method in healthcare. Application of PDSA cycles was assessed against key features of the method, including documentation characteristics, use of iterative cycles, prediction-based testing of change, initial small-scale testing and use of data over time.

Results

73 of 409 individual articles identified met the inclusion criteria. Of the 73 articles, 47 documented PDSA cycles in sufficient detail for full analysis against the whole framework. Many of these studies reported application of the PDSA method that failed to accord with primary features of the method. Less than 20% (14/73) fully documented the application of a sequence of iterative cycles. Furthermore, a lack of adherence to the notion of small-scale change is apparent and only 15% (7/47) reported the use of quantitative data at monthly or more frequent data intervals to inform progression of cycles.

Discussion

To progress the development of the science of improvement, a greater understanding of the use of improvement methods, including PDSA, is essential to draw reliable conclusions about their effectiveness. This would be supported by the development of systematic and rigorous standards for the application and reporting of PDSAs.

Understanding the components of quality improvement collaboratives: a systematic literature review

CONTEXT

In response to national efforts to improve quality of care, policymakers and health care leaders have increasingly turned to quality improvement collaboratives (QICs) as an efficient approach to improving provider practices and patient outcomes through the dissemination of evidence-based practices. This article presents findings from a systematic review of the literature on QICs, focusing on the identification of common components of QICs in health care and exploring, when possible, relations between QIC components and outcomes at the patient or provider level.

METHODS

A systematic search of five major health care databases generated 294 unique articles, twenty-four of which met our criteria for inclusion in our final analysis. These articles pertained to either randomized controlled trials or quasi-experimental studies with comparison groups, and they reported the findings from twenty different studies of QICs in health care. We coded the articles to identify the components reported for each collaborative.

FINDINGS

We found fourteen crosscutting components as common ingredients in health care QICs (e.g., in-person learning sessions, phone meetings, data reporting, leadership involvement, and training in QI methods). The collaboratives reported included, on average, six to seven of these components. The most common were in-person learning sessions, plan-do-study-act (PDSA) cycles, multidisciplinary QI teams, and data collection for QI. The outcomes data from these studies indicate the greatest impact of QICs at the provider level; patient-level findings were less robust.

CONCLUSIONS

Reporting on specific components of the collaborative was imprecise across articles, rendering it impossible to identify active QIC ingredients linked to improved care. Although QICs appear to have some promise in improving the process of care, there is great need for further controlled research examining the core components of these collaboratives related to patient- and provider-level outcomes.

Coaching interprofessional health care improvement teams: the coachee, the coach and the leader perspectives

AIM

To investigate health care improvement team coaching activities from the perspectives of coachees, coaches and unit leaders in two national improvement collaboratives.

BACKGROUND

Despite numerous methods to improve health care, inconsistencies in success have been attributed to factors that include unengaged staff, absence of supportive improvement resources and organisational inertia.

METHODS

Mixed methods sequential exploratory study design, including quantitative and qualitative data from interprofessional improvement teams who received team coaching. The coachees (n = 382), coaches (n = 9) and leaders (n = 30) completed three different data collection tools identifying coaching actions perceived to support improvement activities.

RESULTS

Coachees, coaches and unit leaders in both collaboratives reported generally positive perceptions about team coaching. Four categories of coaching actions were perceived to support improvement work: context, relationships, helping and technical support.

CONCLUSIONS

All participants agreed that regardless of who the coach is, emphasis should include the four categories of team coaching actions.

IMPLICATIONS FOR NURSING MANAGEMENT

Leaders should reflect on their efforts to support improvement teams and consider the four categories of team coaching actions. A structured team coaching model that offers needed encouragement to keep the team energized, seems to support health care improvement.

Understanding the components of quality improvement collaboratives: a systematic literature review

CONTEXT

In response to national efforts to improve quality of care, policymakers and health care leaders have increasingly turned to quality improvement collaboratives (QICs) as an efficient approach to improving provider practices and patient outcomes through the dissemination of evidence-based practices. This article presents findings from a systematic review of the literature on QICs, focusing on the identification of common components of QICs in health care and exploring, when possible, relations between QIC components and outcomes at the patient or provider level.

METHODS

A systematic search of five major health care databases generated 294 unique articles, twenty-four of which met our criteria for inclusion in our final analysis. These articles pertained to either randomized controlled trials or quasi-experimental studies with comparison groups, and they reported the findings from twenty different studies of QICs in health care. We coded the articles to identify the components reported for each collaborative.

FINDINGS

We found fourteen crosscutting components as common ingredients in health care QICs (e.g., in-person learning sessions, phone meetings, data reporting, leadership involvement, and training in QI methods). The collaboratives reported included, on average, six to seven of these components. The most common were in-person learning sessions, plan-do-study-act (PDSA) cycles, multidisciplinary QI teams, and data collection for QI. The outcomes data from these studies indicate the greatest impact of QICs at the provider level; patient-level findings were less robust.

CONCLUSIONS

Reporting on specific components of the collaborative was imprecise across articles, rendering it impossible to identify active QIC ingredients linked to improved care. Although QICs appear to have some promise in improving the process of care, there is great need for further controlled research examining the core components of these collaboratives related to patient- and provider-level outcomes.

A retrospective evaluation of the Perfecting Patient Care University training program for health care organizations

This study evaluated how the Perfecting Patient Care (PPC) University, a quality improvement (QI) training program for health care leaders and clinicians, affected the ability of organizations to improve the health care they provide. This training program teaches improvement methods based on Lean concepts and principles of the Toyota Production System and is offered in several formats. A retrospective evaluation was performed that gathered data on training, other process factors, and outcomes after staff completed the PPC training. A majority of respondents reported gaining QI competencies and cultural achievements from the training. Organizations had high average scores for the success measures of "outcomes improved" and "sustainable monitoring" but lower scores for diffusion of QI efforts. Total training dosage was significantly associated with the measures of QI success. This evaluation provides evidence that organizations gained the PPC competencies and cultural achievements and that training dosage is a driver of QI success.

Interprofessional education: effects on professional practice and healthcare outcomes (update)

BACKGROUND

The delivery of effective, high-quality patient care is a complex activity. It demands health and social care professionals collaborate in an effective manner. Research continues to suggest that collaboration between these professionals can be problematic. Interprofessional education (IPE) offers a possible way to improve interprofessional collaboration and patient care.

OBJECTIVES

To assess the effectiveness of IPE interventions compared to separate, profession-specific education interventions; and to assess the effectiveness of IPE interventions compared to no education intervention.

SEARCH METHODS

For this update we searched the Cochrane Effective Practice and Organisation of Care Group specialised register, MEDLINE and CINAHL, for the years 2006 to 2011. We also handsearched the Journal of Interprofessional Care (2006 to 2011), reference lists of all included studies, the proceedings of leading IPE conferences, and websites of IPE organisations.

SELECTION CRITERIA

Randomised controlled trials (RCTs), controlled before and after (CBA) studies and interrupted time series (ITS) studies of IPE interventions that reported objectively measured or self reported (validated instrument) patient/client or healthcare process outcomes.

DATA COLLECTION AND ANALYSIS

At least two review authors independently assessed the eligibility of potentially relevant studies. For included studies, at least two review authors extracted data and assessed study quality. A meta-analysis of study outcomes was not possible due to heterogeneity in study designs and outcome measures. Consequently, the results are presented in a narrative format.

MAIN RESULTS

This update located nine new studies, which were added to the six studies from our last update in 2008. This review now includes 15 studies (eight RCTs, five CBA and two ITS studies). All of these studies measured the effectiveness of IPE interventions compared to no educational intervention. Seven studies indicated that IPE produced positive outcomes in the following areas: diabetes care, emergency department culture and patient satisfaction; collaborative team behaviour and reduction of clinical error rates for emergency department teams; collaborative team behaviour in operating rooms; management of care delivered in cases of domestic violence; and mental health practitioner competencies related to the delivery of patient care. In addition, four of the studies reported mixed outcomes (positive and neutral) and four studies reported that the IPE interventions had no impact on either professional practice or patient care.

AUTHORS' CONCLUSIONS

This updated review reports on 15 studies that met the inclusion criteria (nine studies from this update and six studies from the 2008 update). Although these studies reported some positive outcomes, due to the small number of studies and the heterogeneity of interventions and outcome measures, it is not possible to draw generalisable inferences about the key elements of IPE and its effectiveness. To improve the quality of evidence relating to IPE and patient outcomes or healthcare process outcomes, the following three gaps will need to be filled: first, studies that assess the effectiveness of IPE interventions compared to separate, profession-specific interventions; second, RCT, CBA or ITS studies with qualitative strands examining processes relating to the IPE and practice changes; third, cost-benefit analyses.

Improving organizational climate for quality and quality of care: does membership in a collaborative help?

BACKGROUND

The lack of quality-oriented organizational climates is partly responsible for deficiencies in patient-centered care and poor quality more broadly. To improve their quality-oriented climates, several organizations have joined quality improvement collaboratives. The effectiveness of this approach is unknown.

OBJECTIVE

To evaluate the impact of collaborative membership on organizational climate for quality and service quality.

SUBJECTS

Twenty-one clinics, 4 of which participated in a collaborative sponsored by the Institute for Clinical Systems Improvement.

RESEARCH DESIGN

Pre-post design. Preassessments occurred 2 months before the collaborative began in January 2009. Postassessments of service quality and climate occurred about 6 months and 1 year, respectively, after the collaborative ended in January 2010. We surveyed clinic employees (eg, physicians, nurses, receptionists, etc.) about the organizational climate and patients about service quality.

MEASURES

Prioritization of quality care, high-quality staff relationships, and open communication as indicators of quality-oriented climate and timeliness of care, staff helpfulness, doctor-patient communication, rating of doctor, and willingness to recommend doctor's office as indicators of service quality.

RESULTS

There was no significant effect of collaborative membership on quality-oriented climate and mixed effects on service quality. Doctors' ratings improved significantly more in intervention clinics than in control clinics, staff helpfulness improved less, and timeliness of care declined more. Ratings of doctor-patient communication and willingness to recommend doctor were not significantly different between intervention and comparison clinics.

CONCLUSION

Membership in the collaborative provided no significant advantage for improving quality-oriented climate and had equivocal effects on service quality.

Chronic Care Model Decision Support and Clinical Information Systems interventions for people living with HIV: a systematic review

BACKGROUND

The Chronic Care Model is an effective framework for improving chronic disease management. There is scarce literature describing this model for people living with HIV. Decision Support (DS) and Clinical Information Systems (CIS) are two components of this model that aim to improve care by changing health care provider behavior.

OBJECTIVE

Our aim was to assess the effectiveness of DS and CIS interventions for individuals with HIV, through a systematic literature review.

DESIGN

We performed systematic electronic searches from 1996 to February 2011 of the medical (E.g. Medline, EMBASE, CINAHL) and grey literature. Effectiveness was measured by the frequency of statistically significant outcome improvement. Data and key equity indicator extraction and synthesis was completed.

PARTICIPANTS AND INTERVENTIONS

We included comparative studies of people living with HIV that examined the impact of DS or CIS interventions on outcomes.

MAIN MEASURES

The following measures were assessed: outcome (immunological/virological, medical, psychosocial, economic measures) and health care process/performance measures.

KEY RESULTS

Records were screened for relevance (n = 10,169), full-text copies of relevant studies were obtained (n = 123), and 16 studies were included in the review. Overall, 5/9 (55.6%) and 17/41 (41.5%) process measures and 5/12 (41.7%) and 3/9 (33.3%) outcome measures for DS and CIS interventions, respectively, were statistically significantly improved. DS-explicit mention of implementation of guidelines and CIS-reminders showed the most frequent improvement in outcomes. DS-only interventions were more effective than CIS-only interventions in improving both process and outcome measures. Clinical, statistical and methodological heterogeneity among studies precluded meta-analysis. Primary studies were methodologically weak and often included multifaceted interventions that made assessment of effectiveness challenging.

CONCLUSIONS

Overall, DS and CIS interventions may modestly improve care for people living with HIV, having a greater impact on process measures compared to outcome measures. These interventions should be considered as part of strategies to improve HIV care through changing provider performance.

Unhealthy alcohol and illicit drug use are associated with decreased quality of HIV care

BACKGROUND

HIV-infected patients with substance use experience suboptimal health outcomes, possibly because of variations in care.

OBJECTIVES

To assess the association between substance use and the quality of HIV care (QOC) received.

RESEARCH DESIGN

Retrospective cohort study.

SUBJECTS

HIV-infected patients enrolled in the Veterans Aging Cohort Study.

MEASURES

We collected self-report substance use data and abstracted 9 HIV quality indicators (QIs) from medical records. Independent variables were unhealthy alcohol use (AUDIT-C score ≥4) and illicit drug use (self-report of stimulants, opioids, or injection drug use in past year). Main outcome was the percentage of QIs received, if eligible. We estimated associations between substance use and QOC using multivariable linear regression.

RESULTS

The majority of the 3410 patients were male (97.4%) and black (67.0%) with a mean age of 49.1 years (SD = 8.8). Overall, 25.8% reported unhealthy alcohol use, 22% illicit drug use, and participants received 81.5% (SD = 18.9) of QIs. The mean percentage of QIs received was lower for those with unhealthy alcohol use versus not (59.3% vs. 70.0%, P < 0.001) and those using illicit drugs vs. not (57.8% vs. 70.7%, P < 0.001). In multivariable models, unhealthy alcohol use (adjusted β -2.74; 95% confidence interval: -4.23 to -1.25) and illicit drug use (adjusted β -3.51; 95% CI: -4.99 to -2.02) remained inversely associated with the percentage of QIs received.

CONCLUSIONS

Although the overall QOC for these HIV-infected Veteran patients was high, gaps persist for those with unhealthy alcohol and illicit drug use. Interventions that address substance use in HIV-infected patients may improve the QOC received.

Quality of comprehensive HIV care in underserved communities: does clinical training lead to improvement

The AIDS Education and Training Centers National Evaluation Center led collaborative research to evaluate whether Minority AIDS Initiative (MAI)-funded clinical training changes clinical practice. Chart abstraction and feedback (34 clinics; n = 530) were used to assess adherence to clinical practice guidelines, identify training needs, and assess change in clinical practice (14 clinics, n = 271). Generalized estimating equations were used to account for repeated measures within each clinic. At baseline, clinics displayed 49% (95% confidence interval [CI] = 44-53) adherence to clinical practice guidelines. After feedback associated with the baseline chart review and subsequent implementation of MAI-funded clinical training, an 11% increase (95% CI = 7-16) in adherence to clinical practice guidelines was observed. MAI-funded clinical training was associated with increased adherence to clinical practice guidelines for HIV care. Chart abstraction is useful to assess clinical practice, facilitate conversations about quality improvement, and evaluate the effectiveness of clinical training.

Electronic health information technology as a tool for improving quality of care and health outcomes for HIV/AIDS patients

PURPOSE

This paper presents research on the interplay of health information technology (HIT), quality improvement and progression of health status. The purpose of the research was to determine whether electronic exchange of health information impacts quality of care and, by extension, health outcomes of patients with HIV/AIDS. The research was supported as a demonstration project under the Information Technology Networks of Care Initiative sponsored by the U.S. Health Resources and Services Administration, HIV/AIDS Bureau, Special Projects of National Significance (SPNS). The City of Paterson, New Jersey, Department of Health and Human Services administered the project as the grant recipient, secured and managed through the City of Paterson's Ryan White Part A Program of Bergen and Passaic Counties.

METHODS

We implemented a web-based health information support system, e2, to facilitate rigorous quality improvement activities associated with care and treatment of HIV/AIDS patients. We used e2 to monitor patient care in the clinic setting. We observed five quality and two health status indicators relating to the care of 263 HIV/AIDS medical patients at three HIV/AIDS medical clinics from 2008 to 2010. The quality indicators conformed to HIV/AIDS Bureau (HAB) Groups 1 and 2 definitions of two or more CD4 T-cell counts performed in the measurement year, AIDS patients prescribed HAART, two or more medical visits in the measurement year, PCP prophylaxis administered to AIDS patients with CD4 T-cell counts <200, and adults screened for syphilis within the measurement year. CD4 T-cell count and viral load suppression indicators were used as health status indicators. Frequency analysis and logistic generalized estimating equations were used to assess correlation. We also assessed user satisfaction with e2 as a quality improvement tool using qualitative methods.

RESULTS

Availability of the e2 system was a leading contributing factor in successful quality management activities, leading to improved quality of care and health status of HIV/AIDS patients across all three clinics. Significant improvements were observed in three of the five quality indicators and in both of the two health status indicators.

CONCLUSIONS

We conclude that health information technology as a tool for rigorous application of quality improvement methods can positively impact quality of care and health outcomes. We found that health outcomes improved over time when quality improvement methods were initiated and facilitated by a responsive electronic information support system.

All teach, all learn, all improve?: the role of interorganizational learning in quality improvement collaboratives

BACKGROUND

Quality improvement collaboratives are an increasingly common strategy for implementing evidence-based practices in health care. However, research shows that many participating organizations do not achieve the level of performance improvement desired.

PURPOSE

This study examined the use of interorganizational learning activities (inter-OLAs) as an explanation for mixed performance improvement among collaborative participants. We tested the hypotheses that inter-OLA use is positively associated with participants' performance improvement and that this relationship is moderated by the use of intraorganizational learning activities (intra-OLAs) and quality-focused human resource (Q-HR) practices.

METHODOLOGY

We conducted a survey of organizational teams participating in 4 Institute for Healthcare Improvement Breakthrough Series collaboratives. Survey responses from 52 teams, regarding the use of inter-OLAs, intra-OLAs and Q-HR practices, were linked to performance improvement data obtained from the Institute for Healthcare Improvement and demographic data obtained from secondary sources.

FINDINGS

The more collaborative teams used inter-OLAs, the more their organizations' performance improved. Contrary to our hypothesis, the use of intra-OLAs did not moderate this relationship; teams' use of intra-OLAs added to, but did not multiply, the effect of inter-OLA use. In contrast, an organization's use of Q-HR practices multiplied the performance benefit of inter-OLA use.

PRACTICE IMPLICATIONS

Our findings suggest that organizations that participate in collaboratives are more likely to improve their performance if they use the inter-OLAs offered by the collaborative. Our results also suggest that complementing high use of inter-OLAs with intra-OLA use and Q-HR practices enhances performance improvement. For collaborative sponsors, our findings imply that including activities that facilitate interorganizational and intraorganizational learning are worthwhile.