Assessing the impact of continuous quality improvement on clinical practice: what it will take to accelerate progress

Can continuous quality improvement be assessed using randomized trials? [see comment]

STUDY QUESTION

Continuous quality improvement (CQI) has been implemented at least to some degree in many health care settings, yet randomized controlled trials (RCTs) of CQI are rare. We ask whether, when, and how RCTs of CQI might be designed.

STUDY DESIGN

We consider two applications of CQI: as a general philosophy of management and (by analogy with the use of conceptual models from the behavioral sciences) as a conceptual model for developing specific interventions. The example of warfarin therapy for stroke prevention among patients with atrial fibrillation is used throughout.

PRINCIPAL FINDINGS

While it is impractical to use RCTs to study CQI as a general management philosophy, RCT methodology is appropriate for studying CQI as a conceptual model for generating interventions. RCTs of CQI might be considered when the process change under consideration is very large, its implications (e.g., in terms of cost, outcomes of care, etc.) are very great, and the best approach is uncertain. When designing RCTs of CQI, critical decisions include (1) the unit of randomization; (2) whether the focus is on CQI as a method for generating interventions or, instead, is on specific interventions in and of themselves; and (3) the flexibility available to local personnel to modify the intervention's operational details.

CONCLUSIONS

RCTs of CQI as a conceptual model for generating interventions are feasible.

Total quality leadership in the U.S. Navy: effective for health promotion activities?

BACKGROUND

Although the U.S. Navy mandated Total Quality Leadership (TQL) as a management strategy throughout its medical department in the early 1990s, it was unknown to what extent it was being used for health promotion activities and, if so, how effectively.

METHODS

A brief mail survey of 204 Navy commands supplemented by 97 telephone interviews to TQL-for-health-promotion-using commands and nonusing controls provided worksite information on TQL implementation. Responses from a Navywide health and fitness survey provided perceptions and health behavior attitudes from the individuals at commands.

RESULTS

A total of 32% of commands surveyed had used TQL specifically for improving health- and fitness-related processes and outcomes between 1991 and 1995. Participants at commands that had used TQL for health- and fitness-related processes reported a higher importance of good health (P < 0.05) and were more certain that they would reach and or maintain their ideal weight (P < 0.05) than participants at non-TQL commands. However, there were no significant differences in perceptions of command support for health and fitness between TQL and non-TQL commands.

CONCLUSIONS

Several factors and organizational arrangements that were pertinent to the development and practice of TQL in the Navy were identified. The use of TQL specifically for health promotion was not consistently related to health-related perceptions or health behavior attitudes.

Reviewing audit: barriers and facilitating factors for effective clinical audit

OBJECTIVE

To review the literature on the benefits and disadvantages of clinical and medical audit, and to assess the main facilitators and barriers to conducting the audit process.

DESIGN

A comprehensive literature review was undertaken through a thorough review of Medline and CINAHL databases using the keywords of "audit", "audit of audits", and "evaluation of audits" and a handsearch of the indexes of relevant journals for key papers.

RESULTS

Findings from 93 publications were reviewed. These ranged from single case studies of individual audit projects through retrospective reviews of departmental audit programmes to studies of interface projects between primary and secondary care. The studies reviewed incorporated the experiences of a wide variety of clinicians, from medical consultants to professionals allied to medicine and from those involved in unidisciplinary and multidisciplinary ventures. Perceived benefits of audit included improved communication among colleagues and other professional groups, improved patient care, increased professional satisfaction, and better administration. Some disadvantages of audit were perceived as diminished clinical ownership, fear of litigation, hierarchical and territorial suspicions, and professional isolation. The main barriers to clinical audit can be classified under five main headings. These are lack of resources, lack of expertise or advice in project design and analysis, problems between groups and group members, lack of an overall plan for audit, and organisational impediments. Key facilitating factors to audit were also identified: they included modern medical records systems, effective training, dedicated staff, protected time, structured programmes, and a shared dialogue between purchasers and providers.

CONCLUSIONS

Clinical audit can be a valuable assistance to any programme which aims to improve the quality of health care and its delivery. Yet without a coherent strategy aimed at nurturing effective audits, valuable opportunities will be lost. Paying careful attention to the professional attitudes highlighted in this review may help audit to deliver on some of its promise.

The art and science of chart review

BACKGROUND

Explicit chart review was an integral part of an ongoing national cooperative project, "Using Achievable Benchmarks of Care to Improve Quality of Care for Outpatients with Depression," conducted by a large managed care organization (MCO) and an academic medical center. Many investigators overlook the complexities involved in obtaining high-quality data. Given a scarcity of advice in the quality improvement (QI) literature on how to conduct chart review, the process of chart review was examined and specific techniques for improving data quality were proposed.

METHODS

The abstraction tool was developed and tested in a prepilot phase; perhaps the greatest problem detected was abstractor assumption and interpretation. The need for a clear distinction between symptoms of depression or anxiety and physician diagnosis of major depression or anxiety disorder also became apparent. In designing the variables for the chart review module, four key aspects were considered: classification, format, definition, and presentation. For example, issues in format include use of free-text versus numeric variables, categoric variables, and medication variables (which can be especially challenging for abstraction projects). Quantitative measures of reliability and validity were used to improve and maintain the quality of chart review data. Measuring reliability and validity offers assistance with development of the chart review tool, continuous maintenance of data quality throughout the production phase of chart review, and final documentation of data quality. For projects that require ongoing abstraction of large numbers of clinical records, data quality may be monitored with control charts and the principles of statistical process control.

RESULTS

The chart review module, which contained 140 variables, was built using MedQuest software, a suite of tools designed for customized data collection. The overall interrater reliability increased from 80% in the prepilot phase to greater than 96% in the final phase (which included three abstractors and 465 unique charts). The mean time per chart was calculated for each abstractor, and the maximum value was 13.7 +/- 13 minutes.

CONCLUSIONS

In general, chart review is more difficult than it appears on the surface. It is also project specific, making a "cookbook" approach difficult. Many factors, such as imprecisely worded research questions, vague specification of variables, poorly designed abstraction tools, inappropriate interpretation by abstractors, and poor or missing recording of data in the chart, may compromise data quality.

Getting new evidence into medicine

The rate of transfer of the knowledge gained from health and medical research into evidence-based practice is determined by many factors. Preconditions for the uptake of new evidence are the availability of good evidence, ready access to the evidence, a supportive organisational environment, and effective mechanisms for promoting knowledge uptake. Evidence-based medicine is being promoted in Australia by a body of enthusiastic clinicians, public health practitioners and consumer advocates, supported by initiatives from national, State and local healthcare services and professional bodies. The short to medium term future of evidence-based medicine in Australia is likely to be shaped by three major factors: a reduction in cost and technical barriers which limit access to computerised databases; a trend towards shared decision-making between clinicians and patients; and increased demand for information to fill the gaps in research-based evidence on specific problems.

Assessing the impact of total quality management and organizational culture on multiple outcomes of care for coronary artery bypass graft surgery patients

OBJECTIVES

To assess the impact of total quality management (TQM) and organizational culture on a comprehensive set of endpoints of care for coronary artery bypass graft surgery (CABG) patients, including risk-adjusted adverse outcomes, clinical efficiency, patient satisfaction, functional health status, and cost of care.

METHODS

Prospective cohort study of 3,045 eligible CABG patients from 16 hospitals using risk-adjusted clinical outcomes, functional health status, patient satisfaction, and cost measures. Implementation of TQM was measured by a previously validated instrument based on the Baldridge national quality award criteria. Organizational culture was measured by a previously validated 20-item instrument. Generalized estimating equations were used to control for potential selection bias, repeated measures, and intraclass correlation.

RESULTS

A 2- to 4-fold difference in all major clinical CABG care endpoints was observed among the 16 hospitals, but little of this variation was associated with TQM or organizational culture. Patients receiving CABG from hospitals with high TQM scores were more satisfied with their nursing care (P = 0.005) but were more likely to have lengths of stay >10 days (P = 0.0003). A supportive group culture was associated with shorter postoperative intubation times (P = 0.01) but longer operating room times (P = 0.004). A supportive group culture was also associated with higher patient physical (P = 0.005) and mental (P = 0.01) functional health status scores 6 months after CABG.

CONCLUSIONS

There was little effect of TQM and organizational culture on multiple endpoints of care for CABG patients. There is a need to examine further the relationships among individual professional skills and motivations, group and microsystem team processes, specifically tailored interventions, and organization-wide culture, decision support processes, and incentives. Assessing the impact of such multifaceted approaches is an important area for further research.

Benchmarking and quality improvement: the Harvard Emergency Department Quality Study

PURPOSE

To determine whether feedback of comparative information was associated with improvement in medical record and patient-based measures of quality in emergency departments.

SUBJECTS AND METHODS

During 1-month study periods in 1993 and 1995, all medical records for patients who presented to five Harvard teaching hospital emergency departments with one of six selected chief complaints (abdominal pain, shortness of breath, chest pain, hand laceration, head trauma, or vaginal bleeding) were reviewed for the percent compliance with process-of-care guidelines. Patient-reported problems and patient ratings of satisfaction with emergency department care were collected from eligible patients using patient questionnaires. After reviewing benchmark information, emergency department directors designed quality improvement interventions to improve compliance with the process-of-care guidelines and improve patient-reported quality measures.

RESULTS

In the preintervention period, 4,876 medical records were reviewed (99% of those eligible), 2,327 patients completed on-site questionnaires (84% of those eligible), and 1,386 patients completed 10-day follow-up questionnaires (80% of a random sample of eligible participants). In the postintervention period, 6,005 medical records were reviewed (99% of those eligible), 2,899 patients completed on-site questionnaires (84% of those eligible), and 2,326 patients completed 10-day follow-up questionnaires (80% of all baseline participants). In multivariate analyses, adjusting for age, urgency, chief complaint, and site, compliance with process-of-care guidelines increased from 55.9% (preintervention) to 60.4% (postintervention, P = 0.0001). We also found a 4% decrease (from 24% to 20%) in the rate of patient-reported problems with emergency department care (P = 0.0001). There were no significant improvements in patient ratings of satisfaction.

CONCLUSION

Feedback of benchmark information and subsequent quality improvement efforts led to small, although significant, improvement in compliance with process-of-care guidelines and patient-reported measures of quality. The measures that relied on patient reports of problems with care, rather than patient ratings of satisfaction with care, seemed to be more responsive to change. These results support the value of benchmarking and collaboration.

Developing effective clinical audit

Clinical audit received a mixed press when it was formalized as part of the health-care reforms in the early 1990's, and has continued to mixed reviews ever since. However, the latest governmental stipulations on clinical governance envisage a revitalized role for clinical audit. This paper advises on avoiding some of the pitfalls that await the unwary in developing effective clinical audit projects.

Musings on using evidence to guide CQI efforts toward success: the computerized firm system as primary care microunit

BACKGROUND

The attempt to transfer classic industrial CQI (continuous quality improvement) theory into the clinical arena has proved to be more difficult than originally promised. A new "computerized firm system" approach to incorporating CQI efforts into mainstream practice settings, which has been able to obviate many of these shortcomings, is described.

METHODS

To make it easier for CQI efforts to be successful, the scope of activities undertaken in completing the Shewhart cycle popularly referred to as PDSA (plan change, do change, study results, act on results) was delimited. Rather than plan the intervention themselves, staff worked with experts on tailoring a preselected change idea with already established efficacy--a computerized reminder system. Because the clinic was divided into two small group practices known as firms, a controlled time-series trial (CTST) design was used by initially turning the reminders on for one firm but not the other. The clinic was thereby also relieved of the responsibility of conducting a study to determine whether the intended improvement in quality had been achieved. In essence, one clinic was asked to do just DA (that is, do-act).

RESULTS

This approach engendered the successful completion of a streamlined Shewhart cycle in a busy clinic setting at remarkably low cost. The compelling nature of controlled evaluation results aided leadership in rapidly disseminating the reminder system to the remaining 11 primary care clinics associated with the university's 2 academic medical centers.

CONCLUSION

Computerized firm systems can be developed to conduct CTSTs as part of streamlined CQI cycles guided by both published and local evidence, and they are worth developing.

Evidence-based implementation of evidence-based medicine

BACKGROUND

The slow and haphazard process of translating research findings into clinical practice compromises the potential benefits of clinical research. Most quality improvement (QI) initiatives are based on the beliefs of decision makers rather than on the growing theoretical and empirical knowledge about organizational and provider behavior change. If future QI activities are to improve the translation of evidence into practice, they should be based on an understanding of the different models and strategies for implementing research evidence and the evidence base supporting their use. Evidence-based medicine should be complemented by evidence-based implementation.

THE EVIDENCE FOR DIFFERENT STRATEGIES OF IMPLEMENTING CHANGE

A general framework for changing practice based on theoretical perspectives and research evidence considers a variety of theoretical approaches and their contribution to an understanding of provider behavior change. The framework summarizes evidence from systematic reviews of provider behavior change, which suggest the potential of several dissemination and implementation strategies that are effective under certain conditions. Passive dissemination approaches are largely ineffective; specific strategies to implement research-based recommendations appear to be necessary to ensure practice change. Multifaceted interventions that address specific barriers to change are more likely to lead to changes in practice. PRACTICAL, FIVE-STAGE FRAMEWORK: A practical, five-stage framework for changing practice, which is illustrated with experiences from a comprehensive program on implementing evidence-based clinical guidelines in primary care, includes development of a concrete proposal for change; analysis of the target setting and group to identify obstacles to change; linking interventions to needs, facilitators, and obstacles to change; development of an implementation plan; and monitoring progress with implementation.

Clinical governance: bridging the gap between managerial and clinical approaches to quality of care

Clinical governance has been introduced as a new approach to quality improvement in the UK national health service. This article maps clinical governance against a discussion of the four main approaches to measuring and improving quality of care: quality assessment, quality assurance, clinical audit, and quality improvement (including continuous quality improvement). Quality assessment underpins each approach. Whereas clinical audit has, in general, been professionally led, managers have driven quality improvement initiatives. Quality assurance approaches have been perceived to be externally driven by managers or to involve professional inspection. It is discussed how clinical governance seeks to bridge these approaches. Clinical governance allows clinicians in the UK to lead a comprehensive strategy to improve quality within provider organisations, although with an expectation of greatly increased external accountability. Clinical governance aims to bring together managerial, organisational, and clinical approaches to improving quality of care. If successful, it will define a new type of professionalism for the next century. Failure by the professions to seize the opportunity is likely to result in increasingly detailed external control of clinical activity in the UK, as has occurred in some other countries.