Assessing the impact of continuous quality improvement on clinical practice: what it will take to accelerate progress

The quantitative measurement of organizational culture in health care: a review of the available instruments

OBJECTIVE

To review the quantitative instruments available to health service researchers who want to measure culture and cultural change.

DATA SOURCES

A literature search was conducted using Medline, Cinahl, Helmis, Psychlit, Dhdata, and the database of the King's Fund in London for articles published up to June 2001, using the phrase "organizational culture." In addition, all citations and the gray literature were reviewed and advice was sought from experts in the field to identify instruments not found on the electronic databases. The search focused on instruments used to quantify culture with a track record, or potential for use, in health care settings.

DATA EXTRACTION

For each instrument we examined the cultural dimensions addressed, the number of items for each questionnaire, the measurement scale adopted, examples of studies that had used the tool, the scientific properties of the instrument, and its strengths and limitations.

PRINCIPAL FINDINGS

Thirteen instruments were found that satisfied our inclusion criteria, of which nine have a track record in studies involving health care organizations. The instruments varied considerably in terms of their grounding in theory, format, length, scope, and scientific properties.

CONCLUSIONS

A range of instruments with differing characteristics are available to researchers interested in organizational culture, all of which have limitations in terms of their scope, ease of use, or scientific properties. The choice of instrument should be determined by how organizational culture is conceptualized by the research team, the purpose of the investigation, intended use of the results, and availability of resources.

Sustainability in changing clinical practice promotes evidence-based nursing care

AIM

To examine the relationship between sustained work with quality improvement (QI) and factors related to research utilization in a group of nurses.

DESIGN

The study was designed as a comparative survey that included 220 nurses from various health care organizations in Sweden. These nurses had participated in uniformly designed 4-day basic training courses to manage a method for QI.

METHODS

A validated questionnaire covering different aspects of research utilization was employed. The response rate was 70% (154 of 220). Nurses in managerial positions at the departmental level were excluded. Therefore, the final sample consisted of 119 respondents. Four years after the training courses, 39% were still involved in audit-related activities, while 61% reported that they had discontinued the QI work (missing = 1).

RESULTS

Most nurses (80-90%) had a positive attitude to research. Those who had continued the QI work over a 4-year period reported more activity in searching research literature compared with those who had discontinued the QI work (P = 0.005). The QI-sustainable nurses also reported more frequent participation in research-related activities, particularly in implementing specific research findings in practice (P = 0.001). Some contextual differences were reported: the QI-sustainable nurses were more likely to obtain support from their chief executive (P = 0.001), consultation from a skilled researcher (P = 0.005) and statistical support (P = 0.001). Within the broader health care organization, the existence of a research committee and a research and development strategy, as well as access to research assistant staff, had a tendency to be more common for nurses who had continued the QI work.

CONCLUSION

Sustainability in QI work was significantly related to supportive leadership, facilitative human resources, increased activity in seeking new research and enhanced implementation of research findings in clinical practice. It appears that these factors constitute a necessary prerequisite for professional development and the establishment of evidence-based practice.

Do local opinion leaders augment hospital quality improvement efforts? A randomized trial to promote adherence to unstable angina guidelines

BACKGROUND

The influence of an opinion leader intervention on adherence to Unstable Angina (UA) guidelines compared with a traditional quality improvement model was investigated.

RESEARCH DESIGN

A group-randomized controlled trial with 2210 patients from 21 hospitals was designed. There were three intervention arms: (1) no intervention (NI); (2) a traditional Health Care Quality Improvement Program (HCQIP); and (3) a physician opinion leader in addition to the HCQIP model (OL). Quality indicators included: electrocardiogram within 20 minutes, antiplatelet therapy within 24 hours and at discharge, and heparin and beta-blockers during hospitalization. Hospitals could determine the specific indicators they wished to target. Potential cases of UA were identified from Medicare claims data. UA confirmation was determined by a clinical algorithm based on data abstracted from medical records. Data analyses included both hospital level analysis (analysis of variance) and patient level analysis (generalized linear models).

RESULTS

The only statistically significant postintervention difference in percentage compliant was greater improvement for the OL group in the use of antiplatelet therapy at 24 hours in both hospital level (P = 0.01) and patient level analyses (P <0.05) compared with the HCQIP and NI groups. When analyses were confined to hospitals that targeted specific indicators, compared with the HCQIP hospitals, the OL hospitals showed significantly greater change in percentage compliant postintervention in both antiplatelet therapy during the first 24 hours (20.2% vs. -3.9%, P = 0.02) and heparin (31.0% vs.9.1%, P = 0.05).

CONCLUSIONS

The influence of physician opinion leaders was unequivocally positive for only one of five quality indicators. To maximize adherence to best practices through physician opinion leaders, more research on how these physicians influence health care delivery in their organizations will be required.

Perspectives on the value of American Society of Clinical Oncology clinical guidelines as reported by oncologists and health maintenance organizations

PURPOSE

Although the American Society of Clinical Onoclogy's (ASCO) Health Services Research (HSR) committee activities have primarily focused on clinical guideline development, little is known about the value placed on these guidelines by the desired end users. ASCO members and Health Maintenance Organizations (HMOs) were surveyed on the value and implementation of ASCO guidelines. In this article, we summarize our findings.

METHODS

ASCO members (n = 1500) were queried about whether they had read ASCO's first four clinical guidelines and technology assessment; whether they agreed with the recommendations; whether they used guidelines in clinical practice; and how guidelines had affected reimbursement. HMOs (n = 131) were queried on how they identify, implement, and value the first four ASCO clinical guidelines.

RESULTS

The membership survey indicated that ASCO guidelines were read more often by physicians in private healthcare settings compared with physicians in academic practices (P <.02). Disagreement rates were low for all guidelines (range, 1% to 7%). One quarter of respondents reported that the guidelines were difficult to find and difficult to apply to the practice setting, and approximately one tenth of respondents indicated that the guidelines were difficult to evaluate, interpret, or read. The HMO survey indicated that one third of HMOs reported use of ASCO guidelines, with higher rates of usage by larger HMOs and by those with higher National Committee on Quality Assurance (NCQA) ratings. Respondent HMOs valued guidelines for various purposes and used multiple methods of guideline identification and implementation.

CONCLUSION

ASCO guidelines are generally highly supported by physicians and HMOs. Additional studies are needed to identify implementation barriers and to see whether guidelines have resulted in improvements in healthcare.

Quality improvement implementation in the nursing home

OBJECTIVE

To examine quality improvement (QI) implementation in nursing homes, its association with organizational culture, and its effects on pressure ulcer care.

DATA SOURCES/STUDY SETTING

Primary data were collected from staff at 35 nursing homes maintained by the Department of Veterans Affairs (VA) on measures related to QI implementation and organizational culture. These data were combined with information obtained from abstractions of medical records and analyses of an existing database.

STUDY DESIGN

A cross-sectional analysis of the association among the different measures was performed.

DATA COLLECTION/EXTRACTION METHODS

Completed surveys containing information on QI implementation, organizational culture, employee satisfaction, and perceived adoption of guidelines were obtained from 1,065 nursing home staff. Adherence to best practices related to pressure ulcer prevention was abstracted from medical records. Risk-adjusted rates of pressure ulcer development were calculated from an administrative database.

PRINCIPAL FINDINGS

Nursing homes differed significantly (p<.001) in their extent of QI implementation with scores on this 1 to 5 scale ranging from 2.98 to 4.08. Quality improvement implementation was greater in those nursing homes with an organizational culture that emphasizes innovation and teamwork. Employees of nursing homes with a greater degree of QI implementation were more satisfied with their jobs (a 1-point increase in QI score was associated with a 0.83 increase on the 5-point satisfaction scale, p<.001) and were more likely to report adoption of pressure ulcer clinical guidelines (a 1-point increase in QI score was associated with a 28 percent increase in number of staff reporting adoption, p<.001). No significant association was found, though, between QI implementation and either adherence to guideline recommendations as abstracted from records or the rate of pressure ulcer development.

CONCLUSIONS

Quality improvement implementation is most likely to be successful in those VA nursing homes with an underlying culture that promotes innovation. While QI implementation may result in staff who are more satisfied with their jobs and who believe they are providing better care, associations with improved care are uncertain.

What do collaborative improvement projects do? Experience from seven countries

BACKGROUND

Health care organizations are increasingly adopting multiorganizational collaborative approaches to quality improvement. Collaboratives have been conducted in many countries. There are large variations in the way collaboratives are structured and run, but there is no widely accepted framework for describing the components of collaboratives. Thus, it is difficult to study which approaches are most effective.

METHOD

The authors conducted semistructured interviews with 15 leaders of collaboratives to ascertain the common components of collaboratives and identify variations in the ways these components are implemented.

RESULTS

The study identified seven features of collaboratives that the leaders interviewed thought were critical determinants of how effective the collaboratives were: sponsorship, topic, ideas for improvements, participants, senior leadership support, preliminary work and learning, and strategies for learning about and making improvements. For example, every interviewee mentioned that having participants collect data, perform audit work, or analyze the system they were in before the collaboration started was important to understanding their organization and the nature of the problems they had and to developing baseline data for later comparison. The authors describe variations in how these features have been implemented and possible functions of these features.

CONCLUSION

Systematically studying the impact of variations in the seven key features of collaboratives could yield important information about their role and impact.

A research agenda to advance quality measurement and improvement

BACKGROUND

In developing a conceptual framework for the design of a national quality measurement and reporting system (NQMRS), the Strategic Framework Board (SFB) recommends that such a system be built on a strong evidence base.

OBJECTIVES

To identify critical gaps in the evidence needed for a fully functional NQMRS and to recommend a starting point for the development of a research agenda.

RESEARCH DESIGN

Selective review of literature in quality of care measurement and reporting and identification of strategic issues that must be addressed.

FINDINGS

There is some limited evidence that measurement and reporting can improve quality. Substantial advances have been made in the science of measurement and reporting but important gaps remain, specifically in (1) measurement methods and tools, (2) uses of quality performance data, (3) organizational and cultural factors, (4) information and informatics, and (5) impact evaluation/research.

CONCLUSIONS

To achieve a sustainable research agenda, three strategic issues will have to be addressed: (1) the policy rationale for the research agenda, (2) adequate levels of public-sector funding, and (3) sustainability in a rapidly changing environment.

Using performance measurement to drive improvement: a road map for change

BACKGROUND

Performance measures and reporting have not been adopted throughout the US health care system despite their central role in encouraging increased participation by consumers in decision-making. Understanding whether the failure of measurement and reporting to diffuse throughout the health system can be overcome is critical for determining future policy in this area.

OBJECTIVES

To create a conceptual framework for analyzing the current rate of adoption and evaluating alternatives for accelerating adoption, and to recommend a set of concrete steps that can be taken to increase the use of performance measurement and reporting.

RESEARCH DESIGN

Review of three theoretic models (Rogers, Prochaska/DiClemente, Gladwell), examination of the literature on previous experiences with quality measurement and reporting, and interviews with select stakeholders.

FINDINGS

The three theoretic models provide a valuable framework for understanding why the use of performance measures is stalled ("the circle of unaccountability") and for generating ideas about concrete steps that could be taken to accelerate adoption. Six steps are recommended: (1) raise public awareness, (2) redesign measures and reports, (3) make the delivery of information timely, (4) require public reporting, (5) develop and implement systems to reward quality, and (6) actively court leaders.

CONCLUSIONS

The recommended six steps are interconnected; action on all will be required to drive significant acceleration in rates of adoption of performance measurement and reporting. Leadership and coordination are necessary to ensure these steps are taken and that they work in concert with one another.

Strengthening research to improve the practice and management of long-term care

Past investments in long-term care (LTC) research have improved the quality of care and the quality of life for LTC recipients by conceptualizing the goals and measuring the outcomes of care, designing practical assessment tools, testing clinical interventions, and evaluating new service delivery programs and models. To build a balanced portfolio of LTC research that will yield and sustain increased dividends in quality and outcomes will require (1) increasing investment in both basic and applied LTC research to ensure that critical service delivery issues are addressed in a rigorous and timely fashion, (2) fostering better communication between researchers and users to ensure research salience and credibility, and (3) dedicating more resources to identifying and implementing successful methods for translating LTC research into practice.

Progress of unit based quality improvement: an evaluation of a support strategy

OBJECTIVE

To evaluate a strategy for supporting nurses to work with quality improvement (QI).

DESIGN

Post-intervention evaluation. Study participants and intervention: 240 nurses participated in a uniformly designed 4 day basic training course in applying a model for QI. Of these, 156 nurses from over 50 healthcare institutions constituted the generic education (GE) group while 84 nurses from 42 neonatal units took part in a project to develop national guidelines, constituting the targeted intervention (TI) group.

METHOD

Postal questionnaire 4 years after the training courses.

RESULTS

The response rate was 80% in the TI group and 64% in the GE group. Nurses in the TI group had a significantly higher rate in completing all phases of the QI cycle (p=0.0002). With no differences between the groups, 39% of all nurses were still involved in QI work 4 years after the training courses. Three factors were significantly related to nurses continuing their involvement in QI projects: remaining employed on the same unit (OR 11.3), taking courses in nursing science (OR 4.1), and maintenance of the same QI model (OR 3.1). Reported motives for remaining active in QI work were the enhancement of knowledge, influence over clinical practice, and development as a nurse. Reasons for discontinuation were organisational restructuring, a lack of facilitation and knowledge, and change of workplace.

CONCLUSIONS

Participation in a national guideline project, including a common focus for improvement, facilitation and opportunities for networking, seems to have enhanced the ability to carry out the process of QI, but not to sustain the QI work over a longer period.

Development of a healthcare quality improvement measurement tool: results of a content validity study

Current methods of measuring continuous quality improvement (CQI) implementation are too long and not comprehensive. A new survey for CQI implementation was developed and tested for content validity using a panel of 8 experts--7 from the United States and 1 from England. The survey was reduced from 70 items to 22. The resultant survey had a clarity interrater agreement (IR) of .91, a representativeness IR of .93, a clarity content validity index (CVI) of .73, and a representativeness CVI of .91. Content validity served as an excellent data reduction method in building a valid, concise, and comprehensive measure of CQI implementation.

Public health quality measurement: concepts and challenges

Public health agencies increasingly are recognizing the need to formally and quantitatively assess and improve the quality of their programs, information, and policies. Measuring quality can help organizations monitor their progress toward public health goals and become more accountable to both the populations they serve and policy makers. Yet quality assessment is a complex task that involves precise determination and specification of useful measures. We discuss a well-established conceptual framework for organizing quality assessment in the context of planning and delivery of programs and services by local health departments, and consider the strengths and limitations of this approach for guiding quality improvement. We review several past and present quality measurement-related initiatives designed for public health department use, and discuss current and future challenges in this evolving area of public health practice.

The Patient Visits Program: a strategy to highlight patient satisfaction and refocus organizational culture

BACKGROUND

Seeking patient input may improve patients' perceptions of the quality of care and provide managers with helpful information for strategic decision making. In addition, the involvement of senior hospital leadership is critical to successful implementation of quality improvement initiatives and illustrates an organization's commitment to enhancing quality from the top down. IMPLEMENTING THE PVP: Senior management's Patient Visits Program (PVP) at Tufts-New England Medical Center is a structured, ongoing initiative in which senior clinicians are paired with nonclinician administrators. During an initial evaluation period (Aug 1999-Feb 2001), PVP teams visited with patients and their families on a monthly basis to talk to them about their experiences. Patient suggestions were then evaluated and acted on.

DISCUSSION

The PVP has been beneficial for patients and for the hospital team members--clinicians and nonclinicians alike--who participated in the patient interviews. The PVP may serve as a mechanism to enhance organizational awareness of the importance of patient satisfaction. The program provides opportunities for immediate service recovery, and faster, broader-reaching responses to quality complaints due to the multispecialty nature of the PVP teams. In addition, based on early available data, the PVP shows promise as an interventional strategy to improve patient satisfaction scores.

CONCLUSIONS

A structured, ongoing program such as the PVP is an effective strategy to highlight the value of patient satisfaction, refocus organizational culture, and generate specific suggestions for improving the quality of patient care.

The combined effect of public profiling and quality improvement efforts on heart failure management

BACKGROUND

A before-and-after study was conducted to examine the combined effect of public profiling and quality improvement activities on management of heart failure (HF) in the hospital setting.

METHODS

Thirty-one hospitals in southeastern Michigan participated in this profiling and quality improvement study. One hospital closed after the baseline measurement. Two quality indicators were developed to evaluate the key processes of HF care, and one profiling indicator was designed for public profiling. The baseline results of the profiling indicator were publicly released. The individual hospitals were identified in the profiling report by name as "having statistically higher (or lower) rates than average." Remeasurement results were compared to the baseline results by using t-tests for the individual hospitals and all 30 hospitals as an aggregate.

RESULTS

Two-thirds of the hospitals improved ejection fraction documentation; the aggregate result improved 5.4 percentage points (p < 0.05). No change was observed in the aggregate measure of prescribing angiotensin-converting enzyme inhibitors (ACEIs) to eligible HF patients at discharge. Hospitals with low baseline rates made improvement in ACEI use at discharge, but those with good baseline performance tended to decline in performance. There was a 2.2 percentage point increase (p < 0.05) in the profiling indicator.

SUMMARY AND CONCLUSIONS

There seemed to be differential impacts of interventions across indicators and hospitals. Public profiling may have the most positive impact on hospitals with low performance at baseline. Maintaining the baseline good practice was a struggle for hospitals with relatively high baseline rates.

Evaluation of quality improvement programmes

In response to increasing concerns about quality, many countries are carrying out large scale programmes which include national quality strategies, hospital programmes, and quality accreditation, assessment and review processes. Increasing amounts of resources are being devoted to these interventions, but do they ensure or improve quality of care? There is little research evidence as to their effectiveness or the conditions for maximum effectiveness. Reasons for the lack of evaluation research include the methodological challenges of measuring outcomes and attributing causality to these complex, changing, long term social interventions to organisations or health systems, which themselves are complex and changing. However, methods are available which can be used to evaluate these programmes and which can provide decision makers with research based guidance on how to plan and implement them. This paper describes the research challenges, the methods which can be used, and gives examples and guidance for future research. It emphasises the important contribution which such research can make to improving the effectiveness of these programmes and to developing the science of quality improvement.

What patient population does visit-based sampling in primary care settings represent?

BACKGROUND

Evaluations of outpatient interventions often rely on consecutive sampling of clinic visitors, and assume that study results generalize to the population of patients cared for.

OBJECTIVE

The representativeness of such visit-based sampling compared with the population of patients seen during the same year, in terms of sociodemographic and clinical characteristics of the user groups that visit-based sampling yielded were assessed.

METHODS

One thousand five hundred forty-six continuing patients visiting the primary care firms in an urban VA medical center were consecutively sampled, and visit frequencies were compared for these patients with subsets of the patient population. Administrative and survey data was then used to describe the types of patients visit-based sampling most represented compared with the types of patients sampled less frequently.

RESULTS

The average sampled patient visited the firms significantly more often than patients in the reference population (18.7 vs. 9.5). Sampled patients were significantly older (>55 years), in poorer health (higher prevalence of cancer, stroke, hypertension), less likely to smoke, and more likely to be single than the average patient visiting the firms (P<0.05). Adjusting for age and sickness, frequent visitors were more apt to have experienced continuity of care during the prior year, to prefer VA care, and to be unemployed.

CONCLUSIONS

Consecutive visit-based sampling actually selected patients with a visit pattern more typical of the patient population visiting four or more times a year. Studies using sampling of consecutive visitors will typically under-represent low users of care and should account for the degree to which results may not generalize to the broader practice population.

A qualitative study of the cultural changes in primary care organisations needed to implement clinical governance

BACKGROUND

It is commony claimed that changing the culture of health organisations is a fundamental prerequisite for improving the National Health Service (NHS). Little is currently known about the nature or importance of culture and cultural change in primary care groups and trusts (PCG/Ts) or their constituent general practices.

AIMS

To investigate the importance of culture and cultural change for the implementation of clinical governance in general practice by PCG/Ts, to identify perceived desirable and undesirable cultural attributes of general practice, and to describe potential facilitators and barriers to changing culture.

DESIGN

Qualitative: case studies using data derived from semi-structured interviews and review of documentary evidence.

SETTING

Fifty senior non-clinical and clinical managers from 12 purposely sampled PCGs or trusts in England.

RESULTS

Senior primary care managers regard culture and cultural change as fundamental aspects of clinical governance. The most important desirable cultural traits were the value placed on a commitment to public accountability by the practices, their willingness to work together and learn from each other, and the ability to be self-critical and learn from mistakes. The main barriers to cultural change were the high level of autonomy of practices and the perceived pressure to deliver rapid measurable changes in general practice.

CONCLUSIONS

The culture of general practice is perceived to be an important component of health system reform and quality improvement. This study develops our understanding of a changing organisational culture in primary care; however, further work is required to determine whether culture is a useful practical lever for initiating or managing improvement.

Understanding team-based quality improvement for depression in primary care

OBJECTIVE

To assess the impacts of the characteristics of quality improvement (QI) teams and their environments on team success in designing and implementing high quality, enduring depression care improvement programs in primary care (PC) practices.

STUDY SETTING/DATA SOURCES

Two nonprofit managed care organizations sponsored five QI teams tasked with improving care for depression in large PC practices. Data on characteristics of the teams and their environments is from observer process notes, national expert ratings, administrative data, and interviews.

STUDY DESIGN

Comparative formative evaluation of the quality and duration of implementation of the depression improvement programs developed by Central Teams (CTs) emphasizing expert design and Local Teams (LTs) emphasizing participatory local clinician design, and of the effects of additional team and environmental factors on each type of team. Both types of teams depended upon local clinicians for implementation.

PRINCIPAL FINDINGS

The CT intervention program designs were more evidence-based than those of LTs. Expert team leadership, support from local practice management, and support from local mental health specialists strongly influenced the development of successful team programs. The CTs and LTs were equally successful when these conditions could be met, but CTs were more successful than LTs in less supportive environments.

CONCLUSIONS

The LT approach to QI for depression requires high local support and expertise from primary care and mental health clinicians. The CT approach is more likely to succeed than the LT approach when local practice conditions are not optimal.

Why is quality assurance so difficult? A review of issues in quality assurance over the last decade

The quality assurance movement has been unable to produce major improvements in the realm of clinical practice because of an inability to make satisfactory measurements of process and/or outcome, together with the intrinsic difficulties associated with producing change. Progress in both these areas is likely to be slow. Improvements in the quality of care occurring as a result of the introduction of information technology into clinical practice may be seen more quickly.

National sentinel clinical audit of evidence-based prescribing for older people: methodology and development

RATIONALE, AIMS AND OBJECTIVES

This national clinical audit aimed to develop and implement a methodology to assess the appropriateness of prescribing for patients over the age of 65 in hospitals, general practice and nursing homes.

METHODS

Organizations providing health care in the National Health Service in these three sectors were recruited into multi-disciplinary and inter-organizational local coalition teams. Prescription data and relevant clinical data were collected electronically on a customized database. The appropriateness of prescribing for specific conditions among the patients sampled was assessed by simple computerized algorithms, and users were provided with feedback to stimulate discussion and change. Use of the software tool was demonstrated to be feasible and its data reliable. Participants were re-audited, after a period of nationally guided and locally driven intervention, to evaluate levels of change. Local efforts to stimulate change and barriers to change were collected qualitatively.

RESULTS AND CONCLUSIONS

The investigation revealed encouraging results and demonstrated the ability of audit to improve the quality of clinical services in given circumstances, although a multiplicity of questions relating to cost and methodology remain to be addressed.

Evidence-based management: using serial firm trials to improve diabetes care quality

BACKGROUND

The design of delivery systems that can truly conduct continuous quality improvement (CQI) as a routine part of clinical care provision remains a vexing problem. The effectiveness of the "computerized firm system" approach to chronic disease CQI was examined, with diabetes as the focus of a 5-year case study.

METHODS

A large family medical center had been divided into two parallel group practices for reasons of efficiency. These frontline structures (also known as primary care "firms") were supported to serially adapt and evaluate selected CQI interventions by first introducing process changes on one firm but not the other and comparing the groups. Because all the required longitudinal data were contained in a computerized repository, it was possible to conduct these controlled "firm trials" in a matter of months at low cost.

RESULTS

During a 3-year period, implementation of point-of-service reminders and a pharmacist out-reach program increased recommended glycohemoglobin (HbA1c) testing by 50% (p = 0.02) and reduced the number of diabetic patients inadequately controlled by 43% (p < 0.01). Following this outcome improvement, patients exhibited a 16% reduction in ambulatory visit rates (p = 0.04). The observed outcome improvement, however, was reversed during the subsequent 2 years, when staffing austerities forced by unrelated declines in clinic revenue caused the withdrawal of trial interventions.

CONCLUSIONS

The processes and outcomes of diabetes care were improved, demonstrating that CQI and controlled trials are not mutually exclusive in moving toward the practice of evidence-based management. Health care systems can, by conducting serial firm trials, become learning organizations. CQI programs of all kinds will likely never flourish, however, until quality improvement and reimbursement mechanisms have become better aligned.

Implementing clinical governance in English primary care groups/trusts: reconciling quality improvement and quality assurance

OBJECTIVES

To investigate the concept of clinical governance being advocated by primary care groups/trusts (PCG/Ts), approaches being used to implement clinical governance, and potential barriers to its successful implementation in primary care.

DESIGN

Qualitative case studies using semi-structured interviews and documentation review.

SETTING

Twelve purposively sampled PCG/Ts in England.

PARTICIPANTS

Fifty senior staff including chief executives, clinical governance leads, mental health leads, and lay board members.

MAIN OUTCOME MEASURES

Participants' perceptions of the role of clinical governance in PCG/Ts.

RESULTS

PCG/Ts recognise that the successful implementation of clinical governance in general practice will require cultural as well as organisational changes, and the support of practices. They are focusing their energies on supporting practices and getting them involved in quality improvement activities. These activities include, but move beyond, conventional approaches to quality assessment (audit, incentives) to incorporate approaches which emphasise corporate and shared learning. PCG/Ts are also engaged in setting up systems for monitoring quality and for dealing with poor performance. Barriers include structural barriers (weak contractual levers to influence general practices), resource barriers (perceived lack of staff or money), and cultural barriers (suspicion by practice staff or problems overcoming the perceived blame culture associated with quality assessment).

CONCLUSION

PCG/Ts are focusing on setting up systems for implementing clinical governance which seek to emphasise developmental and supportive approaches which will engage health professionals. Progress is intentionally incremental but formidable challenges lie ahead, not least reconciling the dual role of supporting practices while monitoring (and dealing with poor) performance.

Evaluation of a guided protocol for quality improvement in identifying common geriatric problems

OBJECTIVES

Many common geriatric problems are underrecognized and undertreated. A simple and reliable tool to facilitate a standard approach to evaluating geriatric patients might improve the quality of medical care delivered to geriatric patients. The objective of this study was to evaluate a standardized, semistructured quality-improvement protocol (the guided geriatric care protocol) for the assessment of common geriatric problems.

DESIGN

Sequential comparison cohorts, with chart review to evaluate study measures before and after introduction of the guided geriatric care protocol.

SETTING

The outpatient consultative geriatric assessment center of Yale-New Haven Hospital in New Haven, Connecticut.

PARTICIPANTS

One hundred consecutive new patients before and 100 consecutive new patients after introduction of the guided geriatric care protocol.

MEASUREMENTS

Number and type of problems identified and recommendations made during the clinical encounter, duration of the clinical encounter, clinician acceptance.

RESULTS

The two patient groups were similar in sociodemographics, cognitive and functional status, and reasons for evaluation. Significantly more problems were identified after (mean 5.51) than before (mean 3.49) introduction of the guided geriatric care protocol (P< .001); likewise, significantly more recommendations were made after (mean 10.45) than before (mean 8.48) introduction of the protocol (P< .001). The duration of the clinical encounter did not differ significantly between the two groups. The protocol was well accepted by participating clinicians.

CONCLUSIONS

Use of the guided geriatric care protocol assured a standard approach to evaluating common geriatric problems and may have led to the identification and treatment of more problems than usual care without increasing the duration of the clinical encounter. A quality-improvement tool that standardizes the evaluation of common geriatric problems, if validated in other clinical settings, holds the potential to improve the quality of care for vulnerable older patients.

Processes and outcomes of evidence-based practice

OBJECTIVES

To describe knowledge utilization, the models created to apply knowledge and promote evidence-based practice, and outcomes of integrating evidence into practice.

DATA SOURCES

Textbooks, research and review articles, and professional experience.

CONCLUSIONS

Evidence-based practice is both a process, which requires activities to evaluate evidence, and a product, which is the translation of evidence into a practice change.

IMPLICATIONS FOR NURSING PRACTICE

The integration of evidence into nursing practice will strengthen nursing's theoretical base, decrease the variation in processes of care, improve patient outcomes, empower nurses, and help identify areas for research.

Development of the continuous quality improvement questionnaire (CQIQ)

PURPOSE

Increasingly, hospitals are engaging in continuous quality improvement (CQI) endeavors, aimed at optimizing patient care. Physician involvement is critical to the success of such initiatives. Little is known about any mediating factors that affect physician participation in these projects, though such knowledge may be potentially important for targeting approaches to maximize physician involvement. The purpose of this study was to develop a reliable and valid instrument to assess physicians' knowledge of and attitudes toward CQI.

MATERIALS AND METHODS

Items for the questionnaire were generated by using interviews and literature re-view and covered areas of knowledge, attitude, and facilitators and barriers to involvement in CQI projects. Five physicians participated in the interviews, 64 participated in the survey, and 9 participated in the assessment of test-retest reliability. Main outcomes were reliability and validity.

RESULTS

The CQI questionnaire (CQIQ) had acceptable internal consistency and Cronbach's alpha correlation coefficient exceeded.70 for all scales. Item-total correlation ranged from.30 to.63 for all scales except for 1 item. Pearson's correlation coefficient for test-retest reliability was 0.85 (P =.02). A 76% response rate was achieved.

CONCLUSIONS

There appears to be complex interactions among psychologic and environmental mediators that influence physician participation in hospital quality initiatives. The CQIQ shows reasonable measurement properties and our findings should be generalizable to physicians in other academic institutions. The CQIQ provides additional information on the implementation of programs and processes that should be validated in other institutional settings to enhance the interpretability of the instrument.

The failure of a controlled trial to improve depression care: a qualitative study

BACKGROUND

The DIAMOND Project (Depression Is A MANageable Disorder), a nonrandomized controlled effectiveness trial, was intended to improve the long-term management of depression in primary care medical clinics. The project tested whether a quality improvement (QI) intervention could implement a systems approach-so that there would be more reliable and effective monitoring of patients with depression, leading to better outcomes. THE QUALITATIVE STUDY: A study was conducted in 1998-2000 to determine why a quality improvement intervention to improve depression care did not have a significant impact. Data consisted of detailed notes from observations of 12 project-related events (for example, team meetings and presentations) and open-ended interviews with a purposive sampling of 17 key informants. Thematic analytic methods were used to identify themes in the contextual data.

PRINCIPAL FINDINGS

Overall, the project implementation was very limited. Five themes emerged: (1) The project received only lukewarm support from clinic and medical group leadership. (2) Clinicians did not perceive an urgent need for the new care system, and therefore there was a lack of impetus to change. (3) The improvement initiative was perceived as too complex by the physicians. (4) There was an inherent disconnect between the commitment of the improvement team and the unresponsiveness of most other clinic staff. (5) The doctor focus in clinic culture created a catch-22 dilemma-the involvement and noninvolvement of physicians were both problematic.

CONCLUSION

Problems in both predisposing and enabling factors accounted for the ultimate failure of the DIAMOND quality improvement effort.

A survey of 92 quality improvement projects

BACKGROUND

Studies focusing on the impact of improvement efforts on the organization have yielded mixed results, which has increased interest in comparing the processes of improvement used. Data for a convenience sample of 92 quality improvement (QI) projects in 32 organizations were gathered from interviews and self-reported surveys from 1998 to 2000. A self-administered questionnaire was developed to measure 70 characteristics of improvement projects.

RESULTS

Most (80%) of the improvement projects were conducted by hospitals or clinics affiliated with hospitals. The projects took an average of 13 months from the team's first meeting to the end of the pilot study. Project teams met 14 times (approximately once a month) and spent 1.5 hours per meeting. Some projects did not measure the impact, others did not intend to have a specific impact, and still others measured but did not achieve the planned impact.

DISCUSSION

Patients and employees may be benefitting from improvement projects, but organizations may not be leveraging these improvements to reduce cost of delivery or increase market share. Considerable variation in the projects' impact raises the question of the need to improve the improvement methods. Generalization from this study should be made with caution, as data were based on a self-selected convenience sample of organizations. Furthermore, respondents did not complete all items, and missing information may affect the conclusions. The data on current improvement practices that are provided in this study can serve as baseline data against which rapid improvement efforts can be judged.

Clinical governance: a convincing strategy for quality improvement?

Clinical governance is a new policy introduced by the UK government to improve quality of care in the National Health Service; it imposes a "duty of quality" on all NHS organisations, and aims to bring together managerial, organisational and clinical approaches to improving quality of care. Infrastructures have been established to support quality improvement in NHS organisations and priorities for quality improvement have been established. Initial approaches are largely educational. However, information on quality of care is starting to be shared, and experiments are being conducted with a range of financial and contractual incentives for quality improvement. For widespread cultural change to occur, a "no blame" approach to quality improvement will be necessary; this may be incompatible with the need to identify and eliminate bad practice. Other tensions include the rapid pace of change being centrally driven and uneven development of the infrastructure to support clinical governance. What has not yet been shown is that quality of care has improved. It is too early to say this yet. Given the magnitude both of the vision and the work required, it is unlikely that change will be rapid, or seen on a widespread scale.

Quality improvement interventions: what works?

Quality improvement efforts in healthcare focus on the use of interventions and tools to change identified processes. What interventions and tools are effective? A survey of the literature yields information on interventions and tools that have been used to successfully create quality improvement. Effective methods for changing clinical practice include face-to-face education outreach visits, involvement of local opinion leaders, reminder systems, repeated feedback from the senior medical staff, patient-mediated interventions, and a combination of interventions deployed simultaneously. Participation in an organized continuous quality improvement process is beneficial in conjunction with additional interventions.

Evaluating an integrated approach to clinical quality improvement: clinical guidelines, quality measurement, and supportive system design

BACKGROUND

Implementing clinical practice guidelines to change patient outcomes presents a challenge. Studies of single interventions focused on changing provider behavior demonstrate modest effects, suggesting that effective guideline implementation requires a multifaceted approach. Traditional biomedical research designs are not well suited to evaluating systems interventions.

OBJECTIVES

RAND and the Army Medical Department collaborated to develop and evaluate a system for implementing guidelines and documenting their effects on patient care.

RESEARCH DESIGN

The evaluation design blended quality improvement, case study, and epidemiologic methods. A formative evaluation of implementation process and an outcome evaluation of patient impact were combined.

SUBJECTS

Guidelines were implemented in 3 successive demonstrations targeting low back pain, asthma, and diabetes. This paper reports on the first wave of 4 facilities implementing a low back pain guideline.

METHODS

Organizational climate and culture, motivation, leadership commitment, and resources were assessed. Selected indicators of processes and outcomes of care were compared before, during, and after guideline implementation at the demonstration facilities and at comparison facilities. Logistic regression analysis was used to test for guideline effects on patient care.

RESULTS

Process evaluation documented varied approaches to quality improvement across sites. Outcome evaluation revealed a significant downward trend in the percentage of acute low back pain patients referred to physical therapy or chiropractic care (10.7% to 7.2%) at demonstration sites and no such trend at control sites.

CONCLUSIONS

Preliminary results suggest the power of this design to stimulate improvements in guideline implementation while retaining the power to evaluate rigorously effects on patient care.

Exploring the pathology of quality failings: measuring quality is not the problem--changing it is

The USA can boast a long history of investigation into quality failings in health care. From Ernest Codman and Abraham Flexner in the opening decades of this century through to the intense activity of the 1980s and 1990s, much careful study has exposed extraordinary and at times scandalous deficiencies in the quality of care (Millenson 1997; Chassin & Galvin 1998; Schuster et al. 1998). Yet we are still far from developing 'industrial strength' quality in health care: in all but a few isolated areas, such as general anaesthesia, 'six sigma quality' (i.e. a handful of errors per million) seems wishful thinking (Chassin 1998). Pockets of excellence and innovation notwithstanding, the dominant experience of the past two decades has been an increasing ability to document quality failings and a seeming inability to mobilize effective action (Coye & Detmer 1998). The rich literature on health-care quality that has sprung up over the past few decades has largely failed to provide a clear direction for quality improvement activity. This paper analyses some of the reasons why this might be so. Contrasting the relative absence of progress on health-care quality with the relative success of disease epidemiology provides some illuminating parallels. In essence, study of the quality of care has focused largely on providing a 'descriptive epidemiology'. Much more work is needed yet to unravel the underlying pathology of quality failings, in order to empower development of an 'aetiological epidemiology' of quality in health care. Such understanding is essential as a precursor to targeted and effective preventative and remedial action.

Quality improvement efforts in oncology: are we ready to begin?

Large variations in the quality of cancer care are a matter of concern in the United States. Despite spending over 15% of our GNP on health care, more than any other country in the world, some cancer patients face significant risks of dying from their treatment precisely because of their choice of physician. The Institute of Medicine has reported that variations in the quality of cancer are large, and that low-experienced providers are more likely to provide a lower quality of medical care. Increased pressures to contain costs have led to concern that the quality and outcomes of cancer care may only worsen. One reaction to this situation is a greater reliance on "report cards." In an effort to address both quality and cost issues, providers are looking outside the health care sector for guidance for more acceptable alternatives to report cards, which are often viewed as punitive. The approach that they most often have selected recently is termed continuous quality improvement (CQI) or total quality management (TQM). In this article, we describe the potential benefits and drawbacks of CQI efforts in oncology, review experiences with four different CQI cancer programs, and make recommendations about future CQI efforts.

Are managed care plans organizing for quality?

This article examines the degree to which managed care organizations (MCOs) are reorganizing to take responsibility for the quality of care and service they provide. Specifically, factors prompting plans to focus on quality improvement (QI) and how they may be building the capacity to improve quality are considered. The authors' analysis is based on executive interviews with the plan medical directors, QI directors, and chief executive officers (CEOs) in a sample of 24 health plans. The overall response rate was 58.3 percent (medical director = 62.5 percent, QI director = 79.2 percent, CEO = 33.3 percent). The authors queried respondents about (1) perceived drivers and obstacles to the development of an effective QI program, (2) plan organizational structure for QI, and (3) technical capacities for data collection, management, and performance measurement. The results suggest that MCOs are responding to outside pressures to engage in QI. They are reorganizing their management structures and more slowly and tentatively are building technical capacity for QI.

Quality measures for mental health care: results from a national inventory

The National Inventory of Mental Health Quality Measures was funded by the Agency for Healthcare Research and Quality to (1) inventory process measures for assessing the quality of mental health care; (2) identify clinical, administrative, and quality domains where measures have been developed; and (3) identify areas where further research and development is needed. Among the 86 measures identified, most evaluated treatment of major mental disorders, for example, schizophrenia (24 percent) and major depression (21 percent). A small proportion focused on children (8 percent) or the elderly (9 percent). Domains of quality included treatment appropriateness (65 percent), continuity (26 percent), access (26 percent), coordination (13 percent), detection (12 percent), and prevention (6 percent). Few measures were evaluated for reliability (12 percent) or validity (3 percent). Measures imposing a lower burden were more likely to be in use (chi 2 = 4.41, p = .036). Further measures are needed to assess care for several priority clinical and demographic groups. Research should focus on measure validity, reliability, and implementation costs. In order to foster quality improvement activities and use of common measures and specifications for mental health care, the inventory of quality measures will be made available at www.challiance.org/cqaimh.

Humanism in health care service: the role of stakeholder management

In the era of technological proliferation the potential for the holistic intention and practice of nursing to become overshadowed is immense. This article presents an overview of the stakeholder theory of management as a useful and important management model for the creation of health care delivery environments where the human condition is celebrated, exemplary service is cultivated, and human caring becomes an enterprisewide value.

A report card on quality improvement for children's health care

OBJECTIVE

Improving the quality of health care is a national priority. Nonetheless, no systematic effort has assessed the status of quality improvement (QI) initiatives for children or reviewed past research in child health care QI. This assessment is necessary to establish priorities for QI programs and research.

METHODS

To assess the status of QI initiatives and research, we reviewed the literature and interviewed experts experienced in QI for child health services. We defined QI as activities intended to close the gap between desired processes and outcomes of care and what is actually delivered. We classified reports published between 1985 and 1997 by publication characteristics, study design, clinical problem addressed, site of intervention, the QI method(s) used, and explicit association with a continuous quality improvement program.

RESULTS

We reviewed 68 reports meeting our definition of QI. More than half (48) were published after 1994. The reviewed reports included controlled evaluations in 36% of all identified interventions, and 3% of the reports were associated with continuous quality improvement. QI methods demonstrating some effectiveness included reminder systems for office-based preventive services and inpatient pathways for complex care. Reportedly successful QI initiatives more commonly described improvement in administrative measures such as rate of hospitalization or length of stay rather than functional status or quality of life. Interviews found that barriers to QI for children were similar to those for adults, but were compounded by difficulties in measuring child health outcomes, limited resources among public organizations and small provider groups, and relative lack of competition for pediatric tertiary care providers. Research and dissemination of QI for children were seen as less well developed than for adults.

CONCLUSIONS

Attempts to improve the quality of child health services have been increasing, and the evidence we reviewed suggests that it is possible to improve the quality of care for children. Nonetheless, numerous gaps remain in the understanding of QI for children, and widespread improvement in the quality of health services for children faces significant barriers.

Local implementation of clinical practice guidelines and continuous quality improvement: challenges and opportunities

Implementing clinical practice guidelines (CPGs), disease management programs, and quality improvement projects requires an understanding of the trends that will facilitate adoption at the local level. These trends include consumerism and the building of a national information technology infrastructure. The recommendations of the Presidential Advisory Commission on Consumer Protection and Quality in the Health Care Industry and the Institute of Medicine (IOM) report entitled "To Err Is Human: Building a Safer Health System" will frame developments and implementation. The relative roles and responsibilities of national entities, with coordination and integration of these activities with local efforts will need to be carefully orchestrated. Utilizing the Web to connect dialysis facilities, patients, and members of the health care delivery team has the potential to reap large benefits. Utilizing information technology at the point of care and retrospectively to improve decision making by all parties has the real potential to improve patient outcomes. Implementation of what at a local level? And what do we mean when local level is referred to? This article addresses these two questions and provides a framework for the engagement of patients, physicians, and members of the health care delivery team and organizations, at both the local and national levels, in improving patient outcomes. All have their roles, rights, and responsibilities. We are challenged to provide the right treatment, to the right patient, at the right time. And importantly, to provide the right information, to the right person, at the right time.

Impact of breast cancer treatment guidelines on surgeon practice patterns: results of a hospital-based intervention

BACKGROUND

Despite evidence regarding the effectiveness of post-surgical treatments for early-stage breast cancer, older women are less likely to receive appropriate therapy. We evaluated the impact of surgeon-specific "performance reports" on adherence to treatment guidelines among older women with breast cancer.

METHODS

We obtained diagnostic and treatment data from hospital tumor registries supplemented with self-reported adjuvant therapy information on 1099 patients with stage I or II breast cancer diagnosed between November 1, 1992, and January 31, 1997, at 6 Rhode Island hospitals. We compared rates of appropriate treatment receipt before and after distribution of performance reports. Hierarchical analysis was used to account for the nesting of patients within surgeons. Separate analyses of mastectomy and breast-conserving surgery were performed.

RESULTS

Age was negatively associated with post-surgical treatment, with patients who had breast-conserving surgery and who were older than 80 years significantly less likely to undergo radiation therapy (adjusted odds ratio = 0.08 [0.04, 0.14]) or appropriate adjuvant therapies (adjusted odds ratio = 0.14 [0.08, 0.22]) or both relative to 70- to 79-year-old patients. This effect did not improve post-intervention. While there was much variability in compliance with guidelines, surgeons' characteristics did not explain this variation.

CONCLUSIONS

In Rhode Island, advanced age continues to be associated with less than adequate breast cancer therapy. Providing surgeons with "feedback" on the appropriateness of adjuvant treatment for older patients was insufficient to alter established practices. Using guideline compliance data as standard "quality indicators" of physician practice may be required.

Guideline implementation: what the literature doesn't tell us

BACKGROUND

Despite large numbers of studies and literature reviews about guideline implementation, it remains unclear whether and how clinical guidelines can be used to improve the quality of medical care. This study sought to learn whether these studies and reviews have recognized the importance of systems thinking and organizational change for implementation.

METHODS

A literature search was conducted for systematic reviews of guideline implementation or practice improvement studies. Each review was studied for the extent to which it identified or discussed the value of systems changes, organizational support, practice environmental factors, and use of a change process.

RESULTS

Forty-seven good-quality systematic reviews were found. They largely concurred that using reminders and perhaps using feedback in the course of clinical encounters were the most effective ways of implementing guidelines. However, these same reviews rarely identified these strategies as systems changes, and there was little discussion about any need for organizational support or attention to various environmental variables that might affect implementation. The change process required to introduce a new or changed practice system received even less attention.

CONCLUSION

Reviews of guideline implementation trials have focused on how to change the behavior of individual clinicians. There has been little attention to the impact of practice systems or organizational support of clinician behavior, the process by which change is produced, or the role of the practice environmental context within which change is being attempted. New attention to these issues may help us to better understand and undertake the process of improving medical care delivery.