Audit and feedback: effects on professional practice and healthcare outcomes

BACKGROUND

Audit and feedback is widely used as a strategy to improve professional practice either on its own or as a component of multifaceted quality improvement interventions. This is based on the belief that healthcare professionals are prompted to modify their practice when given performance feedback showing that their clinical practice is inconsistent with a desirable target. Despite its prevalence as a quality improvement strategy, there remains uncertainty regarding both the effectiveness of audit and feedback in improving healthcare practice and the characteristics of audit and feedback that lead to greater impact.

OBJECTIVES

To assess the effects of audit and feedback on the practice of healthcare professionals and patient outcomes and to examine factors that may explain variation in the effectiveness of audit and feedback.

SEARCH METHODS

We searched the Cochrane Central Register of Controlled Trials (CENTRAL) 2010, Issue 4, part of The Cochrane Library. www.thecochranelibrary.com, including the Cochrane Effective Practice and Organisation of Care (EPOC) Group Specialised Register (searched 10 December 2010); MEDLINE, Ovid (1950 to November Week 3 2010) (searched 09 December 2010); EMBASE, Ovid (1980 to 2010 Week 48) (searched 09 December 2010); CINAHL, Ebsco (1981 to present) (searched 10 December 2010); Science Citation Index and Social Sciences Citation Index, ISI Web of Science (1975 to present) (searched 12-15 September 2011).

SELECTION CRITERIA

Randomised trials of audit and feedback (defined as a summary of clinical performance over a specified period of time) that reported objectively measured health professional practice or patient outcomes. In the case of multifaceted interventions, only trials in which audit and feedback was considered the core, essential aspect of at least one intervention arm were included.

DATA COLLECTION AND ANALYSIS

All data were abstracted by two independent review authors. For the primary outcome(s) in each study, we calculated the median absolute risk difference (RD) (adjusted for baseline performance) of compliance with desired practice compliance for dichotomous outcomes and the median percent change relative to the control group for continuous outcomes. Across studies the median effect size was weighted by number of health professionals involved in each study. We investigated the following factors as possible explanations for the variation in the effectiveness of interventions across comparisons: format of feedback, source of feedback, frequency of feedback, instructions for improvement, direction of change required, baseline performance, profession of recipient, and risk of bias within the trial itself. We also conducted exploratory analyses to assess the role of context and the targeted clinical behaviour. Quantitative (meta-regression), visual, and qualitative analyses were undertaken to examine variation in effect size related to these factors.

MAIN RESULTS

We included and analysed 140 studies for this review. In the main analyses, a total of 108 comparisons from 70 studies compared any intervention in which audit and feedback was a core, essential component to usual care and evaluated effects on professional practice. After excluding studies at high risk of bias, there were 82 comparisons from 49 studies featuring dichotomous outcomes, and the weighted median adjusted RD was a 4.3% (interquartile range (IQR) 0.5% to 16%) absolute increase in healthcare professionals' compliance with desired practice. Across 26 comparisons from 21 studies with continuous outcomes, the weighted median adjusted percent change relative to control was 1.3% (IQR = 1.3% to 28.9%). For patient outcomes, the weighted median RD was -0.4% (IQR -1.3% to 1.6%) for 12 comparisons from six studies reporting dichotomous outcomes and the weighted median percentage change was 17% (IQR 1.5% to 17%) for eight comparisons from five studies reporting continuous outcomes. Multivariable meta-regression indicated that feedback may be more effective when baseline performance is low, the source is a supervisor or colleague, it is provided more than once, it is delivered in both verbal and written formats, and when it includes both explicit targets and an action plan. In addition, the effect size varied based on the clinical behaviour targeted by the intervention.

AUTHORS' CONCLUSIONS

Audit and feedback generally leads to small but potentially important improvements in professional practice. The effectiveness of audit and feedback seems to depend on baseline performance and how the feedback is provided. Future studies of audit and feedback should directly compare different ways of providing feedback.

How valid are medical records and patient questionnaires for physician profiling and health services research? A comparison with direct observation of patients visits

OBJECTIVES

This study was designed to determine the optimal nonobservational method of measuring the delivery of outpatient medical services.

METHODS

As part of a multimethod study of the content of primary care practice, research nurses directly observed consecutive patient visits to 138 practicing family physicians. Data on services delivered were collected using a direct observation checklist, medical record review, and patient exit questionnaires. For each medical service, the sensitivity, specificity, and Kappa statistic were calculated for medical record review and patient exit questionnaires compared with direct observation. Interrater reliability among eight research nurses was calculated using the Kappa statistic for a separate sample of videotaped visits and medical records.

RESULTS

Visits by 4,454 patients were observed. Exit questionnaires were returned by 74% of patients. Research nurse interrater reliabilities were generally high. The specificity of both the medical record and the patient exit questionnaire was high for most services. The sensitivity of the medical record was low for measuring health habit counseling and moderate for physical examination, laboratory testing, and immunization. The patient exit questionnaire showed moderate to high sensitivity for health habit counseling and immunization and variable sensitivity for physical examination and laboratory services.

CONCLUSIONS

The validity of the medical record and patient questionnaire for measuring delivery of different health services varied with the service. This report can be used to choose the optimal nonobservational method of measuring the delivery of specific ambulatory medical services for research and physician profiling and to interpret existing health services research studies using these common measures.

Reviewing audit: barriers and facilitating factors for effective clinical audit

OBJECTIVE

To review the literature on the benefits and disadvantages of clinical and medical audit, and to assess the main facilitators and barriers to conducting the audit process.

DESIGN

A comprehensive literature review was undertaken through a thorough review of Medline and CINAHL databases using the keywords of "audit", "audit of audits", and "evaluation of audits" and a handsearch of the indexes of relevant journals for key papers.

RESULTS

Findings from 93 publications were reviewed. These ranged from single case studies of individual audit projects through retrospective reviews of departmental audit programmes to studies of interface projects between primary and secondary care. The studies reviewed incorporated the experiences of a wide variety of clinicians, from medical consultants to professionals allied to medicine and from those involved in unidisciplinary and multidisciplinary ventures. Perceived benefits of audit included improved communication among colleagues and other professional groups, improved patient care, increased professional satisfaction, and better administration. Some disadvantages of audit were perceived as diminished clinical ownership, fear of litigation, hierarchical and territorial suspicions, and professional isolation. The main barriers to clinical audit can be classified under five main headings. These are lack of resources, lack of expertise or advice in project design and analysis, problems between groups and group members, lack of an overall plan for audit, and organisational impediments. Key facilitating factors to audit were also identified: they included modern medical records systems, effective training, dedicated staff, protected time, structured programmes, and a shared dialogue between purchasers and providers.

CONCLUSIONS

Clinical audit can be a valuable assistance to any programme which aims to improve the quality of health care and its delivery. Yet without a coherent strategy aimed at nurturing effective audits, valuable opportunities will be lost. Paying careful attention to the professional attitudes highlighted in this review may help audit to deliver on some of its promise.

Protocol biopsy of the stable renal transplant: a multicenter study of methods and complication rates

BACKGROUND

Clinical trials in renal transplantation must use surrogate markers of long-term graft survival if conclusions are to be drawn at acceptable speed and cost. Morphologic changes in transplant biopsies provide the earliest available evidence of damage, and "protocol" biopsies from stable grafts can be used to reduce the number of patients needed in clinical trials. This approach has been inhibited by concerns over safety, but the risk of biopsy of a stable kidney, with no active inflammation or acute functional impairment, has never been formally estimated.

METHODS

In accordance with a predefined set of questions, a retrospective audit of a sequential series of protocol biopsies was performed in four major transplant centers.

RESULTS

A total of 2,127 biopsy events were assessed for major complications, and 1,486 were assessed for minor ones. There were no deaths. One graft was lost, under circumstances indicating that the loss should have been prevented. Three episodes of hemorrhage required direct intervention. Three further patients required transfusion. There were two episodes of peritonitis, but one was arguably an unrelated event. All serious complications presented within 4 hr of biopsy.

CONCLUSIONS

The incidence of clinically significant complications after protocol biopsy of a stable renal transplant is low. Direct benefits to the patients concerned (irrespective of the benefit that may accrue in clinical trials) were not formally assessed but seem likely to outweigh the risk of the procedure. We believe that it is ethically justifiable to ask renal transplant recipients to undergo protocol biopsies in clinical trials and routine care.

Outcome measures in palliative care for advanced cancer patients: a review

Information generated using outcome measures to measure the effectiveness of palliative care interventions is potentially invaluable. Depending on the measurement tool employed the results can be used to monitor clinical care, carry out comparative research, provide audit data or inform purchasing decisions. However, the data collected can only ever be as good as the method used to obtain them. This review aimed to systematically identify and examine outcome measures that have been used, or proposed for use in the clinical audit of palliative care of patients with advanced cancer. Database searches were performed using MEDLINE (1991-1995), CANCERLIT (1991-1995), Healthplan (1985-1995), and 'Oncolink' on the internet. Further measures were located with the assistance of other professionals working in palliative care. The criteria for the inclusion and assessment of measures were a measure assessing more than one domain and a target population of advanced disease or palliative care Forty-one measures were identified, 12 of which satisfied the inclusion criteria. These contained between five and 56 items and covered aspects of physical, psychological and spiritual domains. Each measure meets some but not all of the objectives of measurement in palliative care, and fulfils some but not all of our criteria for validity, reliability, responsiveness and appropriateness.

Medication errors in neonatal and paediatric intensive-care units

In a 4-year prospective quality assurance study, 315 iatrogenic medication errors were reported among the 2147 neonatal and paediatric intensive-care admissions, an error rate of 1 per 6.8 admissions (14.7%). The frequency of iatrogenic injury of any sort due to a medication error was 66/2147 (3.1%)--1 injury for each 33 intensive-care admissions. 33 (10.5%) errors were potentially serious, 32 (10.2%) caused mild patient injuries, and 1 patient had acute aminophylline poisoning after receiving five intravenous doses of the drug at a dosage ten times higher than prescribed, owing to a calculation error during dilution. A longitudinal monitoring system helps to identify iatrogenic complications due to medication errors and may help in implementing preventive measures.

Identifying adverse events caused by medical care: degree of physician agreement in a retrospective chart review

OBJECTIVE

To 1) assess the degree of agreement among physicians on the cause of previously flagged adverse outcomes and 2) relate the findings to systems of quality assurance and performance assessment and proposals for no-fault compensation for medical injuries.

DESIGN

Observational study of 7533 pairs of "structured implicit" reviews (subjective opinions based on guidelines) of medical records done by 127 physicians working independently.

SETTING

Random sample of 51 inpatient facilities in New York State.

PATIENTS

Random sample of inpatient medical records from the selected facilities.

MEASUREMENTS

1) Number of agreed-upon adverse events compared with the number of cases of extreme disagreement and 2) internally and indirectly standardized rates at which physician reviewers found adverse events (injuries to patients caused at least in part by medical management).

RESULTS

In 12.9% of cases (971 of 7533), the two physicians in a pair had extreme disagreement about the occurrence of an adverse event. These cases outnumbered those in which both reviewers found an adverse event (10%; n = 757). Agreement was highest for wound infections and lowest for adverse events attributed to failure to diagnose or lack of therapy. The amount of experience the physicians had in reviewing records tended to increase the level of agreement. Even after standardization to the results of the entire sample, individual physicians' rates of finding at least slight evidence of an adverse event varied widely (range, 9.9% to 43.7%) (P < 0.001).

CONCLUSIONS

Structured implicit reviews produced disagreement on the causes of adverse patient outcomes. If systems of quality assurance, performance audits, or no-fault patient compensation are to succeed, methods for overcoming the common tendency toward disagreement among experts must be developed.

Adherence to local hospital guidelines for surgical antimicrobial prophylaxis: a multicentre audit in Dutch hospitals

OBJECTIVE

To study the adherence to local hospital guidelines for antimicrobial prophylaxis in surgery, and explore reasons for non-adherence.

METHODS

A prospective, multicentre audit of elective procedures, without prior suspicion of infection, was carried out in 13 Dutch hospitals. By reviewing medical, anaesthetic and nursing records, and medication charts, the prescription of antibiotics was compared with the local hospital guideline on antibiotic choice, duration of prophylaxis, dose, dosing interval and timing of the first dose.

RESULTS

Between January 2000 and January 2001, 1763 procedures were studied. Antibiotic choice, duration, dose, dosing interval and timing of the first dose were concordant with the hospital guideline in 92%, 82%, 89%, 43% and 50%, respectively. Overall adherence to all aspects of the guideline, however, was achieved in only 28%. The most important barriers to local guideline adherence were lack of awareness due to ineffective distribution of the most recent version of the guidelines, lack of agreement by surgeons with the local hospital guidelines, and environmental factors, such as organizational constraints in the surgical suite and in the ward.

CONCLUSION

This study shows that, although adherence to separate aspects of local hospital guidelines for surgical prophylaxis in the Netherlands is favourable, overall adherence to all parameters is hard to achieve. Adherence to guidelines on dosing interval and timing needs improvement, in particular. To increase the quality of antimicrobial prophylaxis in surgery, effort should be put into developing guidelines acceptable to surgeons, in adequately distributing the guidelines and to facilitating logistics. Audits of surgical prophylaxis may help hospitals identify barriers to guideline adherence.

Guidelines from the European Society of Breast Imaging for diagnostic interventional breast procedures

The aim of the breast team is to obtain a definitive, nonoperative diagnosis of all potential breast abnormalities in a timely and cost-effective way. Percutaneous needle biopsy with its high sensitivity and specificity should now be standard practice, removing the need for open surgical biopsy or frozen section. For patients with cancer, needle biopsy provides a cost-effective and rapid way of providing not only a definitive diagnosis but prognostic information, allowing prompt discussion of treatment options, be they surgical or medical. Early removal of uncertainty also allows better psychosocial adjustment to the disease. Patients with benign conditions found either by themselves or as a result of population or opportunistic screening can be promptly reassured and discharged, removing the health care and psychological costs of surgical biopsy or repeated follow-up. Radiologists involved in breast imaging should ensure that they have the necessary skills to carry out core biopsy and/or fine-needle aspiration (FNA) under all forms of image guidance. This paper provides guidelines on best practice for diagnostic interventional breast procedures and standards, against which all practitioners should audit themselves, from the European Society of Breast Imaging.

The art and science of chart review

BACKGROUND

Explicit chart review was an integral part of an ongoing national cooperative project, "Using Achievable Benchmarks of Care to Improve Quality of Care for Outpatients with Depression," conducted by a large managed care organization (MCO) and an academic medical center. Many investigators overlook the complexities involved in obtaining high-quality data. Given a scarcity of advice in the quality improvement (QI) literature on how to conduct chart review, the process of chart review was examined and specific techniques for improving data quality were proposed.

METHODS

The abstraction tool was developed and tested in a prepilot phase; perhaps the greatest problem detected was abstractor assumption and interpretation. The need for a clear distinction between symptoms of depression or anxiety and physician diagnosis of major depression or anxiety disorder also became apparent. In designing the variables for the chart review module, four key aspects were considered: classification, format, definition, and presentation. For example, issues in format include use of free-text versus numeric variables, categoric variables, and medication variables (which can be especially challenging for abstraction projects). Quantitative measures of reliability and validity were used to improve and maintain the quality of chart review data. Measuring reliability and validity offers assistance with development of the chart review tool, continuous maintenance of data quality throughout the production phase of chart review, and final documentation of data quality. For projects that require ongoing abstraction of large numbers of clinical records, data quality may be monitored with control charts and the principles of statistical process control.

RESULTS

The chart review module, which contained 140 variables, was built using MedQuest software, a suite of tools designed for customized data collection. The overall interrater reliability increased from 80% in the prepilot phase to greater than 96% in the final phase (which included three abstractors and 465 unique charts). The mean time per chart was calculated for each abstractor, and the maximum value was 13.7 +/- 13 minutes.

CONCLUSIONS

In general, chart review is more difficult than it appears on the surface. It is also project specific, making a "cookbook" approach difficult. Many factors, such as imprecisely worded research questions, vague specification of variables, poorly designed abstraction tools, inappropriate interpretation by abstractors, and poor or missing recording of data in the chart, may compromise data quality.

A real-time audit of radiation therapy in a regional cancer center

PURPOSE

To report the development, structure, and implementation of a real-time clinical radiotherapy audit of the practice of radiation oncology in a regional cancer center.

METHODS AND MATERIALS

Radiotherapy treatment plans were audited by a real-time peer-review process over an 8-year period (1989-1996). The overall goal of the audit was to establish a process for quality assurance (QA) of radiotherapy planning and prescription for individual patients. A parallel process was developed to audit the implementation of intervention-specific radiotherapy treatment policies.

RESULTS

A total of 3052 treatment plans were audited. Of these, 124 (4.1%) were not approved by the audit due to apparent errors in radiation planning. The majority of the nonapproved plans (79%) were modified prior to initiating treatment; the audit provided important clinical feedback about individual patient care in these instances. Most of the remaining nonapproved plans were deviations from normal practice due to patient-specific considerations. A further 110 (3.6% of all audited plans) were not approved by the audit due to deviations from radiotherapy treatment policy. A minority of these plans (22%) were modified prior to initiating treatment and the remainder provided important feedback for continuous quality improvement of treatment policies.

CONCLUSION

A real-time audit of radiotherapy practice in a regional cancer center setting proved feasible and provided important direct and indirect patient benefits.

Guidelines for medical audit: seven principles

The government, general managers, and professional bodies all agree that medical audit should be implemented throughout the United Kingdom. Nevertheless, it is not yet decided either nationally or locally how audit should be defined and what its implications will be. In an analysis to find ways of measuring the design and effectiveness of hospital audit, therefore, seven main measures emerged that might serve as practical criteria. These were the definition of medical and managerial responsibilities; medical organisation; scope of audit; essential characteristics; resources needed; record keeping; and evaluation. Though generally consistent with the proposals of the government and the Department of Health, these seven principles offer some alternative approaches.

Near miss audit in obstetrics

PURPOSE OF REVIEW

Over the past decade, there has been increasing interest internationally in studying maternal near misses, or severe morbidity, to complement traditional audit of maternal mortality. This review summarizes studies in this field published during 2005-2006.

RECENT FINDINGS

There is wide variation among published studies in terms of definitions of near miss, sources of data, and assessment of quality of care. Some investigators focus on single categories of near miss (e.g. postpartum haemorrhage, obstetric hysterectomy, intensive care unit admission), whereas others include multiple categories (ranging from two to 14). Some groups identify cases from routinely collected administrative data; whereas others search hospital registers and individual case records. Many investigators make no attempt to assess quality of care or preventability, but restrict their studies to the reporting of rates of severe morbidity. Others assess care by means of interviews with survivors or case note review, but study only a sample of cases and cannot report incidence rates. A minority of investigators both report incidence rates and assess quality of care.

SUMMARY

Near miss audit is increasingly used to complement maternal mortality review. Standardization of inclusion criteria and of methods for case assessment would facilitate comparisons over time and among countries.

Management and morbidity of cellulitis of the leg

Ascending cellulitis of the leg is a common emergency. An audit was conducted in two district general hospitals to determine how it is managed and the long-term morbidity, and to formulate a treatment strategy. Case notes were reviewed for 92 patients admitted to hospital under adult specialties. Mean duration of inpatient therapy was 10 days. A likely portal of entry was identified in 51/92 cases, of which the commonest were minor injuries and tinea pedis. Pathogens were rarely identified, group G streptococci being the single most frequent organism. Benzylpenicillin was administered in only 43 cases. Long-term morbidity, identified in 8 of 70 patients with over six months' follow-up, included persistent oedema (6) and leg ulceration (2); an additional 19 patients had either suffered previous episodes or experienced a further episode subsequently. Ascending cellulitis of the leg has substantial short-term and long-term morbidity. Important but often neglected therapeutic suggestions are the inclusion of benzylpenicillin in all cases without a contraindication, assessment and treatment of tinea pedis, use of support hosiery, and serological testing for streptococci to confirm the diagnosis in retrospect. The high frequency of recurrent episodes suggests that longer courses of penicillin, or penicillin prophylaxis, might be useful.

Falls: an examination of three reporting methods in nursing homes

OBJECTIVE

To examine the concordance of various fall reporting methods and to use the results to recommend a preferred method of ascertaining fall frequency for residents of nursing homes, both for research and in the collection of federally mandated nursing home data.

DESIGN

A cohort study followed for 858 patient months, with a mean individual follow-up of 6.6 months.

MEASUREMENTS

Falls were independently ascertained monthly by three methods: review of administrative incident reports, nursing home chart abstraction, and structured interview of subjects. Concordance of events was assessed using measures of simple agreement and Kendall's Tau-b. Simple correlation and multiple regression were used to evaluate the relation of age, sex, gender, depression, mental status, and functional status with degree of concordance between self-reported falls and chart-recorded falls.

SETTING

One academic and six community nursing homes in San Antonio, Texas.

PARTICIPANTS

131 long-stay nursing home residents, greater than 60 years of age, dependent in at least two activities of daily living, and mildly cognitively impaired.

RESULTS

Falls were ascertained in 74 of the 131 individuals; 53 subjects fell 124 times by incident report, 58 had 140 falls according to chart review, and 66 subjects self-reported 232 falls. Greatest agreement between reporting methods was shown for incident report and chart review, with a Kendall's Tau-b of 0.88; self-report and chart-review agreement was 0.56; and self-report and incident agreement was 0.53. Estimated total fall events were more often (P = 0.001) identified by chart review (92%) than incident report (82%). Although concordance was higher for non-fallers, no significant relationships were observed between concordance and age, sex, race, depression, mental status, and functional status. Also, there was no systematic relationship between length of follow-up and degree of concordance.

CONCLUSIONS

Fall frequency varies by ascertainment method, with chart review reflecting a greater number of fall events than the traditionally counted incident reports.

Systematic review of the quality of surgical mortality monitoring

BACKGROUND

Mortality is the most tightly defined and used adverse event for audit and performance monitoring in surgery. However, to identify cause and therefore scope for improvement, accurate and timely data are required. The aim of this study was to perform a systematic review of the quality of measurement, reporting and monitoring of mortality as an outcome after surgery.

METHODS

A systematic review of published literature was undertaken for the 7-year interval 1993-1999. Grey and unpublished literature was obtained through the Royal College of Surgeons of England, from UK national audits and routine national hospital data collections.

RESULTS

Eligible monitoring systems included six UK national surgical audits, and cardiac and vascular surgery monitoring systems from North America and the UK. The definitions of 'surgical death' varied in several respects and deaths after discharge from hospital were rarely ascertained unless there was routine linkage to national death registers. There were very few published studies on validation of the completeness and accuracy of the data collection.

CONCLUSION

A comprehensive data collection system is needed for improving clinical performance, with ownership, but not necessarily data collection, resting with the surgeons concerned. Recording of risk factors and deaths after discharge from hospital is essential, whatever data collection system is used.

The role of performance indicators in changing the autonomy of the general practice profession in the UK

Performance indicators (PIs) are widely used across the UK public sector, but they have only recently been applied to clinical care. In doing so, they challenge a previously guarded aspect of clinical autonomy-the assessment of work performance. This "challenge" is specific to a primary care setting and in the general practice profession. This paper reviews the qualitative findings from an empirical study within one English primary care group on the response to a set of clinical PIs relating to general practitioners (GPs) in terms of the effect upon their clinical autonomy. Prior to interviews with GPs, primary care teams received feedback on their clinical performance as judged by indicators. Five themes were crucial in understanding GPs responses: the credibility of PIs, the growing need to demonstrate competence, perceptions of autonomy, the ulterior purpose of PIs, and the identity of the assessor of their performance. PIs are playing a key role in changing the locus of performance assessment along two dimensions: location and expertise. As the locus helps to determine the nature of clinical autonomy, it is likely to have implications for the nature of the general practice profession.

The Mammography Quality Standards Act. An overview of the regulations and guidance

The MQSA has imposed a set of federal regulations on every facility and radiologist that performs or interprets mammograms. Because annual inspections and paperwork are somewhat burdensome, many hope that in the future the process of FDA inspections might be streamlined for facilities with good track records and few, if any, findings at inspection. This would serve to reduce facility inconvenience and the costs of regulatory compliance without compromising patient care and outcomes. Currently, however, although not all personnel are required to be familiar with the regulations, key individuals should be conversant with the rules, FDA guidance, and methods that the facility has devised to comply with the regulations.

An audit of audits: are we completing the cycle?

Clinical audit plays an important part in the drive to improve quality of patient care and thus forms a cornerstone of clinical governance. We evaluated the standard of clinical audits conducted by all departments in a teaching hospital between 1996 and 1997. Of a total of 213 audits carried out, 102 (48%) were 'partial' and only 29 (14%) were 'full'. Recommendations for improvement emerged from 134 (63%) of the audits performed. In only 51 audits (24%) was the cycle completed by re-auditing, during the subsequent 3 years. Most departments undertake clinical audits but failure to close the loop undermines their effectiveness and wastes resources.

APACHE II, data accuracy and outcome prediction

From review of 122 intensive care charts, Acute Physiology and Chronic Health Evaluation (APACHE) II points were determined for eight physiological values. Using a strict interpretation of APACHE II criteria, an average of 20.6% of these points were higher and 6.7% lower than the points entered originally into an intensive care database. The resulting 1.73 points mean increase in APACHE II score increased predicted mortality from 24.8% to 27.8% and decreased the mortality ratio (observed hospital deaths devided by predicted deaths) from 1.52 (95% confidence interval: 1.11-2.03) to 1.35 (95% confidence interval: 0.99-1.81). There were few errors entering the data recorded on the audit form into the intensive care unit database with an optical mark reader and keyboard. Inaccuracy and inconsistency in data collection must be excluded before differences in mortality ratios are ascribed to intensive care unit performance.

Audit: the third clinical science?

In summary, we believe that if clinical audit is accepted and prosecuted as the third clinical science, it has the potential to deliver substantial benefits to patients and health professionals.

Improving benzodiazepine prescribing for elderly hospital inpatients using audit and multidisciplinary feedback

BACKGROUND

Benzodiazepines are commonly prescribed for older people. Inappropriate benzodiazepine prescribing may result in serious adverse effects, including confusion and falls.

AIMS

To determine the prevalence and appropriateness of benzodiazepine prescribing for elderly inpatients at nine public hospitals in Victoria, Australia, and to assess the impact of multidisciplinary feedback on quality of benzodiazepine prescribing.

METHODS

Clearance for the study was obtained from ethics committees of participating hospitals. Medication charts for 1301 patients aged 65 years and over were reviewed to identify patients prescribed a benzodiazepine. Using an evidence-based indicator of prescribing appropriateness, cross-sectional notes-based audits were conducted at three time points (baseline, 4-8 weeks and 6 months). Feedback of baseline audit results was provided to medical, pharmacy and nursing staff at multidisciplinary meetings. Changes in benzodiazepine prescribing over the following 6 months were assessed.

RESULTS

At baseline, benzodiazepines were prescribed for 36% of patients, and 20% of prescriptions were appropriate. Four to six weeks after feedback, the proportion of patients prescribed a benzodiazepine had not changed significantly (31%, P = 0.125); however, more prescriptions were appropriate (44%, P < 0.0001). For patients who were using a benzodiazepine prior to admission and had a contraindication, there were more attempts to withdraw or reduce the dose (47% vs 21%, P = 0.002) and more prescriptions were for acceptable indications (7.4% vs 2.6%, P = 0.024). Six months after feedback, appropriateness of prescribing remained improved compared with baseline (50%, P = 0.002).

CONCLUSIONS

Benzodiazepines were prescribed for approximately one in three elderly hospital inpatients and a large proportion was inappropriate. A multidisciplinary intervention based on audit and feedback improved prescribing and the impact of the intervention was still evident after 6 months.

The comparative effectiveness and efficiency of cognitive behaviour therapy and generic counselling in the treatment of depression: evidence from the 2nd UK National Audit of psychological therapies

BACKGROUND

Cognitive Behaviour Therapy (CBT) is the front-line psychological intervention for step 3 within UK psychological therapy services. Counselling is recommended only when other interventions have failed and its effectiveness has been questioned.

METHOD

A secondary data analysis was conducted of data collected from 33,243 patients across 103 Improving Access to Psychological Therapies (IAPT) services as part of the second round of the National Audit of Psychological Therapies (NAPT). Initial analysis considered levels of pre-post therapy effect sizes (ESs) and reliable improvement (RI) and reliable and clinically significant improvement (RCSI). Multilevel modelling was used to model predictors of outcome, namely patient pre-post change on PHQ-9 scores at last therapy session.

RESULTS

Counselling received more referrals from patients experiencing moderate to severe depression than CBT. For patients scoring above the clinical cut-off on the PHQ-9 at intake, the pre-post ES (95% CI) for CBT was 1.59 (1.58, 1.62) with 46.6% making RCSI criteria and for counselling the pre-post ES was 1.55 (1.52, 1.59) with 44.3% of patients meeting RCSI criteria. Multilevel modelling revealed a significant site effect of 1.8%, while therapy type was not a predictor of outcome. A significant interaction was found between the number of sessions attended and therapy type, with patients attending fewer sessions on average for counselling [M = 7.5 (5.54) sessions and a median (IQR) of 6 (3-10)] than CBT [M = 8.9 (6.34) sessions and a median (IQR) of 7 (4-12)]. Only where patients had 18 or 20 sessions was CBT significantly more effective than counselling, with recovery rates (95% CIs) of 62.2% (57.1, 66.9) and 62.4% (56.5, 68.0) respectively, compared with 44.4% (32.7, 56.6) and 42.6% (30.0, 55.9) for counselling. Counselling was significantly more effective at two sessions with a recovery rate of 34.9% (31.9, 37.9) compared with 22.2% (20.5, 24.0) for CBT.

CONCLUSIONS

Outcomes for counselling and CBT in the treatment of depression were comparable. Research efforts should focus on factors other than therapy type that may influence outcomes, namely the inherent variability between services, and adopt multilevel modelling as the given analytic approach in order to capture the naturally nested nature of the implementation and delivery of psychological therapies. It is of concern that half of all patients, regardless of type of intervention, did not show reliable improvement.