An audit of audits: are we completing the cycle?

Community action projects: community-engaged quality improvement for medical students

BACKGROUND

Healthcare Quality Improvement (QI) is an essential skill for medical students to acquire, although there is insufficient empirical research which suggests the best educational methods to do this. This study explored the experiences of medical students participating in two versions of a Community Action Project (CAP) which gave medical students the opportunity to learn QI skills in a community setting. The first version (GPCAP) was pre-pandemic where students identified and delivered QI projects on placement in general practice to improve local population health. The second version (Digi-CAP) ran remotely where students worked on QI projects identified by local voluntary sector organisations focused on local community priorities during COVID-19.

METHODS

Semi-structured interviews were conducted with volunteers from the two cohorts of students who had taken part in quality improvement initiatives. Transcriptions were independently coded by two researchers and analysed through thematic analysis.

RESULTS

Sixteen students were interviewed. Whilst students had mixed experiences of completing their CAP, engagement and successful learning was associated with the following themes from the two versions of QI CAP projects: finding a sense of purpose and meaning in QI projects; preparedness for responsibility and service-driven learning; the importance of having supportive partnerships throughout the project duration and making a sustainable difference.

CONCLUSIONS AND IMPLICATIONS

The study provides valuable insights into the design and implementation of these community-based QI projects, which enabled students to learn new and often hard to teach skills, whilst working on projects which have a sustainable impact on local community outcomes.

Improving the reporting of orthodontic clinical audits: an evaluation

Aims The aim of this study was to evaluate the reporting of orthodontic audits published between 2013-2019 following the introduction of a submission template in 2015.Methods An audit reporting checklist was developed, with each audit independently assessed by two assessors. Based on the previous quality checklist, an overall score of 4 or less represented poor reporting, 5-8 fair reporting and 9 or greater good reporting. All data variables were collected in a pre-piloted Excel data collection sheet.Results One hundred and fifty-nine audits were identified. A range of reporting scores were evident. The overall mean score was 10.1 (SD 1.5). Reporting scores showed improvement during the study timeframe, with a general increase in scores evident from 2015. Higher scores were achieved by multi-cycle audits (coefficient [coef]: 2.0, 95% CI: 1.38, 2.62, p <0.001). Lower scores were achieved by partial audits (coef: -1.8, 95% CI: -2.23, -1.36, p <0.001), but scores increased every year (coef: 0.2, 95% CI: 0.12, 0.27, p <0.001).Conclusions The reporting of orthodontic audits is rated as good, with yearly improvement in scores evident. The introduction of a submission template had a positive effect on the reporting of audits. Recommendations to further improve the quality of audits are outlined.

Quality improvement education for medical students: a near-peer pilot study

BACKGROUND

Quality improvement (QI) is an essential component of modern clinical practice. Front-line professionals offer valuable perspectives on areas for improvement and are motivated to deliver change. In the UK, all junior doctors are expected to participate in QI in order to advance to the next stage of their training. However, UK undergraduates receive no standardized training in QI methods. This is perpetuated within medical schools by a lack of teaching capacity and competing priorities, and may lead to tokenistic engagement with future QI projects.

METHODS

We describe a near-peer teaching programme designed to introduce students to QI methods. This pilot study was conceived and delivered in full by junior doctors and used existing resources to ensure high quality teaching content. 111 fifth-year medical students from the University of Cambridge were taught in interactive, participative workshops that encourage them to develop their own QI change ideas and projects. Core topics included the model for improvement, driver diagrams, stakeholder engagement, measurement for improvement and analysing and presenting data. Students completed surveys before and immediately after this intervention to assess their understanding of and confidence in utilizing QI methods. Questionnaires were also completed by junior doctor tutors.

RESULTS

Analysis of questionnaires completed before and immediately after the intervention revealed statistically significant improvements in students' self-reported understanding of QI (p < 0.05) and confidence in applying techniques to their own work (p < 0.05). Students expressed a preference for QI teaching delivered by junior doctors, citing a relaxed learning environment and greater relevance to their stage of training. Tutors reported increased confidence in using QI techniques and a greater willingness to engage with QI in future.

CONCLUSIONS

In this single-centre study, near-peer teaching produced significant improvements in students' self-reported understanding of QI and confidence in applying QI methods. Near-peer teaching may constitute a sustainable means of teaching essential QI skills at undergraduate level. Future work must evaluate objective measures of student engagement with and competence in conducting QI.

Plasma HbA1c in the investigation of suspected heart failure in general practice: An audit of the 2018 NICE guidelines update

Introduction:

Diabetes mellitus is a known risk factor for heart failure (HF); nevertheless, many HF patients remain undiagnosed. The National Institute for Health and Care Excellence in England updated their HF guidelines in 2018, replacing the use of fasting plasma glucose with glycated hemoglobin (HbA1c), in suspected HF investigation. This audit aimed to assess this update's uptake at a general practice partnership in Colchester, England.

Materials and Methods:

The audit cycle consisted of a two-round electronic record search, for approximately 29,000 patients registered at the partnership. From November 1, 2017 to November 1, 2018 for the first round and from November 1, 2018 to March 6, 2019 for the second round, patients who had their NT pro-brain natriuretic peptide levels measured for the initial investigation of suspected HF were included in the study. Interventions put in place after the first round included an oral presentation and an illustrated guide for the general practitioners (GPs).

Results:

One hundred and ten patients, 19 in cycle 1 and 91 in cycle 2, were identified and included in the analysis. At the first round, only 31.6% of the patients had their HbA1c level measured, while 36.8% had no diabetic investigation done. At the second round, the percentage of patients who had their HbA1c level assessed increased to 59.3%. A decrease from 36.8% of patients without diabetic status assessment to 20% was observed.

Conclusions:

Lack of awareness among GPs regarding this guideline update was identified and simple interventions achieved an increase in the guideline's uptake. Regular and complete audit cycles can help GPs adhere to up-to-date guidelines. Primary care can help other organizations such as pathology laboratories keep up to date with guidelines, while primary care technology can be amended in-house to help adherence to new guidelines. We recommend the National UK Heart Failure Audit considers auditing the use of HbA1c testing in inpatients investigated for new HF.

Closing the loop between evidence-based medicine and care delivery: a possible role for clinical audits in spinal surgery

BACKGROUND CONTEXT

Evidence-based medicine (EBM) should be the ultimate force driving change in clinical practice. This process generally occurs through a trickle-down phenomenon by which practice recommendations are revised, modified, and/or changed based on the best published data. Recommendations are subsequently incorporated by individual physicians. The fundamental assumption that drives this paradigm is that adopting evidence-based recommendations and/or treatment guidelines will result in improved outcomes. Unfortunately, to date, the paradigm does not have an effective feedback loop that would then evaluate whether the changes did, in fact, improve outcomes.

PURPOSE

To explore the process of clinical audits as a mechanism by which to provide a feedback loop to evaluate the results of spinal surgery on an individual basis and whether those results can be improved.

STUDY DESIGN

Review article, discussion.

METHODS

A literature review of the current data regarding clinical audits was performed, and a discussion of how they may apply to spinal surgery is offered.

RESULTS

Clinical audits have been used outside the United States, particularly in the United Kingdom, to fulfill this function. A clinical audit would allow a practicing spinal surgeon to examine his or her individual experience and determine if it is achieving the expected outcome based on published results. In the most important feature of a clinical audit, the reaudit, if an individual's results are found to be inconsistent with published results, it presents an opportunity to identify if there are reconcilable differences from which potential improvements can be made. Effectively, this "closes the loop" between EBM and actual clinical practice.

CONCLUSIONS

Documenting improved outcomes through the audit process can impact spinal care in several ways. Patients would receive a clear message that their doctors are interested in improving care. Hospitals will use the information to optimize treatment algorithms. Finally, insurers might make the audit process more tenable or attractive by indicating a physician's voluntary participation as a criterion to be a preferred provider.

Audit of audit: review of a clinical audit programme in a teaching hospital intensive care unit

OBJECTIVE

A comprehensive review of the clinical audit programme in a teaching hospital intensive care unit.

DESIGN

A retrospective analysis of the clinical audit projects undertaken within the intensive care unit over the preceding 2 years and compared with published national guidelines for clinical audit.

SETTING

A 27-bedded teaching hospital intensive care unit in the UK.

MEASUREMENTS

Each audit project was reviewed independently by two assessors. The following questions were assessed. 1. Were the projects true audits? 2. Were they prospective of retrospective? 3. Did the projects have input from appropriate members of the multidisciplinary team. 4. How many of the audit projects were re-audits? 5. Of the re-audits how many showed evidence of service improvement? each audit project was also scored against the Audit Project Assessment Tool produced by the UK Clinical Governance Support Team.

RESULTS

Of the twenty five audit projects reviewed twenty two were considered to be true audits. All of the projects used only retrospective data. Audit projects were contributed from all sections of the multidisciplinary critical care team but there were few truly multidisciplinary projects. Four of the audit projects were re-audits, of these three showed service improvement and one showed deterioration. Of the twenty two true audit projects reviewed, eleven were classified as good quality projects using the Audit Project Assessment Tool.

CONCLUSIONS

Despite the clinical audit programme being active and well supported, objective evidence of clinical governance benefit was lacking. The overall clinical audit programme has been revitalised by a series of improvements since undertaking this review and this approach is recommended to other organizations who are interested in improving their clinical audit performance.

Use of high-intensity statin therapy with simvastatin 80 mg and atorvastatin 80 mg in primary care

AIMS

Cardiovascular disease (CVD) is the most common cause of death worldwide. Pharmaceutical risk reduction with high-intensity statin therapy is advisable for high-risk patients. Clinicians face a conflict between prescribing for cost (simvastatin 80 mg) or for efficacy (atorvastatin 80 mg). The aim of this audit was to examine the use, efficacy and tolerability of high intensity statin treatment (simvastatin 80 mg; atorvastatin 80 mg) in primary care.

METHODOLOGY

Electronic medical records were examined from two general practitioners' surgeries. Analyses involved Mann-Whitney U and χ(2) tests.

RESULTS

A total of 116 patients had taken simvastatin 80 mg or atorvastatin 80 mg. Patients were similar between treatment groups: mostly men (62.9%), over 60 years old (68.1%), non-smokers (81.0%) taking statins for secondary prevention (56.9%). More patients on simvastatin withdrew from treatment as a result of inefficacy (49.3% vs. 23.2%, p=0.025) compared with the atorvastatin group. Furthermore, patients on simvastatin were more likely to be failing conventional targets of lipid control, compared with patients on atorvastatin 80 mg (43.5% vs. 21.3%, p=0.006). Tolerability was similar between the two groups.

DISCUSSION

UK guidelines recommend simvastatin 80 mg as an economic choice, despite scant evidence at this dose and recent safety concerns. Conversely, robust evidence exists for atorvastatin 80 mg. Head-to-head clinical trials or clinical studies comparing these agents are lacking. The present study suggests that atorvastatin 80 mg compares favourably to simvastatin in terms of efficacy and has a similar tolerability profile.

CONCLUSION

This retrospective observational study suggests that despite national guidelines, atorvastatin 80 mg is used in clinical practice and is more effective and at least as well tolerated as simvastatin 80 mg.

Audit of orthopaedic audits in an english teaching hospital: are we closing the loop?

BACKGROUND

Clinical audit is an important tool to improve patient care and outcomes in health service. A significant proportion of time and economic resources are spent on activities related to clinical audit. Completion of audit cycle is essential to confirm the improvements in healthcare delivery. We aimed this study to evaluate audits carried out within trauma and orthopaedic unit of a teaching hospital over the last 4 years, and establish the proportions which were re-audited as per recommendations.

METHODS

Data was collected from records of the clinical audit department. All orthopaedic audit projects from 2005 to 2009 were included in this study. The projects were divided in to local, regional and national audits. Data regarding audit lead clinicians, completion and presentation of projects, recommendations and re-audits was recorded.

RESULTS

Out of 61 audits commenced during last four years, 19.7% (12) were abandoned, 72.1% (44) were presented and 8.2 % (5) were still ongoing. The audit cycle was completed in only 29% (13) projects.

CONCLUSION

Change of junior doctors every 4~6 months is related to fewer re-audits. Active involvement by supervising consultant, reallocation of the project after one trainee has finished, and full support of audit department may increase the ratio of completion of audit cycles, thereby improving the patient care.

The outcome of a rapid hepatitis B vaccination programme in a methadone treatment clinic

AIM

Injecting drug users are a high-risk population for hepatitis B (HBV), but are difficult to engage in vaccination programmes. This study examines the completion rates of a HBV vaccination schedule and seroconversion in a group of patients in methadone maintenance treatment.

METHODS

Patients at a public methadone maintenance programme in Sydney, Australia, were screened for viral hepatitis (hepatitis A, B and C) and offered a rapid HBV vaccination schedule (0, 1 and 2 months). Hepatitis B surface antibody (antiHBs) was retested on completion of the vaccination schedule.

RESULTS

A total of 143 patients [71.3% male, mean age 33.1 (standard deviation +/- 8.3)] enrolled in the project. Forty-nine per cent of patients were HAV antibody (Ab) positive, 81.1% hepatitis C virus (HCV) antibody (Ab) positive and 38.9% antiHBs positive. Exposure to multiple hepatitis viruses was common, with 24.5% testing positive for all three viruses. Seventy-three (83%) of the 88 antiHBs negative patients completed the vaccination schedule. Post-vaccination serology indicated a seroconversion rate of 75.4% (55 of 73) of completors, or 62.5% of eligible participants (55 of 88).

CONCLUSION

While there was a high rate of completion of the rapid vaccination schedule in this population, a moderate seroconversion rate was achieved. Further work is required to identify an optimal vaccination schedule in opioid substitution patients.

Auditing orthopaedic audit

INTRODUCTION

Clinical audit plays an important role in the drive to improve the quality of patient care and thus forms a cornerstone of clinical governance. Assurance that the quality of patient care has improved requires completion of the audit cycle. A considerable sum of money and time has been spent establishing audit activity in the UK. Failure to close the loop undermines the effectiveness of the audit process and wastes resources.

PATIENTS AND METHODS

We analysed the effectiveness of audit in trauma and orthopaedics at a local hospital by comparing audit projects completed over a 6-year period to criteria set out in the NHS National Audit and Governance report.

RESULTS

Of the 25 audits performed since 1999, half were presented to the relevant parties and only 20% completed the audit cycle. Only two of these were audits against national standards and 28% were not based on any standards at all. Only a third of the audits led by junior doctors resulted in implementation of their action plan compared to 75% implementation for consultant-led and 67% for nurse-led audits.

CONCLUSIONS

A remarkably large proportion of audits included in this analysis failed to meet accepted criteria for effective audit. Audits completed by junior doctors were found to be the least likely to complete the cycle. This may relate to the lack of continuity in modern medical training and little incentive to complete the cycle. Supervision by permanent medical staff, principally consultants, and involvement of the audit department may play the biggest role in improving implementation of change.

Hearing half the message? A re-audit of the care of patients with acute asthma by emergency ambulance crews in London

PROBLEM

An initial audit of the care provided to emergency asthma patients by the ambulance service was carried out in 1996. Some under-recognition and under-treatment of severe asthma was found as well as a lack of documentation of patient condition on scene. A re-audit was undertaken in 1999.

DESIGN

A multidisciplinary advisory group was reconvened. The same method was adopted as for the first audit. Patients included were those administered nebulised salbutamol by crews in the catchment areas of four hospitals and those diagnosed with asthma at the Accident & Emergency (A&E) departments of those hospitals between January and March 1999.

SETTING

London Ambulance Service.

KEY MEASURES FOR IMPROVEMENT

(1) Accuracy of diagnosis and appropriateness of treatment, and (2) adherence to protocol.

STRATEGIES FOR CHANGE

Following the first audit, treatment protocols were widened and brought into line with the British Thoracic Society guidelines for care of acute asthma patients. The results were widely disseminated within the service and training was initiated for all operational staff.

EFFECTS OF CHANGE

The number of patients included in the re-audit more than doubled (audit 1: n = 252, audit 2: n = 532). The increase occurred exclusively in those administered nebulised salbutamol by ambulance crews but diagnosed with conditions other than asthma in A&E (audit 1: n = 15, audit 2: n = 161). The proportion of patients diagnosed with asthma in A&E who were administered nebulised salbutamol by their attending crew rose from 58% to 75%. However, 43 asthma patients were not treated; several of these were not recognised as suffering from asthma and others fell within the changed protocols for treatment. Adherence to protocol for administration of salbutamol remained high. Pre-hospital documentation of key observations did not improve.

LESSONS LEARNT

Messages from the first audit seem to have been acted upon selectively. Implementing change is complex, and re-audit is necessary to understand the effects of the changes made.

Evolution of trends in risk management

In the past, the detection and response to adverse clinical events were viewed as an inherent part of professionalism; and, if perceived problems were not sorted out at that level, the ultimate expression of dissatisfaction was litigation. There are now demands for the adoption of more transparent and effective processes for risk management. Reviews of surgical practice have highlighted the presence of unacceptable levels of avoidable adverse events. This is being resolved in two ways. First, attention is being directed to the extent that training and experience have on outcomes after surgery, and both appear to be important. Second, a greater appreciation of human factors engineering has promoted a greater involvement of surgeons in processes involving teamwork and non-technical skills. The community wants surgeons who are competent and health-care systems that minimize risk. In recent times attention has been focused on the turmoil associated with change; but, when events are viewed over a period of several decades, there has been considerable progress towards these ideals. Further advancement would be aided by removing the adversarial nature of malpractice systems that have failed to maintain standards.

Interface flow process audit: using the patient's career as a tracer of quality of care and of system organisation

Objectives

This case study aims to demonstrate the method's feasibility and capacity to improve quality of care. Several drawbacks attached to tracer condition and selected procedure audits oblige clinicians to rely on external evaluators. Interface flow process audit is an alternative method, which also favours integration of health care across institutions divide.

Methods

An action research study was carried out to test the feasibility of interface flow process audit and its impact on quality improvement. An anonymous questionnaire was carried out to assess the participants' perception of the process.

Results

In this study, interface flow process audit brought together general practitioners and hospital doctors to analyse the co-ordination of their activities across the primary-secondary interface. Human factors and organisational characteristics had a clear influence on implementation of the solutions. In general, the participants confirmed that the interface flow process audit helped them to analyse the quality of case management both at primary and secondary care level.

Conclusions

The interface flow process audit appears a useful method for regular in-service self-evaluation. Its practice enabled to address a wide scope of clinical, managerial and economical problems. Bridging the primary-secondary care gap, interface flow process audit's focus on the patient's career combined with the broad scope of problems that can be analysed are particularly powerful features. The methodology would benefit from an evaluation of its practice on larger scale.

Quality assessment for three common conditions in primary care: validity and reliability of review criteria developed by expert panels for angina, asthma and type 2 diabetes

OBJECTIVES

To field test the reliability, validity, and acceptability of review criteria for angina, asthma, and type 2 diabetes which had been developed by expert panels using a systematic process to combine evidence with expert opinion.

DESIGN

Statistical analysis of data derived from a clinical audit, and postal questionnaire and semi-structured interviews with general practitioners and practice nurses in a representative sample of general practices in England.

SETTING

60 general practices in England.

MAIN OUTCOME MEASURES

Clinical audit results for angina, asthma, and type 2 diabetes. General practitioner and practice nurse validity ratings from the postal questionnaire.

RESULTS

54%, 59%, and 70% of relevant criteria rated valid by the expert panels for angina, asthma, and type 2 diabetes, respectively, were found to be usable, valid, reliable, and acceptable for assessing quality of care. General practitioners and practice nurses agreed with panellists that these criteria were valid but not that they should always be recorded in the medical record.

CONCLUSION

Quality measures derived using expert panels need field testing before they can be considered valid, reliable, and acceptable for use in quality assessment. These findings provide additional evidence that the RAND panel method develops valid and reliable review criteria for assessing clinical quality of care.