An assessment of the clinical effects of reporting accident and emergency radiographs

Formalising written preliminary image evaluation by Australian radiographers: a review of practice value

The Medical Radiation Practice Board of Australia (MRPBA) minimum competency framework requires all Australian radiographers to identify significant pathology in radiological images and take appropriate action to alert these urgent findings and ensure patient safety. Despite professional bodies endorsing the provision of preliminary image evaluations (PIE) in written format, radiographer image interpretation often remains inconsistent, informal, or undocumented. The purpose of this narrative review was to assess the literature to determine if PIE in the form of written radiographer comments is of value to the Australian healthcare system. A structured search was completed using four health research databases: CINAHL, Medline, Scopus and Web of Science. Studies have suggested that there is a contextual need for commenting due to increased imaging service pressures, radiologist shortages and subsequent reporting delays. Radiographers appear well placed and willing to provide accurate initial input with evidence that this would be valued and appreciated within the multidisciplinary team. Radiographer commenting has also been shown to reduce diagnostic and communicative errors with the potential to improve patient management. Finally, it was shown that participation in image interpretation practices can enhance recruitment, retention and job satisfaction among radiographers. Therefore, the current literature supports implementation of radiographer commenting within the Australian healthcare system.

After-hour trauma-radiograph interpretation in the emergency centre of a District Hospital

Introduction

Plain radiographs remain a first-line trauma investigation. Most trauma radiographs worldwide are reported by junior doctors. This study assesses the accuracy of after-hour acute trauma radiograph reporting by emergency centre (EC) doctors in an African district hospital.

Methods

An institutional review board approved retrospective descriptive study over two consecutive weekends in February 2020. The radiologist report on the admission radiographs of adult trauma patients was compared with the initial EC interpretation. The accuracy, sensitivity, specificity, positive predictive value (PPV) and negative predictive value (NPV) for EC interpretation were calculated with 95% confidence intervals (95%CI). The association between reporting accuracy and anatomical region, mechanism of injury, time of investigation, and the number of abnormalities per radiograph was assessed.

Results

140 radiographs were included, of which 49 (35%) were abnormal. EC doctors recorded (95%CI) 77% (69-84%) accuracy, 38% (25-54%) sensitivity, 97% (91-99%) specificity, 86% (65-95%) PPV and 76% (71-80%) NPV. Performance was associated with the anatomical region (p=0.02), mechanism of injury (p=<0.01) time of day (p=0.04) and the number of abnormalities on the film (p=<0.01). The highest sensitivity was achieved in reports of the appendicular skeleton (42%) and in the setting of simple blunt trauma (62%). Overall accuracy was in line with the range (44%-99%) reported in the international literature.

Discussion

Accurate reporting of acute trauma radiographs is challenging. Key factors impact performance. Further training of junior doctors in this area of clinical practice is recommended. Future work should focus on assessing the impact of such training on reporting performance.

Reducing risk in the emergency department: a 12-month prospective longitudinal study of radiographer preliminary image evaluations

INTRODUCTION

Innovations are necessary to accommodate the increasing demands on emergency departments whilst maintaining a high level of patient care and safety. Radiographer Preliminary Image Evaluation (PIE) is one such innovation. The purpose of this study was to determine the accuracy of radiographer PIE in clinical practice within an emergency department over 12 months.

METHODS

A total of 6290 radiographic examinations were reviewed from 15 January 2016 to 15 January 2017. The range of adult and paediatric examinations incorporated in the review included the appendicular and axial skeleton including the chest and abdomen. Each examination was compared to the radiologist's report this allowed calculated mean sensitivity and specificity values to indicate if the radiographer's PIE was of a true negative/positive or false negative/positive value. Cases of no PIE participation or series' marked as unsure for pathology by the radiographer were also recorded. This allowed mean sensitivity, specificity and diagnostic accuracy to be calculated.

RESULTS

The study reported a mean ± 95% confidence level (standard deviation) for sensitivity, specificity, accuracy, no participation and unsure of 71.1% ± 2.4% (6.1), 98.4% ± 0.04% (0.9), 92.0% ± 0.68% (1.9), 5.1% (1.6) and 3.6% (0.14) respectively.

CONCLUSIONS

This study has demonstrated that the participating radiographers provided a consistent PIE service while maintaining a reasonably high diagnostic accuracy. This form of image interpretation can complement an emergency referrer's diagnosis when a radiologist's report is unavailable at the time of patient treatment. PIE promotes a reliable enhancement of the radiographer's role with the multi-disciplinary team.

A Retrospective Review of Patients with Radiological Missed Fractures in an Emergency Department in Hong Kong

Aim

To review the clinical spectrum and outcome of radiological missed fractures in the Accident and Emergency Department of United Christian Hospital (UCH) in 2002.

Method

In UCH, radiologists report all X-Rays taken in the Accident and Emergency Department (AED) within 48 hours. The study period was from 1st January 2002 to 31st December 2002. AED notes, relevant clinical records and all X-rays of patients with suspected missed fractures as reported by radiologists were reviewed for information on clinical features, treatments and outcomes.

Results

A total of 286 cases of missed fractures were found. Fourteen (4.9%) involved the skull and maxillofacial region, 83 (29.0%) involved the chest region, 53 (18.5%) involved the spinal region, 72 (25.2%) involved the upper limbs and 64 (22.4%) involved the lower limbs. Of these 286 cases, 137 (47.9%) were followed up in AED, 90 (31.5%) were referred to specialist clinics for further management, 26 (9.1%) required admission to hospital for further assessment and treatment, and 33 (11.5%) defaulted follow up. Furthermore, 87 (30.4%) of these 286 missed fractures required a change in management plan: 3 missed fractures required operative intervention (internal fixation) and 84 missed fractures required some form of external immobilisation. This group of patient did not lodge any complaint or claim.

Conclusion

A&E doctors missed quite a number of fractures that might result in significant morbidity. However, a reporting system by radiologists within 48 hours from discharge can pick up all these missed fractures, and may prevent complaints and litigations.

Towards task shifting? A comparison of the accuracy of acute trauma-radiograph reporting by medical officers and senior radiographers in an African hospital

INTRODUCTION

Due to the universal shortage of radiologists, medical officers are largely responsible for acute trauma radiograph reporting in public sector healthcare facilities in well-resourced countries. In poorly-resourced countries, a shortage of medical officers results in most acute trauma radiographs being unreported. In the European Union (EU), experienced radiographers with no specific training have been shown to be more accurate than medical officers in trauma radiograph reporting, while EU radiographers who receive additional training can reach accuracies comparable to radiologists. In some EU countries, the role of the radiographer has been extended to include trauma reporting. However, there has been no study of the accuracy of trauma radiograph reporting by radiographers in Africa, where task-shifting could yield potentially greater benefits, due to shortages of both radiologists and medical officers. The aim of this study was therefore to compare the accuracy of acute trauma-radiograph reporting by medical officers and senior radiographers in an African setting.

METHODS

A prospective study was conducted at a South African hospital from November 2013-April 2014. Medical officers and senior radiographers reported the same set of appendicular skeleton trauma radiographs. Reporting accuracy, sensitivity and specificity were calculated using a consultant radiologist's report as the reference standard. Differences were evaluated using the Mann-Whitney U test, with p < 0.05 significant.

RESULTS

Senior radiographers achieved significantly higher reporting accuracy and sensitivity than medical officers (81.5% vs 67.8%, p = 0.002).

CONCLUSION

Senior radiographers represent a potentially important resource for acute trauma-radiograph reporting in the public healthcare sector in Africa.

Accuracy of Non-Contrast CT Brain Interpretation by Emergency Physicians: A cohort study

OBJECTIVE

To assess the accuracy of emergency physicians (EPs) in the interpretation of non-contrast CT Brain (NCCT Brain) by examining the inter rater reliability between EPs and radiology specialists.

METHODOLOGY

A four months prospective cohort study was conducted at emergency department of King Khalid University Hospital (KKUH), Riyadh, Saudi Arabia. We studied the daily performance of our EPs, and compared it to the radiological report issued within the week after. Data were analyzed by calculating sensitivity, specificity, accuracy and agreement (kappa statistic), using radiology report as the reference standard. Results : Out of 241 cases eligible for the study, 210 (87.14%) were concordant, and 31 (12.86%) were discordant. The agreement (kappa) was to be 0.64. Conclusion : We concluded that our EPs are moderately accurate at interpreting NCCT Brain studies. Further education and training programs were necessary for all our EPs to improve the accuracy. Further studies are required to determine the most cost-effective method of minimizing consequential misinterpretations.

The impact of immediate reporting on interpretive discrepancies and patient referral pathways within the emergency department: a randomised controlled trial

OBJECTIVE

To determine whether an immediate reporting service for musculoskeletal trauma reduces interpretation errors and positively impacts on patient referral pathways.

METHODS

A pragmatic multicentre randomised controlled trial was undertaken. 1502 patients were recruited and randomly assigned to an immediate or delayed reporting arm and treated according to group assignment. Assessment was made of concordance in image interpretation between emergency department (ED) clinicians and radiology; discharge and referral pathways; and patient journey times.

RESULTS

1688 radiographic examinations were performed (1502 patients). 91 discordant interpretations were identified (n=91/1688; 5.4%) with a greater number of discordant interpretations noted in the delayed reporting arm (n=67/849, 7.9%). In the immediate reporting arm, the availability of a report reduced, but did not eliminate, discordance in interpretation (n=24/839, 2.9%). No significant difference in number of patients discharged, referred to hospital clinics or admitted was identified. However, patient ED recalls were significantly reduced (z=2.66; p=0.008) in the immediate reporting arm, as were the number of short-term inpatient bed days (5 days or less) (z=3.636; p<0.001). Patient journey time from ED arrival to discharge or admission was equivalent (z=0.79, p=0.432).

CONCLUSION

Immediate reporting significantly reduced ED interpretive errors and prevented errors that would require patient recall. However, immediate reporting did not eliminate ED interpretative errors or change the number of patients discharged, referred to hospital clinics or admitted overall.

ADVANCES IN KNOWLEDGE

This is the first study to consider the wider impact of immediate reporting on the ED patient pathway as a whole and hospital resource usage.

How good are emergency department Senior House Officers at interpreting X-rays following radiographers?? triage?

OBJECTIVE

To assess the accuracy of Senior House Officers at interpreting plain X-rays following their triage by radiographers in an emergency department.

METHOD

We collected 2593 patients' records by systematic sampling of all those seen by emergency physicians between January 2002 and April 2002 (ca 10 000 patients) in a UK emergency department. The variables recorded included evidence of X-ray investigations and, when present, the Senior House Officer's diagnosis, the presence (abnormal) or absence of a radiographers red dot and the reference standard diagnosis. A separate category of uncertain (inconclusive) was applied to the Senior House Officer and reference standard diagnosis where appropriate. Diagnostic performance was measured by likelihood ratios with associated pre-test and post-test probabilities.

RESULTS

Including the uncertain category as abnormal gave the following results: there were 967 X-rays and those with a red dot had a probability of an abnormality of 80%. Although a further opinion of abnormal by a Senior House Officer increased this probability to 89% when they overrode the red dot opinion of the radiographer, it was incorrect in 26% of cases.

CONCLUSION

Currently, the Senior House Officer contributes to the red dot system by improving on the radiographer in rates of diagnosis of both abnormal and normal X-rays. Further reductions in error rates, however, are unlikely to be achieved until there is a change to the existing system. This may ultimately involve removing some of the responsibility of X-ray interpretation from the Senior House Officer. Any future research should consider the methodological issues highlighted by this study.

First year radiology residents not taking call: will there be a difference?

Currently, there is a debate in the academic radiology community about whether or not first year residents should take overnight call. The purpose of this study was to track discrepancies on overnight resident preliminary reads on radiographs from the emergency department to see if the experience level of the resident makes a difference. From October 1, 2005 to September 22, 2006, 13,213 radiographs were prospectively interpreted by residents at night at a Level I Trauma Center. Discrepancies were documented after review of the films with the staff radiologist in the morning. The patient's medical record was then examined to determine if there was any adverse clinical outcome as a result of the reading. Of the 13,184 radiographs interpreted, 120 total discrepancies were identified (overall discrepancy rate 0.9%). First year residents showed a discrepancy rate of 1.59%, higher than other residents, which were ranged from 0.39 to 0.56%. Of the 54 patients with follow-up imaging, the abnormality that was felt to be present by staff persisted on follow-up imaging in 22 cases; however, the abnormality was not present on follow up of the other 32 patients (59.2% of discrepancies with follow-up imaging). Although there is higher rate of discrepancy among reports generated by first year residents, the difference compared to the other levels of experience is small, and its overall significance can be debated. Follow-up imaging often showed that staff interpretations were false positives when there was a discrepancy reported.

Accuracy of radiographers red dot or triage of accident and emergency radiographs in clinical practice: a systematic review

AIM

To determine the accuracy of radiographers red dot or triage of accident and emergency (A&E) radiographs in clinical practice.

MATERIALS AND METHODS

Eligible studies assessed radiographers red dot or triage of A&E radiographs in clinical practice compared with a reference standard and provided accuracy data to construct 2 x 2 tables. Data were extracted on study eligibility and characteristics, quality, and accuracy. Pooled sensitivities and specificities and chi-square tests of heterogeneity were calculated.

RESULT

Three red dot and five triage studies were eligible for inclusion. Radiographers' red dot of A&E radiographs in clinical practice compared with a reference standard is 0.87 [95% confidence interval (CI) 0.85-0.89] and 0.92 (0.91-0.93) sensitivity and specificity, respectively. Radiographers' triage of A&E radiographs of the skeleton is 0.90 (0.89-0.92) and 0.94 (0.93-0.94) sensitivity and specificity, respectively; and for chest and abdomen is 0.78 (0.74-0.82) and 0.91 (0.88-0.93). Radiographers' red dot of skeletal A&E radiographs without training is 0.71 (0.62-0.79) and 0.96 (0.93-0.97) sensitivity and specificity, respectively; and with training is 0.81 (0.72-0.87) and 0.95 (0.93-0.97). Pooled sensitivity and specificity for radiographers without training for the triage of skeletal A&E radiographs is 0.89 (0.88-0.91) and 0.93 (0.92-0.94); and with training is 0.91 (0.88-0.94) and 0.95 (0.93-0.96).

CONCLUSION

Radiographers red dot or triage of A&E radiographs in clinical practice is affected by body area, but not by training.

Cranial computed tomography in trauma: the accuracy of interpretation by staff in the emergency department

OBJECTIVES

Cranial computed tomography (CT) is replacing skull radiography in head trauma. Rapid radiological opinions on these images may not always be available. We assessed the ability of our permanent emergency department staff to interpret the images.

METHODS

A retrospective series of 100 consecutive cases was reviewed and interpreted by five permanent emergency department medical staff, and their interpretation compared with the consensus opinion of two radiologists.

RESULTS

An overall agreement of 86.6% (95% confidence interval (CI) 83.4 to 89.9) was achieved, with a false negative rate of 4.2% (95% CI 3.9 to 4.3). No findings that would have changed the overnight management of any patient were missed.

CONCLUSIONS

Our results for CT scans are similar to studies of interpretation of other radiographic images in emergency departments. Our emergency staff could safely make the initial interpretation of cranial CT images in trauma out of hours, and formal reporting may wait until a suitably experienced radiologist is available.

The costs and effects of introducing selectively trained radiographers to an A&E reporting service: a retrospective controlled before and after study

The costs and effects of introducing selectively trained radiographers reporting accident and emergency (A&E) radiographs of the appendicular skeleton in a district general hospital were assessed using a retrospective controlled before and after design. Reference standard reports were compared with a random stratified sample of 200 A&E and 200 general practitioner (GP) reports before and after the intervention. GP reports were used as a non-intervention, non-equivalent control group. An A&E specialist registrar judged whether incorrect A&E reports might have a clinically important effect on patient management. The effect of incorrect A&E reports on outcome was assessed by patient re-attendance to the hospital because of missed abnormalities. The annual, average and incremental costs of radiographers and radiologists reporting A&E radiographs were calculated and a sensitivity analysis was undertaken. The introduction of the radiographers resulted in a 1% (95% CI -7.9 to 5.9) fall in A&E radiograph reporting accuracy and 11% (95% CI -33.7 to 11.3) reduction of cases in which incorrect A&E reports might have a clinically important effect on patient management. Only two A&E reports (one before and one after the intervention) affected patient outcome in that a fracture missed at the first visit resulted in patient re-attendance to the X-ray Department. There was a saving of 361 pounds per annum to the X-ray Department. In conclusion this study provides further evidence that selectively trained radiographers can accurately report A&E plain radiographs and at no additional cost.

Radiographers and radiologists reporting plain radiograph requests from accident and emergency and general practice

AIM

To assess selectively trained radiographers and consultant radiologists reporting plain radiographs for the Accident and Emergency Department (A&E) and general practitioners (GPs) within a typical hospital setting.

METHODS

Two radiographers, a group of eight consultant radiologists, and a reference standard radiologist independently reported under controlled conditions a retrospectively selected, random, stratified sample of 400 A&E and 400 GP plain radiographs. An independent consultant radiologist judged whether the radiographer and radiologist reports agreed with the reference standard report. Clinicians then assessed whether radiographer and radiologist incorrect reports affected confidence in their diagnosis and treatment plans, and patient outcome.

RESULTS

For A&E and GP plain radiographs, respectively, there was a 1% (95% confidence interval (CI) -2 to 5) and 4% (95% CI -1 to 8) difference in reporting accuracy between the two professional groups. For both A&E and GP cases there was an 8% difference in the clinicians' confidence in their diagnosis based on radiographer or radiologist incorrect reports. For A&E and GP cases, respectively, there was a 2% and 8% difference in the clinicians' confidence in their management plans based on radiographer or radiologist incorrect reports. For A&E and GP cases, respectively, there was a 1% and 11% difference in effect on patient outcome of radiographer or radiologist incorrect reports.

CONCLUSION

There is the potential to extend the reporting role of selectively trained radiographers to include plain radiographs for all A&E and GP patients. Further research conducted during clinical practice at a number of sites is recommended.

What is the effect of reporting all emergency department radiographs?

OBJECTIVES

To evaluate the effect of formal radiological reporting of all emergency department (ED) radiographs on clinical practice and patient outcome, and to consider whether a selective reporting policy might prove safe and effective.

METHODS

All radiographs taken in a single ED over a six month period were prospectively studied simultaneously in both the emergency and radiology departments to detect cases where a radiograph that was considered normal by ED staff was then reported as abnormal by the reporting radiologist. Whenever such a discrepancy occurred the patient's records were scrutinised to ascertain the source of the discrepancy, with a gold standard interpretation derived from senior clinical review and additional investigations where indicated. The clinical impact of the radiologist's formal report was then assessed. Accuracy of interpretation was considered in relation to the grade of ED staff and the radiographic examination obtained.

RESULTS

During the study period, 19468 new patient attendances to the ED generated 11749 radiographic examinations. Discrepancies were detected in 175 patients (1.5% of all radiographic examinations). Of these, 136 (1.2%) were subsequently shown to have been incorrectly interpreted in the ED (ED false negatives), with 40 patients (0.3%) undergoing a change in management as a result. In the remaining 39 the ED interpretation was judged to be correct (radiology false positives), with 16 patients undergoing further investigations or visits to the ED to confirm this.

CONCLUSIONS

The formal reporting of ED radiographs by the radiology department detects a number of clinically important abnormalities that have been overlooked. However, this formal reporting also generates a number of incorrect interpretations that may lead to further unnecessary investigations. Some groups of ED radiographs (such as those interpreted by an ED consultant and films of the fingers and toes) may not require formal radiological reporting. The adoption of a selective reporting policy may reduce the reporting workload of the radiology department without compromising patient care.

Are accident and emergency consultants as accurate as consultant radiologists in interpreting plain skeletal radiographs taken at a minor injury unit?

The objective of this study was to compare the accuracy of an accident and emergency (A&E) consultant in interpreting plain skeletal radiographs with that of a consultant radiologist (CR). It took the form of a retrospective study of 2133 radiographs taken in a Minor Injury Unit (MIU). A&E consultant reports on these films were compared with those of a CR and also with a gold standard. The A&E consultant diagnoses achieved an accuracy of 98.5% (CR 97.8%), sensitivity of 97.8% (CR 98.1%), specificity of 98.8% (CR 97.7%), positive predictive value of 97.3% (CR 95.1%) and negative predictive value of 98.97% (CR 99.07%) (gold standard of 100%). In conclusion, the A&E consultant reports of plain skeletal radiographs generated from an MIU were as accurate as those of a consultant radiologist. This could have significant implications for the wet reporting of A&E departmental radiographs.

Measuring the effects of image interpretation: an evaluative framework

The relaxing of restrictions on reporting films has resulted in radiographers and other health care professionals becoming increasingly involved in the interpretation of images in areas such as mammography, ultrasound and plain film radiography. However, errors and variation in the interpretation of images now represents the weakest area of clinical imaging. This has been highlighted by the difficulty of establishing standards to measure the film reading performance of radiographers as part of role extension initiatives. Despite a growing literature of studies that evaluate the film reading performance of different health care professionals, there is a paucity of evidence of the subsquent effects on the referring clinician's diagnosis, management plans and patient outcome. This paper proposes an evaluative framework that can be used to measure the chain of events from the initial technical assessment of observers' potential to interpret images using search behaviour techniques, through to the potential costs and benefits to society. Evaluating the wider implications of alternative or complementary reporting policies is essential for generating the evidence base to comprehensively underpin policy and practice and direct future research. Brealey, S.(2001). Clinical Radiology56, 341-347.

Radiological review of accident and emergency radiographs: a 1-year audit

AIM

To assess the impact and cost effectiveness of a system of radiological review of accident and emergency (A&E) plain films.

MATERIALS AND METHODS

Review documentation was studied retrospectively over a 1-year period. Six hundred and eighty-four actual or suspected errors in the initial radiological interpretation by A&E staff were highlighted by radiologists in training. These selected 'red reports' were then further reviewed by a musculoskeletal radiologist and a more senior member of the A&E team.

RESULTS

Three hundred and fifty-one missed or strongly suspected fractures were detected, with ankle, finger and elbow lesions predominating. Other errors included 11 missed chest radiograph abnormalities and 24 A&E false-positives. Radiologists in training tended to over-report abnormalities with an 18% false-positive rate when compared to the subsequent musculoskeletal radiology opinion. Following review, further action was taken by A&E staff in 286 (42.6%) of cases. No operative intervention was required in those patients with a delayed or missed A&E diagnosis. Consideration is given to the cost of providing this form of review and the impact of medico-legal factors.

CONCLUSION

Compared with the large numbers of patients seen and radiographed in a busy A&E department, the number of radiological errors was small. There were even fewer changes in management. Despite this, concern over litigation, clinical governance and future work patterns in A&E make this form of review a useful means of risk reduction in a teaching hospital.Williams, S. M. (2000). Clinical Radiology55, 861-865

Interpretation of selected accident and emergency radiographic examinations by radiographers: a review of 11000 cases

Two specially trained radiographers took part in the radiologists' rota for "cold" reporting skeletal radiographs of patients who attended the Accident and Emergency (A&E) Department at times when there was no "hot" reporting service operating, and who were not admitted for treatment or referred to fracture clinics for follow-up. These radiographs had initially been seen by A&E medical staff. At the end of an 18 month period during which the two radiographers reported on 11,322 skeletal examinations, a retrospective search was made to detect interpretive errors. The radiological history of all 11,322 patients was reviewed over a follow-up period of at least 3 months subsequent to the attendance reported by the radiographers. 48% of patients did not re-attend, 42% re-attended for unrelated examinations, and 10% re-attended for repeat examinations of the same anatomical area, or for different procedures (bone scintigraphy or CT) related to the original injury. The second (or subsequent) report was discrepant with that of the first attendance in only 29 cases out of 1103. Of these 29 patients, 13 had occult fractures which were undetectable at first attendance even in retrospect, six had new injuries accounting for the new findings, six had been the subject of false positive calls at an earlier visit, one had a fracture of the hamate missed at first attendance, and in three cases no consensus could be reached as to the cause of the discrepancy, owing to incomplete films or insufficient clinical data. Analysis of the patients' postcodes showed at least 89% were of local origin and only 1% were from outside Yorkshire, suggesting that the review should have identified the great majority of erroneous reports. It is concluded that appropriately trained and supervised radiographers can successfully undertake diagnostic reporting of selected skeletal examinations on A&E patients.

Radiologists' review of radiographs interpreted confidently by emergency physicians infrequently leads to changes in patient management

STUDY OBJECTIVE

To determine whether radiologists' review of radiographs is unwarranted when emergency physicians are confident in their interpretations.

METHODS

This was a prospective, descriptive study. Treating emergency physicians at a high-volume, urban teaching hospital and a moderate-volume nonteaching hospital indicated their confidence or lack of confidence in their plain-film radiographic study interpretations. All radiographs were then reviewed by radiologists according to the preexisting practice of each hospital. A total of 16,410 emergency department radiographic studies were included, comprising consecutive patients at each hospital. Charts of all discordant readings in the confident group were reviewed and judged clinically significant if treatment was altered. Charges for radiologic review of the confident ED interpretations were calculated.

RESULTS

The ED physicians were confident in 9,599 sets of radiographs out of a total of 16,410 (58%). Discordant interpretation rates for the "confident" and "not confident" groups were 1.2% and 3.1%, respectively (difference, 1.9%; 95% confidence interval [CI] of the difference, 1.44% to 2.36%). Review of the 118 discordant interpretations in the confident group demonstrated that 11 were significant. Few management changes were made as a result of radiologists' review of these radiographs. Total radiology review charges for the confident group were $215,338. Therefore the average radiology charge for each significant discordant interpretation was $19,576.

CONCLUSIONS

Of 9,599 sets of radiographs interpreted confidently by the emergency physicians in this study, there were 11 clinically significant discordant interpretations (0.1%). The standard practice of radiologists' review of all ED radiographs may not be justifiable.

Interpretation of trauma radiographs by junior doctors in accident and emergency departments: a cause for concern?

OBJECTIVES

To investigate how well junior doctors in accident and emergency (A&E) were able to diagnose significant x ray abnormalities after trauma and to compare their results with those of more senior doctors.

METHODS

49 junior doctors (senior house officers) in A&E were tested with an x ray quiz in a standard way. Their results were compared with 34 consultants and senior registrars in A&E and radiology, who were tested in the same way. The quiz included 30 x rays (including 10 normal films) that had been taken after trauma. The abnormal films all had clinically significant, if sometimes uncommon, diagnoses. The results were compared and analysed statistically.

RESULTS

The mean score for the abnormal x rays for all the junior doctors was only 32% correct. The 10 junior doctors were more experience scored significantly better (P < 0.001) but their mean score was only 48%. The mean score of the senior doctors was 80%, which was significantly higher than the juniors (P < 0.0001).

CONCLUSIONS

The majority of junior doctors misdiagnosed significant trauma abnormalities on x ray. Senior doctors scored well, but were not infallible. This suggests that junior doctors are not safe to work on their own in A&E departments. There are implications for training, supervision, and staffing in A&E departments, as well as a need for fail-safe mechanisms to ensure adequate patient care and to improve risk management.

A scale for measuring the severity of diagnostic errors in accident and emergency departments

OBJECTIVE

To design and test a simple scale for measuring the severity of diagnostic errors occurring in accident and emergency (A&E) departments.

METHODS

Empirical design of a scale which indicates the severity of errors on a scale of 1 to 7. It is obtained by adding two scores which indicate the additional treatment which a patient would have received and the follow up which would have been organised if the correct diagnosis had been made initially.

RESULTS

The misdiagnosis severity score (MSS) revealed 166 diagnostic errors in injuries treated in an A&E department over one years. The scoring system allowed the more significant errors to be separated from the less significant ones.

CONCLUSIONS

The MSS proved useful in describing the errors made in an A&E department.

False-positive preliminary radiograph interpretations in a pediatric emergency department: clinical and economic impact

A prospective, case control study at a university-affiliated, academic pediatric emergency department was undertaken to determine the clinical impact and cost of false-positive preliminary radiograph interpretations and to compare the cost of false-positive interpretations with the estimated cost of a 24-hour on-site pediatric radiologist. Data were collected on all patients undergoing radiography of the chest, abdomen, lateral (soft tissue) neck, cervical spine, or extremities during a 5-month period. A total of 1,471 radiograph examinations was performed, and 200 (14%) misinterpretations (false-positive and false-negative) by the pediatric emergency medicine physicians were identified. As reported previously, 20 (10%) of the false-negative interpretations were noted to be clinically significant, in the current analysis, 103 (7%) false-positive radiograph interpretations were identified. False-positive interpretations were noted more frequently (14%) for soft tissue lateral neck radiographs than for any other radiograph type. Of the 103 total false-positive radiographs, nine (0.6%) resulted in an increased patient cost totaling $764.75. These data show that false-positive radiograph interpretations have limited economic and clinical impact.

An assessment of inter-observer agreement and accuracy when reporting plain radiographs

This study investigates the inter-observer agreement and accuracy of plain radiograph reporting. Five consultant radiologists were asked to report 50 plain radiographs where the diagnosis had been established. Thirty-one of the radiographs showed an abnormality; 19 were normal. The radiographs were viewed on two separate occasions, 5 months apart. Initially clinical details were omitted but on the second occasion they were available. The accuracy of the report to the known diagnosis was assessed using McNemar test and inter-observer agreement was assessed using Kappa statistics. The precision (mean accuracy) improved in the light of the clinical information from 77% (range 62% to 86%) to 80% (range 78% to 84%), although this improvement did not reach statistical significance. Sensitivity improved from 80% to 83% and specificity from 72% to 76%. The Kappa value for inter-observer agreement improved from 0.31 to 0.58 for all radiographs, from 0.40 to 0.70 for the abnormal radiographs and from 0.24 to 0.43 for the normal radiographs. There was a trend to improved accuracy of diagnosis and inter-observer agreement with knowledge of clinical details.

The rate of clinically significant plain radiograph misinterpretation by faculty in an emergency medicine residency program

OBJECTIVE

To estimate the rate of clinically significant discrepancies between radiograph interpretations by attending radiologists and emergency medicine (EM) faculty in 2 academic EDs, using a unique scoring system.

METHODS

A retrospective comparison of radiographic agreement between EM and radiology faculty members was performed. All plain films initially interpreted by EM faculty or by EM residents with immediate reinterpretation by EM faculty were subsequently reviewed by attending radiologists. All discrepancies between these readings were reported to the ED on the following day for review by an EM faculty member (usually different from the initial EM faculty reader) who determined the need for treatment or follow-up changes. A secondary chart review by a quality assurance faculty member determined whether radiographic findings not noted on the x-ray log were present on the ED record All discrepancies from February to June 1994 were reviewed. A severity score was assigned based on the following criteria. Q-0: There was no change in treatment or follow-up; or the initial interpretation by EM faculty was validated by repeat or additional views. Q-1: Discrepancy is minor. Q-2: Discrepancy is significant, with potential for injury or bad outcome. Q-3: Discrepancy is significant, with actual injury or bad outcome.

RESULTS

Of 14,046 radiographic studies eligible for enrollment, there were 134 discrepancies (0.95%). Only 28 cases (0.2%) were found to be clinically significant. Of these, 25 were scored Q-1, 3 were scored Q-2, and 0 were scored Q-3. These clinically significant discrepancy rates were highest for the finger, skull, elbow, hand, and lumbar spine.

CONCLUSION

Emergency medicine faculty provide highly accurate rates of plain radiograph interpretation, particularly when adjusted for clinical significance and actual impact on patient care.

Radiographic interpretation in the emergency department

This study examined the concordance of radiographic readings between emergency department (ED) attending physicians and radiologists in a community teaching hospital. In addition, the incidents of misinterpretations leading to an alteration in patient care were also reviewed. All radiographs obtained from January through October 1993 were initially interpreted by ED attending physicians with subsequent final review by attending radiology staff. Misread radiographs were placed into one of three categories. The groupings included overread radiographs with no change in treatment, underread radiographs with no change in treatment, and radiograph misinterpretations with a change in treatment. Of 15,585 radiographs obtained during the study period, there were 120 misreads; 12,099 (77.6%) of the 15,585 radiographs had an initial emergency physician interpretation. Radiographic misinterpretations included 7 (5.78%) overreads, 57 (47.1%) underreads, and 57 (47.51%) misreads requiring follow-up (MR-FU). The five most frequently misread radiographs were: abdominal, 12/247 (4.4%); rib, 3/99 (3.0%); foot, 13/621 (2.1%); hip, 3/152 (1.9%); and ankle 11/758 (1.4%). The most frequently obtained radiographs included: chest, 7,012 (0.33% MR-FU); cervical spine, 1,112 (0.18% MR-FU); ankle, 758 (0.66% MR-FU); knee, 633 (0.32% MR-FU); and foot, 621 (0.97% MR-FU). In this study, 99.0% of all emergency department radiographs were read correctly on initial review by ED attending physicians. Of all misread radiographs, less than half (46%) were deemed clinically significant and required a follow-up intervention.

Pediatric emergency physician interpretation of plain radiographs: Is routine review by a radiologist necessary and cost-effective?

STUDY OBJECTIVE

To determine the concordance rate of plain radiograph interpretations by pediatric emergency physicians and pediatric radiologists, to evaluate the effect of incorrect radiologic diagnosis on patient management, and to evaluate the necessity and cost-effectiveness of routine follow-up review of all plain radiographs by a radiologist.

METHODS

We assembled a prospective series of all patients who presented to the emergency department of an urban tertiary care children's hospital and underwent plain radiography between October 1 and October 31, 1994. Pediatric emergency physicians documented their interpretations. Within 24 hours, films were reviewed by a pediatric radiologist. The two interpretations were classified as concordant or discordant and were further assessed for medical significance and subsequent change in management.

RESULTS

During the study period, 707 radiographic examinations were performed: chest, 56%; skeletal excluding spine, 20.1%; abdomen, 11.9%; sinus, 4.2%; spine, 3.6%; and other, 4%. The accuracy or concordance rate was 90.2% (638 of 707) for pediatric emergency physician interpretations; clinical management was unchanged in 96.9% (685 of 707) of the cases. Of the 69 discordant interpretations, 48 were clinically significant, with 22 requiring changes in management. They included 9 false-negative interpretations by pediatric emergency physicians: (5 fractures, 2 cases of pneumonia, 1 case of sinusitis, 1 case of cardiomegaly); 10 false-positive interpretations by pediatric emergency physicians (5 fractures, 4 cases of pneumonia, 1 case of sinusitis), and 3 false-positive interpretations by radiologists (1 case of C-2 spine subluxation, 1 retropharyngeal abscess, and 1 case of necrotizing enterocolitis). No adverse outcomes resulted from these misinterpretations. Routine review of all plain radiographs by a radiologist represents an estimated $210,000 annual cost to the patients and payers.

CONCLUSION

Radiograph interpretations by pediatric emergency physicians were generally accurate, and no adverse outcomes occurred as a result of misinterpretation. Clinical assessment probably assisted these physicians in interpreting the radiographs of high-risk patients. Judicious consultation with a radiologist during the initial presentation of a high-risk patient, when deemed warranted by the pediatric emergency physician, will help the emergency physician deliver high-quality, cost-effective health care. Given the overall clinical accuracy rate of radiograph interpretations by the pediatric emergency physicians and the cost of routine review of all plain radiographs in the ED by a radiologist, routine review versus selective specialty consultation must be further evaluated.

Out-of-hours radiographs: the effect of resident registrar cover

A retrospective study was undertaken to assess the effect of the presence of a resident compared with a non-resident registrar on the selection of patients for radiography outside normal working hours at a north London district general hospital accident and emergency (A&E) department. During periods when the registrar was resident there was a significant reduction in the proportion of patients referred for radiography.

Reporting of fracture radiographs by radiographers: the impact of a training programme

In order to determine what influence training would have on their ability to interpret skeletal radiographs from the accident and emergency department, a 6 months training programme was established for three radiographers in various aspects of the radiology of orthopaedics and skeletal trauma. During the study the radiographers reported on radiographs from the accident and emergency department and each month an evaluation of their accuracy was undertaken. The overall radiographer error rate for fracture detection (false positive and false negative) declined during the training period. This was highly significant (p < 0.001). The sensitivity for fracture detection improved from 81.1% at the commencement of the trial to 95.9% at the end. This was also highly significant (p < 0.001). Radiographer specificity for the exclusion of fractures also improved from 94.4% during the first 2 months to 96.6% in the final 2 months, and this was also significant (p < 0.05). The overall error rate of two of the three radiographers improved significantly (p < 0.001) but for one radiographer the improvement did not reach a level of statistical significance. The difference in sensitivity for fracture detection at the commencement of the trial period between radiologist and radiographer was highly significant (p < 0.001), but there was no statistically significant difference during the last two months of the trial. The difference in specificity between radiologist and radiographer remained highly significant both at the beginning and the end of the trial (p < 0.001). Experienced radiographers who receive supplementary training in the radiology of skeletal trauma can significantly improve their diagnostic skills and can report such radiographs with a high degree of accuracy. A programme of training and certification of radiographers in fracture reporting could help alleviate the diagnostic radiologists' workload of plain film reporting.

The importance of clinical details when reporting accident and emergency radiographs

The effect of clinical information on the accuracy of reporting accident and emergency (A&E) radiographs was studied in two A&E officers and five radiologists involved in their daily reporting. Each was shown 50 sets of radiographs, 30 subtle fractures and 20 controls. In one half the clinical history and precise localization was provided, and in the other half no such information was given. After an interval of 6 months the radiographs were viewed again with the amount of information reversed. The observers were asked to determine the presence of injury, describe its site, and indicate how sure they were of their diagnosis. Correct diagnosis was improved from 72.3 per cent to 80.3 per cent overall, and from 68.1 per cent to 81.4 per cent in the fracture group. All observers improved their performance with clinical details. The results confirm that accurate clinical details improve injury location.

Value of radiograph audit in an accident service department

In our accident service department all trauma radiographs are reported acutely and those misinterpreted by the casualty officers are presented at the daily clinicoradiological conference. We have reviewed this practice over a 6-month period. From over 25,000 patients attending the accident service, 16,246 radiographs were requested and reported. Of these, 456 (2.8 per cent) were considered to have been potentially misinterpreted. The errors included 167 (1 per cent) missed fractures, 55 (0.3 per cent) suspected fractures and 72 (0.4 per cent) false-positive diagnoses of fracture. Subsequently, 114 (0.7 per cent) patients required recall for treatment or further imaging. Incorrect diagnoses were seen most frequently in the more commonly injured anatomical sites--the ankle, wrist, foot, elbow and hand. However, the incidence of misinterpretation was highest in examination of the fingers, especially in children. We believe that these low figures are principally the result of involving both orthopaedic surgeons and radiologists at the formal daily conference. We regard our system of audit as beneficial to patients' care and anticipate reduced litigation which may offset the increased cost of audit.

Clinically significant radiograph misinterpretations at an emergency medicine residency program

Radiographic misinterpretation rates have been suggested as a quality assurance tool for assessing emergency departments and individual physicians, but have not been defined for emergency medicine residency programs. A study was conducted to define misinterpretation rates for an emergency medicine residency program, compare misinterpretation rates among various radiographic studies, and determine differences with respect to level of training. A total of 12,395 radiographic studies interpreted by emergency physicians during a consecutive 12-month period were entered into a computerized data base as part of our quality assurance program. The radiologist's interpretation was defined as correct. Clinical significance of all discrepancies was determined prospectively by ED faculty. Four hundred seventy-five (3.4%) total errors and 350 (2.8%) clinically significant errors were found. There was a difference in clinically significant misinterpretation rates among the seven most frequently obtained radiograph studies (P less than .0005, chi 2), accounted for by the 9% misinterpretation rates for facial films. No difference (P = .421) was noted among full-time, part-time, third-year, second-year, and "other" physicians. This finding is likely due to faculty review of residents' readings. Evaluation of misinterpretation rates as a quality assurance tool is necessary to determine the role of radiographic quality assurance in emergency medicine resident training. Educational activities should be directed toward radiographic studies with higher-than-average reported misinterpretation rates.

Accident and emergency reporting in UK teaching departments

A survey of 44 UK training departments shows that trainee radiologists commence accident and emergency (A & E) reporting after an average of 10.2 months in radiology and perform a considerable proportion of the A & E reporting workload. Most respondents consider that they supervise junior staff but only 11 departments directly check their work and only two departments offer supervision to staff with more than 2 years' experience. All A & E radiographs are reported in 79.9% of hospitals and the other hospitals operate selective reporting policies. The delay between the radiographic examination and the radiology report reaching the A & E department is less than 48 hours in about two-thirds of departments when the examination is performed Monday to Friday, but exceeds 48 hours in two-thirds of departments when it is performed at weekends. Misinterpretation of radiographs is one of the commonest errors in A & E. Improved supervision of less experienced trainees is required. Consultants should contribute and one should be responsible for this area. Radiological conferences are important and the secretarial services require major improvement.

Accuracy of interpretations of emergency department radiographs: effect of confidence levels

We conducted a prospective study to assess the relationship between the interpretive agreement rate for emergency department radiographs and the degree of interpretive confidence. We hoped to identify a subset of ED radiographs that did not require mandatory review by a radiologist. For each of the 1,872 plain radiographs studied, emergency physicians assigned a confidence level to the interpretation before comparing it with the radiologist's dictated report. A second radiologist was consulted to resolve disagreements. The overall rate of interpretive agreement was 94.6%. Agreement varied significantly (P less than .001) as a function of confidence level and by type of radiograph, but not by training level. Agreement varied significantly (P less than .001) as a function of confidence level for some types of radiographs (eg, chest, extremities) and for some types of radiographic findings (acute positive). No subset of radiographs had 100% agreement. Treatment was potentially altered in 38 patients as a result of the interpretive disagreement that occurred in 2% of studied radiographs. We conclude that the interpretive agreement rate increases in relation to interpretive confidence but that confidence levels cannot safely exclude certain radiographs from mandatory review by a radiologist.

Accuracy of detection of radiographic abnormalities by junior doctors

This study assessed the ability of junior doctors in accident and emergency to detect radiographic abnormalities. Their assessments of 505 radiographs taken at nights and weekends over a period of 8 months and showing abnormalities were examined. Each assessment by a senior house officer (SHO) was compared with the subsequent diagnosis of a radiologist of senior registrar or consultant status. An error rate of 35% was found. For abnormalities with clinically significant consequences the error rate was 39%. Although this error rate appears high the results are consistent with those of earlier studies in that missed positive radiographs constitute 2.8% of the total number of radiographs taken in the period. It is considered that the proportion of missed abnormalities gives a truer index of SHOs' abilities. No improvement in performance was evident over the 6-month period of the SHOs tenure of post. It is argued that it is unrealistic to expect accident and emergency SHOs to acquire this complex skill simply through experience and that more formal training and guidance is needed.

Diagnostic radiology in disaster medicine: implications for design, planning and organization of X-ray departments

Diagnostic radiology plays an important role in the evaluation of disaster casualties; these are referred to X-ray departments, shortly after their arrival at hospital, in large numbers, frequently overwhelming facilities and resources of the department. The study of relevant literature and the experience of the authors suggest that there are implications for design, planning and organization of X-ray departments to be considered in disaster-prone areas. Departments should be sited as near as possible to triage areas, on the same floor, with broad, unobstructed passages. There should be provision for easy passage of equipment and trolleys through doors and corridors and large circulation areas, working rooms and waiting spaces with plugs, for electricity, piped oxygen and suction and devices for hanging infusion sets. Two entrances to the department would enable one-way flow through the department. All work should be done in the department and examinations performed with mobile units avoided. A radiologist at the entrance to the department and a control post at the exit ensure adequate processing of casualties. Interpretation and reporting should be done by assigned radiologists dictating directly to typists; films and reports should accompany casualties.

Can more efficient use be made of x ray examinations in the accident and emergency department?

Increasing workloads in our radiology department prompted a study of casualty officers' use of x ray examinations, of which there were 5463 in the period. While casualty officers were in post referrals for x ray examination did not become more selective, but skills in interpreting films improved. Overall, 4.9% of trauma radiographs were misinterpreted, but this fell from 7.1% to 2.9% during tenure of post. One in four errors was clinically important. Clinical guidelines for selective radiography produced a significant and sustained reduction in the number of x ray examinations requested by the department. Analysis of one common injury indicated that the quality of patient care was not adversely affected. The number of x ray examinations carried out in the accident and emergency department can be reduced by using guidelines, and this does not compromise the quality of patient care. Appreciable savings may be made in patients' waiting times and radiodiagnostic expenditure.

Assessing the quality of care in a hospital emergency unit: a framework and its application

This article presents a specific application of a general approach to quality assessment by describing in detail both the theoretical and the practical aspects of quality assessment in a hospital emergency unit. The theoretical framework is established by specifying the level and scope of assessment, adopting a definition of quality, modeling emergency care as a succession of phases, specifying evaluative attributes, choosing a method for case selection, and selecting one or more methods of assessment. The practical application of this framework is illustrated using two methods, "tracer" and "trajectory," and selecting specific conditions amenable to assessment with each method.