Radiology residents' on-call interpretation of chest radiographs for pneumonia

An Evaluation of the Quality of Plain Radiograph Interpretations by Radiology Trainees: A Single Institution Experience

Introduction: The primary communication between the radiologist and referrer is through the radiological report. However, there are incidents of misinterpretation during radiologist training. Therefore, the present study aimed to evaluate the accuracy level and incidence of interpretation errors for plain radiographs among radiology trainees at our institution. Materials and Methods: The present study retrospectively reviewed 508 reported plain radiographs for one year, and two radiologists subsequently evaluated these plain radiographs. The initial diagnosis by the trainee was compared with the radiologists’ evaluation, and the results were categorized as either ‘accurate’, ‘minor discrepancy’, or ‘major discrepancy’. The data were analyzed concerning the overall performance, year of trainee, anatomic area, patient age group, and radiograph type. A chi-square test was performed, with p < 0.05 indicating statistical significance. Results: The overall accuracy rate was 69%, with minor and major discrepancy rates of 21% and 10%, respectively. There was an insignificant increase in overall accuracy with increased years of training, despite a reduction to 58% accuracy among Year 3 trainees. The accuracy level increased between Year 1, Year 2 and Year 4 by 70%, 71% and 75%, respectively (p > 0.05). The accuracy rates for both the adult and pediatric age groups were not statistically significant. The mobile radiographs showed lower accuracy rate of reporting than the plain radiographs. Conclusion: The radiological trainee interpretations for plain radiographs had an average rating with low discrepancy rates. The Year 3 trainees had the lowest accuracy compared to the other trainee groups. However, the present study suggests the need for further research to determine if the current outcomes are outliers or are indicative of a real phenomenon.

Added Value of Bone Suppression Image in the Detection of Subtle Lung Lesions on Chest Radiographs with Regard to Reader's Expertise

BACKGROUND

Chest radiographs (CXR) are the most commonly used imaging techniques by various clinicians and radiologists. However, detecting lung lesions on CXR depends largely on the reader's experience level, so there have been several trials to overcome this problem using post-processing of CXR. We investigated the added value of bone suppression image (BSI) in detecting various subtle lung lesions on CXR with regard to reader's expertise.

METHODS

We applied a software program to generate BSI in 1,600 patients in the emergency department. Of them, 80 patients with subtle lung lesions and 80 patients with negative finding on CXR were retrospectively selected based on the subtlety scores on CXR and CT findings. Ten readers independently rated their confidence in deciding the presence or absence of a lung lesion at each of 960 lung regions on the two separated imaging sessions: CXR alone vs. CXR with BSI.

RESULTS

The additional use of BSI for all readers significantly increased the mean area under the curve (AUC) in detecting subtle lung lesions (0.663 vs. 0.706; P < 0.001). The less experienced readers were, the more AUC differences increased: 0.067 (P < 0.001) for junior radiology residents; 0.064 (P < 0.001) for non-radiology clinicians; 0.044 (P < 0.001) for senior radiology residents; and 0.019 (P = 0.041) for chest radiologists. The additional use of BSI significantly increased the mean confidence regarding the presence or absence of lung lesions for 213 positive lung regions (2.083 vs. 2.357; P < 0.001) and for 747 negative regions (1.217 vs. 1.195; P = 0.008).

CONCLUSION

The use of BSI increases diagnostic performance and confidence, regardless of reader's expertise, reduces the impact of reader's expertise and can be helpful for less experienced clinicians and residents in the detection of subtle lung lesions.

The Value of Supine Chest X-Ray in the Diagnosis of Pneumonia in the Basal Lung Zones

RATIONALE AND OBJECTIVES

Basal lung opacities are frequently observed on supine chest x-ray (SCXR) of intensive care patients, causing insecurity among clinicians and radiologists. We sought to determine the diagnostic accuracy of SCXR for basal pneumonia.

MATERIALS AND METHODS

We identified 172 patients who received both SCXR and computed tomography within 1 hour. Two readers examined the SCXR and rated findings in both basal zones according to the following scale: 0 = "no pneumonia," 1 = "possible pneumonia," 2 = "highly suspected pneumonia." Computed tomography served as standard of reference. Sensitivity, specificity, and positive and negative predictive values (PPV/NPV) were calculated once pooling 0 and 1 as negative and once pooling 1 and 2 as positive finding.

RESULTS

When pooling 0 and 1 as negative, sensitivity was 0.45 (right)/0.38 (left), specificity was 0.94/0.97, PPV was 0.76/0.79, and NPV was 0.81/0.84. When pooling 1 and 2 as positive, sensitivity was 0.80/0.75, specificity was 0.62/0.58, PPV was 0.45/0.35, and NPV was 0.88/0.89. The most common findings in false-positive cases were combined pleural effusions and lower lobe atelectasis.

CONCLUSIONS

Interpreting only highly suspicious basal opacities as pneumonia considerably increases the PPV with almost constant NPV. Clinicians and radiologists should be aware of the limitations of SCXR regarding basal pneumonia.

Final year physiotherapy student's reliability in chest X-ray interpretation

BACKGROUND

Newly qualified physiotherapists may be required to interpret an unreported chest X-ray (CXR) as part of their physiotherapy assessment in "on call" situations. Their interpretation may influence the patient management strategies they adopt. There is no research published which have tested the reliability of final year physiotherapy students in CXR interpretation.

METHODS

Twenty-five final year physiotherapy students were asked to view and interpret the findings of six CXRs, together with a brief vignette, typical of a single commonly encountered diagnosis. Students were also asked if they had received additional CXR training on placement or had a desire to specialize in respiratory care.

RESULTS

The CXR interpretations were scored as incorrect 0, partially correct 1 (abnormality detected but not able to diagnose or missed some detail) and 2 correct. Scores for each of the six CXRs were added to give a total score (out of 12). The median score was 3 out of 12, (range 0-9). Median scores were slightly higher at 4 out of 12 in those students with additional training or a desire to specialize (range 1-7), but this was not statistically significant (p = 0.43).

CONCLUSIONS

Final year physiotherapy students were not able to reliably interpret CXRs. These findings were consistent with previous published research involving medical students. Therefore on graduation before starting "on call" duties it is recommended newly qualified physiotherapists receive additional training in CXR interpretation.

Radiologist-initiated double reading of abdominal CT: retrospective analysis of the clinical importance of changes to radiology reports

Background

Misinterpretation of radiological examinations is an important contributing factor to diagnostic errors. Consultant radiologists in Norwegian hospitals frequently request second reads by colleagues in real time. Our objective was to estimate the frequency of clinically important changes to radiology reports produced by these prospectively obtained double readings.

Methods

We retrospectively compared the preliminary and final reports from 1071 consecutive double-read abdominal CT examinations of surgical patients at five public hospitals in Norway. Experienced gastrointestinal surgeons rated the clinical importance of changes from the preliminary to final report. The severity of the radiological findings in clinically important changes was classified as increased, unchanged or decreased.

Results

Changes were classified as clinically important in 146 of 1071 reports (14%). Changes to 3 reports (0.3%) were critical (demanding immediate action), 35 (3%) were major (implying a change in treatment) and 108 (10%) were intermediate (requiring further investigations). The severity of the radiological findings was increased in 118 (81%) of the clinically important changes. Important changes were made less frequently when abdominal radiologists were first readers, more frequently when they were second readers, and more frequently to urgent examinations.

Conclusion

A 14% rate of clinically important changes made during double reading may justify quality assurance of radiological interpretation. Using expert second readers and a targeted selection of urgent cases and radiologists reading outside their specialty may increase the yield of discrepant cases.

Lingular pneumonia obscured by implanted cardioverter-defibrillator: Lateral thinking

A 56-year-old female with an implanted cardioverter-defibrillator was admitted with a short history suggestive of a diagnosis of pneumonia. An AP radiograph did not identify an area of consolidation. A subsequent lateral radiograph highlighted an extensive left-lingular-lobe consolidation that had been obscured by the cardiac device. This case highlights the fact that large devices can obscure significant pathology, and that lateral or cross-sectional imaging may be helpful in reaching a diagnosis.

Improved detection of focal pneumonia by chest radiography with bone suppression imaging

OBJECTIVE

To evaluate radiologists' ability to detect focal pneumonia by use of standard chest radiographs alone compared with standard plus bone-suppressed chest radiographs.

METHODS

Standard chest radiographs in 36 patients with 46 focal airspace opacities due to pneumonia (10 patients had bilateral opacities) and 20 patients without focal opacities were included in an observer study. A bone suppression image processing system was applied to the 56 radiographs to create corresponding bone suppression images. In the observer study, eight observers, including six attending radiologists and two radiology residents, indicated their confidence level regarding the presence of a focal opacity compatible with pneumonia for each lung, first by use of standard images, then with the addition of bone suppression images. Receiver operating characteristic (ROC) analysis was used to evaluate the observers' performance.

RESULTS

The mean value of the area under the ROC curve (AUC) for eight observers was significantly improved from 0.844 with use of standard images alone to 0.880 with standard plus bone suppression images (P < 0.001) based on 46 positive lungs and 66 negative lungs.

CONCLUSION

Use of bone suppression images improved radiologists' performance for detection of focal pneumonia on chest radiographs.

KEY POINTS

Bone suppression image processing can be applied to conventional digital radiography systems. Bone suppression imaging (BSI) produces images that appear similar to dual-energy soft tissue images. BSI improves the conspicuity of focal lung disease by minimizing bone opacity. BSI can improve the accuracy of radiologists in detecting focal pneumonia.

Factors affecting attending agreement with resident early readings of computed tomography and magnetic resonance imaging of the head, neck, and spine

RATIONALE AND OBJECTIVES

This study examines the joint effect of several factors on radiology resident performance in the task of interpreting after-hours neuroradiology examinations.

MATERIALS AND METHODS

As part of a quality assessment process, we conducted a prospective evaluation of all (N = 21,796) after-hours preliminary readings of neuroradiology examinations performed by radiology residents over a 62-month period at our academic medical center. Each reading was scored by the interpreting neuroradiologist as "agree," "disagree with minimal clinical impact," and "disagree with significant clinical impact." Coded resident and attending identities were also recorded for each case along with modality, body area studied, and the date of examination. These raw data were used to create an analytic data set with level of resident/attending agreement as the outcome and six predictors, including two date-derived variables: months 1-62 representing when the case occurred during the study and quartiles 1-4 accounting for the timing of the case in each resident's own experience. Cross tabulations, plots, bivariate statistics, and logistic regression were used to examine the relationships between study variables and the outcome (level of agreement).

RESULTS

Over about 5 years of the study, the absolute number of significant disagreements remained stable at about three per month. The total caseload increased at a rate of 4.1 per month with most of the increase falling into the agree category, whereas the minimal disagreements actually decreased slightly (0.2 per month). In the logistic model for disagreement, three of the factors accounted for most of the variance: attending (61%), resident (15%), and month (15%). Study type (modality and area examined) accounted for another 10%. There was no significant contribution from the variable (quartile) constructed to test for individual resident learning during the on-call experience.

CONCLUSION

Although residents differ somewhat in the extent of attending agreement with their on-call work, evaluation or remediation made on the basis of simple comparison of these rates should be done with caution. Improved agreement over time seems to be a collective experience shared by residents.

Academic Radiologists’ On-Call and Late-Evening Duties

On-call and late-evening duties have increased dramatically for radiologists, be they in private practice, at academic medical centers, or at state or federal government health care facilities. Most busy medical centers in North America require around-the-clock radiology interpretations for emergent or urgent patients, particularly if they are level 1 trauma centers. Coverage by attending radiologists around the clock is expensive and difficult to implement. In this study, an e-mail questionnaire was sent to 83 members of the Society of Chairmen of Academic Radiology Departments concerning general radiologists' on-call and after-hours duties. Detailed replies were received from 29 academic medical centers, all of which were university owned or affiliated. There was complex variation on how academic radiology departments approached their after-hours commitments, but only 10% of academic institutions (3 of 29) answering the survey had 24-hour in-house coverage by general radiologists. Coverage by attending radiologists around the clock at academic medical centers is not the current standard of practice at most academic medical centers.

Radiology residents' on-call interpretation of chest radiographs for congestive heart failure

RATIONALE AND OBJECTIVES

This study was designed to evaluate the performance of radiology residents in interpreting emergency department (ED) chest radiographs for congestive heart failure and to characterize the factors associated with a subsequent amended interpretation by an attending radiologist.

MATERIALS AND METHODS

We retrospectively reviewed all amended reports for ED chest radiographs between January 2004 and July 2005 and identified those with discrepant interpretations regarding the diagnosis of congestive heart failure. A total of 1.9% (476 of 24,600) of chest radiographs were amended over the study period. Forty-eight patients (75% female, mean age 66 years) whose chest radiograph was amended for the diagnosis of congestive heart failure and were available for review formed the study population. A control group of 35 patients (69% female, mean age 67 years) were individually matched to a convenience subset of patients by age, gender, clinical indication, and radiographic projection. Chest radiographs were in the anteroposterior projection in 62% (30 of 48) of study patients and 60% (21 of 35) of controls. A blinded expert panel of three board-certified cardiothoracic radiologists jointly reviewed each chest radiograph for the presence or absence of congestive heart failure and its specific radiographic findings.

RESULTS

The expert panel diagnosed congestive heart failure in 19% (9 of 48) of study patients and in 23% (8 of 35) of controls (P = .65). When present, congestive heart failure was mild to moderate in severity in both the study and control groups (P = 1.00). There was a significant difference in the expert panel agreement between the attending versus the resident interpretation (65% versus 35%, P = .008), for the study group. This resulted in fair agreement (kappa = 0.29) between the expert panel and the attending interpretation and no agreement (kappa = -0.29) between the expert panel and the resident interpretation. In contrast, the expert panel agreed with the joint resident/attending interpretation in 83% (29 of 35) of controls, yielding substantial agreement (kappa = 0.72).

CONCLUSION

Interpretation of chest radiographs for congestive heart failure by radiology residents has a low error rate. The majority of chest radiographs with discrepant resident and attending interpretations were portable films of female patients with subtle radiographic findings of congestive heart failure, and were inherently difficult to interpret.

A computer-based radiology simulator as a learning tool to help prepare first-year residents for being on call

RATIONALE AND OBJECTIVES

The start of call is a stressful time for radiology residents. Traditional teaching methods are not ideal for call preparation because they are radically different than the task performed on call. The purpose of this study is to determine if a computer-based radiology simulator would have an effect on resident confidence level or diagnostic abilities.

MATERIALS AND METHODS

A simulator was created to mimic the picture archive and communication system (PACS) at our hospital. Typical call-level cases were selected, anonymized, and entered into the database. The first-year residents were randomly split into a control group and a study group that used the simulator. Each resident took a survey 1 month before and after beginning call to measure his or her subjective feeling of preparedness and nervousness. Objective measures were also obtained through the use of discordance levels from on-call cases.

RESULTS

Seventy-one cases were entered into the simulator. Of the 12 residents in the first-year class, 7 were placed in the study group and 5 in the control group. The residents in both groups claimed they felt more prepared and less nervous 1 month after starting call. The differences at survey were not significant, but the residents in the study group trended toward feeling more prepared and less nervous. There was no statistical difference in the discordance rates for on-call cases between the two groups.

CONCLUSIONS

Although statistical significance was not reached between the users of the radiology simulator and the control group, there was a subjective feeling that the simulator was useful for call preparation and as an interactive learning tool. A larger sample study group size may show statistical significance.

Preliminary reports in the emergency department: is a subspecialist radiologist more accurate than a radiology resident?

RATIONALE AND OBJECTIVES

To determine whether emergency department (ED) preliminary reports rendered by subspecialist attending radiologists who are reading outside their field of expertise are more accurate than reports rendered by radiology residents, and to compare error rates between radiologists and nonradiologists in the ED setting.

MATERIALS AND METHODS

The study was performed at a large academic medical center with a busy ED. An electronic preliminary report generator was used in the ED to capture preliminary interpretations rendered in a clinical setting by radiology residents, junior attendings (within 2 years of taking their oral boards), senior attendings, and ED clinicians between August 1999 and November 2004. Each preliminary report was later reviewed by a final interpreting radiologist, and the preliminary interpretation was adjudicated for the presence of substantial discordances, defined as a difference in interpretation that might immediately impact the care of the patient. Of the 612,890 preliminary reports in the database, 65,780 (11%) met inclusion criteria for this study. A log-linear analysis was used to assess the effects of modality and type of author on preliminary report error rates.

RESULTS

ED clinicians had significantly higher error rates when compared with any type of radiologist, regardless of modality. Within the radiologists, residents and junior attendings had lower error rates than did senior attendings, but the differences were not statistically significant.

CONCLUSION

Subspecialized attending radiologists who interpret ED examinations outside their area of expertise have error rates similar to those of radiology residents. Nonradiologists have significantly higher error rates than radiologists and radiology residents when interpreting examinations in the ED.

Part I: preparing first-year radiology residents and assessing their readiness for on-call responsibilities

RATIONALE AND OBJECTIVES

The aim of the study is to evaluate the effectiveness of an Emergency Radiology (ER) Core Curriculum training module and a Digital Imaging and Communications in Medicine (DICOM)-based interactive examination system to prepare first-year (postgraduate year 2 [PGY-2]) radiology residents and assess their readiness for taking overnight radiology call.

MATERIALS AND METHODS

Institutional review board approval was obtained, and the study was compliant with Health Insurance Portability and Accountability Act (HIPAA) regulations. A dedicated month-long ER curriculum was designed to prepare new radiology residents for overnight radiology call that includes interpretation of off-hour urgent and emergent studies without immediate direct attending supervision. Lectures of the curriculum, provided by department staff, were based on the American Society of Emergency Radiology core curriculum. The lecture series was implemented after PGY-2 residents had completed formal introductory resident rotations during their first 6 months of training. A DICOM-based interactive computer-based testing module was developed and administered at the end of the lecture series. The module consisted of 19 actual emergency department cases with entire series of images, simulating an on-call setting. Tests were scored by two staff members blinded to resident identifying information. Upper-level residents also were tested, and comparison was made between first-year and upper-level resident test scores to determine the effectiveness of the test in determining first-year resident preparedness for call. Statistical analysis of results was performed by using t-test (P < .05).

RESULTS

All residents in the residency program present during the month (nine PGY-2, six PGY-3, seven PGY-4, seven PGY-5 residents) attended the lecture series and finished the testing module at the end of the lecture series. Of 19 actual emergency cases on the testing module, five cases were neuroradiology, three cases were thoracic imaging, eight cases were body imaging, and three cases were musculoskeletal. PGY-2 residents scored an average of 73.0% (range, 63.2%-81.6%) of total points possible. PGY-3 residents scored an average of 76.8% (range, 68.4%-86.8%); PGY-4 residents scored an average of 77.4% (range, 65.8%-100%), and PGY-5 residents scored an average of 81.2% (range, 68.4%-94.7%). There was no statistically significant difference in scores according to level of training.

CONCLUSION

First-year radiology residents who underwent 6 months of formal radiology training followed by an intensive ER lecture series before taking overnight call had scores similar to upper-level colleagues on an interactive computer-based ER simulation module.