Opportunities for Targeted Education: Critical Neuroradiologic Findings Missed or Misinterpreted by Residents and Fellows

Am I Ready to Be an Independent Neuroradiologist? Objective Trends in Neuroradiology Fellows' Performance during the Fellowship Year

BACKGROUND AND PURPOSE

Aside from basic Accreditation Council for Graduate Medical Education guidelines, few metrics are in place to monitor fellows' progress. The purpose of this study was to determine objective trends in neuroradiology fellowship training on-call performance during an academic year.

MATERIALS AND METHODS

We retrospectively reviewed the number of cross-sectional neuroimaging studies dictated with complete reports by neuroradiology fellows during independent call. Monthly trends in total call cases, report turnaround times, relationships between volume and report turnaround times, and words addended to preliminary reports by attending neuroradiologists were evaluated with regression models. Monthly variation in frequencies of call-discrepancy macros were assessed via χ² tests. Changes in frequencies of specific macro use between fellowship semesters were assessed via serial 2-sample tests of proportions.

RESULTS

From 2012 to 2017, for 29 fellows, monthly median report turnaround times significantly decreased during the academic year: July (first month) = 79 minutes (95% CI, 71-86 minutes) and June (12th month) = 55 minutes (95% CI, 52-60 minutes; P value = .023). Monthly report turnaround times were inversely correlated with total volumes for CT (r = -0.70, F = 9.639, P value = .011) but not MR imaging. Words addended to preliminary reports, a surrogate measurement of report clarity, slightly improved and discrepancy rates decreased during the last 6 months of fellowship. A nadir for report turnaround times, discrepancy errors, and words addended to reports was seen in December and January.

CONCLUSIONS

Progress through fellowship correlates with a decline in report turnaround times and discrepancy rates for cross-sectional neuroimaging call studies and slight improvement in indirect quantitative measurement of report clarity. These metrics can be tracked throughout the academic year, and the midyear would be a logical time point for programs to assess objective progress of fellows and address any deficiencies.

Diagnostic Performance of Ultrasonography for Pediatric Appendicitis: A Night and Day Difference?

RATIONALE AND OBJECTIVES

For imaging pediatric appendicitis, ultrasonography (US) is preferred because of its lack of ionizing radiation, but is limited by operator dependence. This study investigates the US diagnostic performance during night shifts covered by radiology trainees compared to day shifts covered by attending radiologists.

MATERIALS AND METHODS

Appy-Scores (1 = completely visualized normal appendix; 2 = partially visualized normal appendix; 3 = nonvisualized appendix with no inflammatory changes in the expected region of the appendix; 4 = equivocal; 5a = nonperforated appendicitis; 5b = perforated appendicitis) from 2935 US examinations (2161:774, day-to-night) from July 2013 to 2014 were correlated with the intraoperative diagnoses and the clinical follow-up. The diagnostic performance of trainees and attendings was compared with Fisher exact test. Interobserver agreement was measured by Cohen kappa coefficient.

RESULTS

Appendicitis prevalence was 25.3% (day) and 22.5% (night). Sensitivity, specificity, accuracy, negative predictive value, and positive predictive vale were 94.0%, 93.7%, 93.8%, 97.9%, and 83.4% during the day and 92.0%, 91.2%, 91.3%, 97.5%, and 75.2% at night. Specificity (P = .048) and positive predictive value (P = .011) differed, with more false positives at night (7%) than during the day (4.7%). Trainee and attending agreement was high (k = 0.995), with Appy-Scores of 1, 4, and 5a most frequently discordant.

CONCLUSIONS

US has a high diagnostic performance and interobserver agreement for pediatric appendicitis when interpreted by radiology trainees during night shifts or attending radiologists during day shifts. However, lower specificity and positive predictive value at night warrants a thorough trainee education to avoid false-positive examinations.

Attending Radiologist Variability and Its Effect on Radiology Resident Discrepancy Rates

RATIONALE AND OBJECTIVES

Discrepancy rates for interpretations produced in a call situation are one metric to evaluate residents during training. Current benchmarks, reported in previous studies, do not consider the effects of practice pattern variability among attending radiologists. This study aims to investigate the impact of attending variability on resident discrepancy rates to determine if the current benchmarks are an accurate measure of resident performance and, if necessary, update discrepancy benchmarks to accurately identify residents performing below expectations.

MATERIALS AND METHODS

All chest radiographs, musculoskeletal (MSK) radiographs, chest computed tomographies (CTs), abdomen and pelvis CTs, and head CTs interpreted by postgraduate year-3 residents in a call situation over 5 years were reviewed for the presence of a significant discrepancy and composite results compared to prior findings. Simulations of the expected discrepancy distribution for an "average resident" were then performed using Gibbs sampling, and this distribution was compared to the actual resident distribution.

RESULTS

A strong inverse correlation between resident volume and discrepancy rates was found. There was wide variability among attendings in both overread volume and propensity to issue a discrepancy, although there was no significant correlation. Simulations show that previous benchmarks match well for chest radiographs, abdomen and pelvis CTs, and head CTs but not for MSK radiographs and chest CTs. The simulations also demonstrate a large effect of attending practice patterns on resident discrepancy rates.

CONCLUSIONS

The large variability in attending practice patterns suggests direct comparison of residents using discrepancy rates is unlikely to reflect true performance. Current benchmarks for chest radiographs, abdomen and pelvis CTs, and head CTs are appropriate and correctly flag residents whose performance may benefit from additional attention, whereas those for MSK radiographs and chest CTs are likely too strict.

Comparing Preliminary and Final Neuroradiology Reports: What Factors Determine the Differences?

BACKGROUND AND PURPOSE

Trainees' interpretations of neuroradiologic studies are finalized by faculty neuroradiologists. We aimed to identify the factors that determine the degree to which the preliminary reports are modified.

MATERIALS AND METHODS

The character length of the preliminary and final reports and the percentage character change between the 2 reports were determined for neuroradiology reports composed during November 2012 to October 2013. Examination time, critical finding flag, missed critical finding flag, trainee level, faculty experience, imaging technique, and native-versus-non-native speaker status of the reader were collected. Multivariable linear regression models were used to evaluate the association between mean percentage character change and the various factors.

RESULTS

Of 34,661 reports, 2322 (6.7%) were read by radiology residents year 1; 4429 (12.8%), by radiology residents year 2; 3663 (10.6%), by radiology residents year 3; 2249 (6.5%), by radiology residents year 4; and 21,998 (63.5%), by fellows. The overall mean percentage character change was 14.8% (range, 0%-701.8%; median, 6.6%). Mean percentage character change increased for a missed critical finding (+41.6%, P < .0001), critical finding flag (+1.8%, P < .001), MR imaging studies (+3.6%, P < .001), and non-native trainees (+4.2%, P = .018). Compared with radiology residents year 1, radiology residents year 2 (-5.4%, P = .002), radiology residents year 3 (-5.9%, P = .002), radiology residents year 4 (-8.2%, P < .001), and fellows (-8.7%; P < .001) had a decreased mean percentage character change. Senior faculty had a lower mean percentage character change (-6.88%, P < .001). Examination time and non-native faculty did not affect mean percentage character change.

CONCLUSIONS

A missed critical finding, critical finding flag, MR imaging technique, trainee level, faculty experience level, and non-native-trainee status are associated with a higher degree of modification of a preliminary report. Understanding the factors that influence the extent of report revisions could improve the quality of report generation and trainee education.