Out of hours multidetector computed tomography pulmonary angiography: are specialist resident reports reliable?

Resident-faculty overnight discrepancy rates as a function of number of consecutive nights during a week of night float

Objectives

In 2018, the ACGME (Accreditation Council for Graduate Medical Education) made a change to the maximum permissible number of consecutive nights a resident trainee can be on “night float,” from six to seven nights. To our knowledge, although investigators have studied overall discrepancy rates and discrepancy rates as a function of shift length or perceived workload of a particular shift, no study has been performed to evaluate resident-faculty discrepancy rates as a quality/performance proxy, to see whether resident performance declines as a function of the number of consecutive nights. Our hypothesis is that we would observe a progressive increase in significant overnight resident – attending discrepancies over the 7 days’ time.

Methods

A total of 8,488 reports were extracted between 4/26/2019 to 8/22/2019 retrospectively. Data was obtained from the voice dictation system report server. Exported query was saved as a .csv file format and analyzed using Python packages. A “discrepancy checker” was created to search all finalized reports for the departmental standard heading of “Final Attending Report,” used to specify any significant changes from the preliminary interpretation.

Results

Model estimates varied on different days however there were no trends or patterns to indicate a deterioration in resident performance throughout the week. There were comparable probabilities throughout the week, with 2.17% on Monday, 2.35% on Thursday and 2.05% on Friday.

Conclusions

Our results reveal no convincing trend in terms of overnight report discrepancies between the preliminary report generated by the night float resident and the final report issued by a faculty the following morning. These results are in support of the ACGME’s recent change in the permissible number of consecutive nights on night float. We did not prove our hypothesis that resident performance on-call in the domain of report accuracy would diminish over seven consecutive nights while on the night float rotation. Our results found that performance remained fairly uniform over the course of the week.

Radiologist-initiated double reading of abdominal CT: retrospective analysis of the clinical importance of changes to radiology reports

Background

Misinterpretation of radiological examinations is an important contributing factor to diagnostic errors. Consultant radiologists in Norwegian hospitals frequently request second reads by colleagues in real time. Our objective was to estimate the frequency of clinically important changes to radiology reports produced by these prospectively obtained double readings.

Methods

We retrospectively compared the preliminary and final reports from 1071 consecutive double-read abdominal CT examinations of surgical patients at five public hospitals in Norway. Experienced gastrointestinal surgeons rated the clinical importance of changes from the preliminary to final report. The severity of the radiological findings in clinically important changes was classified as increased, unchanged or decreased.

Results

Changes were classified as clinically important in 146 of 1071 reports (14%). Changes to 3 reports (0.3%) were critical (demanding immediate action), 35 (3%) were major (implying a change in treatment) and 108 (10%) were intermediate (requiring further investigations). The severity of the radiological findings was increased in 118 (81%) of the clinically important changes. Important changes were made less frequently when abdominal radiologists were first readers, more frequently when they were second readers, and more frequently to urgent examinations.

Conclusion

A 14% rate of clinically important changes made during double reading may justify quality assurance of radiological interpretation. Using expert second readers and a targeted selection of urgent cases and radiologists reading outside their specialty may increase the yield of discrepant cases.

Residents' performance in the interpretation of on-call "triple-rule-out" CT studies in patients with acute chest pain

RATIONALE AND OBJECTIVES

To evaluate the performance of radiology residents in the interpretation of on-call, emergency "triple-rule-out" (TRO) computed tomographic (CT) studies in patients with acute chest pain.

MATERIALS AND METHODS

The study was institutional review board-approved and Health Insurance Portability and Accountability Act compliant. Data from 617 on-call TRO studies were analyzed. Dedicated software enables subspecialty attendings to grade discrepancies in interpretation between preliminary trainee reports and their final interpretation as "unlikely to be significant" (minor discrepancies) or "likely to be significant" for patient management (major discrepancies). The frequency of minor, major and all discrepancies in resident's TRO interpretations was compared to 609 emergent non-electrocardiography (ECG)-synchronized chest CT studies using Pearson χ(2) test.

RESULTS

Minor discrepancies occurred more often in the TRO group (9.1% vs. 3.9%, P < .001), but there was no difference in the frequency of major discrepancies (2.1% vs. 2.8%, P = .55). Minor discrepancies in the TRO group most commonly resulted from missed extrathoracic findings with missed liver lesions being the most frequent. Major discrepancies mostly encompassed cardiac and extracardiac vascular findings but did not result in unnecessary interventions, significant immediate changes in management, or adverse patient outcomes.

CONCLUSIONS

On-call resident interpretation of TRO CT studies in patients with acute chest pain is congruent with final subspecialty attending interpretation in the overwhelming majority of cases. The rate of discrepancies likely to affect patient management in this domain is not different from emergent non-ECG-synchronized chest CT.

CT in adults: systematic review and meta-analysis of interpretation discrepancy rates

PURPOSE

To use meta-analysis to determine the discrepancy rate when interpreting computed tomography (CT) studies performed in adult patients and to determine whether discrepancy rate differs on the basis of body region or level of radiologist training.

MATERIALS AND METHODS

MEDLINE and EMBASE were searched from 1946 to June 2012 by using the combination "radiology AND (error OR peer review)." Two reviewers independently selected studies that met the inclusion criteria and extracted study data. Total and major discrepancy rates were investigated with a random-effects meta-analysis, and subgroups were compared by using the χ(2) Q statistic. Subgroup analyses were performed on the basis of the level of training of the initial radiologist and the body system scanned.

RESULTS

Fifty-eight studies met the inclusion criteria (388 123 CT examinations). The pooled total discrepancy rate was 7.7% (95% confidence interval [CI]: 5.6%, 10.3%), and the major discrepancy rate was 2.4% (95% CI: 1.7%, 3.2%). The pooled major discrepancy rate was comparable for staff (2.9%; 95% CI: 1.2%, 6.7%) and residents (2.2%; 95% CI: 1.7%, 2.9%) (Q = 0.92, P = .633). The pooled major discrepancy rates for head CT (0.8%; 95% CI: 0.4%, 1.6%) and spine CT (0.7%; 95% CI: 0.2%, 2.7%) were lower than those for chest CT (2.8%; 95% CI: 1.5%, 5.4%) and abdominal CT (2.6%; 95% CI: 1.0%, 6.7%) (Q = 8.28, P = .041). Lack of blinding of the reference radiologist to the initial report was associated with a lower major discrepancy rate (2.0%; 95% CI: 1.4%, 2.7%; 43 studies) than when blinding was present (12.1%; 95% CI: 4.4%, 29.4%; five studies) (Q = 10.65, P = .001).

CONCLUSION

Potentially useful reference ranges were identified in the subgroup analyses on the basis of body region scanned at adult CT. However, considerable heterogeneity that is only partially explained by subgroup analysis signifies that further research is necessary--particularly regarding the question of blinding of the reference radiologist.