Effect of work hours, caseload, shift type, and experience on resident call performance

A different way to diagnosis acute appendicitis: machine learning

<b><br>Indroduction:</b> Machine learning is a branch of artificial intelligence based on the idea that systems can learn from data, identify patterns, and make decisions with minimal human intervention.</br> <b><br>Aim:</b> Our aim is to predict acute appendicitis, which is the most common indication for emergency surgery, using machine learning algorithms with an easy and inexpensive method.</br> <b><br>Materials and methods:</b> Patients who were treated surgically with a prediagnosis of acute appendicitis in a single center between 2011 and 2021 were analyzed. Patients with right lower quadrant pain were selected. A total of 189 positive and 156 negative appendectomies were found. Gender and hemogram were used as features. Machine learning algorithms and data analysis were made in Python (3.7) programming language.</br> <b><br>Results:</b> Negative appendectomies were found in 62% (n = 97) of the women and in 38% (n = 59) of the men. Positive appendectomies were present in 38% (n = 72) of the women and 62% (n = 117) of the men. The accuracy in the test data was 82.7% in logistic regression, 68.9% in support vector machines, 78.1% in k-nearest neighbors, and 83.9% in neural networks. The accuracy in the voting classifier created with logistic regression, k-nearest neighbor, support vector machines, and artificial neural networks was 86.2%. In the voting classifier, the sensitivity was 83.7% and the specificity was 88.6%.</br> <b><br>Conclusions:</b> The results of our study show that machine learning is an effective method for diagnosing acute appendicitis. This study presents a practical, easy, fast, and inexpensive method to predict the diagnosis of acute appendicitis.</br>.

After-hours radiology reporting in an Australian tertiary hospital: Factors influencing the rate of discrepancy for computed tomography reporting

INTRODUCTION

Demand for after-hours radiology services across Australia has been rapidly increasing for many years. Public hospital radiology departments are generally staffed by trainees after hours, often 1-2 doctors. This study reviewed 1 year of after-hours data at a tertiary hospital in Sydney, to determine overall discrepancy rates and factors which influenced the rate of discrepancy after hours. Overall radiology and after-hours case volumes are increasing, and the goal was to ensure that our department practices are within safe limits at current staffing levels.

METHODS

After-hours computed tomography scan reports issued by registrars were reviewed over 1 year between 01 January 2019 and 01 January 2020. Data recorded included overall case numbers on each shift, discrepancy rates and rates of clinically significant discrepancy. Caseload was measured by equivalent ascribed reporting time as described by Pitman et al. (2018).

RESULTS

There were 10,886 cases reported after hours during the study period and 1,412 discrepancies (13% cases). Of these, 300 (3.7%) were clinically significant. Factors that increased the rate of significant discrepancy included shifts with caseloads greater than 3 h of equivalent consultant reporting time per 8-h shift and the radiology trainee's level of training. The frequency of discrepancy for trauma cases was similar to non-trauma after adjusting for study type; however, the frequency of significant discrepancy was higher.

CONCLUSION

In all instances, the rates of discrepancy and clinically significant discrepancy remained well within ranges previously described.

Diagnostic Accuracy Rates of Appendicitis Scoring Systems for the Stratified Age Groups

BACKGROUND

Many scoring systems have been developed for acute appendicitis, which is the most common emergent disorder in surgical practice. Considering the physiological changes and chronic diseases occurring with advancing age, an applied scoring system may not produce the same score in similar patients in all age groups.

OBJECTIVES

We aimed to compare the predictive values of scoring systems in different age groups.

METHODS

In this prospective study, the patients operated on in our clinic with a prediagnosis of acute appendicitis between March 2020 and March 2021 were included. We divided them into three age groups as 18-45 years (group 1), 46-65 years (group 2), and >65 years (group 3). We compared the scores of the nine acute appendicitis scoring systems most commonly used in the literature for these age groups.

RESULTS

A total of 203 patients were included in our study. The Alvarado scoring system yielded the most accurate results for group 1, whereas the Fenyo-Linberg scoring system was the most accurate system for group 2 and the Eskelinen scoring system for group 3.

CONCLUSION

Age should be considered as a major parameter during the selection of the scoring system to be applied for patients with prediagnosis of acute appendicitis. Our study revealed the Alvarado and the Fenyo-Lindberg scoring systems as the most accurate systems for the differential diagnosis of appendicitis in the 18-45 and 46-65 years age groups, respectively. Although we found the Eskelinen scoring system as the most accurate one in the >65 years age group, the confidence intervals indicated that it may not be appropriate for use alone in this group.

Interpretation discrepancies of abdominal imaging by on-call radiology residents: Evaluation of risk factors

The aim of this study was to determine the rate, important findings, and risk factors related to discrepancies between on-call residents’ and attending radiologists’ interpretations of abdominal examinations. We identified 1132 eligible patients with abdominal radiology findings that were preliminary interpreted by on-call residents between February 2016 and September 2019. The preliminary interpretations were compared with the final interpretations by abdominal attending radiologists, including clinical data. The preliminary interpretations were analyzed by three radiologists in consensus, who categorized the reports according to organs, important findings (i.e., active bleeding, bowel obstruction, organ ischemia or infarction, and organ rupture), clinical outcomes, and discrepancies with respect to final interpretations. Multiple logistic regression analysis was used to evaluate the risk factors for important discrepant findings. Of 1132 patients, the bowel (n = 567, 50.1%) was the most common organ interpreted by on-call residents, followed by gallbladder/bile duct/pancreas (n = 139, 12.3%) and liver (n = 116, 10.2%). Of 1132, 359 patients (31.7%) had disease with 379 important findings: active bleeding (n = 222), organ rupture (n = 77), bowel obstruction (n = 52), bowel ischemia (n = 24), and organ infarction (n = 4). Sixty-four patients (5.6%) showed discrepancies, and 30 (2.6%) showed 32 important discrepant findings comprising 14 active bleeding, 10 bowel obstructions, 6 organ ruptures, and 2 cases of bowel ischemia. Of the 64 discrepant patients, 33 underwent delayed surgery (n = 18, 28.1%) or interventional treatment (n = 15, 23.4%). In multivariable analysis, bowel obstruction (adjusted odds ratio, 2.52; p = 0.049) was an independent risk factor for determining discrepancy between preliminary and final interpretations. The rate of overall and important discrepancies between on-call residents’ and final interpretations was low. However, given that the bowel was the most frequently interpreted organ, bowel obstruction was identified as a risk factor for discrepant interpretations. The identified risk factor and findings may be useful for residents to minimize discrepancies.

Comparison of scoring systems regarding the gender as a parameter with the traditional scoring systems for predicting appendicitis

Although acute appendicitis remains the most common cause of acute abdomen in General Surgery practice, negative appendectomy rates are still high in particularly female patients. Appendicitis scoring systems considering gender can help the clinician to reduce negative appendectomy rates in females. This present study aims to compare the Lintula, Ripasa, Fenyo-Lindberg scoring systems, which use gender as a variable, with the Alvarado, Karaman, scoring systems to evaluate which CSS is more successful in the differential diagnosis of appendicitis in females. We analyzed the records of the patients operated on with a prediagnosis of acute appendicitis in our clinic between 2020 and 2021, retrospectively. Alvarado, adult appendicitis score (AAS), appendicitis inflammatory response score (AIRS), Ripasa, Karaman, Lintula, and Fenyo Lindberg scores were calculated for each patient. The patients were divided into two groups as male and female, according to gender. Receiver operator characteristic (ROC) curve analysis was used to identify the best cut-off value and assess the performance of the test score for appendicitis. Three hundred and sixty-three patients were included in the study. One hundred seventy-two (47.4%) of the patients were male, and 191 (52.6%) were female. Alvarado and AAS were the most valuable score in female (AUC: 0.805, sensitivity: 0.63, specificity: 0.83; and area under curve (AUC): 0.794, Sensitivity 0.71, Specificity: 0.76, respectively), male group (AUC: 0.828, Sensitivity: 0.71, Specificity: 0.83; and AUC: 0.834, Sensitivity 0.74, Specificity: 0.77, respectively), and when patients were not categorized by gender (AUC: 0.818, Sensitivity: 0.67 Specificity: 0.83; and AUC: 0.794, Sensitivity 0.71, Specificity: 0.76, respectively). Although the Alvarado scoring system is the first defined appendicitis scoring system, it seems as superior to the many scoring systems defined after it in predicting appendicitis, even in female patients.

The Accuracy of On-Call CT Reporting in Teleradiology Networks in Comparison to In-House Reporting

(1) Background: We aimed to compare the accuracy of after-hours CT reports created in a traditional in-house setting versus a teleradiology setting by assessing the discrepancy rates between preliminary and final reports. (2) Methods: We conducted a prospective study to determine the number and severity of discrepancies between preliminary and final reports for 7761 consecutive after-hours CT scans collected over a 21-month period. CT exams were performed during on-call hours and were proofread by an attending the next day. Discrepancies between preliminary and gold-standard reports were evaluated by two senior attending radiologists, and differences in rates were assessed for statistical significance. (3) Results: A total of 7209 reports were included in the analysis. Discrepancies occurred in 1215/7209 cases (17%). Among these, 433/7209 reports (6%) showed clinically important differences between the preliminary and final reports. A total of 335/5509 of them were in-house reports (6.1%), and 98/1700 were teleradiology reports (5.8%). The relative frequencies of report changes were not significantly higher in teleradiology. (4) Conclusions: The accuracy of teleradiology reports was not inferior to that of in-house reports, with very similar clinically important differences rates found in both reporting situations.

Quantifying Radiology Resident Fatigue: Analysis of Preliminary Reports

Background Workloads in radiology departments have constantly increased over the past decades. The resulting radiologist fatigue is considered a rising problem that affects diagnostic accuracy. Purpose To investigate whether data mining of quantitative parameters from the report proofreading process can reveal daytime and shift-dependent trends in report similarity as a surrogate marker for resident fatigue. Materials and Methods Data from 117 402 radiology reports written by residents between September 2017 and March 2020 were extracted from a report comparison tool and retrospectively analyzed. Through calculation of the Jaccard similarity coefficient between residents' preliminary and staff-reviewed final reports, the amount of edits performed by staff radiologists during proofreading was quantified on a scale of 0 to 1 (1: perfect similarity, no edits). Following aggregation per weekday and shift, data were statistically analyzed by using simple linear regression or one-way analysis of variance (significance level, P < .05) to determine relationships between report similarity and time of day and/or weekday reports were dictated. Results Decreasing report similarity with increasing work hours was observed for day shifts (r = -0.93 [95% CI: -0.73, -0.98]; P < .001) and weekend shifts (r = -0.72 [95% CI: -0.31, -0.91]; P = .004). For day shifts, negative linear correlation was strongest on Fridays (r = -0.95 [95% CI: -0.80, -0.99]; P < .001), with a 16% lower mean report similarity at the end of shifts (0.85 ± 0.24 at 8 am vs 0.69 ± 0.32 at 5 pm). Furthermore, mean similarity of reports dictated on Fridays (0.79 ± 0.35) was lower than that on all other weekdays (range, 0.84 ± 0.30 to 0.86 ± 0.27; P < .001). For late shifts, report similarity showed a negative correlation with the course of workweeks, showing a continuous decrease from Monday to Friday (r = -0.98 [95% CI: -0.70, -0.99]; P = .007). Temporary increases in report similarity were observed after lunch breaks (day and weekend shifts) and with the arrival of a rested resident during overlapping on-call shifts. Conclusion Decreases in report similarity over the course of workdays and workweeks suggest aggravating effects of fatigue on residents' report writing performances. Periodic breaks within shifts potentially foster recovery. © RSNA, 2021.

Resident-faculty overnight discrepancy rates as a function of number of consecutive nights during a week of night float

Objectives

In 2018, the ACGME (Accreditation Council for Graduate Medical Education) made a change to the maximum permissible number of consecutive nights a resident trainee can be on “night float,” from six to seven nights. To our knowledge, although investigators have studied overall discrepancy rates and discrepancy rates as a function of shift length or perceived workload of a particular shift, no study has been performed to evaluate resident-faculty discrepancy rates as a quality/performance proxy, to see whether resident performance declines as a function of the number of consecutive nights. Our hypothesis is that we would observe a progressive increase in significant overnight resident – attending discrepancies over the 7 days’ time.

Methods

A total of 8,488 reports were extracted between 4/26/2019 to 8/22/2019 retrospectively. Data was obtained from the voice dictation system report server. Exported query was saved as a .csv file format and analyzed using Python packages. A “discrepancy checker” was created to search all finalized reports for the departmental standard heading of “Final Attending Report,” used to specify any significant changes from the preliminary interpretation.

Results

Model estimates varied on different days however there were no trends or patterns to indicate a deterioration in resident performance throughout the week. There were comparable probabilities throughout the week, with 2.17% on Monday, 2.35% on Thursday and 2.05% on Friday.

Conclusions

Our results reveal no convincing trend in terms of overnight report discrepancies between the preliminary report generated by the night float resident and the final report issued by a faculty the following morning. These results are in support of the ACGME’s recent change in the permissible number of consecutive nights on night float. We did not prove our hypothesis that resident performance on-call in the domain of report accuracy would diminish over seven consecutive nights while on the night float rotation. Our results found that performance remained fairly uniform over the course of the week.

Radiology Education in the 21st Century: Threats and Opportunities

Over the past 25 years, radiology has seen tremendous growth in interpretive demands, including increasing imaging volumes and shorter turnaround times, as well as increased noninterpretive demands often targeting value-adding opportunities. These mounting pressures have led to increased burnout among radiologists nationwide and, in the academic setting, have begun to threaten the core educational mission. Despite these threats, radiology has also proven itself over the years to be a leader when it comes to innovation, and as such, no other field is better suited to confront these challenges with innovative solutions. This article explores the impact these workload trends have had on radiology education and educators, as well as opportunities to confront these challenges.

A review of current approaches for evaluating impaired performance in around-the-clock medical professionals

The need for data to study the relationship between fatigued healthcare professionals and performance outcomes is evident, however, it is unclear which methodology is most appropriate to provide these insights. To address this issue, we performed a systematic review of relevant articles by searching the MEDLINE, EMBASE, Cochrane, Web of Science, and CINAHL databases. The literature search identified 2960 unique references, of which 82 were identified eligible. The impact on performance was studied on clinical outcomes, medical simulation, neurocognitive performance, sleep quantification and subjective assessment. In general results on performance are conflicting; impairment, no effect, and improvement were found. This review outlines the various methods currently available for assessing fatigue-impaired performance. The contrasting outcomes can be attributed to three main factors: differences in the operationalisation of fatigue, incomplete control data, and the wide variety in the methods used. We recommend the implementation of a clinically applicable tool that can provide uniform data. Until these data become available, caution should be used when developing regulations that can have implications for physicians, education, manpower planning, and ‒ ultimately ‒ patient care.

Resident Case Volume Correlates with Clinical Performance: Finding the Sweet Spot

RATIONALE AND OBJECTIVES

To determine whether the total number of studies interpreted during radiology residency correlates with clinical performance as measured by objective criteria.

MATERIALS AND METHODS

We performed a retrospective cohort study of three graduating classes of radiology residents from a single residency program between the years 2015-2017. The total number of studies interpreted by each resident during residency was tracked. Clinical performance was determined by tracking an individual resident's major discordance rate. A major discordance was recorded when there was a difference between the preliminary resident interpretation and final attending interpretation that could immediately impact patient care. Accreditation council for graduate medical education milestones at the completion of residency, Diagnostic radiology in-training scores in the third year, and score from the American board of radiology core exam were also tabulated. Pearson correlation coefficients and polynomial regression analysis were used to identify correlations between the total number of interpreted films and clinical, test, and milestone performance.

RESULTS

Thirty-seven residents interpreted a mean of 12,709 studies (range 8898-19,818; standard deviation [SD] 2351.9) in residency with a mean major discordance rate of 1.1% (range 0.34%-2.54%; stand dev 0.49%). There was a nonlinear correlation between total number of interpreted films and performance. As the number of interpreted films increased to approximately 16,000, clinical performance (p = 0.004) and test performance (p = 0.01) improved, but volumes over 16,000 correlated with worse performance.

CONCLUSION

The total number of studies interpreted during radiology training correlates with performance. Residencies should endeavor to find the "sweet spot": the amount of work that maximizes clinical exposure and knowledge without overburdening trainees.

Radiologist Quality Assurance by Nonradiologists at Tumor Board

PURPOSE

To explore the use of nonradiologists as a method to efficiently reduce bias in the assessment of radiologist performance using a hepatobiliary tumor board as a case study.

MATERIALS AND METHODS

Institutional review board approval was obtained for this HIPAA-compliant prospective quality assurance (QA) effort. Consecutive patients with CT or MR imaging reviewed at one hepatobiliary tumor board between February 2016 and October 2016 (n = 265) were included. All presentations were assigned prospective anonymous QA scores by an experienced nonradiologist hepatobiliary provider based on contemporaneous comparison of the imaging interpretation at a tumor board and the original interpretation(s): concordant, minor discordance, major discordance. Major discordance was defined as a discrepancy that may affect clinical management. Minor discordance was defined as a discrepancy unlikely to affect clinical management. All discordances and predicted management changes were retrospectively confirmed by the liver tumor program medical director. Logistic regression analyses were performed to determine what factors best predict discordant reporting.

RESULTS

Approximately one-third (30% [79 of 265]) of reports were assigned a discordance, including 51 (19%) minor and 28 (11%) major discordances. The most common related to mass size (41% [32 of 79]), tumor stage and extent (24% [19 of 79]), and assigned LI-RADS v2014 score (22% [17 of 79]). One radiologist had 11.8-fold greater odds of discordance (P = .002). Nine other radiologists were similar (P = .10-.99). Radiologists presenting their own studies had 4.5-fold less odds of discordance (P = .006).

CONCLUSIONS

QA conducted in line with tumor board workflow can enable efficient assessment of radiologist performance. Discordant interpretations are commonly (30%) reported by nonradiologist providers.

Reporting of CT cervical spine after office hours by radiology trainees-analysis of discrepancy rates and RADPEER scores

PURPOSE

To review the errors made by radiology trainees in the reporting of cervical spine CTs (CCT) and to compare the discrepancy rates between the stages of training.

METHODS

All CCTs reported by trainees after office hours between January 2015 and December 2015 were retrospectively reviewed by a team of five musculoskeletal consultants with experience ranging between 7 and 15 years. Discrepancies between the provisional report by the trainee and the findings by the musculoskeletal consultants were graded according to the RADPEER scoring system. Sensitivity and specificity of the trainees were assessed.

RESULTS

Of 254 CCT provisional reports, there were 12 (4.7%) discrepancies, of which 5 (2.0%) discrepancies were likely to be clinically significant. We found a clinically significant difference between the stage of training of the trainee and RADPEER score (P = 0.023). The sensitivity and specificity of the senior radiology trainees were 97.0 and 98.1%, respectively, and that of the junior radiology trainees were 80 and 98.0% respectively (P = 0.039). Conditions misinterpreted as fractures include degenerative changes (n = 2) and nutrient vessel (n = 1). Other missed abnormalities include ossification of the posterior longitudinal ligament (n = 1), fracture of the foramen transversarium (n = 2), vertebral body fractures (n = 2), articular facet fractures (n = 2), and transverse process fractures (n = 2).

CONCLUSION

Cervical spine CTs performed after office hours can be safely interpreted by senior radiology trainees to a reasonable degree, although a targeted intervention to improve diagnostic performance of junior radiology trainees may be of clinical benefit.

Radiologic discrepancies in children with special healthcare needs in a pediatric emergency department

BACKGROUND

After-hours radiologic interpretation by nonradiology attendings or resident radiologists introduces the risk of discrepancies. Clinical outcomes following radiologic discrepancies among pediatric emergency department (ED) patients are poorly described. In particular, children with special healthcare needs (CSHCN), have more opportunities for discrepancies and potential consequences than non- CSHCN. Our objective was to determine the rates and types of radiologic discrepancies, and to compare CSHCN to non-CSHCN.

METHODS

From July 2014 to February 2015, all children who underwent a diagnostic imaging study at a free-standing children's ED were included. Data collected included radiologic studies - type and location - and clinical details - chief complaint and CSHCN type. Differences between preliminary reads and final pediatric radiology attending reads were defined as discrepancies, and categorized by clinical significance. Descriptive statistics, z-tests, and chi-square were used.

RESULTS

Over 8months, 8310 visits (7462 unique patients) had radiologic studies (2620 CSHCN, 5690 non-CSHCN). A total of 198 (2.4%) radiologic discrepancies [56 (28.3%) CSHCN, 142 (71.7%) non-CSHCN] were found. Chief complaints for CSCHN were more often within the cardiac, pulmonary and neurologic systems (p<0.001 for each), whereas non-CSHCN presented with more trauma (p<0.001). The rates of discrepancies (CSHCN 2.1%, non- CSHCN 2.5%, p=0.3) and severity of clinical consequences (p=0.6) were not significantly different between CSHCN and non-CSHCN.

CONCLUSION

Though the frequency and type of radiologic studies performed between CSHCN and non-CSHCN were different, we found no significant difference in the rate of radiologic discrepancies or the rate of clinically significant radiologic discrepancies.

Preliminary Interpretations of Transthoracic Echocardiograms by Cardiology Fellows

BACKGROUND

Echocardiograms are often obtained after business hours on an urgent or emergent basis to assist in the care of patients with complex presentations. Considerable variation exists among academic medical centers with regard to who performs and interprets these studies, with different levels of cardiology fellow involvement in scanning and/or interpreting. On-call echocardiographic interpretation can be educationally valuable for cardiologists in training but may come at the expense of patient care. The aim of this study was to examine the agreement of preliminary fellows' interpretations of weekend on-call transthoracic echocardiograms with official attending cardiologists' interpretations.

METHODS

Cardiology fellows perform preliminary interpretations of sonographer-obtained echocardiograms obtained on weekends, with final reports performed by attending cardiologists the following business day. In this study, 358 consecutive echocardiograms obtained on weekends over a 12-month period were reviewed. Discrepancies between the preliminary and final interpretations were categorized as either major (diagnoses with implications for urgent change in management) or minor (diagnoses without such implications). All discrepancies were also categorized as a missed diagnosis, an overcall (of severity), or an undercall.

RESULTS

No preliminary interpretation was identified in 18.4% of the studies (66 of 358). Of the remaining on-call echocardiograms (n = 292), the overall discrepancy rate in interpretations between fellows and attending cardiologists was 16.8%. Out of these, the minor discrepancy rate was 14.4% (42 of 292), and the major discrepancy rate was 2.4% (seven of 292). Misses, overcalls, and undercalls accounted for 29%, 31%, and 40% of all discrepancies, respectively.

CONCLUSIONS

The results indicate that although minor discrepancies between fellows' and attending cardiologists' interpretations were common (14.4%), major discrepancies were uncommon (2.4%) and similar to major discrepancy rates from the radiology literature. In general, discrepant interpretations were more likely to result from changes in severity, but misses accounted for almost all of the major discrepancies. Further research is needed to compare the clinical impact of different models of on-call echocardiographic services.

Systemic Error in Radiology

OBJECTIVE

Interpretive errors in diagnostic imaging result in significant patient morbidity and mortality, but the importance of errors and process failures in the imaging cycle other than during image interpretation is underappreciated. In this article, we describe these errors and potential solutions, providing a framework to improve patient safety and understand the changing roles of radiologists beyond image interpretation.

CONCLUSION

For comprehensive improvements to health care delivery, other failures in the cycle besides diagnostic interpretive error-such as ordering inappropriate studies, PACS failures, and a lack of accurate clinician contact information (with resultant communication failure)-should be recognized as contributors to patient harm because they lead to wasted resources and delayed care. By taking ownership of the entire imaging cycle, radiologists can increase their net worth to patient care and cement their roles as experts in the effective, evidence-based use of imaging technologies.

Implementation of a Longitudinal Introduction to Radiology Course During Internship Year Improves Diagnostic Radiology Residents' Academic and Clinical Skills: A Canadian Experience

RATIONALE AND OBJECTIVES

In order to ease the transition from internship to diagnostic radiology residency, a year-long didactic introduction to radiology course was offered to post-graduate year one (PGY-1) diagnostic radiology residents during their internship, which consisted of 27 hours of lecture over 9 months. The purpose of this study was to determine the quantitative and qualitative educational value of this course and its effect with respect to on-call preparedness.

MATERIALS AND METHODS

Two consecutive cohorts of Diagnostic Radiology residents were included: the first cohort (PGY-1s in 2011-2012) did not participate in the new course (Old Curriculum Residents) and the second cohort (PGY-1s in 2012-2013) completed the new course (New Curriculum Residents). These two cohorts were compared both qualitatively and quantitatively. Scores were compared from the standardized Canadian National Pre-Call Observed Standardized Clinical Examination and American College of Radiology Diagnostic Radiology In-Training examination, which are taken in the PGY-2 year, at months 5 and 7, respectively. In addition, staff observation of on-call resident performance and resident self-reported preparedness were considered. Cohorts were compared using Mann-Whitney U test with significance defined as P value <0.05. P values from 0.05 to 0.10 were noted as possibly significant and further analyzed using a Cohen d test where the difference was determined to be small (0.2), medium (0.5), or large (0.8).

RESULTS

New Curriculum Residents reported that the content of the PGY1 curriculum was more appropriate than the old curriculum to prepare them for call in PGY2 (P = 0.013). New Curriculum Residents scored better than the Old Curriculum Residents on the Diagnostic Radiology In-Training examination (P = 0.039) and on the emergency cases of the Canadian National Pre-Call Observed Standardized Clinical Examination (P = 0.035). Staff radiologists, who were not blinded, reported that the New Curriculum Residents were better prepared for daytime (P = 0.006) and overnight (P = 0.008) independent call were better prepared to perform common ultrasound examinations alone (P = 0.049), and required less guidance while on call for nine competency areas. There was, however, no statistical difference between the residents' self-reported preparedness for independent call.

CONCLUSIONS

Participation in a lecture-based introductory radiology curriculum during the PGY-1 internship year improved both radiology residents' preparedness for call and their performance in PGY-2.

"Concordance" Revisited: A Multispecialty Appraisal of "Concordant" Preliminary Abdominopelvic CT Reports

PURPOSE

To determine whether resident abdominopelvic CT reports considered prospectively concordant with the final interpretation are also considered concordant by other blinded specialists and abdominal radiologists.

METHODS

In this institutional review board-approved retrospective cohort study, 119 randomly selected urgent abdominopelvic CT examinations with a resident preliminary report deemed prospectively "concordant" by the signing faculty were identified. Nine blinded specialists from Emergency Medicine, Internal Medicine, and Abdominal Radiology reviewed the preliminary and final reports and scored the preliminary report with respect to urgent findings as follows: 1.) concordant; 2.) discordant with minor differences; 3.) discordant with major differences that do not alter patient management; or 4.) discordant with major differences that do alter patient management. Predicted management resulting from scores of 4 was recorded. Consensus was defined as majority agreement within a specialty. Consensus major discrepancy rates (ie, scores 3 or 4) were compared to the original major discrepancy rate of 0% (0/119) using the McNemar test.

RESULTS

Consensus scores of 4 were assigned in 18% (21/119, P < .001, Emergency Medicine), 5% (6/119, P = .03, Internal Medicine), and 13% (16/119, P < .001, Abdominal Radiology) of examinations. Consensus scores of 3 or 4 were assigned in 31% (37/119, P < .001, Emergency Medicine), 14% (17/119, P < .001, Internal Medicine), and 18% (22/119, P < .001, Abdominal Radiology). Predicted management alterations included hospital status (0-4%), medical therapy (1%-4%), imaging (1%-10%), subspecialty consultation (3%-13%), nonsurgical procedure (3%), operation (1%-3%), and other (0-3%).

CONCLUSIONS

The historical low major discrepancy rate for urgent findings between resident and faculty radiologists is likely underreported.

On-Call Radiology Resident Discrepancies: Categorization by Patient Location and Severity

PURPOSE

To report discrepancy rates for examinations interpreted by on-call residents overall and by resident training level, and to describe a novel discrepancy classification system based on patient location and severity that facilitates recording of discrepancy data, helps ensure proper communication of report changes, and allows our radiology department to assume responsibility for contacting discharged patients with non-time-dependent results.

METHODS

A HIPAA-compliant, institutional review board-exempt review of two years (January 2013 to December 2014) of discrepancy data was retrospectively performed for total number of examination interpreted, discrepancy rates, resident training level, and discrepancy categories. Most common diagnoses and means of results communication for discharged patients were also recorded.

RESULTS

Radiology residents interpreted 153,420 examinations after hours and had 2169 discrepancies, for an overall discrepancy rate of 1.4%. Discrepancy rates for postgraduate year (PGY)-3, PGY-4, and PGY-5 residents were 1.31%, 1.65%, and 1.88%, respectively. The rate of critical discrepancies was extremely low (10/153,420 or 0.007%). A total of 502 patients (23.2% of all discrepancies) were discharged at the time their discrepancy was identified, 60% of whom had non-time-dependent discrepancies that were communicated by radiologists; 32.4% of these had addended results telephoned to a PCP, 43.4% had addended results telephoned to the patient, and the remaining 24.2% required a registered letter. Eight percent of patients with non-time-dependent findings were lost to follow-up.

CONCLUSIONS

Our resident discrepancy rates were comparable to those published previously, with extremely low rates of critical discrepancies. Radiologists assumed responsibility for contacting the majority of discharged patients with discrepant results, a minority of whom were lost to follow-up.

Radiology resident preliminary reporting in an independent call environment: multiyear assessment of volume, timeliness, and accuracy

PURPOSE

The objective of this paper is to assess the volume, accuracy, and timeliness of radiology resident preliminary reports as part of an independent call system. This study seeks to understand the relationship between resident year in training, study modality, and discrepancy rate.

METHODS

Resident preliminary interpretations on radiographs, ultrasound, CT, and MRI from October 2009 through December 2013 were prospectively scored by faculty on a modified RADPEER scoring system. Discrepancy rates were evaluated based on postgraduate year of the resident and the study modality. Turnaround times for reports were also reviewed. Differences between groups were compared with a chi-square test with a significance level of 0.05. Institutional review board approval was waived as only deidentified data were used in the study.

RESULTS

A total of 416,413 studies were reported by 93 residents, yielding 135,902 resident scores. The rate of major resident-faculty assessment discrepancies was 1.7%. Discrepancy rates improved with increasing experience, both overall (PGY-3: 1.8%, PGY-4: 1.7%, PGY-5: 1.5%) and for each individual modality. Discrepancy rates were highest for MR (3.7%), followed by CT (2.4%), radiographs (1.4%), and ultrasound (0.6%). Emergency department report turnaround time averaged 31.7 min. The average graduating resident has been scored on 2,746 ± 267 reports during residency.

CONCLUSIONS

Resident preliminary reports have a low rate of major discrepancies, which improves over 3 years of call-taking experience. Although more complex cross-sectional studies have slightly higher discrepancy rates, discrepancies were still within the range of faculty report variation.

The Effect of Restricting Residents' Duty Hours on Patient Safety, Resident Well-Being, and Resident Education: An Updated Systematic Review

BACKGROUND

Despite 25 years of implementation and a sizable amount of research, the impact of resident duty hour restrictions on patients and residents still is unclear. Advocates interpret the research as necessitating immediate change; opponents draw competing conclusions.

OBJECTIVE

This study updates a systematic review of the literature on duty hour restrictions conducted 1 year prior to the implementation of the Accreditation Council for Graduate Medical Education's 2011 regulations.

METHODS

The review draws on reports catalogued in MEDLINE and PreMEDLINE from 2010 to 2013. Interventions that dealt with the duty hour restrictions included night float, shortened shifts, and protected time for sleep. Outcomes were patient care, resident well-being, and resident education. Studies were excluded if they were not conducted in patient care settings.

RESULTS

Twenty-seven studies met the inclusion criteria. Most frequently, the studies concluded that the restrictions had no impact on patient care (50%) or resident wellness (47%), and had a negative impact on resident education (64%). Night float was the most frequent means of implementing duty hour restrictions, yet it yielded the highest proportion of unfavorable findings.

CONCLUSIONS

This updated review, including 27 recent applicable studies, demonstrates that focusing on duty hours alone has not resulted in improvements in patient care or resident well-being. The added duty hour restrictions implemented in 2011 appear to have had an unintended negative impact on resident education. New approaches to the issue of physician fatigue and its relationship to patient care and resident education are needed.

Sink or Night Float: University of British Columbia Radiology Residents' Experience with Overnight Call

Purpose

In July 2012, in response to residents' concerns regarding the impact of the traditional 24-hour call system on their personal well-being and educational experience, the University of British Columbia Radiology residency program adopted a 12-hour night float system. This shift takes place in the context of increasing concerns, both across Canada and internationally, about resident well-being and the impact of prolonged duty hours on patient care.

Methods

An anonymous survey was distributed to all 25 postgraduate years 2-5 University of British Columbia radiology residents 12 months after the introduction of night float. This study sought to solicit residents' feedback about these changes and to identify potential future changes to optimize the call system.

Results

The response rate was 100%; 96% of residents were in favor of continuing with night float rather than the traditional call system; 72% of residents reported that their judgement was affected secondary to being on night float. Although most residents described varying degrees of impairment, the rate of acute discrepancies between resident preliminary and attending radiologist final reports decreased by more than half, from 2% to less than 1%.

Conclusions

The vast majority of our residents were in favor of maintaining the night float call system. Night float had a beneficial effect on the resident educational experience: by eliminating the pre-call morning and post-call day off rotation, residents gained an additional 24 days per year on other clinical rotations.

The discrepancy rate between preliminary and official reports of emergency radiology studies: a performance indicator and quality improvement method

BACKGROUND

At teaching hospitals, radiology residents give preliminary reports for imaging studies requested from the Emergency Department (ED). Discrepancy rates between preliminary and final reports represent an important performance indicator.

PURPOSE

To present a system for feedback and follow-up of discrepancies, identify the variables associated with the rate and severity of such discrepancies, target the weaknesses, and suggest the need of a standard reference value for comparison among institutions.

MATERIAL AND METHODS

A monitoring and communication system between the Department of Diagnostic Radiology and Emergency Department was initiated to mark and follow all studies from the ED for which the official reading was different than the preliminary interpretation. Data analysis was performed on all studies from 1 June 2011 to 31 May 2012, based on the severity of the discrepancy, imaging modality, resident training level, and organ system. The distribution of the number of discrepancies among the different resident levels and imaging modalities was determined, as well as the distribution of three severity scores in correlation with other variables.

RESULTS

The overall discrepancy rate was 1.62%. The discrepancy rate was higher for first and second year residents (1.62% and 1.96%) than for third and fourth year residents (1.35% and 1.24%). It was higher for computed tomography (2.13%) than for radiographs (1.29%) and ultrasound (0.8%) (P value < 0.01), and higher for musculoskeletal (1.61%) than non-musculoskeletal (0.99%) radiographs (P value = 0.0003). Discrepancies with severity score one constituted 35.5% of the total discrepancies, those with severity scores two and three constituted 22.9% and 41.6%, respectively.

CONCLUSION

We have demonstrated a system for follow-up of discrepancy in interpreting emergency radiology studies, and recorded the discrepancy rate, with further analysis based on different variables. In terms of quality assurance, a periodical analysis might help to reduce the number of discrepant reports by targeted intervention.

Increasing neuroradiology exam volumes on-call do not result in increased major discrepancies in primary reads performed by residents

Background and Purpose:

A common perception is that increased on-call workload leads to increased resident mistakes. To test this, we evaluated whether increased imaging volume has led to increased errors by residents.

Materials and Methods:

A retrospective review was made of all overnight neuroradiology CT exams with a primary resident read from 2006-2010. All studies were over-read by staff neuroradiologists next morning. As the volume is higher on Friday through Sunday nights, weekend studies were examined separately. Discrepancies were classified as either minor or major. “Major” discrepancy was defined as a discrepancy that the staff radiologist felt was significant enough to potentially affect patient care, necessitating a corrected report and phone contact with the ordering physician and documentation. The total number of major discrepancies was recorded by quarter. In addition, the total number of neuroradiology CT studies read overnight on-call was noted.

Results:

The mean number of cases per night during the weekday increased from 3.0 in 2006 to 5.2 in 2010 (p<0.001). During the weekend, the mean number of cases per night increased from 5.4 in 2006 to 7.6 in 2010 (p<0.001). Despite this increase, the major discrepancy rate decreased from 2.7% in 2006 to 2.3% in 2010 (p=0.34).

Conclusion:

Despite an increase in neuroradiology exam volumes, there continues to be a low major discrepancy rate for primary resident interpretations. While continued surveillance of on-call volumes is crucial to the educational environment, concern of increased major errors should not be used as sole justification to limit autonomy.

Anatomical and/or pathological predictors for the "incorrect" classification of red dot markers on wrist radiographs taken following trauma

OBJECTIVE

To establish the prevalence of red dot markers in a sample of wrist radiographs and to identify any anatomical and/or pathological characteristics that predict "incorrect" red dot classification.

METHODS

Accident and emergency (A&E) wrist cases from a digital imaging and communications in medicine/digital teaching library were examined for red dot prevalence and for the presence of several anatomical and pathological features. Binary logistic regression analyses were run to establish if any of these features were predictors of incorrect red dot classification.

RESULTS

398 cases were analysed. Red dot was "incorrectly" classified in 8.5% of cases; 6.3% were "false negatives" ("FNs")and 2.3% false positives (FPs) (one decimal place). Old fractures [odds ratio (OR), 5.070 (1.256-20.471)] and reported degenerative change [OR, 9.870 (2.300-42.359)] were found to predict FPs. Frykman V [OR, 9.500 (1.954-46.179)], Frykman VI [OR, 6.333 (1.205-33.283)] and non-Frykman positive abnormalities [OR, 4.597 (1.264-16.711)] predict "FNs". Old fractures and Frykman VI were predictive of error at 90% confidence interval (CI); the rest at 95% CI.

CONCLUSION

The five predictors of incorrect red dot classification may inform the image interpretation training of radiographers and other professionals to reduce diagnostic error. Verification with larger samples would reinforce these findings.

ADVANCES IN KNOWLEDGE

All healthcare providers strive to eradicate diagnostic error. By examining specific anatomical and pathological predictors on radiographs for such error, as well as extrinsic factors that may affect reporting accuracy, image interpretation training can focus on these "problem" areas and influence which radiographic abnormality detection schemes are appropriate to implement in A&E departments.

Residents' performance in the interpretation of on-call "triple-rule-out" CT studies in patients with acute chest pain

RATIONALE AND OBJECTIVES

To evaluate the performance of radiology residents in the interpretation of on-call, emergency "triple-rule-out" (TRO) computed tomographic (CT) studies in patients with acute chest pain.

MATERIALS AND METHODS

The study was institutional review board-approved and Health Insurance Portability and Accountability Act compliant. Data from 617 on-call TRO studies were analyzed. Dedicated software enables subspecialty attendings to grade discrepancies in interpretation between preliminary trainee reports and their final interpretation as "unlikely to be significant" (minor discrepancies) or "likely to be significant" for patient management (major discrepancies). The frequency of minor, major and all discrepancies in resident's TRO interpretations was compared to 609 emergent non-electrocardiography (ECG)-synchronized chest CT studies using Pearson χ(2) test.

RESULTS

Minor discrepancies occurred more often in the TRO group (9.1% vs. 3.9%, P < .001), but there was no difference in the frequency of major discrepancies (2.1% vs. 2.8%, P = .55). Minor discrepancies in the TRO group most commonly resulted from missed extrathoracic findings with missed liver lesions being the most frequent. Major discrepancies mostly encompassed cardiac and extracardiac vascular findings but did not result in unnecessary interventions, significant immediate changes in management, or adverse patient outcomes.

CONCLUSIONS

On-call resident interpretation of TRO CT studies in patients with acute chest pain is congruent with final subspecialty attending interpretation in the overwhelming majority of cases. The rate of discrepancies likely to affect patient management in this domain is not different from emergent non-ECG-synchronized chest CT.

Incidence of delayed and missed diagnoses in whole-body multidetector CT in patients with multiple injuries after trauma

BACKGROUND

Whole-body CT (WBCT) is the imaging modality of choice during the initial diagnostic work-up of multiple injured patients in order to identify serious injuries and initiate adequate treatment immediately. However, delayed diagnosed or even missed injuries have been reported frequently ranging from 1.3% to 47%.

PURPOSE

To highlight commonly missed lesions in WBCT of patients with multiple injuries.

MATERIAL AND METHODS

A total of 375 patients (age 42.8 ± 17.9 years, ISS 26.6 ± 17.0) with a WBCT (head to symphysis) were included. The final CT report was compared with clinical and operation reports. Discrepant findings were recorded and grouped as relevant and non-relevant to further treatment. In both groups, an experienced trauma radiologist read the CT images retrospectively, whether these lesions were missed or truly not detectable.

RESULTS

In 336 patients (89.6%), all injuries in the regions examined were diagnosed correctly in the final reports of the initial CT. Forty-eight patients (12.8%) had injuries in regions of the body that were not included in the CT. Fourteen patients (3.7%) had injuries that did not require further treatment. Twenty-five patients (6.7%) had injuries that required further treatment. With secondary interpretation, 85.4% of all missed lesions could be diagnosed in retrospect from the primary CT data-set. Small pancreatic and bowel contusions were identified as truly non-detectable.

CONCLUSION

In multiple traumas, only a few missed injuries in initial WBCT reading are clinically relevant. However, as the vast majority of these injuries are detectable, the radiologist has to be alert for commonly missed findings to avoid a delayed diagnosis.

Introductory lecture series for first-year radiology residents: implementation, investment and assessment

RATIONALE AND OBJECTIVES

A lecture series aimed at providing new radiology residents a rapid course on the fundamental concepts of professionalism, safety, and interpretation of diagnostic imaging was established. Evaluation of the course's educational value was attempted through surveys.

MATERIALS AND METHODS

Twenty-six live 45-minute lectures presented by 16 or 17 faculty members were organized exclusively for the first class of radiology residents, held over a 2-month period at the beginning of certain weekdays. Online surveys were conducted after the course to gather feedback from residents. Average resident rotation evaluation scores were measured over the first semester for the two classes before and after this new course implementation.

RESULTS

The lecture series was successfully organized and implemented. A total of 33 residents sat through the course over three summers. Faculty reported a reasonable number of preparation hours, and 100% of residents indicated they valued the course. Comparison of class average evaluation scores before and after the existence of this 2-month course did not significantly change.

CONCLUSIONS

This collection of introductory lectures on professionalism, safety, and diagnostic imaging, delivered early in the first year of the radiology residency, requires a reasonable number of invested preparation hours by the faculty but results in a universal increase in resident confidence. However, we were unable to demonstrate an objective improvement in resident performance on clinical rotations.

Increased error rates in preliminary reports issued by radiology residents working more than 10 consecutive hours overnight

RATIONALE AND OBJECTIVES

To determine if the rate of major discrepancies between resident preliminary reports and faculty final reports increases during the final hours of consecutive 12-hour overnight call shifts.

MATERIALS AND METHODS

Institutional review board exemption status was obtained for this study. All overnight radiology reports interpreted by residents on-call between January 2010 and June 2010 were reviewed by board-certified faculty and categorized as major discrepancies if they contained a change in interpretation with the potential to impact patient management or outcome. Initial determination of a major discrepancy was at the discretion of individual faculty radiologists based on this general definition. Studies categorized as major discrepancies were secondarily reviewed by the residency program director (M.H.S.) to ensure consistent application of the major discrepancy designation. Multiple variables associated with each report were collected and analyzed, including the time of preliminary interpretation, time into shift study was interpreted, volume of studies interpreted during each shift, day of the week, patient location (inpatient or emergency department), block of shift (2-hour blocks for 12-hour shifts), imaging modality, patient age and gender, resident identification, and faculty identification. Univariate risk factor analysis was performed to determine the optimal data format of each variable (ie, continuous versus categorical). A multivariate logistic regression model was then constructed to account for confounding between variables and identify independent risk factors for major discrepancies.

RESULTS

We analyzed 8062 preliminary resident reports with 79 major discrepancies (1.0%). There was a statistically significant increase in major discrepancy rate during the final 2 hours of consecutive 12-hour call shifts. Multivariate analysis confirmed that interpretation during the last 2 hours of 12-hour call shifts (odds ratio (OR) 1.94, 95% confidence interval (CI) 1.18-3.21), cross-sectional imaging modality (OR 5.38, 95% CI 3.22-8.98), and inpatient location (OR 1.81, 95% CI 1.02-3.20) were independent risk factors for major discrepancy.

CONCLUSIONS

In a single academic medical center, major discrepancies in resident preliminary reports increased significantly during the final 2 hours of consecutive 12-hour overnight call shifts. This finding could be related to either fatigue or circadian desynchronization. Discrimination of these two potential etiologies requires additional investigation as major discrepancies in resident reports have the potential to negatively impact patient care/outcome. Cross-sectional imaging modalities including computed tomography and ultrasound (versus conventional radiography), as well as inpatient location (versus Emergency Department location), were also associated with significantly higher major discrepancy rates.

Application of the RADPEER™ scoring language to interpretation discrepancies between diagnostic radiology residents and faculty radiologists

PURPOSE

The objectives of this study were to assess the rate of discrepancies between radiology residents and faculty radiologists at an academic hospital using the RADPEER(™) scoring language and to determine if a correlation existed between assigned RADPEER score (2, 3, or 4) and the clinical significance of the error.

METHODS

Over 19 months, preliminary resident interpretations were graded using the RADPEER scoring system. A retrospective review of discrepant cases was performed to assess the percentage of discrepancy, change in clinical management, and identification of error patterns.

RESULTS

Of 2,255 preliminary interpretations, 29 discrepancies (1.29%) were judged to be potentially clinically significant. Of these, 14 (0.62%) resulted in immediate changes in clinical management. Discrepancies assigned RADPEER scores of 3 or 4 were significantly more likely to be judged clinically significant than those assigned scores of 2 (54.5% of 33 studies graded 3 or 4 and 7.7% of 142 studies graded 2, P < .0001). CT imaging generated a higher percentage of discrepancies that were predicted to be clinically significant than plain-film radiography, as well as a higher percentage of discrepancies that resulted in immediate changes in management, but the incidence of each remained low overall (≤2.1%).

CONCLUSIONS

Discrepancy rates in this study are comparable with previously reported data for discrepancies between attending radiologists and those between attending radiologists and residents data. A significant correlation was observed between increasing RADPEER scores and the clinical significance of discrepancies. This study supports the use of the RADPEER scoring language as both a resident quality assurance measure and an educational tool for quality improvement.

Should registrars be reporting after-hours CT scans? A calculation of error rate and the influencing factors in South Africa

BACKGROUND

There is a heavy reliance on registrars for after-hours CT reporting with a resultant unavoidable error rate.

PURPOSE

To determine the after-hours CT reporting error rate by radiology registrars and influencing factors on this error rate.

MATERIAL AND METHODS

A 2-month prospective study was undertaken at two tertiary, level 1 trauma centers in Johannesburg, South Africa. Provisional CT reports issued by the registrar on call were reviewed by a qualified radiologist the following morning and information relating to the number, time and type of reporting errors made as well as the body region scanned, indication for the scan, year of training of the registrar, and workload during the call were recorded and analyzed.

RESULTS

A total of 1477 CT scans were performed with an overall error rate of 17.1% and a major error rate of 7.7%. The error rate for 2nd, 3rd, and 4th year registrars was 19.4%, 15.1%, and 14.5%, respectively. A significant difference was found between the error rate in reporting trauma scans (15.8%) compared to non-trauma scans (19.2%) although the difference between emergency scans (16.9%) and elective scans (22.6%) was found to be not significant, a finding likely due to the low number of elective scans performed. Abdominopelvic scans elicited the highest number of errors (33.9%) compared to the other body regions such as head (16.5%) and cervical, thoracic, or lumbar spine (11.7%). Increasing workload resulted in a significant increase in error rate when analyzed with a generalized linear model. There was also a significant difference noted in the time of scan groups which we attributed to a workload effect. Missed findings were the most frequent errors seen (57.3%).

CONCLUSION

We found an increasing error rate associated with increasing workload and marked increase in errors with the reporting of abdominopelvic scans. There was a decrease in the error rate when looking an increasing year of training although this there was only found to be significant difference between the 2nd and 3rd year registrars.