Radiology residents’ skill level in chest x-ray reading

Integrated cardio-thoracic imaging curriculum for residents: Design, implementation, and analysis of test and evaluation results

As the field of cardiac imaging has demonstrated exceptional growth over the past several decades, radiology departments and residency programs have struggled to integrate cardiac imaging instruction into training curricula.

PURPOSE

To create an integrated cardio-thoracic teaching and lecture curriculum and resident rotation in accordance with AGGME and Society of Thoracic Radiology (STR) guidelines.

MATERIALS AND METHODS

Consecutive PGY-2 to PGY-4 residents (n = 14) rotating through our Cardiothoracic Imaging (CTI) section from 1/1/2021 to 04/18/2022 were give pre- and post- rotation tests of knowledge and feedback evaluations. Attending feedback of the curriculum was obtained at 3-months and 9-months post curriculum implementation. A Wilcxon test was used to evaluate differences in improvement between pre- and post- rotation resident feedback scores, test scores for thoracic and cardiac test questions in addition to attending feedback scores at 3 and 9-months post curriculum implementation.

RESULTS

The overall post-rotation scores in addition to thoracic only and cardiac only scores improved, with the difference between improved versus stable or decreased scores being statistically significant overall (P = 0.039) and for cardiac scores (P = 0.003), but not for thoracic scores (P = 0.22). The overall (P = 0.002), thoracic (P = 0.027), and cardiac (P = 0.026) resident feedback scores were significantly improved post-rotation. Similarly, the overall attending feedback scores significantly improved over time (P = 0.021).

CONCLUSION

An integrated Cardio-thoracic Imaging teaching curriculum was well received by both residents and attendings with significant improvement in post rotation feedback scores by both groups. Moreover, residents demonstrated improved scores on knowledge tests post rotation.

Influence of residency discipline and seniority on traumatic musculoskeletal radiographs interpretation accuracy: a multicenter study

OBJECTIVES

Imaging studies are a significant and integral part of the initial assessment of patients admitted to the emergency department. Developing imaging diagnostic abilities early in residency is of paramount importance. The purpose of this study was to evaluate and compare diagnosis accuracy of common musculoskeletal X-rays (XR) between residency disciplines and seniority.

METHODS

A multicenter study which evaluated orthopedic surgery, emergency medicine (EM), and radiology residents, through a test set of common MSK XR. Residents were classified as "beginner" or "advanced" according to postgraduate year per residency. Residents were asked to answer whether the radiograph shows normal or pathological findings (success rate) and what is the diagnosis ("diagnosis accuracy"). Residents' answers were analyzed and assessed compared to experts' consensus.

RESULTS

A total of 100 residents (62% beginners) participated in this study. Fifty-four were orthopedic surgeons, 29 were EM residents and 17 were radiologists. The entire cohort's overall success rate was 88.5%. The overall mean success rates for orthopedic, EM, and radiology residents were 93.2%, 82.8%, and 83.3%, respectively, and were significantly different (p < 0.0001). Orthopedic residents had significantly higher diagnostic accuracy rates compared with both radiology and EM residents (p < 0.001). Advanced orthopedic and EM residents demonstrated higher diagnostic accuracy rates compared to beginner residents (p = 0.001 and p = 0.03, respectively).

CONCLUSION

Orthopedic residents presented higher diagnosis accuracy of MSK imaging compared to EM and radiology residents. Seniority had a positive effect on diagnosis accuracy. The development of an educational program on MSK XR is necessary to enhance the competency of physicians in their daily practice.

Simple demographic, laboratory and chest radiograph variables can identify COVID-19 patients with pulmonary thromboembolism: a retrospective multicentre United Kingdom study

OBJECTIVES

To (1) identify discriminatory demographic, laboratory and initial CXR findings; (2) explore correlation between D-dimer and radiographic severity scores; and (3) assess accuracy of published D-dimer thresholds to identify pulmonary thromboembolism (PTE) in COVID-19 patients.

METHODS

Retrospective study including all COVID-19 patients admitted from 1st to 30th April 2020 meeting inclusion criteria from 25 (blinded) hospitals. Demographics, blood results, CXR and CTPA findings were compared between positive and negative PTE cohorts using uni- and multivariable logistic regression. Published D-dimer cut-offs were applied.

RESULTS

389 patients were included [median age 63; 237 males], of which 26.2% had a PTE. Significant univariable discriminators for PTE were peak D-dimer, sex, neutrophil count at the time of the D-dimer and at admission, abnormal CXR, and CXR zonal severity score. Only neutrophil count at peak D-dimer remained significant for predicting PTE on multivariable analysis (p = 0.008). When compared with the published literature, sensitivity for PTE were lower than those published at all cut-off values, however specificity at different cut-offs was variable.

CONCLUSIONS

In this multicentre COVID-19 cohort, univariable admission factors that could indicate pulmonary thromboembolism were male sex, high neutrophil count and abnormal CXR with a greater CXR zonal severity score. The accuracy levels of published D-dimer thresholds were not reproducible in our population.

ADVANCES IN KNOWLEDGE

This is a large multicentre study looking at the discriminatory value of simple variables to determine if a patient with COVID-19 has PTE or not, in addition to comparing D-dimer cut off values against published values.

Learning from the machine: AI assistance is not an effective learning tool for resident education in chest x-ray interpretation

OBJECTIVES

To assess whether a computer-aided detection (CADe) system could serve as a learning tool for radiology residents in chest X-ray (CXR) interpretation.

METHODS

Eight radiology residents were asked to interpret 500 CXRs for the detection of five abnormalities, namely pneumothorax, pleural effusion, alveolar syndrome, lung nodule, and mediastinal mass. After interpreting 150 CXRs, the residents were divided into 2 groups of equivalent performance and experience. Subsequently, group 1 interpreted 200 CXRs from the "intervention dataset" using a CADe as a second reader, while group 2 served as a control by interpreting the same CXRs without the use of CADe. Finally, the 2 groups interpreted another 150 CXRs without the use of CADe. The sensitivity, specificity, and accuracy before, during, and after the intervention were compared.

RESULTS

Before the intervention, the median individual sensitivity, specificity, and accuracy of the eight radiology residents were 43% (range: 35-57%), 90% (range: 82-96%), and 81% (range: 76-84%), respectively. With the use of CADe, residents from group 1 had a significantly higher overall sensitivity (53% [n = 431/816] vs 43% [n = 349/816], p < 0.001), specificity (94% [i = 3206/3428] vs 90% [n = 3127/3477], p < 0.001), and accuracy (86% [n = 3637/4244] vs 81% [n = 3476/4293], p < 0.001), compared to the control group. After the intervention, there were no significant differences between group 1 and group 2 regarding the overall sensitivity (44% [n = 309/696] vs 46% [n = 317/696], p = 0.666), specificity (90% [n = 2294/2541] vs 90% [n = 2285/2542], p = 0.642), or accuracy (80% [n = 2603/3237] vs 80% [n = 2602/3238], p = 0.955).

CONCLUSIONS

Although it improves radiology residents' performances for interpreting CXRs, a CADe system alone did not appear to be an effective learning tool and should not replace teaching.

CLINICAL RELEVANCE STATEMENT

Although the use of artificial intelligence improves radiology residents' performance in chest X-rays interpretation, artificial intelligence cannot be used alone as a learning tool and should not replace dedicated teaching.

KEY POINTS

• With CADe as a second reader, residents had a significantly higher sensitivity (53% vs 43%, p < 0.001), specificity (94% vs 90%, p < 0.001), and accuracy (86% vs 81%, p < 0.001), compared to residents without CADe. • After removing access to the CADe system, residents' sensitivity (44% vs 46%, p = 0.666), specificity (90% vs 90%, p = 0.642), and accuracy (80% vs 80%, p = 0.955) returned to that of the level for the group without CADe.

Point-of-Care Lung Ultrasound in the Intensive Care Unit-The Dark Side of Radiology: Where Do We Stand?

Patients in intensive care units (ICUs) are critically ill and require constant monitoring of clinical conditions. Due to the severity of the underlying disease and the need to monitor devices, imaging plays a crucial role in critically ill patients' care. Given the clinical complexity of these patients, who typically need respiratory assistance as well as continuous monitoring of vital functions and equipment, computed tomography (CT) can be regarded as the diagnostic gold standard, although it is not a bedside diagnostic technique. Despite its limitations, portable chest X-ray (CXR) is still today an essential diagnostic tool used in the ICU. Being a widely accessible imaging technique, which can be performed at the patient's bedside and at a low healthcare cost, it provides additional diagnostic support to the patient's clinical management. In recent years, the use of point-of-care lung ultrasound (LUS) in ICUs for procedure guidance, diagnosis, and screening has proliferated, and it is usually performed at the patient's bedside. This review illustrates the role of point-of-care LUS in ICUs from a purely radiological point of view as an advanced method in ICU CXR reports to improve the interpretation and monitoring of lung CXR findings.

The Effects of Various Teaching Methods on Chest X-ray Interpretation Skills Among Medical Students and Interns: A Systematic Review

Chest X-ray (CXR) is a common tool used in medical practice. Medical students and interns should acquire knowledge of CXR interpretation, as it is an essential diagnostic tool for a large spectrum of diseases. This systematic review aimed to compare the effect of different intervention techniques on the competency of medical students and interns to demonstrate the level of confidence and competence in interpreting common presentations of CXRs. The population, intervention, comparison, and outcomes (PICO) framework was used to formulate the review question. All related articles in five databases (PubMed, Web of Science, Scopus, Medline, and Embase) were retrieved and the search was completed in March 2023 with no limiters on date and time. The number of relevant studies was 469. A multi-level approach through the Rayyan platform was used for the screening and exclusion processes. Eleven articles were included in the systematic review consisting of eight randomized controlled trials, one quasi-experimental study, one cross-sectional study, and one interventional cohort. Results showed significant effects of teaching methods utilizing deductive or inductive approach, clinical history, patient care comfort survey, and SAFMEDS (Say-All-Fast-Minute-Every-Day-Shuffled). Contrarily, no significant effect was shown by flipped classroom models and mixed and blocked practice, peer-assisted learning vs. expert-assisted learning, and Chester, an artificial intelligence tool. This review identified beneficial approaches that may enhance the learning outcomes of interpreting CXRs for medical students and interns, highlighting the remarkable impact of SAFMEDS on medical students' ability to identify CXR findings as well as the availability and practicality of online and e-learning resources for students.

Chest radiographic interpretation - an essential competency for Africa

-

A Performance-Based Competency Assessment of Pediatric Chest Radiograph Interpretation Among Practicing Physicians

INTRODUCTION

There is limited knowledge on pediatric chest radiograph (pCXR) interpretation skill among practicing physicians. We systematically determined baseline interpretation skill, the number of pCXR cases physicians required complete to achieve a performance benchmark, and which diagnoses posed the greatest diagnostic challenge.

METHODS

Physicians interpreted 434 pCXR cases via a web-based platform until they achieved a performance benchmark of 85% accuracy, sensitivity, and specificity. Interpretation difficulty scores for each case were derived by applying one-parameter item response theory to participant data. We compared interpretation difficulty scores across diagnostic categories and described the diagnoses of the 30% most difficult-to-interpret cases.

RESULTS

240 physicians who practice in one of three geographic areas interpreted cases, yielding 56,833 pCXR case interpretations. The initial diagnostic performance (first 50 cases) of our participants demonstrated an accuracy of 68.9%, sensitivity of 69.4%, and a specificity of 68.4%. The median number of cases completed to achieve the performance benchmark was 102 (interquartile range 69, 176; min, max, 54, 431). Among the 30% most difficult-to-interpret cases, 39.2% were normal pCXR and 32.3% were cases of lobar pneumonia. Cases with a single trauma-related imaging finding, cardiac, hilar, and diaphragmatic pathologies were also among the most challenging.

DISCUSSION

At baseline, practicing physicians misdiagnosed about one-third of pCXR and there was up to an eight-fold difference between participants in number of cases completed to achieve the standardized performance benchmark. We also identified the diagnoses with the greatest potential for educational intervention.

Inter- and Intra-Observer Agreement When Using a Diagnostic Labeling Scheme for Annotating Findings on Chest X-rays-An Early Step in the Development of a Deep Learning-Based Decision Support System

Consistent annotation of data is a prerequisite for the successful training and testing of artificial intelligence-based decision support systems in radiology. This can be obtained by standardizing terminology when annotating diagnostic images. The purpose of this study was to evaluate the annotation consistency among radiologists when using a novel diagnostic labeling scheme for chest X-rays. Six radiologists with experience ranging from one to sixteen years, annotated a set of 100 fully anonymized chest X-rays. The blinded radiologists annotated on two separate occasions. Statistical analyses were done using Randolph's kappa and PABAK, and the proportions of specific agreements were calculated. Fair-to-excellent agreement was found for all labels among the annotators (Randolph's Kappa, 0.40-0.99). The PABAK ranged from 0.12 to 1 for the two-reader inter-rater agreement and 0.26 to 1 for the intra-rater agreement. Descriptive and broad labels achieved the highest proportion of positive agreement in both the inter- and intra-reader analyses. Annotating findings with specific, interpretive labels were found to be difficult for less experienced radiologists. Annotating images with descriptive labels may increase agreement between radiologists with different experience levels compared to annotation with interpretive labels.

Selection with Variation in Diagnostic Skill: Evidence from Radiologists

Physicians, judges, teachers, and agents in many other settings differ systematically in the decisions they make when faced with similar cases. Standard approaches to interpreting and exploiting such differences assume they arise solely from variation in preferences. We develop an alternative framework that allows variation in preferences and diagnostic skill and show that both dimensions may be partially identified in standard settings under quasi-random assignment. We apply this framework to study pneumonia diagnoses by radiologists. Diagnosis rates vary widely among radiologists, and descriptive evidence suggests that a large component of this variation is due to differences in diagnostic skill. Our estimated model suggests that radiologists view failing to diagnose a patient with pneumonia as more costly than incorrectly diagnosing one without, and that this leads less skilled radiologists to optimally choose lower diagnostic thresholds. Variation in skill can explain 39% of the variation in diagnostic decisions, and policies that improve skill perform better than uniform decision guidelines. Failing to account for skill variation can lead to highly misleading results in research designs that use agent assignments as instruments.

Comparison of artificial intelligence to the veterinary radiologist's diagnosis of canine cardiogenic pulmonary edema

Application of artificial intelligence (AI) to improve clinical diagnosis is a burgeoning field in human and veterinary medicine. The objective of this prospective, diagnostic accuracy study was to determine the accuracy, sensitivity, and specificity of an AI-based software for diagnosing canine cardiogenic pulmonary edema from thoracic radiographs, using an American College of Veterinary Radiology-certified veterinary radiologist's interpretation as the reference standard. Five hundred consecutive canine thoracic radiographs made after-hours by a veterinary Emergency Department were retrieved. A total of 481 of 500 cases were technically analyzable. Based on the radiologist's assessment, 46 (10.4%) of these 481 dogs were diagnosed with cardiogenic pulmonary edema (CPE+). Of these cases, the AI software designated 42 of 46 as CPE+ and four of 46 as cardiogenic pulmonary edema negative (CPE-). Accuracy, sensitivity, and specificity of the AI-based software compared to radiologist diagnosis were 92.3%, 91.3%, and 92.4%, respectively (positive predictive value, 56%; negative predictive value, 99%). Findings supported using AI software screening for thoracic radiographs of dogs with suspected cardiogenic pulmonary edema to assist with short-term decision-making when a radiologist is unavailable.

Interpretation of Thoracic Radiography Shows Large Discrepancies Depending on the Qualification of the Physician-Quantitative Evaluation of Interobserver Agreement in a Representative Emergency Department Scenario

(1) Background: Chest radiography (CXR) is still a key diagnostic component in the emergency department (ED). Correct interpretation is essential since some pathologies require urgent treatment. This study quantifies potential discrepancies in CXR analysis between radiologists and non-radiology physicians in training with ED experience. (2) Methods: Nine differently qualified physicians (three board-certified radiologists [BCR], three radiology residents [RR], and three non-radiology residents involved in ED [NRR]) evaluated a series of 563 posterior-anterior CXR images by quantifying suspicion for four relevant pathologies: pleural effusion, pneumothorax, pneumonia, and pulmonary nodules. Reading results were noted separately for each hemithorax on a Likert scale (0–4; 0: no suspicion of pathology, 4: safe existence of pathology) adding up to a total of 40,536 reported pathology suspicions. Interrater reliability/correlation and Kruskal–Wallis tests were performed for statistical analysis. (3) Results: While interrater reliability was good among radiologists, major discrepancies between radiologists’ and non-radiologists’ reading results could be observed in all pathologies. Highest overall interrater agreement was found for pneumothorax detection and lowest agreement in raising suspicion for malignancy suspicious nodules. Pleural effusion and pneumonia were often suspected with indifferent choices (1–3). In terms of pneumothorax detection, all readers mainly decided for a clear option (0 or 4). Interrater reliability was usually higher when evaluating the right hemithorax (all pathologies except pneumothorax). (4) Conclusions: Quantified CXR interrater reliability analysis displays a general uncertainty and strongly depends on medical training. NRR can benefit from radiology reporting in terms of time efficiency and diagnostic accuracy. CXR evaluation of long-time trained ED specialists has not been tested.

Assessment of final-year medical students' performance in diagnosing critical findings on chest X-ray

Purpose

Due to the recently emerging shortage in medical staff during the novel corona virus pandemic, several countries have rushed their undergraduate medical students into the emergency department. The accuracy of diagnosing critical findings on X-rays by senior medical students is not well assessed. In this study, we aim to assess the knowledge and accuracy of undergraduate final-year medical students in diagnosing life-threatening emergency conditions on chest x-ray.

Method

This is a cross-sectional nationwide survey across all medical schools in Jordan. Through an electronic questionnaire, participants were sequentially shown a total of six abnormal X-rays and one normal. For each X-ray, participants were asked to choose the most likely diagnosis, and to grade the degree of self-confidence regarding the accuracy of their answer in a score from 0 (not confident) to 10 (very confident).

Results

We included a total of 530 participants. All participants answered at least six out of seven questions correctly, out of them, 139 (26.2%) participants answered all questions correctly. Pneumoperitoneum was the highest correct answer (93.8%), whereas flail chest was the least correctly answered case with only 310 (58.5%) correct answers. Regarding self-confidence for each question, 338 participants (63.8%) reported very high overall self-confidence level. Answers related to tension pneumothorax had the highest confidence level.

Conclusion

Senior Jordanian medical students showed good knowledge with high confidence levels in diagnosing life-threatening conditions on chest x-rays, supporting their incorporation in the emergency department during pandemics and confirming the reliability of information they can extract.

Influence of radiologic expertise in detecting lung tumors on chest radiographs

PURPOSE

To analyze the influence of radiologic expertise in detecting lung tumors on chest radiographs.

MATERIALS AND METHODS

We retrieved posteroanterior chest radiographs and CT examination obtained from 283 patients with solitary primary malignant lung tumors who underwent surgical resection. There were 176 men and 107 women with a mean age of 67.0±9.1 (SD) years (range: 33-88 years). Thirteen first-year post-graduate (PGY-1) trainees and nine pulmonary specialists (three radiologists, three thoracic surgeons, and three pulmonologists) interpreted the chest radiographs. Detection rates among trainees and specialists were compared using Student t test.

RESULTS

The total numbers of detected tumors ranged from 103 (36.4%) to 136 (48.1%) with a mean of 127.9±9.1 (45.2±3.2%) in the trainee group, and 137 (48.4%) to 182 (64.3%) with a mean of 161.6±13.1 (57.1±4.6%) in the specialist group; the intergroup difference was statistically significant (P<0.001). Significant intergroup detectability differences of >10% were noted for tumors in the peripheral zone with (i) ground glass opacity (GGO) ratio ≥10% and <70% and any size, or (ii) GGO ratio <10% and size ≤2cm; and for tumors hidden by the mediastinum, heart, or diaphragm with (i) GGO ratio ≥10% and <30% and size >3cm, or (ii) GGO ratio <10% and size >2cm.

CONCLUSION

Our study demonstrates significant differences in lung tumor detectability on chest radiographs between PGY-1 trainees and pulmonary specialists according to tumor size, extent of GGO, and tumor location.