1
|
Barile MF, Kifjak D, El Kaddouri B, Bankier AA. Integrated cardio-thoracic imaging curriculum for residents: Design, implementation, and analysis of test and evaluation results. Eur J Radiol 2024; 177:111584. [PMID: 38909516 DOI: 10.1016/j.ejrad.2024.111584] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/18/2024] [Revised: 05/20/2024] [Accepted: 06/19/2024] [Indexed: 06/25/2024]
Abstract
As the field of cardiac imaging has demonstrated exceptional growth over the past several decades, radiology departments and residency programs have struggled to integrate cardiac imaging instruction into training curricula. PURPOSE To create an integrated cardio-thoracic teaching and lecture curriculum and resident rotation in accordance with AGGME and Society of Thoracic Radiology (STR) guidelines. MATERIALS AND METHODS Consecutive PGY-2 to PGY-4 residents (n = 14) rotating through our Cardiothoracic Imaging (CTI) section from 1/1/2021 to 04/18/2022 were give pre- and post- rotation tests of knowledge and feedback evaluations. Attending feedback of the curriculum was obtained at 3-months and 9-months post curriculum implementation. A Wilcxon test was used to evaluate differences in improvement between pre- and post- rotation resident feedback scores, test scores for thoracic and cardiac test questions in addition to attending feedback scores at 3 and 9-months post curriculum implementation. RESULTS The overall post-rotation scores in addition to thoracic only and cardiac only scores improved, with the difference between improved versus stable or decreased scores being statistically significant overall (P = 0.039) and for cardiac scores (P = 0.003), but not for thoracic scores (P = 0.22). The overall (P = 0.002), thoracic (P = 0.027), and cardiac (P = 0.026) resident feedback scores were significantly improved post-rotation. Similarly, the overall attending feedback scores significantly improved over time (P = 0.021). CONCLUSION An integrated Cardio-thoracic Imaging teaching curriculum was well received by both residents and attendings with significant improvement in post rotation feedback scores by both groups. Moreover, residents demonstrated improved scores on knowledge tests post rotation.
Collapse
Affiliation(s)
- Maria F Barile
- Department of Radiology, University of Massachusetts Memorial Health and University of Massachusetts Chan Medical School, 55 N. lake Ave., Worcester, MA, United States.
| | - Daria Kifjak
- Department of Radiology, University of Massachusetts Memorial Health and University of Massachusetts Chan Medical School, 55 N. lake Ave., Worcester, MA, United States; Department of Biomedical Imaging and Image-guided Therapy, Medical University of Vienna, Vienna, Austria.
| | - Bilal El Kaddouri
- Department of Radiology, University of Massachusetts Memorial Health and University of Massachusetts Chan Medical School, 55 N. lake Ave., Worcester, MA, United States; Department of Radiology, Erasmus Hospital, Université Libre de Bruxelles, Brussels, Belgium.
| | - Alexander A Bankier
- Department of Radiology, University of Massachusetts Memorial Health and University of Massachusetts Chan Medical School, 55 N. lake Ave., Worcester, MA, United States.
| |
Collapse
|
2
|
Factor S, Albagli A, Bebin A, Druckmann I, Bulkowstein S, Stahl I, Shichman I. Influence of residency discipline and seniority on traumatic musculoskeletal radiographs interpretation accuracy: a multicenter study. Eur J Trauma Emerg Surg 2023; 49:2589-2597. [PMID: 37573536 DOI: 10.1007/s00068-023-02347-0] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/13/2023] [Accepted: 08/01/2023] [Indexed: 08/15/2023]
Abstract
OBJECTIVES Imaging studies are a significant and integral part of the initial assessment of patients admitted to the emergency department. Developing imaging diagnostic abilities early in residency is of paramount importance. The purpose of this study was to evaluate and compare diagnosis accuracy of common musculoskeletal X-rays (XR) between residency disciplines and seniority. METHODS A multicenter study which evaluated orthopedic surgery, emergency medicine (EM), and radiology residents, through a test set of common MSK XR. Residents were classified as "beginner" or "advanced" according to postgraduate year per residency. Residents were asked to answer whether the radiograph shows normal or pathological findings (success rate) and what is the diagnosis ("diagnosis accuracy"). Residents' answers were analyzed and assessed compared to experts' consensus. RESULTS A total of 100 residents (62% beginners) participated in this study. Fifty-four were orthopedic surgeons, 29 were EM residents and 17 were radiologists. The entire cohort's overall success rate was 88.5%. The overall mean success rates for orthopedic, EM, and radiology residents were 93.2%, 82.8%, and 83.3%, respectively, and were significantly different (p < 0.0001). Orthopedic residents had significantly higher diagnostic accuracy rates compared with both radiology and EM residents (p < 0.001). Advanced orthopedic and EM residents demonstrated higher diagnostic accuracy rates compared to beginner residents (p = 0.001 and p = 0.03, respectively). CONCLUSION Orthopedic residents presented higher diagnosis accuracy of MSK imaging compared to EM and radiology residents. Seniority had a positive effect on diagnosis accuracy. The development of an educational program on MSK XR is necessary to enhance the competency of physicians in their daily practice.
Collapse
Affiliation(s)
- Shai Factor
- Division of Orthopedic Surgery, Tel Aviv Medical Center, 6 Weitzman St., 6423906, Tel Aviv, Israel.
- Sackler Faculty of Medicine, Tel Aviv University, Tel Aviv, Israel.
| | - Assaf Albagli
- Division of Orthopedic Surgery, Tel Aviv Medical Center, 6 Weitzman St., 6423906, Tel Aviv, Israel
- Sackler Faculty of Medicine, Tel Aviv University, Tel Aviv, Israel
| | - Alex Bebin
- Division of Orthopedic Surgery, Tel Aviv Medical Center, 6 Weitzman St., 6423906, Tel Aviv, Israel
- Sackler Faculty of Medicine, Tel Aviv University, Tel Aviv, Israel
| | - Ido Druckmann
- Sackler Faculty of Medicine, Tel Aviv University, Tel Aviv, Israel
- Division of Radiology, Tel Aviv Medical Center, 6423906, Tel Aviv, Israel
| | - Shlomi Bulkowstein
- Division of Orthopedics, Soroka University Medical Center, Beer-Sheva, P.O. Box 151, 84101, Beer-Sheva, Israel
- Affiliated to the Faculty of Health Sciences, Ben-Gurion University of the Negev, Beer Sheva, Israel
| | - Ido Stahl
- Division of Orthopedic Surgery, Rambam Healthcare Campus, 3109601, Haifa, Israel
- Affiliated to the Rappaport Faculty of Medicine, Technion-Israeli Institute of Technology, Haifa, Israel
| | - Ittai Shichman
- Division of Orthopedic Surgery, Tel Aviv Medical Center, 6 Weitzman St., 6423906, Tel Aviv, Israel
- Sackler Faculty of Medicine, Tel Aviv University, Tel Aviv, Israel
| |
Collapse
|
3
|
Gangi-Burton A, Chan N, Ashok AH, Nair A. Simple demographic, laboratory and chest radiograph variables can identify COVID-19 patients with pulmonary thromboembolism: a retrospective multicentre United Kingdom study. Br J Radiol 2023; 96:20230082. [PMID: 37747264 PMCID: PMC10646650 DOI: 10.1259/bjr.20230082] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/24/2023] [Revised: 03/20/2023] [Accepted: 04/10/2023] [Indexed: 09/26/2023] Open
Abstract
OBJECTIVES To (1) identify discriminatory demographic, laboratory and initial CXR findings; (2) explore correlation between D-dimer and radiographic severity scores; and (3) assess accuracy of published D-dimer thresholds to identify pulmonary thromboembolism (PTE) in COVID-19 patients. METHODS Retrospective study including all COVID-19 patients admitted from 1st to 30th April 2020 meeting inclusion criteria from 25 (blinded) hospitals. Demographics, blood results, CXR and CTPA findings were compared between positive and negative PTE cohorts using uni- and multivariable logistic regression. Published D-dimer cut-offs were applied. RESULTS 389 patients were included [median age 63; 237 males], of which 26.2% had a PTE. Significant univariable discriminators for PTE were peak D-dimer, sex, neutrophil count at the time of the D-dimer and at admission, abnormal CXR, and CXR zonal severity score. Only neutrophil count at peak D-dimer remained significant for predicting PTE on multivariable analysis (p = 0.008). When compared with the published literature, sensitivity for PTE were lower than those published at all cut-off values, however specificity at different cut-offs was variable. CONCLUSIONS In this multicentre COVID-19 cohort, univariable admission factors that could indicate pulmonary thromboembolism were male sex, high neutrophil count and abnormal CXR with a greater CXR zonal severity score. The accuracy levels of published D-dimer thresholds were not reproducible in our population. ADVANCES IN KNOWLEDGE This is a large multicentre study looking at the discriminatory value of simple variables to determine if a patient with COVID-19 has PTE or not, in addition to comparing D-dimer cut off values against published values.
Collapse
Affiliation(s)
- Anmol Gangi-Burton
- Department of Radiology, Nottingham University Hospitals NHS Trust, Nottingham, United Kingdom
| | - Nathan Chan
- Department of Interventional Neuroradiology, The Royal London Hospital, London, United Kingdom
| | - Abhishekh H Ashok
- Department of Radiology, Addenbrooke’s Hospital, Cambridge University Hospital NHS Foundation Trust, Cambridge, United Kingdom
| | - Arjun Nair
- Department of Radiology, University of Cambridge, Cambridge, United Kingdom
| |
Collapse
|
4
|
Chassagnon G, Billet N, Rutten C, Toussaint T, Cassius de Linval Q, Collin M, Lemouchi L, Homps M, Hedjoudje M, Ventre J, Gregory J, Canniff E, Regnard NE, Bennani S, Revel MP. Learning from the machine: AI assistance is not an effective learning tool for resident education in chest x-ray interpretation. Eur Radiol 2023; 33:8241-8250. [PMID: 37572190 DOI: 10.1007/s00330-023-10043-1] [Citation(s) in RCA: 3] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/03/2023] [Revised: 05/29/2023] [Accepted: 06/20/2023] [Indexed: 08/14/2023]
Abstract
OBJECTIVES To assess whether a computer-aided detection (CADe) system could serve as a learning tool for radiology residents in chest X-ray (CXR) interpretation. METHODS Eight radiology residents were asked to interpret 500 CXRs for the detection of five abnormalities, namely pneumothorax, pleural effusion, alveolar syndrome, lung nodule, and mediastinal mass. After interpreting 150 CXRs, the residents were divided into 2 groups of equivalent performance and experience. Subsequently, group 1 interpreted 200 CXRs from the "intervention dataset" using a CADe as a second reader, while group 2 served as a control by interpreting the same CXRs without the use of CADe. Finally, the 2 groups interpreted another 150 CXRs without the use of CADe. The sensitivity, specificity, and accuracy before, during, and after the intervention were compared. RESULTS Before the intervention, the median individual sensitivity, specificity, and accuracy of the eight radiology residents were 43% (range: 35-57%), 90% (range: 82-96%), and 81% (range: 76-84%), respectively. With the use of CADe, residents from group 1 had a significantly higher overall sensitivity (53% [n = 431/816] vs 43% [n = 349/816], p < 0.001), specificity (94% [i = 3206/3428] vs 90% [n = 3127/3477], p < 0.001), and accuracy (86% [n = 3637/4244] vs 81% [n = 3476/4293], p < 0.001), compared to the control group. After the intervention, there were no significant differences between group 1 and group 2 regarding the overall sensitivity (44% [n = 309/696] vs 46% [n = 317/696], p = 0.666), specificity (90% [n = 2294/2541] vs 90% [n = 2285/2542], p = 0.642), or accuracy (80% [n = 2603/3237] vs 80% [n = 2602/3238], p = 0.955). CONCLUSIONS Although it improves radiology residents' performances for interpreting CXRs, a CADe system alone did not appear to be an effective learning tool and should not replace teaching. CLINICAL RELEVANCE STATEMENT Although the use of artificial intelligence improves radiology residents' performance in chest X-rays interpretation, artificial intelligence cannot be used alone as a learning tool and should not replace dedicated teaching. KEY POINTS • With CADe as a second reader, residents had a significantly higher sensitivity (53% vs 43%, p < 0.001), specificity (94% vs 90%, p < 0.001), and accuracy (86% vs 81%, p < 0.001), compared to residents without CADe. • After removing access to the CADe system, residents' sensitivity (44% vs 46%, p = 0.666), specificity (90% vs 90%, p = 0.642), and accuracy (80% vs 80%, p = 0.955) returned to that of the level for the group without CADe.
Collapse
Affiliation(s)
- Guillaume Chassagnon
- Radiology Department, Hôpital Cochin, AP-HP, 27 Rue du Faubourg Saint-Jacques, 75014, Paris, France.
- Université de Paris, 27 Rue du Faubourg Saint-Jacques, 85 Boulevard Saint-Germain, 75006, Paris, France.
| | - Nicolas Billet
- Radiology Department, Hôpital Cochin, AP-HP, 27 Rue du Faubourg Saint-Jacques, 75014, Paris, France
| | - Caroline Rutten
- Radiology Department, Hôpital Cochin, AP-HP, 27 Rue du Faubourg Saint-Jacques, 75014, Paris, France
| | - Thibault Toussaint
- Radiology Department, Hôpital Cochin, AP-HP, 27 Rue du Faubourg Saint-Jacques, 75014, Paris, France
| | | | - Mégane Collin
- Radiology Department, Hôpital Cochin, AP-HP, 27 Rue du Faubourg Saint-Jacques, 75014, Paris, France
| | - Leila Lemouchi
- Radiology Department, Hôpital Cochin, AP-HP, 27 Rue du Faubourg Saint-Jacques, 75014, Paris, France
| | - Margaux Homps
- Radiology Department, Hôpital Cochin, AP-HP, 27 Rue du Faubourg Saint-Jacques, 75014, Paris, France
| | - Mohamed Hedjoudje
- Radiology Department, Hôpital Cochin, AP-HP, 27 Rue du Faubourg Saint-Jacques, 75014, Paris, France
| | | | - Jules Gregory
- Université de Paris, 27 Rue du Faubourg Saint-Jacques, 85 Boulevard Saint-Germain, 75006, Paris, France
- Radiology Department, FHU MOSAIC, Hôpital Beaujon, 100 Bd du Général Leclerc, 92110, Clichy, France
| | - Emma Canniff
- Radiology Department, Hôpital Cochin, AP-HP, 27 Rue du Faubourg Saint-Jacques, 75014, Paris, France
| | - Nor-Eddine Regnard
- Gleamer, 117 Quai de Valmy, 75010, Paris, France
- Réseau d'Imagerie Sud Francilien, 254 Ter Avenue Henri Barbusse, 91210, Draveil, France
| | - Souhail Bennani
- Radiology Department, Hôpital Cochin, AP-HP, 27 Rue du Faubourg Saint-Jacques, 75014, Paris, France
- Gleamer, 117 Quai de Valmy, 75010, Paris, France
| | - Marie-Pierre Revel
- Radiology Department, Hôpital Cochin, AP-HP, 27 Rue du Faubourg Saint-Jacques, 75014, Paris, France
- Université de Paris, 27 Rue du Faubourg Saint-Jacques, 85 Boulevard Saint-Germain, 75006, Paris, France
| |
Collapse
|
5
|
Di Serafino M, Dell’Aversano Orabona G, Caruso M, Camillo C, Viscardi D, Iacobellis F, Ronza R, Sabatino V, Barbuto L, Oliva G, Romano L. Point-of-Care Lung Ultrasound in the Intensive Care Unit-The Dark Side of Radiology: Where Do We Stand? J Pers Med 2023; 13:1541. [PMID: 38003856 PMCID: PMC10672373 DOI: 10.3390/jpm13111541] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/24/2023] [Revised: 10/23/2023] [Accepted: 10/24/2023] [Indexed: 11/26/2023] Open
Abstract
Patients in intensive care units (ICUs) are critically ill and require constant monitoring of clinical conditions. Due to the severity of the underlying disease and the need to monitor devices, imaging plays a crucial role in critically ill patients' care. Given the clinical complexity of these patients, who typically need respiratory assistance as well as continuous monitoring of vital functions and equipment, computed tomography (CT) can be regarded as the diagnostic gold standard, although it is not a bedside diagnostic technique. Despite its limitations, portable chest X-ray (CXR) is still today an essential diagnostic tool used in the ICU. Being a widely accessible imaging technique, which can be performed at the patient's bedside and at a low healthcare cost, it provides additional diagnostic support to the patient's clinical management. In recent years, the use of point-of-care lung ultrasound (LUS) in ICUs for procedure guidance, diagnosis, and screening has proliferated, and it is usually performed at the patient's bedside. This review illustrates the role of point-of-care LUS in ICUs from a purely radiological point of view as an advanced method in ICU CXR reports to improve the interpretation and monitoring of lung CXR findings.
Collapse
Affiliation(s)
- Marco Di Serafino
- Department of General and Emergency Radiology, “Antonio Cardarelli” Hospital, 80131 Naples, Italy; (G.D.O.); (M.C.); (C.C.); (F.I.); (R.R.); (V.S.); (L.B.); (G.O.); (L.R.)
| | - Giuseppina Dell’Aversano Orabona
- Department of General and Emergency Radiology, “Antonio Cardarelli” Hospital, 80131 Naples, Italy; (G.D.O.); (M.C.); (C.C.); (F.I.); (R.R.); (V.S.); (L.B.); (G.O.); (L.R.)
| | - Martina Caruso
- Department of General and Emergency Radiology, “Antonio Cardarelli” Hospital, 80131 Naples, Italy; (G.D.O.); (M.C.); (C.C.); (F.I.); (R.R.); (V.S.); (L.B.); (G.O.); (L.R.)
| | - Costanza Camillo
- Department of General and Emergency Radiology, “Antonio Cardarelli” Hospital, 80131 Naples, Italy; (G.D.O.); (M.C.); (C.C.); (F.I.); (R.R.); (V.S.); (L.B.); (G.O.); (L.R.)
| | - Daniela Viscardi
- Department of Intensive Care and Resuscitation, “Antonio Cardarelli” Hospital, 80131 Naples, Italy;
| | - Francesca Iacobellis
- Department of General and Emergency Radiology, “Antonio Cardarelli” Hospital, 80131 Naples, Italy; (G.D.O.); (M.C.); (C.C.); (F.I.); (R.R.); (V.S.); (L.B.); (G.O.); (L.R.)
| | - Roberto Ronza
- Department of General and Emergency Radiology, “Antonio Cardarelli” Hospital, 80131 Naples, Italy; (G.D.O.); (M.C.); (C.C.); (F.I.); (R.R.); (V.S.); (L.B.); (G.O.); (L.R.)
| | - Vittorio Sabatino
- Department of General and Emergency Radiology, “Antonio Cardarelli” Hospital, 80131 Naples, Italy; (G.D.O.); (M.C.); (C.C.); (F.I.); (R.R.); (V.S.); (L.B.); (G.O.); (L.R.)
| | - Luigi Barbuto
- Department of General and Emergency Radiology, “Antonio Cardarelli” Hospital, 80131 Naples, Italy; (G.D.O.); (M.C.); (C.C.); (F.I.); (R.R.); (V.S.); (L.B.); (G.O.); (L.R.)
| | - Gaspare Oliva
- Department of General and Emergency Radiology, “Antonio Cardarelli” Hospital, 80131 Naples, Italy; (G.D.O.); (M.C.); (C.C.); (F.I.); (R.R.); (V.S.); (L.B.); (G.O.); (L.R.)
| | - Luigia Romano
- Department of General and Emergency Radiology, “Antonio Cardarelli” Hospital, 80131 Naples, Italy; (G.D.O.); (M.C.); (C.C.); (F.I.); (R.R.); (V.S.); (L.B.); (G.O.); (L.R.)
| |
Collapse
|
6
|
Al Elq A, Alfayez AA, AlQahtani MI, Alshahrani RS, Alotaibi GA, Aldakheel AA, Alhammad AA, Bakr Mohamed RH, Jarrar M, Abusalah MAHA, Al-Bsheish M. The Effects of Various Teaching Methods on Chest X-ray Interpretation Skills Among Medical Students and Interns: A Systematic Review. Cureus 2023; 15:e44399. [PMID: 37791172 PMCID: PMC10542214 DOI: 10.7759/cureus.44399] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 08/30/2023] [Indexed: 10/05/2023] Open
Abstract
Chest X-ray (CXR) is a common tool used in medical practice. Medical students and interns should acquire knowledge of CXR interpretation, as it is an essential diagnostic tool for a large spectrum of diseases. This systematic review aimed to compare the effect of different intervention techniques on the competency of medical students and interns to demonstrate the level of confidence and competence in interpreting common presentations of CXRs. The population, intervention, comparison, and outcomes (PICO) framework was used to formulate the review question. All related articles in five databases (PubMed, Web of Science, Scopus, Medline, and Embase) were retrieved and the search was completed in March 2023 with no limiters on date and time. The number of relevant studies was 469. A multi-level approach through the Rayyan platform was used for the screening and exclusion processes. Eleven articles were included in the systematic review consisting of eight randomized controlled trials, one quasi-experimental study, one cross-sectional study, and one interventional cohort. Results showed significant effects of teaching methods utilizing deductive or inductive approach, clinical history, patient care comfort survey, and SAFMEDS (Say-All-Fast-Minute-Every-Day-Shuffled). Contrarily, no significant effect was shown by flipped classroom models and mixed and blocked practice, peer-assisted learning vs. expert-assisted learning, and Chester, an artificial intelligence tool. This review identified beneficial approaches that may enhance the learning outcomes of interpreting CXRs for medical students and interns, highlighting the remarkable impact of SAFMEDS on medical students' ability to identify CXR findings as well as the availability and practicality of online and e-learning resources for students.
Collapse
Affiliation(s)
- Abdulmohsen Al Elq
- Department of Internal Medicine, College of Medicine, Imam Abdulrahman Bin Faisal University, Dammam, SAU
| | | | | | - Rakan S Alshahrani
- College of Medicine, Imam Abdulrahman Bin Faisal University, Dammam, SAU
| | - Ghazi A Alotaibi
- College of Medicine, Imam Abdulrahman Bin Faisal University, Dammam, SAU
| | | | - Ali A Alhammad
- College of Medicine, Imam Abdulrahman Bin Faisal University, Dammam, SAU
| | - Radwa H Bakr Mohamed
- Vice Deanship for Development and Community Partnership, College of Medicine, Imam Abdulrahman Bin Faisal University, Dammam, SAU
| | - Mu'taman Jarrar
- Vice Deanship for Development and Community Partnership, College of Medicine, Imam Abdulrahman Bin Faisal University, Dammam, SAU
- Department of Medical Education, King Fahd Hospital of the University, Al-Khobar, SAU
| | - Mai Abdel Haleem A Abusalah
- Department of Medical Laboratory Sciences, Faculty of Allied Medical Sciences, Zarqa University, Al-Zarqa, JOR
| | - Mohammad Al-Bsheish
- Department of Health Management, Batterjee Medical College, Jeddah, SAU
- Department of Occupational Health, Al-Nadeem Governmental Hospital, Ministry of Health, Amman, JOR
| |
Collapse
|
7
|
Irusen E. Chest radiographic interpretation - an essential competency for Africa. Afr J Thorac Crit Care Med 2023; 29:10.7196/AJTCCM.2023.v29i1.893. [PMID: 37476655 PMCID: PMC10355186 DOI: 10.7196/ajtccm.2023.v29i1.892] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 07/22/2023] Open
Affiliation(s)
- Elvis Irusen
- Stellenbosch University and Tygerberg Academic Hospital
| |
Collapse
|
8
|
Bregman S, Thau E, Pusic M, Perez M, Boutis K. A Performance-Based Competency Assessment of Pediatric Chest Radiograph Interpretation Among Practicing Physicians. THE JOURNAL OF CONTINUING EDUCATION IN THE HEALTH PROFESSIONS 2022; 44:28-34. [PMID: 36728983 DOI: 10.1097/ceh.0000000000000481] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 07/13/2022] [Accepted: 10/19/2022] [Indexed: 06/18/2023]
Abstract
INTRODUCTION There is limited knowledge on pediatric chest radiograph (pCXR) interpretation skill among practicing physicians. We systematically determined baseline interpretation skill, the number of pCXR cases physicians required complete to achieve a performance benchmark, and which diagnoses posed the greatest diagnostic challenge. METHODS Physicians interpreted 434 pCXR cases via a web-based platform until they achieved a performance benchmark of 85% accuracy, sensitivity, and specificity. Interpretation difficulty scores for each case were derived by applying one-parameter item response theory to participant data. We compared interpretation difficulty scores across diagnostic categories and described the diagnoses of the 30% most difficult-to-interpret cases. RESULTS 240 physicians who practice in one of three geographic areas interpreted cases, yielding 56,833 pCXR case interpretations. The initial diagnostic performance (first 50 cases) of our participants demonstrated an accuracy of 68.9%, sensitivity of 69.4%, and a specificity of 68.4%. The median number of cases completed to achieve the performance benchmark was 102 (interquartile range 69, 176; min, max, 54, 431). Among the 30% most difficult-to-interpret cases, 39.2% were normal pCXR and 32.3% were cases of lobar pneumonia. Cases with a single trauma-related imaging finding, cardiac, hilar, and diaphragmatic pathologies were also among the most challenging. DISCUSSION At baseline, practicing physicians misdiagnosed about one-third of pCXR and there was up to an eight-fold difference between participants in number of cases completed to achieve the standardized performance benchmark. We also identified the diagnoses with the greatest potential for educational intervention.
Collapse
Affiliation(s)
- Stacey Bregman
- Dr. Bregman: Division of Pediatric Emergency Medicine, Department of Pediatrics, Hospital for Sick Children and University of Toronto, Toronto, Ontario, Canada. Dr. Thau: Division of Pediatric Emergency Medicine, Department of Pediatrics, Hospital for Sick Children and University of Toronto, Toronto, Ontario, Canada. Dr. Pusic: Division of Pediatric Emergency Medicine, Department of Pediatrics, Boston Children's Hospital, Harvard University; Boston, MA. Dr. Perez: Department of Diagnostic Imaging, University of Toronto, Ontario, Canada. Dr. Boutis: Division of Pediatric Emergency Medicine, Department of Pediatrics, Hospital for Sick Children and University of Toronto, Toronto, Ontario, Canada
| | | | | | | | | |
Collapse
|
9
|
Li D, Pehrson LM, Tøttrup L, Fraccaro M, Bonnevie R, Thrane J, Sørensen PJ, Rykkje A, Andersen TT, Steglich-Arnholm H, Stærk DMR, Borgwardt L, Hansen KL, Darkner S, Carlsen JF, Nielsen MB. Inter- and Intra-Observer Agreement When Using a Diagnostic Labeling Scheme for Annotating Findings on Chest X-rays-An Early Step in the Development of a Deep Learning-Based Decision Support System. Diagnostics (Basel) 2022; 12:diagnostics12123112. [PMID: 36553118 PMCID: PMC9776917 DOI: 10.3390/diagnostics12123112] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/19/2022] [Revised: 11/21/2022] [Accepted: 11/26/2022] [Indexed: 12/14/2022] Open
Abstract
Consistent annotation of data is a prerequisite for the successful training and testing of artificial intelligence-based decision support systems in radiology. This can be obtained by standardizing terminology when annotating diagnostic images. The purpose of this study was to evaluate the annotation consistency among radiologists when using a novel diagnostic labeling scheme for chest X-rays. Six radiologists with experience ranging from one to sixteen years, annotated a set of 100 fully anonymized chest X-rays. The blinded radiologists annotated on two separate occasions. Statistical analyses were done using Randolph's kappa and PABAK, and the proportions of specific agreements were calculated. Fair-to-excellent agreement was found for all labels among the annotators (Randolph's Kappa, 0.40-0.99). The PABAK ranged from 0.12 to 1 for the two-reader inter-rater agreement and 0.26 to 1 for the intra-rater agreement. Descriptive and broad labels achieved the highest proportion of positive agreement in both the inter- and intra-reader analyses. Annotating findings with specific, interpretive labels were found to be difficult for less experienced radiologists. Annotating images with descriptive labels may increase agreement between radiologists with different experience levels compared to annotation with interpretive labels.
Collapse
Affiliation(s)
- Dana Li
- Department of Diagnostic Radiology, Copenhagen University Hospital, Rigshospitalet, 2100 Copenhagen, Denmark
- Department of Clinical Medicine, University of Copenhagen, 2100 Copenhagen, Denmark
- Correspondence:
| | - Lea Marie Pehrson
- Department of Diagnostic Radiology, Copenhagen University Hospital, Rigshospitalet, 2100 Copenhagen, Denmark
- Department of Computer Science, University of Copenhagen, 2100 Copenhagen, Denmark
| | | | | | | | | | - Peter Jagd Sørensen
- Department of Diagnostic Radiology, Copenhagen University Hospital, Rigshospitalet, 2100 Copenhagen, Denmark
- Department of Clinical Medicine, University of Copenhagen, 2100 Copenhagen, Denmark
| | - Alexander Rykkje
- Department of Diagnostic Radiology, Copenhagen University Hospital, Rigshospitalet, 2100 Copenhagen, Denmark
- Department of Clinical Medicine, University of Copenhagen, 2100 Copenhagen, Denmark
| | - Tobias Thostrup Andersen
- Department of Diagnostic Radiology, Copenhagen University Hospital, Rigshospitalet, 2100 Copenhagen, Denmark
| | - Henrik Steglich-Arnholm
- Department of Diagnostic Radiology, Copenhagen University Hospital, Rigshospitalet, 2100 Copenhagen, Denmark
| | - Dorte Marianne Rohde Stærk
- Department of Diagnostic Radiology, Copenhagen University Hospital, Rigshospitalet, 2100 Copenhagen, Denmark
| | - Lotte Borgwardt
- Department of Diagnostic Radiology, Copenhagen University Hospital, Rigshospitalet, 2100 Copenhagen, Denmark
| | - Kristoffer Lindskov Hansen
- Department of Diagnostic Radiology, Copenhagen University Hospital, Rigshospitalet, 2100 Copenhagen, Denmark
- Department of Clinical Medicine, University of Copenhagen, 2100 Copenhagen, Denmark
| | - Sune Darkner
- Department of Computer Science, University of Copenhagen, 2100 Copenhagen, Denmark
| | - Jonathan Frederik Carlsen
- Department of Diagnostic Radiology, Copenhagen University Hospital, Rigshospitalet, 2100 Copenhagen, Denmark
- Department of Clinical Medicine, University of Copenhagen, 2100 Copenhagen, Denmark
| | - Michael Bachmann Nielsen
- Department of Diagnostic Radiology, Copenhagen University Hospital, Rigshospitalet, 2100 Copenhagen, Denmark
- Department of Clinical Medicine, University of Copenhagen, 2100 Copenhagen, Denmark
| |
Collapse
|
10
|
Chan DC, Gentzkow M, Yu C. Selection with Variation in Diagnostic Skill: Evidence from Radiologists. THE QUARTERLY JOURNAL OF ECONOMICS 2022; 137:729-783. [PMID: 35422677 PMCID: PMC8992547 DOI: 10.1093/qje/qjab048] [Citation(s) in RCA: 9] [Impact Index Per Article: 4.5] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/07/2023]
Abstract
Physicians, judges, teachers, and agents in many other settings differ systematically in the decisions they make when faced with similar cases. Standard approaches to interpreting and exploiting such differences assume they arise solely from variation in preferences. We develop an alternative framework that allows variation in preferences and diagnostic skill and show that both dimensions may be partially identified in standard settings under quasi-random assignment. We apply this framework to study pneumonia diagnoses by radiologists. Diagnosis rates vary widely among radiologists, and descriptive evidence suggests that a large component of this variation is due to differences in diagnostic skill. Our estimated model suggests that radiologists view failing to diagnose a patient with pneumonia as more costly than incorrectly diagnosing one without, and that this leads less skilled radiologists to optimally choose lower diagnostic thresholds. Variation in skill can explain 39% of the variation in diagnostic decisions, and policies that improve skill perform better than uniform decision guidelines. Failing to account for skill variation can lead to highly misleading results in research designs that use agent assignments as instruments.
Collapse
Affiliation(s)
- David C Chan
- Stanford University, Department of Veterans Affairs, and National Bureau of Economic Research, United States
| | - Matthew Gentzkow
- Stanford University and National Bureau of Economic Research, United States
| | - Chuan Yu
- Stanford University, United States
| |
Collapse
|
11
|
Kim E, Fischetti AJ, Sreetharan P, Weltman JG, Fox PR. Comparison of artificial intelligence to the veterinary radiologist's diagnosis of canine cardiogenic pulmonary edema. Vet Radiol Ultrasound 2022; 63:292-297. [PMID: 35048445 DOI: 10.1111/vru.13062] [Citation(s) in RCA: 12] [Impact Index Per Article: 6.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/20/2021] [Revised: 12/08/2021] [Accepted: 12/08/2021] [Indexed: 11/29/2022] Open
Abstract
Application of artificial intelligence (AI) to improve clinical diagnosis is a burgeoning field in human and veterinary medicine. The objective of this prospective, diagnostic accuracy study was to determine the accuracy, sensitivity, and specificity of an AI-based software for diagnosing canine cardiogenic pulmonary edema from thoracic radiographs, using an American College of Veterinary Radiology-certified veterinary radiologist's interpretation as the reference standard. Five hundred consecutive canine thoracic radiographs made after-hours by a veterinary Emergency Department were retrieved. A total of 481 of 500 cases were technically analyzable. Based on the radiologist's assessment, 46 (10.4%) of these 481 dogs were diagnosed with cardiogenic pulmonary edema (CPE+). Of these cases, the AI software designated 42 of 46 as CPE+ and four of 46 as cardiogenic pulmonary edema negative (CPE-). Accuracy, sensitivity, and specificity of the AI-based software compared to radiologist diagnosis were 92.3%, 91.3%, and 92.4%, respectively (positive predictive value, 56%; negative predictive value, 99%). Findings supported using AI software screening for thoracic radiographs of dogs with suspected cardiogenic pulmonary edema to assist with short-term decision-making when a radiologist is unavailable.
Collapse
Affiliation(s)
- Eunbee Kim
- Department of Diagnostic Imaging, The Animal Medical Center, New York, New York, USA
| | - Anthony J Fischetti
- Department of Diagnostic Imaging, The Animal Medical Center, New York, New York, USA
| | | | - Joel G Weltman
- Department of Emergency and Critical Care, The Animal Medical Center, New York, New York, USA
| | - Philip R Fox
- Department of Cardiology, The Animal Medical Center, New York, New York, USA
| |
Collapse
|
12
|
Interpretation of Thoracic Radiography Shows Large Discrepancies Depending on the Qualification of the Physician-Quantitative Evaluation of Interobserver Agreement in a Representative Emergency Department Scenario. Diagnostics (Basel) 2021; 11:diagnostics11101868. [PMID: 34679566 PMCID: PMC8534346 DOI: 10.3390/diagnostics11101868] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/05/2021] [Revised: 09/22/2021] [Accepted: 10/06/2021] [Indexed: 01/31/2023] Open
Abstract
(1) Background: Chest radiography (CXR) is still a key diagnostic component in the emergency department (ED). Correct interpretation is essential since some pathologies require urgent treatment. This study quantifies potential discrepancies in CXR analysis between radiologists and non-radiology physicians in training with ED experience. (2) Methods: Nine differently qualified physicians (three board-certified radiologists [BCR], three radiology residents [RR], and three non-radiology residents involved in ED [NRR]) evaluated a series of 563 posterior-anterior CXR images by quantifying suspicion for four relevant pathologies: pleural effusion, pneumothorax, pneumonia, and pulmonary nodules. Reading results were noted separately for each hemithorax on a Likert scale (0–4; 0: no suspicion of pathology, 4: safe existence of pathology) adding up to a total of 40,536 reported pathology suspicions. Interrater reliability/correlation and Kruskal–Wallis tests were performed for statistical analysis. (3) Results: While interrater reliability was good among radiologists, major discrepancies between radiologists’ and non-radiologists’ reading results could be observed in all pathologies. Highest overall interrater agreement was found for pneumothorax detection and lowest agreement in raising suspicion for malignancy suspicious nodules. Pleural effusion and pneumonia were often suspected with indifferent choices (1–3). In terms of pneumothorax detection, all readers mainly decided for a clear option (0 or 4). Interrater reliability was usually higher when evaluating the right hemithorax (all pathologies except pneumothorax). (4) Conclusions: Quantified CXR interrater reliability analysis displays a general uncertainty and strongly depends on medical training. NRR can benefit from radiology reporting in terms of time efficiency and diagnostic accuracy. CXR evaluation of long-time trained ED specialists has not been tested.
Collapse
|
13
|
Assessment of final-year medical students' performance in diagnosing critical findings on chest X-ray. Emerg Radiol 2021; 28:333-338. [PMID: 33398711 PMCID: PMC7781396 DOI: 10.1007/s10140-020-01893-z] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/30/2020] [Accepted: 12/17/2020] [Indexed: 11/07/2022]
Abstract
Purpose Due to the recently emerging shortage in medical staff during the novel corona virus pandemic, several countries have rushed their undergraduate medical students into the emergency department. The accuracy of diagnosing critical findings on X-rays by senior medical students is not well assessed. In this study, we aim to assess the knowledge and accuracy of undergraduate final-year medical students in diagnosing life-threatening emergency conditions on chest x-ray. Method This is a cross-sectional nationwide survey across all medical schools in Jordan. Through an electronic questionnaire, participants were sequentially shown a total of six abnormal X-rays and one normal. For each X-ray, participants were asked to choose the most likely diagnosis, and to grade the degree of self-confidence regarding the accuracy of their answer in a score from 0 (not confident) to 10 (very confident). Results We included a total of 530 participants. All participants answered at least six out of seven questions correctly, out of them, 139 (26.2%) participants answered all questions correctly. Pneumoperitoneum was the highest correct answer (93.8%), whereas flail chest was the least correctly answered case with only 310 (58.5%) correct answers. Regarding self-confidence for each question, 338 participants (63.8%) reported very high overall self-confidence level. Answers related to tension pneumothorax had the highest confidence level. Conclusion Senior Jordanian medical students showed good knowledge with high confidence levels in diagnosing life-threatening conditions on chest x-rays, supporting their incorporation in the emergency department during pandemics and confirming the reliability of information they can extract.
Collapse
|
14
|
Lesion detection on a combined “All-in-One” window compared to conventional window settings in thoracic oncology chest CT examinations. Diagn Interv Imaging 2020; 101:25-33. [DOI: 10.1016/j.diii.2019.07.009] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/18/2019] [Revised: 07/21/2019] [Accepted: 07/24/2019] [Indexed: 01/11/2023]
|
15
|
Influence of radiologic expertise in detecting lung tumors on chest radiographs. Diagn Interv Imaging 2018; 100:95-107. [PMID: 30266280 DOI: 10.1016/j.diii.2018.08.015] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/29/2018] [Revised: 08/20/2018] [Accepted: 08/30/2018] [Indexed: 11/22/2022]
Abstract
PURPOSE To analyze the influence of radiologic expertise in detecting lung tumors on chest radiographs. MATERIALS AND METHODS We retrieved posteroanterior chest radiographs and CT examination obtained from 283 patients with solitary primary malignant lung tumors who underwent surgical resection. There were 176 men and 107 women with a mean age of 67.0±9.1 (SD) years (range: 33-88 years). Thirteen first-year post-graduate (PGY-1) trainees and nine pulmonary specialists (three radiologists, three thoracic surgeons, and three pulmonologists) interpreted the chest radiographs. Detection rates among trainees and specialists were compared using Student t test. RESULTS The total numbers of detected tumors ranged from 103 (36.4%) to 136 (48.1%) with a mean of 127.9±9.1 (45.2±3.2%) in the trainee group, and 137 (48.4%) to 182 (64.3%) with a mean of 161.6±13.1 (57.1±4.6%) in the specialist group; the intergroup difference was statistically significant (P<0.001). Significant intergroup detectability differences of >10% were noted for tumors in the peripheral zone with (i) ground glass opacity (GGO) ratio ≥10% and <70% and any size, or (ii) GGO ratio <10% and size ≤2cm; and for tumors hidden by the mediastinum, heart, or diaphragm with (i) GGO ratio ≥10% and <30% and size >3cm, or (ii) GGO ratio <10% and size >2cm. CONCLUSION Our study demonstrates significant differences in lung tumor detectability on chest radiographs between PGY-1 trainees and pulmonary specialists according to tumor size, extent of GGO, and tumor location.
Collapse
|
16
|
Affiliation(s)
- G Chassagnon
- Radiology Department, groupe hospitalier Cochin Broca Hôtel-Dieu-Université Paris Descartes, 27, rue du Faubourg Saint-Jacques, 75014 Paris, France
| | - M-P Revel
- Radiology Department, groupe hospitalier Cochin Broca Hôtel-Dieu-Université Paris Descartes, 27, rue du Faubourg Saint-Jacques, 75014 Paris, France.
| |
Collapse
|