1
|
Vanderbecq Q, Gelard M, Pesquet JC, Wagner M, Arrive L, Zins M, Chouzenoux E. Deep learning for automatic bowel-obstruction identification on abdominal CT. Eur Radiol 2024; 34:5842-5853. [PMID: 38388719 DOI: 10.1007/s00330-024-10657-z] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/21/2023] [Revised: 01/03/2024] [Accepted: 01/23/2024] [Indexed: 02/24/2024]
Abstract
RATIONALE AND OBJECTIVES Automated evaluation of abdominal computed tomography (CT) scans should help radiologists manage their massive workloads, thereby leading to earlier diagnoses and better patient outcomes. Our objective was to develop a machine-learning model capable of reliably identifying suspected bowel obstruction (BO) on abdominal CT. MATERIALS AND METHODS The internal dataset comprised 1345 abdominal CTs obtained in 2015-2022 from 1273 patients with suspected BO; among them, 670 were annotated as BO yes/no by an experienced abdominal radiologist. The external dataset consisted of 88 radiologist-annotated CTs. We developed a full preprocessing pipeline for abdominal CT comprising a model to locate the abdominal-pelvic region and another model to crop the 3D scan around the body. We built, trained, and tested several neural-network architectures for the binary classification (BO, yes/no) of each CT. F1 and balanced accuracy scores were computed to assess model performance. RESULTS The mixed convolutional network pretrained on a Kinetics 400 dataset achieved the best results: with the internal dataset, the F1 score was 0.92, balanced accuracy 0.86, and sensitivity 0.93; with the external dataset, the corresponding values were 0.89, 0.89, and 0.89. When calibrated on sensitivity, this model produced 1.00 sensitivity, 0.84 specificity, and an F1 score of 0.88 with the internal dataset; corresponding values were 0.98, 0.76, and 0.87 with the external dataset. CONCLUSION The 3D mixed convolutional neural network developed here shows great potential for the automated binary classification (BO yes/no) of abdominal CT scans from patients with suspected BO. CLINICAL RELEVANCE STATEMENT The 3D mixed CNN automates bowel obstruction classification, potentially automating patient selection and CT prioritization, leading to an enhanced radiologist workflow. KEY POINTS • Bowel obstruction's rising incidence strains radiologists. AI can aid urgent CT readings. • Employed 1345 CT scans, neural networks for bowel obstruction detection, achieving high accuracy and sensitivity on external testing. • 3D mixed CNN automates CT reading prioritization effectively and speeds up bowel obstruction diagnosis.
Collapse
Affiliation(s)
- Quentin Vanderbecq
- Department of Radiology, AP-HP.Sorbonne, Saint Antoine Hospital, 184 Rue du Faubourg Saint-Antoine, 75012, Paris, France.
- UMR 7371, Université Sorbonne, CNRS, Inserm U114615, rue de l'École de Médecine, 75006, Paris, France.
| | - Maxence Gelard
- Université Paris-Saclay, CentraleSupélec, Gif-sur-Yvette, Inria, CVN, France
| | | | - Mathilde Wagner
- UMR 7371, Université Sorbonne, CNRS, Inserm U114615, rue de l'École de Médecine, 75006, Paris, France
- Department of Radiology, Hospital Pitié Salpêtrière, 47-83 Bd de l'Hôpital, 75013 Paris, Île-de-France, France
| | - Lionel Arrive
- Department of Radiology, AP-HP.Sorbonne, Saint Antoine Hospital, 184 Rue du Faubourg Saint-Antoine, 75012, Paris, France
| | - Marc Zins
- Department of Radiology, Hospital Paris Saint-Joseph, 185 Rue Raymond Losserand, 75014, Paris, Île-de-France, France
| | - Emilie Chouzenoux
- Université Paris-Saclay, CentraleSupélec, Gif-sur-Yvette, Inria, CVN, France
| |
Collapse
|
2
|
Sirén A, Turkia E, Nyman M, Hirvonen J. Accuracy of Intra-Axial Brain Tumor Characterization in the Emergency MRI Reports: A Retrospective Human Performance Benchmarking Pilot Study. Diagnostics (Basel) 2024; 14:1791. [PMID: 39202279 PMCID: PMC11353410 DOI: 10.3390/diagnostics14161791] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/30/2024] [Revised: 08/12/2024] [Accepted: 08/14/2024] [Indexed: 09/03/2024] Open
Abstract
Demand for emergency neuroimaging is increasing. Even magnetic resonance imaging (MRI) is often performed outside office hours, sometimes revealing more uncommon entities like brain tumors. The scientific literature studying artificial intelligence (AI) methods for classifying brain tumors on imaging is growing, but knowledge about the radiologist's performance on this task is surprisingly scarce. Our study aimed to tentatively fill this knowledge gap. We hypothesized that the radiologist could classify intra-axial brain tumors at the emergency department with clinically acceptable accuracy. We retrospectively examined emergency brain MRI reports from 2013 to 2021, the inclusion criteria being (1) emergency brain MRI, (2) no previously known intra-axial brain tumor, and (3) suspicion of an intra-axial brain tumor on emergency MRI report. The tumor type suggestion and the final clinical diagnosis were pooled into groups: (1) glial tumors, (2) metastasis, (3) lymphoma, and (4) other tumors. The final study sample included 150 patients, of which 108 had histopathological tumor type confirmation. Among the patients with histopathological tumor type confirmation, the accuracy of the MRI reports in classifying the tumor type was 0.86 for gliomas against other tumor types, 0.89 for metastases, and 0.99 for lymphomas. We found the result encouraging, given the prolific need for emergency imaging.
Collapse
Affiliation(s)
- Aapo Sirén
- Department of Radiology, Turku University Hospital, and University of Turku, Kiinamyllynkatu 4-8, 20520 Turku, Finland
| | - Elina Turkia
- Department of Radiology, Turku University Hospital, and University of Turku, Kiinamyllynkatu 4-8, 20520 Turku, Finland
| | - Mikko Nyman
- Department of Radiology, Turku University Hospital, and University of Turku, Kiinamyllynkatu 4-8, 20520 Turku, Finland
| | - Jussi Hirvonen
- Department of Radiology, Turku University Hospital, and University of Turku, Kiinamyllynkatu 4-8, 20520 Turku, Finland
- Medical Imaging Center, Department of Radiology, Tampere University Hospital, and Tampere University, 33520 Tampere, Finland
| |
Collapse
|
3
|
Zafar AM, Mathur D, Barnacle BD, Magera RW. Design of overnight radiology shifts - primum non nocere. Emerg Radiol 2024; 31:429-434. [PMID: 38581613 DOI: 10.1007/s10140-024-02224-2] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/19/2023] [Accepted: 03/18/2024] [Indexed: 04/08/2024]
Abstract
Overnight radiology (ONR) is necessary for providing timely patient care but poses unique professional and personal challenges to the radiologists. Maintaining a sustainable, long-term overnight radiology program hinges on the retention of radiologists who grasp the institutional workflow and can adeptly navigate inherent disruptions while consistently delivering high-quality patient care. Design of radiology shifts can significantly impact the performance and well-being of radiologists, with downstream implications for patient care and risk management. We provide a narrative review of literature to make recommendations for optimally designing ONR shifts, with a focus on professional and personal challenges pertinent to overnight radiologists and system-based risk mitigation strategies.
Collapse
Affiliation(s)
- Abdul M Zafar
- Geisel School of Medicine, Dartmouth College, 1 Rope Ferry Rd, 03755, Hanover, NH, USA.
- Department of Radiology, Dartmouth Health, 1 Medical Center Dr, Lebanon, NH, 03766, USA.
| | - Diya Mathur
- Geisel School of Medicine, Dartmouth College, 1 Rope Ferry Rd, 03755, Hanover, NH, USA
- Tuck School of Business, Dartmouth College, 100, Tuck Hall, 03755, Hanover, NH, USA
| | - Brian D Barnacle
- Department of Radiology, Dartmouth Health, 1 Medical Center Dr, Lebanon, NH, 03766, USA
| | - Ruth W Magera
- Geisel School of Medicine, Dartmouth College, 1 Rope Ferry Rd, 03755, Hanover, NH, USA
- Department of Radiology, Dartmouth Health, 1 Medical Center Dr, Lebanon, NH, 03766, USA
| |
Collapse
|
4
|
Schaffer AC, Zawi T, Einbinder JS, Sato L, Sodickson AD. Assessment of Claimant, Clinical, and Financial Characteristics of Teleradiology Medical Malpractice Cases. Radiology 2024; 311:e232806. [PMID: 38563670 DOI: 10.1148/radiol.232806] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 04/04/2024]
Abstract
Background The increasing use of teleradiology has been accompanied by concerns relating to risk management and patient safety. Purpose To compare characteristics of teleradiology and nonteleradiology radiology malpractice cases and identify contributing factors underlying these cases. Materials and Methods In this retrospective analysis, a national database of medical malpractice cases was queried to identify cases involving telemedicine that closed between January 2010 and March 2022. Teleradiology malpractice cases were identified based on manual review of cases in which telemedicine was coded as one of the contributing factors. These cases were compared with nonteleradiology cases that closed during the same time period in which radiology had been determined to be the primary responsible clinical service. Claimant, clinical, and financial characteristics of the cases were recorded, and continuous or categorical data were compared using the Wilcoxon rank-sum test or Fisher exact test, respectively. Results This study included 135 teleradiology and 3474 radiology malpractices cases. The death of a patient occurred more frequently in teleradiology cases (48 of 135 [35.6%]) than in radiology cases (685 of 3474 [19.7%]; P < .001). Cerebrovascular disease was a more common final diagnosis in the teleradiology cases (13 of 135 [9.6%]) compared with the radiology cases (124 of 3474 [3.6%]; P = .002). Problems with communication among providers was a more frequent contributing factor in the teleradiology cases (35 of 135 [25.9%]) than in the radiology cases (439 of 3474 [12.6%]; P < .001). Teleradiology cases were more likely to close with indemnity payment (79 of 135 [58.5%]) than the radiology cases (1416 of 3474 [40.8%]; P < .001) and had a higher median indemnity payment than the radiology cases ($339 230 [IQR, $120 790-$731 615] vs $214 063 [IQR, $66 620-$585 424]; P = .01). Conclusion Compared with radiology cases, teleradiology cases had higher clinical and financial severity and were more likely to involve issues with communication. © RSNA, 2024 See also the editorial by Mezrich in this issue.
Collapse
Affiliation(s)
- Adam C Schaffer
- From the CRICO/Risk Management Foundation of the Harvard Medical Institutions, Boston, Mass (A.C.S., T.Z., J.S.E., L.S.); and Department of Medicine (A.C.S., J.S.E., L.S.) and Department of Radiology, Division of Emergency Radiology (A.D.S.), Brigham and Women's Hospital, Harvard Medical School, 75 Francis St, PBB-B-422, Boston, MA 02215
| | - Tarek Zawi
- From the CRICO/Risk Management Foundation of the Harvard Medical Institutions, Boston, Mass (A.C.S., T.Z., J.S.E., L.S.); and Department of Medicine (A.C.S., J.S.E., L.S.) and Department of Radiology, Division of Emergency Radiology (A.D.S.), Brigham and Women's Hospital, Harvard Medical School, 75 Francis St, PBB-B-422, Boston, MA 02215
| | - Jonathan S Einbinder
- From the CRICO/Risk Management Foundation of the Harvard Medical Institutions, Boston, Mass (A.C.S., T.Z., J.S.E., L.S.); and Department of Medicine (A.C.S., J.S.E., L.S.) and Department of Radiology, Division of Emergency Radiology (A.D.S.), Brigham and Women's Hospital, Harvard Medical School, 75 Francis St, PBB-B-422, Boston, MA 02215
| | - Luke Sato
- From the CRICO/Risk Management Foundation of the Harvard Medical Institutions, Boston, Mass (A.C.S., T.Z., J.S.E., L.S.); and Department of Medicine (A.C.S., J.S.E., L.S.) and Department of Radiology, Division of Emergency Radiology (A.D.S.), Brigham and Women's Hospital, Harvard Medical School, 75 Francis St, PBB-B-422, Boston, MA 02215
| | - Aaron D Sodickson
- From the CRICO/Risk Management Foundation of the Harvard Medical Institutions, Boston, Mass (A.C.S., T.Z., J.S.E., L.S.); and Department of Medicine (A.C.S., J.S.E., L.S.) and Department of Radiology, Division of Emergency Radiology (A.D.S.), Brigham and Women's Hospital, Harvard Medical School, 75 Francis St, PBB-B-422, Boston, MA 02215
| |
Collapse
|
5
|
Czerminski J, Pahade JK, Davis MA, Mezrich JL. The disproportionate impact of peer learning on emergency radiology. Emerg Radiol 2024; 31:133-139. [PMID: 38261134 DOI: 10.1007/s10140-024-02207-3] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/30/2023] [Accepted: 01/17/2024] [Indexed: 01/24/2024]
Abstract
PURPOSE The use of peer learning methods in radiology continues to grow as a means to constructively learn from past mistakes. This study examined whether emergency radiologists receive a disproportionate amount of peer learning feedback entered as potential learning opportunities (PLO), which could play a significant role in stress and career satisfaction. Our institution offers 24/7 attending coverage, with emergency radiologists interpreting a wide range of X-ray, ultrasound and CT exams on both adults and pediatric patients. MATERIALS AND METHODS Peer learning submissions entered as PLO at a single large academic medical center over a span of 3 years were assessed by subspecialty distribution and correlated with the number of attending radiologists in each section. Total number of studies performed on emergency department patients and throughout the hospital system were obtained for comparison purposes. Data was assessed using analysis of variance and post hoc analysis. RESULTS Emergency radiologists received significantly more (2.5 times) PLO submissions than the next closest subspeciality division and received more yearly PLO submissions per attending compared to other subspeciality divisions. This was found to still be true when normalizing for increased case volumes; Emergency radiologists received more PLO submissions per 1000 studies compared to other divisions in our department (1.59 vs. 0.85, p = 0.04). CONCLUSION Emergency radiologists were found to receive significantly more PLO submissions than their non-emergency colleagues. Presumed causes for this discrepancy may include a higher error rate secondary to wider range of studies interpreted, demand for shorter turn-around times, higher volumes of exams read per shift, and hindsight bias in the setting of follow-up review.
Collapse
Affiliation(s)
- Jan Czerminski
- Department of Radiology and Biomedical Imaging, Yale School of Medicine, 333 Cedar Street, TE2, New Haven, CT, 06520, USA
| | - Jay K Pahade
- Department of Radiology and Biomedical Imaging, Yale School of Medicine, 333 Cedar Street, TE2, New Haven, CT, 06520, USA
| | - Melissa A Davis
- Department of Radiology and Biomedical Imaging, Yale School of Medicine, 333 Cedar Street, TE2, New Haven, CT, 06520, USA
| | - Jonathan L Mezrich
- Department of Radiology and Biomedical Imaging, Yale School of Medicine, 333 Cedar Street, TE2, New Haven, CT, 06520, USA.
| |
Collapse
|
6
|
Mezrich JL. Malpractice From Afar: Teleradiology May Increase Malpractice Risk. Radiology 2024; 311:e240651. [PMID: 38563668 DOI: 10.1148/radiol.240651] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 04/04/2024]
Affiliation(s)
- Jonathan L Mezrich
- From the Department of Radiology and Biomedical Imaging, Yale School of Medicine, 333 Cedar St, TE2, New Haven, CT 06520
| |
Collapse
|
7
|
Zhang L, Wen X, Li JW, Jiang X, Yang XF, Li M. Diagnostic error and bias in the department of radiology: a pictorial essay. Insights Imaging 2023; 14:163. [PMID: 37782396 PMCID: PMC10545608 DOI: 10.1186/s13244-023-01521-7] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/27/2023] [Accepted: 09/03/2023] [Indexed: 10/03/2023] Open
Abstract
Diagnostic imaging is an essential and indispensable part of medical diagnosis and treatment, and diagnostic errors or biases are also common in the department of radiology, sometimes even having a severe impact on the diagnosis and treatment of patients. There are various reasons for diagnostic errors or biases in imaging. In this review, we analyze and summarize the causes of diagnostic imaging errors and biases based on practical cases. We propose solutions for dealing with diagnostic imaging errors and reducing their probability, thereby helping radiologists in their clinical practice.Critical relevance statement Diagnostic errors or bias contribute to most medical errors in the radiology department. Solutions for dealing with diagnostic imaging errors are pivotal for patients.Key points• Diagnostic errors or bias contribute to most medical errors in radiology department.• Solutions for dealing with diagnostic imaging errors are pivotal for patients.• This review summarizes the causes of diagnostic errors and offers solutions to them.
Collapse
Affiliation(s)
- Li Zhang
- Department of Diagnostic Radiology, National Cancer Center/National Clinical Research Center for Cancer/Cancer Hospital, Chinese Academy of Medical Sciences and Peking Union Medical College, Beijing, 100021, China
| | - Xin Wen
- Department of Diagnostic Radiology, National Cancer Center/National Clinical Research Center for Cancer/Cancer Hospital, Chinese Academy of Medical Sciences and Peking Union Medical College, Beijing, 100021, China
| | - Jian-Wei Li
- Department of Diagnostic Radiology, National Cancer Center/National Clinical Research Center for Cancer/Cancer Hospital, Chinese Academy of Medical Sciences and Peking Union Medical College, Beijing, 100021, China
| | - Xu Jiang
- Department of Diagnostic Radiology, National Cancer Center/National Clinical Research Center for Cancer/Cancer Hospital, Chinese Academy of Medical Sciences and Peking Union Medical College, Beijing, 100021, China
| | - Xian-Feng Yang
- Department of Radiology, Drum Tower Hospital Affiliated to Medical School of Nanjing University, Nanjing, China
| | - Meng Li
- Department of Diagnostic Radiology, National Cancer Center/National Clinical Research Center for Cancer/Cancer Hospital, Chinese Academy of Medical Sciences and Peking Union Medical College, Beijing, 100021, China.
| |
Collapse
|
8
|
Zhou S, Hanna T, Ma T, Johnson TD, Lamoureux C, Weber S, Johnson JO, Steenburg SD, Dunkle JW, Chong ST. Daytime, evening, and overnight: the 24-h radiology cycle and impact on interpretative accuracy. Emerg Radiol 2023; 30:607-612. [PMID: 37518838 DOI: 10.1007/s10140-023-02161-6] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/24/2022] [Accepted: 07/19/2023] [Indexed: 08/01/2023]
Abstract
PURPOSE To assess the influence of time of day when a study is interpreted on discrepancy rates for common and advanced studies performed in the acute community setting. METHODS This retrospective study used the databank of a U.S. teleradiology company to retrieve studies between 2012 and 2016 with a preliminary report followed by a final report by the on-site client hospital. Neuroradiology, abdominal radiology, and musculoskeletal radiology studies were included. Teleradiologists were fellowship trained in one of these subspecialty areas. Daytime, evening, and overnight times were defined. Associations between major and minor discrepancies, time of day, and whether the study was common or advanced were tested with significance set at p = .05. RESULTS A total of 5,883,980 studies were analyzed. There were 8444 major discrepancies (0.14%) and 17,208 minor discrepancies (0.29%). For common studies, daytime (0.13%) and evening (0.13%) had lower major discrepancy rates compared to overnight (0.14%) (daytime to overnight, RR = 0.57, 95%CI: 0.45, 0.72, p < 0.01 and evening to overnight, RR = 0.57, 95%CI: 0.49,0.67, p < 0.01). Minor discrepancy rates for common studies were decreased for evening (0.29%) compared to overnight (0.30%) (RR = 0.89, 95%CI: 0.80,0.99, p = 0.029). For advanced studies, daytime (.15%) had lower major discrepancy rates compared to evening (0.20%) and overnight (.23%) (daytime to evening, RR = 0.77, 95%CI: 0.61, 0.97, p = 0.028 and daytime to overnight, RR = 0.66, 95%CI: 0.50, 0.87, p ≤ 0.01). CONCLUSION Significantly higher major discrepancy rates for studies interpreted overnight suggest the need for radiologists to exercise greater caution when interpreting studies overnight and may require practice management strategies to help optimize overnight work conditions. The lower major discrepancy rates on advanced studies interpreted during the daytime suggest the need for reserving advanced studies for interpretation during the day when possible.
Collapse
Affiliation(s)
- Shannon Zhou
- Department of Radiology and Imaging Sciences, Indiana University, Indianapolis, IN, USA
| | - Tarek Hanna
- Emergency and Trauma Imaging, Department of Radiology and Imaging Sciences, Emory University, Atlanta, GA, USA
| | - Tianwen Ma
- Department of Biostatistics and Bioinformatics, Emory University, Atlanta, GA, USA
| | - Timothy D Johnson
- Department of Biostatistics, University of Michigan, Ann Arbor, MI, USA
| | | | | | - Jamlik-Omari Johnson
- Keck School of Medicine, Department of Radiology, University of Southern California, Los Angeles, CA, USA
| | - Scott D Steenburg
- Department of Radiology and Imaging Sciences, Indiana University, Indianapolis, IN, USA
| | - Jeffrey W Dunkle
- Department of Radiology and Imaging Sciences, Indiana University, Indianapolis, IN, USA
| | - Suzanne T Chong
- Department of Radiology and Imaging Sciences, Indiana University, Indianapolis, IN, USA.
| |
Collapse
|
9
|
Hsieh SS, Cook DA, Inoue A, Gong H, Sudhir Pillai P, Johnson MP, Leng S, Yu L, Fidler JL, Holmes DR, Carter RE, McCollough CH, Fletcher JG. Understanding Reader Variability: A 25-Radiologist Study on Liver Metastasis Detection at CT. Radiology 2023; 306:e220266. [PMID: 36194112 PMCID: PMC9870852 DOI: 10.1148/radiol.220266] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/03/2022] [Revised: 07/07/2022] [Accepted: 08/17/2022] [Indexed: 01/26/2023]
Abstract
Background Substantial interreader variability exists for common tasks in CT imaging, such as detection of hepatic metastases. This variability can undermine patient care by leading to misdiagnosis. Purpose To determine the impact of interreader variability associated with (a) reader experience, (b) image navigation patterns (eg, eye movements, workstation interactions), and (c) eye gaze time at missed liver metastases on contrast-enhanced abdominal CT images. Materials and Methods In a single-center prospective observational trial at an academic institution between December 2020 and February 2021, readers were recruited to examine 40 contrast-enhanced abdominal CT studies (eight normal, 32 containing 91 liver metastases). Readers circumscribed hepatic metastases and reported confidence. The workstation tracked image navigation and eye movements. Performance was quantified by using the area under the jackknife alternative free-response receiver operator characteristic (JAFROC-1) curve and per-metastasis sensitivity and was associated with reader experience and image navigation variables. Differences in area under JAFROC curve were assessed with the Kruskal-Wallis test followed by the Dunn test, and effects of image navigation were assessed by using the Wilcoxon signed-rank test. Results Twenty-five readers (median age, 38 years; IQR, 31-45 years; 19 men) were recruited and included nine subspecialized abdominal radiologists, five nonabdominal staff radiologists, and 11 senior residents or fellows. Reader experience explained differences in area under the JAFROC curve, with abdominal radiologists demonstrating greater area under the JAFROC curve (mean, 0.77; 95% CI: 0.75, 0.79) than trainees (mean, 0.71; 95% CI: 0.69, 0.73) (P = .02) or nonabdominal subspecialists (mean, 0.69; 95% CI: 0.60, 0.78) (P = .03). Sensitivity was similar within the reader experience groups (P = .96). Image navigation variables that were associated with higher sensitivity included longer interpretation time (P = .003) and greater use of coronal images (P < .001). The eye gaze time was at least 0.5 and 2.0 seconds for 71% (266 of 377) and 40% (149 of 377) of missed metastases, respectively. Conclusion Abdominal radiologists demonstrated better discrimination for the detection of liver metastases on abdominal contrast-enhanced CT images. Missed metastases frequently received at least a brief eye gaze. Higher sensitivity was associated with longer interpretation time and greater use of liver display windows and coronal images. © RSNA, 2022 Online supplemental material is available for this article.
Collapse
Affiliation(s)
- Scott S. Hsieh
- From the Departments of Radiology (S.S.H., A.I., H.G., P.S.P., S.L.,
L.Y., J.L.F., C.H.M., J.G.F.), General Internal Medicine (D.A.C.), Quantitative
Health Services–Clinical Trials and Biostatistics (M.P.J.), and
Physiology and Biomedical Engineering (D.R.H.), Mayo Clinic Rochester, 200 First
St SW, Rochester, MN 55905; and Department of Quantitative Health
Services–Clinical Trials and Biostatistics, Mayo Clinic, Jacksonville,
Fla (R.E.C.)
| | - David A. Cook
- From the Departments of Radiology (S.S.H., A.I., H.G., P.S.P., S.L.,
L.Y., J.L.F., C.H.M., J.G.F.), General Internal Medicine (D.A.C.), Quantitative
Health Services–Clinical Trials and Biostatistics (M.P.J.), and
Physiology and Biomedical Engineering (D.R.H.), Mayo Clinic Rochester, 200 First
St SW, Rochester, MN 55905; and Department of Quantitative Health
Services–Clinical Trials and Biostatistics, Mayo Clinic, Jacksonville,
Fla (R.E.C.)
| | - Akitoshi Inoue
- From the Departments of Radiology (S.S.H., A.I., H.G., P.S.P., S.L.,
L.Y., J.L.F., C.H.M., J.G.F.), General Internal Medicine (D.A.C.), Quantitative
Health Services–Clinical Trials and Biostatistics (M.P.J.), and
Physiology and Biomedical Engineering (D.R.H.), Mayo Clinic Rochester, 200 First
St SW, Rochester, MN 55905; and Department of Quantitative Health
Services–Clinical Trials and Biostatistics, Mayo Clinic, Jacksonville,
Fla (R.E.C.)
| | - Hao Gong
- From the Departments of Radiology (S.S.H., A.I., H.G., P.S.P., S.L.,
L.Y., J.L.F., C.H.M., J.G.F.), General Internal Medicine (D.A.C.), Quantitative
Health Services–Clinical Trials and Biostatistics (M.P.J.), and
Physiology and Biomedical Engineering (D.R.H.), Mayo Clinic Rochester, 200 First
St SW, Rochester, MN 55905; and Department of Quantitative Health
Services–Clinical Trials and Biostatistics, Mayo Clinic, Jacksonville,
Fla (R.E.C.)
| | - Parvathy Sudhir Pillai
- From the Departments of Radiology (S.S.H., A.I., H.G., P.S.P., S.L.,
L.Y., J.L.F., C.H.M., J.G.F.), General Internal Medicine (D.A.C.), Quantitative
Health Services–Clinical Trials and Biostatistics (M.P.J.), and
Physiology and Biomedical Engineering (D.R.H.), Mayo Clinic Rochester, 200 First
St SW, Rochester, MN 55905; and Department of Quantitative Health
Services–Clinical Trials and Biostatistics, Mayo Clinic, Jacksonville,
Fla (R.E.C.)
| | - Matthew P. Johnson
- From the Departments of Radiology (S.S.H., A.I., H.G., P.S.P., S.L.,
L.Y., J.L.F., C.H.M., J.G.F.), General Internal Medicine (D.A.C.), Quantitative
Health Services–Clinical Trials and Biostatistics (M.P.J.), and
Physiology and Biomedical Engineering (D.R.H.), Mayo Clinic Rochester, 200 First
St SW, Rochester, MN 55905; and Department of Quantitative Health
Services–Clinical Trials and Biostatistics, Mayo Clinic, Jacksonville,
Fla (R.E.C.)
| | - Shuai Leng
- From the Departments of Radiology (S.S.H., A.I., H.G., P.S.P., S.L.,
L.Y., J.L.F., C.H.M., J.G.F.), General Internal Medicine (D.A.C.), Quantitative
Health Services–Clinical Trials and Biostatistics (M.P.J.), and
Physiology and Biomedical Engineering (D.R.H.), Mayo Clinic Rochester, 200 First
St SW, Rochester, MN 55905; and Department of Quantitative Health
Services–Clinical Trials and Biostatistics, Mayo Clinic, Jacksonville,
Fla (R.E.C.)
| | - Lifeng Yu
- From the Departments of Radiology (S.S.H., A.I., H.G., P.S.P., S.L.,
L.Y., J.L.F., C.H.M., J.G.F.), General Internal Medicine (D.A.C.), Quantitative
Health Services–Clinical Trials and Biostatistics (M.P.J.), and
Physiology and Biomedical Engineering (D.R.H.), Mayo Clinic Rochester, 200 First
St SW, Rochester, MN 55905; and Department of Quantitative Health
Services–Clinical Trials and Biostatistics, Mayo Clinic, Jacksonville,
Fla (R.E.C.)
| | - Jeff L. Fidler
- From the Departments of Radiology (S.S.H., A.I., H.G., P.S.P., S.L.,
L.Y., J.L.F., C.H.M., J.G.F.), General Internal Medicine (D.A.C.), Quantitative
Health Services–Clinical Trials and Biostatistics (M.P.J.), and
Physiology and Biomedical Engineering (D.R.H.), Mayo Clinic Rochester, 200 First
St SW, Rochester, MN 55905; and Department of Quantitative Health
Services–Clinical Trials and Biostatistics, Mayo Clinic, Jacksonville,
Fla (R.E.C.)
| | - David R. Holmes
- From the Departments of Radiology (S.S.H., A.I., H.G., P.S.P., S.L.,
L.Y., J.L.F., C.H.M., J.G.F.), General Internal Medicine (D.A.C.), Quantitative
Health Services–Clinical Trials and Biostatistics (M.P.J.), and
Physiology and Biomedical Engineering (D.R.H.), Mayo Clinic Rochester, 200 First
St SW, Rochester, MN 55905; and Department of Quantitative Health
Services–Clinical Trials and Biostatistics, Mayo Clinic, Jacksonville,
Fla (R.E.C.)
| | - Rickey E. Carter
- From the Departments of Radiology (S.S.H., A.I., H.G., P.S.P., S.L.,
L.Y., J.L.F., C.H.M., J.G.F.), General Internal Medicine (D.A.C.), Quantitative
Health Services–Clinical Trials and Biostatistics (M.P.J.), and
Physiology and Biomedical Engineering (D.R.H.), Mayo Clinic Rochester, 200 First
St SW, Rochester, MN 55905; and Department of Quantitative Health
Services–Clinical Trials and Biostatistics, Mayo Clinic, Jacksonville,
Fla (R.E.C.)
| | - Cynthia H. McCollough
- From the Departments of Radiology (S.S.H., A.I., H.G., P.S.P., S.L.,
L.Y., J.L.F., C.H.M., J.G.F.), General Internal Medicine (D.A.C.), Quantitative
Health Services–Clinical Trials and Biostatistics (M.P.J.), and
Physiology and Biomedical Engineering (D.R.H.), Mayo Clinic Rochester, 200 First
St SW, Rochester, MN 55905; and Department of Quantitative Health
Services–Clinical Trials and Biostatistics, Mayo Clinic, Jacksonville,
Fla (R.E.C.)
| | - Joel G. Fletcher
- From the Departments of Radiology (S.S.H., A.I., H.G., P.S.P., S.L.,
L.Y., J.L.F., C.H.M., J.G.F.), General Internal Medicine (D.A.C.), Quantitative
Health Services–Clinical Trials and Biostatistics (M.P.J.), and
Physiology and Biomedical Engineering (D.R.H.), Mayo Clinic Rochester, 200 First
St SW, Rochester, MN 55905; and Department of Quantitative Health
Services–Clinical Trials and Biostatistics, Mayo Clinic, Jacksonville,
Fla (R.E.C.)
| |
Collapse
|
10
|
Xiong S, Hu H, Liu S, Huang Y, Cheng J, Wan B. Improving diagnostic performance of rib fractures for the night shift in radiology department using a computer-aided diagnosis system based on deep learning: A clinical retrospective study. JOURNAL OF X-RAY SCIENCE AND TECHNOLOGY 2023; 31:265-276. [PMID: 36806541 DOI: 10.3233/xst-221343] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/18/2023]
Abstract
OBJECTIVE To investigate the application value of a computer-aided diagnosis (CAD) system based on deep learning (DL) of rib fractures for night shifts in radiology department. METHODS Chest computed tomography (CT) images and structured reports were retrospectively selected from the picture archiving and communication system (PACS) for 2,332 blunt chest trauma patients. In all CT imaging examinations, two on-duty radiologists (radiologists I and II) completed reports using three different reading patterns namely, P1 = independent reading during the day shift; P2 = independent reading during the night shift; and P3 = reading with the aid of a CAD system as the concurrent reader during the night shift. The locations and types of rib fractures were documented for each reading. In this study, the reference standard for rib fractures was established by an expert group. Sensitivity and false positives per scan (FPS) were counted and compared among P1, P2, and P3. RESULTS The reference standard verified 6,443 rib fractures in the 2,332 patients. The sensitivity of both radiologists decreased significantly in P2 compared to that in P1 (both p < 0.017). The sensitivities of both radiologists showed no statistical difference between P3 and P1 (both p > 0.017). Radiologist I's FPS increased significantly in P2 compared to P1 (p < 0.017). The FPS of radiologist I showed no statistically significant difference between P3 and P1 (p > 0.017). The FPS of Radiologist II showed no statistical difference among all three reading patterns (p > 0.05). CONCLUSIONS DL-based CAD systems can be integrated into the workflow of radiology departments during the night shift to improve the diagnostic performance of CT rib fractures.
Collapse
Affiliation(s)
- Shan Xiong
- Department of Radiology, Jingzhou Hospital Affiliated to Yangtze University, Jingzhou, China
| | - Hai Hu
- Department of Radiology, Jingzhou Hospital Affiliated to Yangtze University, Jingzhou, China
| | - Sibin Liu
- Department of Radiology, Jingzhou Hospital Affiliated to Yangtze University, Jingzhou, China
| | - Yuanyi Huang
- Department of Radiology, Jingzhou Hospital Affiliated to Yangtze University, Jingzhou, China
| | - Jianmin Cheng
- Department of Radiology, The Second Affiliated Hospital and Yuying Children's Hospital of Wenzhou Medical University, Wenzhou, China
| | - Bing Wan
- Department of Radiology, Renhe Hospital Affiliated to Three Gorges University, Yichang, China
| |
Collapse
|
11
|
Muacevic A, Adler JR, Young A, Gould E. Call to Action: Creating Resources for Radiology Technologists to Capture Higher Quality Portable Chest X-rays. Cureus 2022; 14:e29197. [PMID: 36507112 PMCID: PMC9731552 DOI: 10.7759/cureus.29197] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 09/15/2022] [Indexed: 12/15/2022] Open
Abstract
Background Patient rotation, foreign body overlying anatomy, and anatomy out of field of view can have detrimental impacts on the diagnostic quality of portable chest x-rays (PCXRs), especially as the number of PCXR imaging increases due to the coronavirus disease 2019 (COVID-19) pandemic. Although preventable, these "quality failures" are common and may lead to interpretative and diagnostic errors for the radiologist. Aims In this study, we present a baseline quality failure rate of PCXR imaging as observed at our institution. We also conduct a focus group highlighting the key issues that lead to the problematic images and discuss potential interventions targeting technologists that can be implemented to address imaging quality failure rate. Materials and methods A total of 500 PCXRs for adult patients admitted to a large university hospital between July 12, 2021, and July 25, 2021, were obtained for evaluation of quality. The PCXRs were evaluated by radiology residents for failures in technical image quality. The images were categorized into various metrics including the degree of rotation and obstruction of anatomical structures. After collecting the data, a focus group involving six managers of the technologist department at our university hospital was conducted to further illuminate the key barriers to quality PCXRs faced at our institution.. Results Out of the 500 PCXRs evaluated, 231 were problematic (46.2%). 43.5% of the problematic films with a repeat PCXR within one week showed that there was a technical problem impacting the ability to detect pathology. Most problematic films also occurred during the night shift (48%). Key issues that lead to poor image quality included improper patient positioning, foreign objects covering anatomy, and variances in technologists' training. Three interventions were proposed to optimize technologist performance that can lower quality failure rates of PCXRs. These include a longitudinal educational curriculum involving didactic sessions, adding nursing support to assist technologists, and adding an extra layer of verification by internal medicine residents before sending the films to the radiologist. The rationale for these interventions is discussed in detail so that a modified version can be implemented in other hospital systems. Conclusion This study illustrates the high baseline error rate in image quality of PCXRs at our institution and demonstrates the need to improve on image quality. Poor image quality negatively impacts the interpretive accuracy of radiologists and therefore leads to wrong diagnoses. Increasing educational resources and support for technologists can lead to higher image quality and radiologist accuracy.
Collapse
|
12
|
Zheng Y, Wang H, Li Q, Sun H, Guo L. Discriminating Between Benign and Malignant Solid Ovarian Tumors Based on Clinical and Radiomic Features of MRI. Acad Radiol 2022; 30:814-822. [PMID: 35810066 DOI: 10.1016/j.acra.2022.06.007] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/04/2022] [Revised: 05/31/2022] [Accepted: 06/08/2022] [Indexed: 12/17/2022]
Abstract
RATIONALE AND OBJECTIVES To develop and validate a combined model integrating clinical and radiomic features to non-invasive discriminate between the benign and malignant solid ovarian tumors. MATERIALS AND METHODS A total of 148 patients with 156 solid ovarian tumors (86 benign and 70 malignant tumors) were included in this study. The dataset was split into the training and the test set with a ratio of 8:2 using stratified random sampling. 12 clinical features and 1612 radiomic features were extracted from each tumor. These features were selected by least absolute shrinkage and selection operator (Lasso). Three classification models were built using extreme gradient boosting (XGB) algorithm: clinical model, radiomic model, combined model. The area under the receiver operating characteristic curve (AUC), accuracy, precision and sensitivity were analyzed to evaluate the performance of these models. RESULTS All of the three models obtained good performances in differentiating benign with malignant solid ovarian tumors in both training and test sets. The AUC, accuracy, precision, sensitivity of clinical model and radiomic model in test set were 0.847 (95% confidence interval (CI), 0.707-0.986, p <0.01), 0.774, 0.769, 0.714, and 0.807 (95%CI, 0.652-0.961, p <0.05), 0.677, 0.643, 0.643, respectively. Combined model had the best prediction results, the AUC, accuracy, precision and sensitivity were 0.954 (95%CI, 0.862-1.0, p <0.01), 0.839, 0.909 and 0.714 in test set. CONCLUSION Radiomics based on machine learning can be helpful for radiologists in differentiating the benign and malignant solid ovarian tumors.
Collapse
Affiliation(s)
- Yuemei Zheng
- School of Medical Imaging, Tianjin Medical University, No. 1 Guangdong Road, Tianjin 300203, China
| | - Hong Wang
- Department of Radiology, Tianjin First Central Hospital, Tianjin, China
| | - Qiong Li
- Department of Radiology, Tianjin Medical University General Hospital, Tianjin, China
| | - Haoran Sun
- Department of Radiology, Tianjin Medical University General Hospital, Tianjin, China
| | - Li Guo
- School of Medical Imaging, Tianjin Medical University, No. 1 Guangdong Road, Tianjin 300203, China.
| |
Collapse
|
13
|
Alexander R, Waite S, Bruno MA, Krupinski EA, Berlin L, Macknik S, Martinez-Conde S. Mandating Limits on Workload, Duty, and Speed in Radiology. Radiology 2022; 304:274-282. [PMID: 35699581 PMCID: PMC9340237 DOI: 10.1148/radiol.212631] [Citation(s) in RCA: 27] [Impact Index Per Article: 13.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/25/2023]
Abstract
Research has not yet quantified the effects of workload or duty hours on the accuracy of radiologists. With the exception of a brief reduction in imaging studies during the 2020 peak of the COVID-19 pandemic, the workload of radiologists in the United States has seen relentless growth in recent years. One concern is that this increased demand could lead to reduced accuracy. Behavioral studies in species ranging from insects to humans have shown that decision speed is inversely correlated to decision accuracy. A potential solution is to institute workload and duty limits to optimize radiologist performance and patient safety. The concern, however, is that any prescribed mandated limits would be arbitrary and thus no more advantageous than allowing radiologists to self-regulate. Specific studies have been proposed to determine whether limits reduce error, and if so, to provide a principled basis for such limits. This could determine the precise susceptibility of individual radiologists to medical error as a function of speed during image viewing, the maximum number of studies that could be read during a work shift, and the appropriate shift duration as a function of time of day. Before principled recommendations for restrictions are made, however, it is important to understand how radiologists function both optimally and at the margins of adequate performance. This study examines the relationship between interpretation speed and error rates in radiology, the potential influence of artificial intelligence on reading speed and error rates, and the possible outcomes of imposed limits on both caseload and duty hours. This review concludes that the scientific evidence needed to make meaningful rules is lacking and notes that regulating workloads without scientific principles can be more harmful than not regulating at all.
Collapse
Affiliation(s)
- Robert Alexander
- From the Departments of Ophthalmology (R.A., S.M., S.M.C.), Radiology (S.W.), Neurology (S.M., S.M.C.), and Physiology & Pharmacology (S.M., S.M.C.), SUNY Downstate Health Sciences University, 450 Clarkson Ave, Brooklyn, NY 11203; Department of Radiology, Penn State Milton S. Hershey Medical Center, Hershey, Pa (M.A.B.); Department of Radiology and Imaging Sciences, Emory University, Atlanta, Ga (E.A.K.); and Department of Radiology, Rush University Medical College and University of Illinois, Chicago, Ill (L.B.)
| | - Stephen Waite
- From the Departments of Ophthalmology (R.A., S.M., S.M.C.), Radiology (S.W.), Neurology (S.M., S.M.C.), and Physiology & Pharmacology (S.M., S.M.C.), SUNY Downstate Health Sciences University, 450 Clarkson Ave, Brooklyn, NY 11203; Department of Radiology, Penn State Milton S. Hershey Medical Center, Hershey, Pa (M.A.B.); Department of Radiology and Imaging Sciences, Emory University, Atlanta, Ga (E.A.K.); and Department of Radiology, Rush University Medical College and University of Illinois, Chicago, Ill (L.B.)
| | - Michael A Bruno
- From the Departments of Ophthalmology (R.A., S.M., S.M.C.), Radiology (S.W.), Neurology (S.M., S.M.C.), and Physiology & Pharmacology (S.M., S.M.C.), SUNY Downstate Health Sciences University, 450 Clarkson Ave, Brooklyn, NY 11203; Department of Radiology, Penn State Milton S. Hershey Medical Center, Hershey, Pa (M.A.B.); Department of Radiology and Imaging Sciences, Emory University, Atlanta, Ga (E.A.K.); and Department of Radiology, Rush University Medical College and University of Illinois, Chicago, Ill (L.B.)
| | - Elizabeth A Krupinski
- From the Departments of Ophthalmology (R.A., S.M., S.M.C.), Radiology (S.W.), Neurology (S.M., S.M.C.), and Physiology & Pharmacology (S.M., S.M.C.), SUNY Downstate Health Sciences University, 450 Clarkson Ave, Brooklyn, NY 11203; Department of Radiology, Penn State Milton S. Hershey Medical Center, Hershey, Pa (M.A.B.); Department of Radiology and Imaging Sciences, Emory University, Atlanta, Ga (E.A.K.); and Department of Radiology, Rush University Medical College and University of Illinois, Chicago, Ill (L.B.)
| | - Leonard Berlin
- From the Departments of Ophthalmology (R.A., S.M., S.M.C.), Radiology (S.W.), Neurology (S.M., S.M.C.), and Physiology & Pharmacology (S.M., S.M.C.), SUNY Downstate Health Sciences University, 450 Clarkson Ave, Brooklyn, NY 11203; Department of Radiology, Penn State Milton S. Hershey Medical Center, Hershey, Pa (M.A.B.); Department of Radiology and Imaging Sciences, Emory University, Atlanta, Ga (E.A.K.); and Department of Radiology, Rush University Medical College and University of Illinois, Chicago, Ill (L.B.)
| | - Stephen Macknik
- From the Departments of Ophthalmology (R.A., S.M., S.M.C.), Radiology (S.W.), Neurology (S.M., S.M.C.), and Physiology & Pharmacology (S.M., S.M.C.), SUNY Downstate Health Sciences University, 450 Clarkson Ave, Brooklyn, NY 11203; Department of Radiology, Penn State Milton S. Hershey Medical Center, Hershey, Pa (M.A.B.); Department of Radiology and Imaging Sciences, Emory University, Atlanta, Ga (E.A.K.); and Department of Radiology, Rush University Medical College and University of Illinois, Chicago, Ill (L.B.)
| | - Susana Martinez-Conde
- From the Departments of Ophthalmology (R.A., S.M., S.M.C.), Radiology (S.W.), Neurology (S.M., S.M.C.), and Physiology & Pharmacology (S.M., S.M.C.), SUNY Downstate Health Sciences University, 450 Clarkson Ave, Brooklyn, NY 11203; Department of Radiology, Penn State Milton S. Hershey Medical Center, Hershey, Pa (M.A.B.); Department of Radiology and Imaging Sciences, Emory University, Atlanta, Ga (E.A.K.); and Department of Radiology, Rush University Medical College and University of Illinois, Chicago, Ill (L.B.)
| |
Collapse
|
14
|
Li D, Basilico R, Blanco A, Calli C, Dick E, Kirkpatrick IDC, Nicolaou S, Patlas MN. Emergency Radiology: Evolution, Current Status, and Future Directions. Can Assoc Radiol J 2022; 73:697-703. [PMID: 35470687 DOI: 10.1177/08465371221088924] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/17/2022] Open
Abstract
Emergency Radiology is a clinical practice and an academic discipline that has rapidly gained increasing global recognition among radiology and emergency/critical care departments and trauma services around the world. As with other subspecialties, Emergency Radiology practice has a unique scope and purpose and presents with its own unique challenges. There are several advantages of having a dedicated Emergency Radiology section, perhaps most important of which is the broad clinical skillset that Emergency Radiologists are known for. This multi-society paper, representing the views of Emergency Radiology societies in Canada and Europe, outlines several value-oriented contributions of Emergency Radiologists and briefly discusses the current state of Emergency Radiology as a subspecialty.
Collapse
Affiliation(s)
- David Li
- Division of Emergency/Trauma Radiology, Department of Radiology, 153003McMaster University, Hamilton, ON, Canada
| | | | - Ana Blanco
- University Hospital Morales Meseguer, Murcia, Spain
| | - Cem Calli
- 323336Ege University Medical Faculty, Bornova Izmir, Turkey
| | - Elizabeth Dick
- St Mary's Hospital, 8946Imperial College NHS Trust, London, UK
| | - Iain D C Kirkpatrick
- Department of Diagnostic Imaging, 8664University of Manitoba, Winnipeg, MB, Canada
| | - Savvas Nicolaou
- Division of Emergency Radiology, Vancouver General Hospital, Vancouver, BC, Canada
| | - Michael N Patlas
- Division of Emergency/Trauma Radiology, Department of Radiology, 153003McMaster University, Hamilton, ON, Canada
| |
Collapse
|
15
|
Salca D, Lersy F, Willaume T, Stoessel M, Lefèvre A, Ardellier FD, Nicolaï C, Nouri A, Baloglu S, Bierry G, Chammas A, Kremer S. Evaluation of neuroradiology emergency MRI interpretations: low discrepancy rates between on-call radiology residents' preliminary interpretations and neuroradiologists' final reports. Eur Radiol 2022; 32:7260-7269. [PMID: 35435441 DOI: 10.1007/s00330-022-08789-1] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/19/2021] [Revised: 03/15/2022] [Accepted: 04/01/2022] [Indexed: 11/30/2022]
Abstract
OBJECTIVES To evaluate the performance of on-call radiology residents in interpreting alone brain and spine MRI studies performed after hours, to describe their mistakes, and to identify influencing factors that increased the occurrence of errors. METHODS A total of 328 MRI examinations performed during a 13-month period (from December 1, 2019, to January 1, 2021) were prospectively included. Discrepancies between the preliminary interpretation of on-call radiology residents and the final reports of attending neuroradiologists were noted and classified according to a three-level score: level 1 (perfect interpretation or minor correction), level 2 (important correction without immediate change in patient management), or level 3 (major correction with immediate change in patient management). Categorical data were compared using Fisher's exact test. RESULTS The overall discrepancy rate (level-2 and level-3 errors) was 16%; the rate of major discrepancies (only level-3 errors) was 5.5%. The major-discrepancy rate of second-year residents, when compared with that of senior residents, was significantly higher (p = 0.02). Almost all of the level-3 errors concerned cerebrovascular pathology. The most common level-2 errors involved undescribed aneurysms. We found no significant difference in the major-discrepancy rate regarding time since the beginning of the shift. CONCLUSIONS The great majority of examinations were correctly interpreted. The rate of major discrepancies in our study was comparable to the data in the literature, and there was no adverse clinical outcome. The level of residency has an effect on the rate of serious errors in residents' reports. KEY POINTS • The rate of major discrepancies between preliminary MRI interpretations by on-call radiology residents and final reports by attending neuroradiologists is low, and comparable to discrepancy rates reported for head CT interpretations. • The youngest residents made significantly more serious errors when compared to senior residents. • There was no adverse clinical outcome in patient morbidity as a result of an initial misdiagnosis.
Collapse
Affiliation(s)
- Diana Salca
- Hôpitaux Universitaires de Strasbourg, Service d'imagerie 2, Hôpital de Hautepierre, Strasbourg, France.
| | - François Lersy
- Hôpitaux Universitaires de Strasbourg, Service d'imagerie 2, Hôpital de Hautepierre, Strasbourg, France
| | - Thibault Willaume
- Hôpitaux Universitaires de Strasbourg, Service d'imagerie 2, Hôpital de Hautepierre, Strasbourg, France
| | - Marie Stoessel
- Hôpitaux Universitaires de Strasbourg, Service d'imagerie 2, Hôpital de Hautepierre, Strasbourg, France
| | - Agnieszka Lefèvre
- Hôpitaux Universitaires de Strasbourg, Service d'imagerie 2, Hôpital de Hautepierre, Strasbourg, France
| | - François-Daniel Ardellier
- Hôpitaux Universitaires de Strasbourg, Service d'imagerie 2, Hôpital de Hautepierre, Strasbourg, France
| | - Caroline Nicolaï
- Hôpitaux Universitaires de Strasbourg, Service d'imagerie 2, Hôpital de Hautepierre, Strasbourg, France
| | - Abtine Nouri
- Hôpitaux Universitaires de Strasbourg, Service d'imagerie 2, Hôpital de Hautepierre, Strasbourg, France
| | - Seyyid Baloglu
- Hôpitaux Universitaires de Strasbourg, Service d'imagerie 2, Hôpital de Hautepierre, Strasbourg, France
| | - Guillaume Bierry
- Hôpitaux Universitaires de Strasbourg, Service d'imagerie 2, Hôpital de Hautepierre, Strasbourg, France.,Engineering Science, Computer Science and Imaging Laboratory (ICube), Integrative Multimodal Imaging in Healthcare, UMR 7357, University of Strasbourg-CNRS, Strasbourg, France
| | - Agathe Chammas
- Hôpitaux Universitaires de Strasbourg, Service d'imagerie 2, Hôpital de Hautepierre, Strasbourg, France
| | - Stéphane Kremer
- Hôpitaux Universitaires de Strasbourg, Service d'imagerie 2, Hôpital de Hautepierre, Strasbourg, France.,Engineering Science, Computer Science and Imaging Laboratory (ICube), Integrative Multimodal Imaging in Healthcare, UMR 7357, University of Strasbourg-CNRS, Strasbourg, France
| |
Collapse
|
16
|
Trend in radiologist workload compared to number of admissions in the emergency department. Eur J Radiol 2022; 149:110195. [DOI: 10.1016/j.ejrad.2022.110195] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/11/2021] [Revised: 01/07/2022] [Accepted: 01/31/2022] [Indexed: 12/11/2022]
|
17
|
Alshabibi AS, Suleiman ME, Albeshan SM, Heard R, Brennan PC. Variations in breast cancer detection rates during mammogram-reading sessions: does experience have an impact? Br J Radiol 2022; 95:20210895. [PMID: 34735290 PMCID: PMC8722243 DOI: 10.1259/bjr.20210895] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/03/2023] Open
Abstract
OBJECTIVES To examine whether radiologists' performances are consistent throughout a reading session and whether any changes in performance over the reading task differ depending on experience of the reader. METHODS The performance of ten radiologists reading a test set of 60 mammographic cases without breaks was assessed using an ANOVA, 2 × 3 factorial design. Participants were categorized as more (≥2,000 mammogram readings per year) or less (<2,000 readings per year) experienced. Three series of 20 cases were chosen to ensure comparable difficulty and presented in the same sequence to all readers. It usually takes around 30 min for a radiologist to complete each of the 20-case series, resulting in a total of 90 min for the 60 mammographic cases. The sensitivity, specificity, lesion sensitivity, and area under the ROC curve were calculated for each series. We hypothesized that the order in which a series was read (i.e. fixed-series sequence) would have a significant main effect on the participants' performance. We also determined if significant interactions exist between the fixed-series sequence and radiologist experience. RESULTS Significant linear interactions were found between experience and the fixed sequence of the series for sensitivity (F[1] =5.762, p = .04, partial η2 = .41) and lesion sensitivity. (F[1] =6.993, p = .03, partial η2 = .46). The two groups' mean scores were similar for the first series but progressively diverged. By the end of the third series, significant differences in sensitivity and lesion sensitivity were evident, with the more experienced individuals demonstrating improving and the less experienced declining performance. Neither experience nor series sequence significantly affected the specificity or the area under the ROC curve. CONCLUSIONS Radiologists' performance may change considerably during a reading session, apparently as a function of experience, with less experienced radiologists declining in sensitivity and lesion sensitivity while more experienced radiologists actually improve. With the increasing demands on radiologists to undertake high-volume reporting, we suggest that junior radiologists be made aware of possible sensitivity and lesion sensitivity deterioration over time so they can schedule breaks during continuous reading sessions that are appropriate to them, rather than try to emulate their more experienced colleagues. ADVANCES IN KNOWLEDGE Less-experienced radiologists demonstrated a reduction in mammographic diagnostic accuracy in later stages of the reporting sessions. This may suggest that extending the duration of reporting sessions to compensate for increasing workloads may not represent the optimal solution for less-experienced radiologists.
Collapse
Affiliation(s)
| | - Moayyad E Suleiman
- The Medical Image Optimisation and Perception Group (MIOPeG), Faculty of Medicine and Health, The University of Sydney, Susan Wakil Health Building, Camperdown, Australia
| | - Salman M Albeshan
- Department of Radiology and Medical Imaging, The College of Applied Medical Sciences of King Saud University, Riyadh, Saudi Arabia
| | - Robert Heard
- The Medical Image Optimisation and Perception Group (MIOPeG), Faculty of Medicine and Health, The University of Sydney, Susan Wakil Health Building, Camperdown, Australia
| | - Patrick C Brennan
- The Medical Image Optimisation and Perception Group (MIOPeG), Faculty of Medicine and Health, The University of Sydney, Susan Wakil Health Building, Camperdown, Australia
| |
Collapse
|
18
|
Sicular S, Alpaslan M, Ortega FA, Keathley N, Venkatesh S, Jones RM, Lindsey RV. Reevaluation of missed lung cancer with artificial intelligence. Respir Med Case Rep 2022; 39:101733. [PMID: 36118268 PMCID: PMC9472074 DOI: 10.1016/j.rmcr.2022.101733] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/28/2022] [Revised: 07/22/2022] [Accepted: 08/23/2022] [Indexed: 10/25/2022] Open
|
19
|
Gender Differences in Diagnostic Radiology Practice: An Observational Study. J Am Coll Radiol 2021; 19:90-96. [PMID: 34767795 DOI: 10.1016/j.jacr.2021.09.034] [Citation(s) in RCA: 6] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/28/2021] [Revised: 09/19/2021] [Accepted: 09/22/2021] [Indexed: 11/21/2022]
Abstract
PURPOSE To investigate gender differences in diagnostic radiology practice, specifically, the differences in scope of practice, the frequency of consultations to other colleagues, and the error rates. MATERIAL AND METHODS A retrospective observational study was performed including radiologists working for a European teleradiology provider between 2013 and 2019. Main outcome measures included the adjusted odds ratio of female gender for reporting cases in more than one subspecialty, the adjusted incidence rate ratio (IRR) of female gender for the count of second opinion requests to other colleagues, and the adjusted IRR of female gender for the count of radiologic errors. Multivariable adjustment was performed for covariates associated with experience, type of cases reported, part- or full-time employment, and reporting speed. RESULTS A total of 213 radiologists (36% female) were included in the analysis of gender differences in scope of practice. Female gender was associated with a lower odds of reporting cases in more than one subspecialty with an odds ratio of 0.46 (95% confidence interval, 0.22-0.96). A total of 204 radiologists (36% female) were included in the analysis of gender differences in the count of second opinion requests to colleagues. There was a trend toward an association between female gender and higher odds of requesting a second opinion with an adjusted IRR of 1.6 compared with male gender but it was not statistically significant (P = .08). A total of 199 radiologists were included (37% female) in the analysis of gender differences in the number of radiologic errors. Female gender was associated with a decrease in the odds of error with an IRR of 0.8 (95% confidence interval, 0.64-0.995). CONCLUSIONS Female radiologists tend to have a narrower scope of practice and make fewer mistakes than their male counterparts, even after detailed adjustment for factors that might explain gender differences in scope of practice and errors.
Collapse
|
20
|
Patel MD, Pizzitola VJ, Johnson CD. Response to "Reflection on the Evaluation of Radiologists' Performance during Off-Hours Shifts". Radiology 2021; 300:E350. [PMID: 34254856 DOI: 10.1148/radiol.2021210258] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/11/2022]
Affiliation(s)
- Maitray D Patel
- Department of Radiology, Mayo Clinic Arizona, 5777 E Mayo Blvd, Phoenix, AZ 85054
| | - Victor J Pizzitola
- Department of Radiology, Mayo Clinic Arizona, 5777 E Mayo Blvd, Phoenix, AZ 85054
| | - C Daniel Johnson
- Department of Radiology, Mayo Clinic Arizona, 5777 E Mayo Blvd, Phoenix, AZ 85054
| |
Collapse
|
21
|
Vosshenrich J, Brantner P, Cyriac J, Boll DT, Merkle EM, Heye T. Quantifying Radiology Resident Fatigue: Analysis of Preliminary Reports. Radiology 2021; 298:632-639. [PMID: 33497316 DOI: 10.1148/radiol.2021203486] [Citation(s) in RCA: 13] [Impact Index Per Article: 4.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 02/07/2023]
Abstract
Background Workloads in radiology departments have constantly increased over the past decades. The resulting radiologist fatigue is considered a rising problem that affects diagnostic accuracy. Purpose To investigate whether data mining of quantitative parameters from the report proofreading process can reveal daytime and shift-dependent trends in report similarity as a surrogate marker for resident fatigue. Materials and Methods Data from 117 402 radiology reports written by residents between September 2017 and March 2020 were extracted from a report comparison tool and retrospectively analyzed. Through calculation of the Jaccard similarity coefficient between residents' preliminary and staff-reviewed final reports, the amount of edits performed by staff radiologists during proofreading was quantified on a scale of 0 to 1 (1: perfect similarity, no edits). Following aggregation per weekday and shift, data were statistically analyzed by using simple linear regression or one-way analysis of variance (significance level, P < .05) to determine relationships between report similarity and time of day and/or weekday reports were dictated. Results Decreasing report similarity with increasing work hours was observed for day shifts (r = -0.93 [95% CI: -0.73, -0.98]; P < .001) and weekend shifts (r = -0.72 [95% CI: -0.31, -0.91]; P = .004). For day shifts, negative linear correlation was strongest on Fridays (r = -0.95 [95% CI: -0.80, -0.99]; P < .001), with a 16% lower mean report similarity at the end of shifts (0.85 ± 0.24 at 8 am vs 0.69 ± 0.32 at 5 pm). Furthermore, mean similarity of reports dictated on Fridays (0.79 ± 0.35) was lower than that on all other weekdays (range, 0.84 ± 0.30 to 0.86 ± 0.27; P < .001). For late shifts, report similarity showed a negative correlation with the course of workweeks, showing a continuous decrease from Monday to Friday (r = -0.98 [95% CI: -0.70, -0.99]; P = .007). Temporary increases in report similarity were observed after lunch breaks (day and weekend shifts) and with the arrival of a rested resident during overlapping on-call shifts. Conclusion Decreases in report similarity over the course of workdays and workweeks suggest aggravating effects of fatigue on residents' report writing performances. Periodic breaks within shifts potentially foster recovery. © RSNA, 2021.
Collapse
Affiliation(s)
- Jan Vosshenrich
- From the Department of Radiology, University Hospital Basel, Petersgraben 4, 4031 Basel, Switzerland
| | - Philipp Brantner
- From the Department of Radiology, University Hospital Basel, Petersgraben 4, 4031 Basel, Switzerland
| | - Joshy Cyriac
- From the Department of Radiology, University Hospital Basel, Petersgraben 4, 4031 Basel, Switzerland
| | - Daniel T Boll
- From the Department of Radiology, University Hospital Basel, Petersgraben 4, 4031 Basel, Switzerland
| | - Elmar M Merkle
- From the Department of Radiology, University Hospital Basel, Petersgraben 4, 4031 Basel, Switzerland
| | - Tobias Heye
- From the Department of Radiology, University Hospital Basel, Petersgraben 4, 4031 Basel, Switzerland
| |
Collapse
|
22
|
Flink C, Kaplan SL. Reflection on the Evaluation of Radiologists' Performance during Off-Hours Shifts. Radiology 2021; 298:E165. [PMID: 33464185 DOI: 10.1148/radiol.2021203771] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/11/2022]
Affiliation(s)
- Carl Flink
- Department of Radiology, University of Cincinnati Medical Center, 234 Goodman St, ML 0761, Cincinnati, OH 45267
| | - Summer L Kaplan
- Department of Radiology, Children's Hospital of Philadelphia, Philadelphia, Pa
| |
Collapse
|
23
|
Dym RJ, Forman HP, Scheinfeld MH. Night and Day: Confounding Factors Complicate Comparison and Generalizability of Radiology Error Rates. Radiology 2020; 298:E115-E116. [PMID: 33320068 DOI: 10.1148/radiol.2020203577] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/16/2022]
Affiliation(s)
- R Joshua Dym
- Department of Radiology, Section of Emergency Radiology, University Hospital, Rutgers New Jersey Medical School, Newark, NJ
| | - Howard P Forman
- Department of Radiology and Biomedical Imaging, Yale School of Medicine, Yale University, New Haven, Conn
| | - Meir H Scheinfeld
- Department of Radiology, Division of Emergency Radiology, Montefiore Medical Center, Albert Einstein College of Medicine, 111 E 210 St, Bronx, NY 10467
| |
Collapse
|
24
|
Bruls RJM, Kwee RM. Workload for radiologists during on-call hours: dramatic increase in the past 15 years. Insights Imaging 2020; 11:121. [PMID: 33226490 PMCID: PMC7683675 DOI: 10.1186/s13244-020-00925-z] [Citation(s) in RCA: 67] [Impact Index Per Article: 16.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/18/2020] [Accepted: 10/13/2020] [Indexed: 12/15/2022] Open
Abstract
Background The objective of this study is to investigate the workload for radiologists during on-call hours and to quantify the 15-year trend in a large general hospital in Western Europe. Methods Data regarding the number of X-ray, ultrasound and computed tomography (CT) studies during on-call hours (weekdays between 6.00 p.m. and 7.00 a.m., weekends, and national holidays) between 2006 and 2020 were extracted from the picture archiving and communication system. All studies were converted into relative value units (RVUs) to estimate the on-call workload. The Mann–Kendall test was performed to assess the temporal trend. Results The total RVUs during on-call hours showed a significant increase between 2006 and 2020 (Kendall's tau-b = 0.657, p = 0.001). The overall workload in terms of RVUs during on-call hours has quadrupled. The number of X-ray studies significantly decreased (Kendall's tau-b = − 0.433, p = 0.026), whereas the number of CT studies significantly increased (Kendall's tau-b = 0.875, p < 0.001) between 2006 and 2020. CT studies which increased by more than 500% between 2006 and 2020 are CT for head trauma, brain CTA, brain CTV, chest CT (for suspected pulmonary embolism), spinal CT, neck CT, pelvic CT, and CT for suspected aortic dissection. The number of ultrasound studies did not change significantly (Kendall's tau-b = 0.202, p = 0.298). Conclusions The workload for radiologists during on-call hours increased dramatically in the past 15 years. The growing amount of CT studies is responsible for this increase. Radiologist and technician workforce should be matched to this ongoing increasing trend to avoid potential burn-out and to maintain quality and safety of radiological care.
Collapse
Affiliation(s)
- R J M Bruls
- Department of Radiology, Zuyderland Medical Center, Henri Dunantstraat 5, 6419 PC, Heerlen, The Netherlands.
| | - R M Kwee
- Department of Radiology, Zuyderland Medical Center, Henri Dunantstraat 5, 6419 PC, Heerlen, The Netherlands
| |
Collapse
|
25
|
Little D. Radiologist Errors at Night: Are We Still in the Dark? Radiology 2020; 298:E59. [PMID: 33201794 DOI: 10.1148/radiol.2020203489] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/11/2022]
Affiliation(s)
- David Little
- Department of Radiology, Royal United Hospitals Bath, Combe Park, Bath, BA1 3NG, England
| |
Collapse
|
26
|
Affiliation(s)
- Michael A Bruno
- From the Department of Radiology, Penn State Milton S. Hershey Medical Center and Penn State College of Medicine, 500 University Dr, H-066, Hershey, PA 17033
| |
Collapse
|