1
|
Ivanovic V, Broadhead K, Chang YM, Hamer JF, Beck R, Hacein-Bey L, Qi L. Shift Volume Directly Impacts Neuroradiology Error Rate at a Large Academic Medical Center: The Case for Volume Limits. AJNR Am J Neuroradiol 2024; 45:374-378. [PMID: 38238099 DOI: 10.3174/ajnr.a8119] [Citation(s) in RCA: 3] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/06/2023] [Accepted: 12/18/2023] [Indexed: 04/10/2024]
Abstract
BACKGROUND AND PURPOSE Unlike in Europe and Japan, guidelines or recommendations from specialized radiological societies on workflow management and adaptive intervention to reduce error rates are currently lacking in the United States. This study of neuroradiologic reads at a large US academic medical center, which may hopefully contribute to this discussion, found a direct relationship between error rate and shift volume. MATERIALS AND METHODS CT and MR imaging reports from our institution's Neuroradiology Quality Assurance database (years 2014-2020) were searched for attending physician errors. Data were collected on shift volume specific error rates per 1000 interpreted studies and RADPEER scores. Optimal cutoff points for 2, 3 and 4 groups of shift volumes were computed along with subgroups' error rates. RESULTS A total of 643 errors were found, 91.7% of which were clinically significant (RADPEER 2b, 3b). The overall error rate (errors/1000 examinations) was 2.36. The best single shift volume cutoff point generated 2 groups: ≤ 26 studies (error rate 1.59) and > 26 studies (2.58; OR: 1.63, P < .001). The best 2 shift volume cutoff points generated 3 shift volume groups: ≤ 19 (1.34), 20-28 (1.88; OR: 1.4, P = .1) and ≥ 29 (2.6; OR: 1.94, P < .001). The best 3 shift volume cutoff points generated 4 groups: ≤ 24 (1.59), 25-66 (2.44; OR: 1.54, P < .001), 67-90 (3.03; OR: 1.91, P < .001), and ≥ 91 (2.07; OR: 1.30, P = .25). The group with shift volume ≥ 91 had a limited sample size. CONCLUSIONS Lower shift volumes yielded significantly lower error rates. The lowest error rates were observed with shift volumes that were limited to 19-26 studies. Error rates at shift volumes between 67-90 studies were 226% higher, compared with the error rate at shift volumes of ≤ 19 studies.
Collapse
Affiliation(s)
- Vladimir Ivanovic
- From the Department of Radiology, Section of Neuroradiology (V.I., J.F.H., R.B.), Medical College of Wisconsin, Milwaukee, Wisconsin
| | - Kenneth Broadhead
- Department of Statistics (K.B.), Colorado State University, Fort Collins, Colorado
| | - Yu-Ming Chang
- Department of Radiology, Section of Neuroradiology (Y.-M.C.), Beth Israel Deaconess Medical Center, Boston, Massachusetts
| | - John F Hamer
- From the Department of Radiology, Section of Neuroradiology (V.I., J.F.H., R.B.), Medical College of Wisconsin, Milwaukee, Wisconsin
| | - Ryan Beck
- From the Department of Radiology, Section of Neuroradiology (V.I., J.F.H., R.B.), Medical College of Wisconsin, Milwaukee, Wisconsin
| | - Lotfi Hacein-Bey
- Department of Radiology, Section of Neuroradiology (L.H.-B.), University of California Davis Medical Center, Sacramento, California
| | - Lihong Qi
- Department of Public Health Sciences (L.Q.), School of Medicine, University of California Davis, Davis, California
| |
Collapse
|
2
|
Ivanovic V, Broadhead K, Beck R, Chang YM, Paydar A, Biddle G, Hacein-Bey L, Qi L. Factors Associated With Neuroradiologic Diagnostic Errors at a Large Tertiary-Care Academic Medical Center: A Case-Control Study. AJR Am J Roentgenol 2023; 221:355-362. [PMID: 36988269 DOI: 10.2214/ajr.22.28925] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 03/30/2023]
Abstract
BACKGROUND. Numerous studies have explored factors associated with diagnostic errors in neuroradiology; however, large-scale multivariable analyses are lacking. OBJECTIVE. The purpose of this study was to evaluate associations of interpretation time, shift volume, care setting, day of week, and trainee participation with diagnostic errors by neuroradiologists at a large academic medical center. METHODS. This retrospective case-control study using a large tertiary-care academic medical center's neuroradiology quality assurance database evaluated CT and MRI examinations for which neuroradiologists had assigned RADPEER scores. The database was searched from January 2014 through March 2020 for examinations without (RADPEER score of 1) or with (RADPEER scores of 2a, 2b, 3a, 3b, or 4) diagnostic error. For each examination with error, two examinations without error were randomly selected (unless only one examination could be identified) and matched by interpreting radiologist and examination type to form case and control groups. Marginal mixed-effects logistic regression models were used to assess associations of diagnostic error with interpretation time (number of minutes since the immediately preceding report's completion), shift volume (number of examinations interpreted during the shift), emergency/inpatient setting, weekend interpretation, and trainee participation in interpretation. RESULTS. The case group included 564 examinations in 564 patients (mean age, 50.0 ± 25.0 [SD] years; 309 men, 255 women); the control group included 1019 examinations in 1019 patients (mean age, 52.5 ± 23.2 years; 540 men, 479 women). In the case versus control group, mean interpretation time was 16.3 ± 17.2 [SD] minutes versus 14.8 ± 16.7 minutes; mean shift volume was 50.0 ± 22.1 [SD] examinations versus 45.4 ± 22.9 examinations. In univariable models, diagnostic error was associated with shift volume (OR = 1.22, p < .001) and weekend interpretation (OR = 1.60, p < .001) but not interpretation time, emergency/inpatient setting, or trainee participation (p > .05). However, in multivariable models, diagnostic error was independently associated with interpretation time (OR = 1.18, p = .003), shift volume (OR = 1.27, p < .001), and weekend interpretation (OR = 1.69, p = .02). In subanalysis, diagnostic error showed independent associations on weekdays with interpretation time (OR = 1.18, p = .003) and shift volume (OR = 1.27, p < .001); such associations were not observed on weekends (interpretation time: p = .62; shift volume: p = .58). CONCLUSION. Diagnostic errors in neuroradiology were associated with longer interpretation times, higher shift volumes, and weekend interpretation. CLINICAL IMPACT. These findings should be considered when designing work-flow-related interventions seeking to reduce neuroradiology interpretation errors.
Collapse
Affiliation(s)
- Vladimir Ivanovic
- Department of Radiology, Section of Neuroradiology, Medical College of Wisconsin, 8701 Watertown Plank Rd, Milwaukee, WI 53226
| | - Kenneth Broadhead
- Department of Statistics, Colorado State University, Fort Collins, CO
| | - Ryan Beck
- Department of Radiology, Section of Neuroradiology, Medical College of Wisconsin, 8701 Watertown Plank Rd, Milwaukee, WI 53226
| | - Yu-Ming Chang
- Department of Radiology, Section of Neuroradiology, Beth Israel Deaconess Medical Center, Boston, MA
| | - Alireza Paydar
- Department of Radiology, Section of Neuroradiology, University of California, Davis Medical Center, Sacramento, CA
| | - Garrick Biddle
- Department of Radiology, Section of Neuroradiology, University of California, Davis Medical Center, Sacramento, CA
| | - Lotfi Hacein-Bey
- Department of Radiology, Section of Neuroradiology, University of California, Davis Medical Center, Sacramento, CA
| | - Lihong Qi
- Department of Public Health Sciences, School of Medicine, University of California, Davis, Davis, CA
| |
Collapse
|
3
|
Ivanovic V, Paydar A, Chang YM, Broadhead K, Smullen D, Klein A, Hacein-Bey L. Impact of Shift Volume on Neuroradiology Diagnostic Errors at a Large Tertiary Academic Center. Acad Radiol 2023; 30:1584-1588. [PMID: 36180325 DOI: 10.1016/j.acra.2022.08.035] [Citation(s) in RCA: 7] [Impact Index Per Article: 7.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/25/2022] [Revised: 08/20/2022] [Accepted: 08/30/2022] [Indexed: 11/01/2022]
Abstract
BACKGROUND AND PURPOSE Medical errors can result in significant morbidity and mortality. The goal of our study is to evaluate correlation between shift volume and errors made by attending neuroradiologists at an academic medical center, using a large data set. MATERIALS AND METHODS CT and MRI reports from our Neuroradiology Quality Assurance database (years 2014 - 2020) were searched for attending physician errors. Data were collected on shift volume, category of missed findings, error type, interpretation setting, exam type, clinical significance. RESULTS 654 reports contained diagnostic error. There was a significant difference between mean volume of interpreted studies on shifts when an error was made compared with shifts in which no error was documented (46.58 (SD=22.37) vs 34.09 (SD=18.60), p<0.00001); and between shifts when perceptual error was made compared with shifts when interpretive errors were made (49.50 (SD=21.9) vs 43.26 (SD=21.75), p=0.0094). 59.6% of errors occurred in the emergency/inpatient setting, 84% were perceptual and 91.1% clinically significant. Categorical distribution of errors was: vascular 25.8%, brain 23.4%, skull base 13.8%, spine 12.4%, head/neck 11.3%, fractures 10.2%, other 3.1%. Errors were detected most often on brain MRI (25.4%), head CT (18.7%), head/neck CTA (13.8%), spine MRI (13.7%). CONCLUSION Errors were associated with higher volume shifts, were primarily perceptual and clinically significant. We need National guidelines establishing a range of what is a safe number of interpreted cross-sectional studies per day.
Collapse
Affiliation(s)
- Vladimir Ivanovic
- Department of Radiology, Section of Neuroradiology, Medical College of Wisconsin, Milwaukee, WI.
| | - Alireza Paydar
- Department of Radiology, Section of Neuroradiology, University of California Davis Medical Center, Sacramento, CA
| | - Yu-Ming Chang
- Department of Radiology, Section of Neuroradiology, Beth Israel Deaconess Medical Center, Harvard School of Medicine, Boston, Massachusetts
| | - Kenneth Broadhead
- Department of statistics, School of Medicine, University of California Davis, Davis, CA
| | - David Smullen
- Department of Radiology, Section of Neuroradiology, Medical College of Wisconsin, Milwaukee, WI
| | - Andrew Klein
- Department of Radiology, Section of Neuroradiology, Medical College of Wisconsin, Milwaukee, WI
| | - Lotfi Hacein-Bey
- Department of Radiology, Section of Neuroradiology, University of California Davis Medical Center, Sacramento, CA
| |
Collapse
|
4
|
Isikbay M, Caton MT, Calabrese E. A Deep Learning Approach for Automated Bone Removal from Computed Tomography Angiography of the Brain. J Digit Imaging 2023; 36:964-972. [PMID: 36781588 PMCID: PMC10287884 DOI: 10.1007/s10278-023-00788-y] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/21/2022] [Revised: 01/29/2023] [Accepted: 01/30/2023] [Indexed: 02/15/2023] Open
Abstract
Advanced visualization techniques such as maximum intensity projection (MIP) and volume rendering (VR) are useful for evaluating neurovascular anatomy on CT angiography (CTA) of the brain; however, interference from surrounding osseous anatomy is common. Existing methods for removing bone from CTA images are limited in scope and/or accuracy, particularly at the skull base. We present a new brain CTA bone removal tool, which addresses many of these limitations. A deep convolutional neural network was designed and trained for bone removal using 72 brain CTAs. The model was tested on 15 CTAs from the same data source and 17 CTAs from an independent external dataset. Bone removal accuracy was assessed quantitatively, by comparing automated segmentation results to manual segmentations, and qualitatively by evaluating VR visualization of the carotid siphons compared to an existing method for automated bone removal. Average Dice overlap between automated and manual segmentations from the internal and external test datasets were 0.986 and 0.979 respectively. This was superior compared to a publicly available noncontrast head CT bone removal algorithm which had a Dice overlap of 0.947 (internal dataset) and 0.938 (external dataset). Our algorithm yielded better VR visualization of the carotid siphons than the publicly available bone removal tool in 14 out of 15 CTAs (93%, chi-square statistic of 22.5, p-value of < 0.00001) from the internal test dataset and 15 out of 17 CTAs (88%, chi-square statistic of 23.1, p-value of < 0.00001) from the external test dataset. Bone removal allowed subjectively superior MIP and VR visualization of vascular anatomy/pathology. The proposed brain CTA bone removal algorithm is rapid, accurate, and allows superior visualization of vascular anatomy and pathology compared to other available techniques and was validated on an independent external dataset.
Collapse
Affiliation(s)
- Masis Isikbay
- Department of Radiology and Biomedical Imaging, University of California San Francisco, 505 Parnassus Ave, M-396, San Francisco, CA, 94143, USA.
| | - M Travis Caton
- Cerebrovascular Center, Department of Neurosurgery, Icahn School of Medicine at Mount Sinai, 1450 Madison Ave, New York, NY, 10029, USA
| | - Evan Calabrese
- Department of Radiology and Biomedical Imaging, University of California San Francisco, 505 Parnassus Ave, M-396, San Francisco, CA, 94143, USA
- Department of Radiology, Division of Neuroradiology, Duke University Medical Center, Box 3808 DUMC, Durham, NC, 27710, USA
- Duke Center for Artificial Intelligence in Radiology (DAIR), Duke University Medical Center, Durham, NC, 27710, USA
- Center for Intelligent Imaging, University of California San Francisco, San Francisco, CA, 94143, USA
| |
Collapse
|
5
|
Mamourian A, Bagian J. Ax vs Scalpel: The Role of Quality in Optimizing Neuroradiologist's Workload. Acad Radiol 2023; 30:1228. [PMID: 37061450 DOI: 10.1016/j.acra.2023.03.004] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/16/2022] [Accepted: 03/03/2023] [Indexed: 04/17/2023]
Affiliation(s)
- Alex Mamourian
- Professor of Radiology, Neurology and Neurosurgery, Pennsylvania State-Hershey Medical Center, 500 University Drive, Hershey, PA 17033; Professor of Anesthesiology, Operations and Industrial Engineering, Aerospace Engineering Executive Director Center for Risk Analysis Informed Decision Engineering, University of Michigan.
| | - James Bagian
- Professor of Radiology, Neurology and Neurosurgery, Pennsylvania State-Hershey Medical Center, 500 University Drive, Hershey, PA 17033; Professor of Anesthesiology, Operations and Industrial Engineering, Aerospace Engineering Executive Director Center for Risk Analysis Informed Decision Engineering, University of Michigan
| |
Collapse
|