1
|
Moher J, Delos Reyes A, Drew T. Cue relevance drives early quitting in visual search. Cogn Res Princ Implic 2024; 9:54. [PMID: 39183257 PMCID: PMC11345343 DOI: 10.1186/s41235-024-00587-1] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/30/2023] [Accepted: 08/08/2024] [Indexed: 08/27/2024] Open
Abstract
Irrelevant salient distractors can trigger early quitting in visual search, causing observers to miss targets they might otherwise find. Here, we asked whether task-relevant salient cues can produce a similar early quitting effect on the subset of trials where those cues fail to highlight the target. We presented participants with a difficult visual search task and used two cueing conditions. In the high-predictive condition, a salient cue in the form of a red circle highlighted the target most of the time a target was present. In the low-predictive condition, the cue was far less accurate and did not reliably predict the target (i.e., the cue was often a false positive). These were contrasted against a control condition in which no cues were presented. In the high-predictive condition, we found clear evidence of early quitting on trials where the cue was a false positive, as evidenced by both increased miss errors and shorter response times on target absent trials. No such effects were observed with low-predictive cues. Together, these results suggest that salient cues which are false positives can trigger early quitting, though perhaps only when the cues have a high-predictive value. These results have implications for real-world searches, such as medical image screening, where salient cues (referred to as computer-aided detection or CAD) may be used to highlight potentially relevant areas of images but are sometimes inaccurate.
Collapse
Affiliation(s)
- Jeff Moher
- Psychology Department, Connecticut College, 270 Mohegan Avenue, New London, CT, 06320, USA.
| | | | | |
Collapse
|
2
|
Weber MA. Easily missed pathologies of the musculoskeletal system in the emergency radiology setting. ROFO-FORTSCHR RONTG 2024. [PMID: 39094774 DOI: 10.1055/a-2369-8330] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 08/04/2024]
Abstract
The musculoskeletal region is the main area in terms of easily missed pathologies in the emergency radiology setting, because the majority of diagnoses missed in the emergency setting are fractures.A review of the literature was performed by searching the PubMed and ScienceDirect databases, using the keywords ('missed injuries' or 'missed fractures') and ('emergency radiology' or 'emergency room') and ('musculoskeletal' or 'bone' or 'skeleton') for the title and abstract query. The inclusion criteria were scientific papers presented in the English and German languages. Among the 347 relevant hits between 1980 and 2024 as identified by the author of this review article, there were 114 relevant articles from the years between 2018 and 2024. Based on this literature search and the author's personal experience, this study presents useful information for reducing the number of missed pathologies in the musculoskeletal system in the emergency radiology setting.Predominant factors that make up the majority of missed fractures are 'subtle but still visible fractures' and 'radiographically imperceptible fractures'. Radiologists are able to minimize the factors contributing to fractures being missed. For example, implementing a 'four-eyes principle', i.e., two readers read the radiographs, would help to overcome the missing of 'subtle but still visible fractures' and the additional use of cross-sectional imaging would help to overcome the missing of 'radiographically imperceptible fractures'. Knowledge of what is commonly missed and evaluation of high-risk areas with utmost care also increase the diagnostic performance of radiologists. · Radiological imaging in an emergency setting increases the likelihood of radiological diagnostic errors, such as missing musculoskeletal pathologies.. · The majority of diagnoses missed in the emergency setting are fractures.. · To lessen the number of easily missed pathologies in the musculoskeletal system in the emergency radiology setting, a systematic approach is necessary.. · Adequate training of radiologists in emergency radiology and close collaboration with clinical partners are important measures to decrease the number of missed musculoskeletal injuries.. · Weber MA. Easily missed pathologies of the musculoskeletal system in the emergency radiology setting. Fortschr Röntgenstr 2024; DOI 10.1055/a-2369-8330.
Collapse
Affiliation(s)
- Marc-André Weber
- Institute of Diagnostic and Interventional Radiology, Pediatric Radiology and Neuroradiology, University Medical Center Rostock, Rostock, Germany
| |
Collapse
|
3
|
Castner N, Arsiwala-Scheppach L, Mertens S, Krois J, Thaqi E, Kasneci E, Wahl S, Schwendicke F. Expert gaze as a usability indicator of medical AI decision support systems: a preliminary study. NPJ Digit Med 2024; 7:199. [PMID: 39068241 PMCID: PMC11283514 DOI: 10.1038/s41746-024-01192-8] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/10/2023] [Accepted: 07/12/2024] [Indexed: 07/30/2024] Open
Abstract
Given the current state of medical artificial intelligence (AI) and perceptions towards it, collaborative systems are becoming the preferred choice for clinical workflows. This work aims to address expert interaction with medical AI support systems to gain insight towards how these systems can be better designed with the user in mind. As eye tracking metrics have been shown to be robust indicators of usability, we employ them for evaluating the usability and user interaction with medical AI support systems. We use expert gaze to assess experts' interaction with an AI software for caries detection in bitewing x-ray images. We compared standard viewing of bitewing images without AI support versus viewing where AI support could be freely toggled on and off. We found that experts turned the AI on for roughly 25% of the total inspection task, and generally turned it on halfway through the course of the inspection. Gaze behavior showed that when supported by AI, more attention was dedicated to user interface elements related to the AI support, with more frequent transitions from the image itself to these elements. When considering that expert visual strategy is already optimized for fast and effective image inspection, such interruptions in attention can lead to increased time needed for the overall assessment. Gaze analysis provided valuable insights into an AI's usability for medical image inspection. Further analyses of these tools and how to delineate metrical measures of usability should be developed.
Collapse
Affiliation(s)
- Nora Castner
- Carl Zeiss Vision International GmbH, Tübingen, Germany.
- University of Tübingen, Tübingen, Germany.
| | | | - Sarah Mertens
- Charité - Univesitätsmedizin, Oral Diagnostics, Digital Health and Services Research, Berlin, Germany
| | - Joachim Krois
- Charité - Univesitätsmedizin, Oral Diagnostics, Digital Health and Services Research, Berlin, Germany
| | - Enkeleda Thaqi
- Technical University of Munich, Human-Centered Technologies for Learning, Munich, Germany
| | - Enkelejda Kasneci
- Technical University of Munich, Human-Centered Technologies for Learning, Munich, Germany
| | - Siegfried Wahl
- Carl Zeiss Vision International GmbH, Tübingen, Germany
- Institute for Ophthalmic Research, University of Tübingen, Tübingen, Germany
| | - Falk Schwendicke
- Ludwig Maximilian University, Operative, Preventative and Pediatric Dentistry and Periodontology, Munich, Germany
| |
Collapse
|
4
|
Kang D, Raviprasad A, Pierre K, Talati J, Kent T, Batmunh B, Lanier L, Slater RM, Sistrom CL, Mancuso AA, Davis I, Rajderkar DA. Challenges in diagnosis of calcaneal fractures: an examination using the WIDI SIM platform. Emerg Radiol 2024:10.1007/s10140-024-02267-5. [PMID: 38969914 DOI: 10.1007/s10140-024-02267-5] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/10/2024] [Accepted: 07/01/2024] [Indexed: 07/07/2024]
Abstract
INTRODUCTION The calcaneus is the most commonly fractured tarsal bone. Diagnosis is often challenging due to subtle radiographic changes and requires timely identification to prevent complications, including subtalar arthritis, neurovascular injury, malunion, osteomyelitis, and compartment syndrome. Treatment varies based on fracture type, with non-surgical methods for non-displaced stress fractures and surgical interventions for displaced or intra-articular fractures. METHODS This study utilized the Wisdom in Diagnostic Imaging Simulation (WIDI SIM) platform, an emergency imaging simulation designed to assess radiology resident preparedness for independent call. During an 8-hour simulation, residents were tested on 65 cases across various imaging modalities of varying complexity, including normal studies. A single, unique case of calcaneal fracture was included within the simulation in four separate years of testing. Cases were assessed using a standardized grading rubric by subspecialty radiology faculty, with errors subsequently classified by type. RESULTS A total of 1279 residents were tested in five separate years on the findings of calcaneal fractures of 5 different patients. Analysis revealed a consistent pattern of missed diagnoses across all training years, primarily attributed to observational errors. There was limited improvement with training progression as all training years exhibited similar average performance levels. CONCLUSIONS Calcaneal fractures pose a diagnostic challenge due to their frequent subtle radiographic findings, especially in stress fractures. Simulation-based evaluations using WIDI SIM highlighted challenges in radiology residents' proficiency in diagnosing calcaneal fractures. Addressing these challenges through targeted education and exposure to diverse cases is essential to improve diagnostic accuracy and reduce complications with calcaneal fractures.
Collapse
Affiliation(s)
- Dahyun Kang
- College of Medicine, University of Florida, Gainesville, FL, USA
| | - Abheek Raviprasad
- Department of Radiology, University of Florida College of Medicine, PO Box 100374, Gainesville, FL, 32610, USA.
| | - Kevin Pierre
- Department of Radiology, University of Florida College of Medicine, PO Box 100374, Gainesville, FL, 32610, USA
| | - Jay Talati
- College of Medicine, University of Florida, Gainesville, FL, USA
| | - Thomas Kent
- College of Medicine, University of Florida, Gainesville, FL, USA
| | - Bayar Batmunh
- Department of Radiology, University of Florida College of Medicine, PO Box 100374, Gainesville, FL, 32610, USA
| | - Linda Lanier
- Department of Radiology, University of Florida College of Medicine, PO Box 100374, Gainesville, FL, 32610, USA
| | - Roberta M Slater
- Department of Radiology, University of Florida College of Medicine, PO Box 100374, Gainesville, FL, 32610, USA
| | - Christopher L Sistrom
- Department of Radiology, University of Florida College of Medicine, PO Box 100374, Gainesville, FL, 32610, USA
| | - Anthony A Mancuso
- Department of Radiology, University of Florida College of Medicine, PO Box 100374, Gainesville, FL, 32610, USA
| | - Ivan Davis
- Department of Radiology, University of Florida College of Medicine, PO Box 100374, Gainesville, FL, 32610, USA
| | - Dhanashree A Rajderkar
- Department of Radiology, University of Florida College of Medicine, PO Box 100374, Gainesville, FL, 32610, USA
| |
Collapse
|
5
|
Böhner AMC, Effland A, Jacob AM, Böhner KAM, Abdullah Z, Brähler S, Attenberger UI, Rumpf M, Kurts C. Determining individual glomerular proteinuria and periglomerular infiltration in a cleared murine kidney by a 3-dimensional fast marching algorithm. Kidney Int 2024; 105:1254-1262. [PMID: 38458475 DOI: 10.1016/j.kint.2024.01.043] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/27/2023] [Revised: 11/30/2023] [Accepted: 01/09/2024] [Indexed: 03/10/2024]
Abstract
Three-dimensional (3D) imaging has advanced basic research and clinical medicine. However, limited resolution and imperfections of real-world 3D image material often preclude algorithmic image analysis. Here, we present a methodologic framework for such imaging and analysis for functional and spatial relations in experimental nephritis. First, optical tissue-clearing protocols were optimized to preserve fluorescence signals for light sheet fluorescence microscopy and compensated attenuation effects using adjustable 3D correction fields. Next, we adapted the fast marching algorithm to conduct backtracking in 3D environments and developed a tool to determine local concentrations of extractable objects. As a proof-of-concept application, we used this framework to determine in a glomerulonephritis model the individual proteinuria and periglomerular immune cell infiltration for all glomeruli of half a mouse kidney. A correlation between these parameters surprisingly did not support the intuitional assumption that the most inflamed glomeruli are the most proteinuric. Instead, the spatial density of adjacent glomeruli positively correlated with the proteinuria of a given glomerulus. Because proteinuric glomeruli appear clustered, this suggests that the exact location of a kidney biopsy may affect the observed severity of glomerular damage. Thus, our algorithmic pipeline described here allows analysis of various parameters of various organs composed of functional subunits, such as the kidney, and can theoretically be adapted to processing other image modalities.
Collapse
Affiliation(s)
- Alexander M C Böhner
- Institute for Molecular Medicine and Experimental Immunology, University Hospital Bonn, Bonn, Germany; Department of Diagnostic and Interventional Radiology, University Hospital Bonn, Bonn, Germany
| | - Alexander Effland
- Institute for Applied Mathematics, University of Bonn, Bonn, Germany
| | - Alice M Jacob
- Institute for Molecular Medicine and Experimental Immunology, University Hospital Bonn, Bonn, Germany
| | - Karin A M Böhner
- Institute for Molecular Medicine and Experimental Immunology, University Hospital Bonn, Bonn, Germany
| | - Zeinab Abdullah
- Institute for Molecular Medicine and Experimental Immunology, University Hospital Bonn, Bonn, Germany
| | - Sebastian Brähler
- Department of Internal Medicine II, University Hospital Cologne, Cologne, Germany
| | - Ulrike I Attenberger
- Department of Diagnostic and Interventional Radiology, University Hospital Bonn, Bonn, Germany
| | - Martin Rumpf
- Institute for Numerical Simulation, University of Bonn, Bonn, Germany
| | - Christian Kurts
- Institute for Molecular Medicine and Experimental Immunology, University Hospital Bonn, Bonn, Germany; Department of Microbiology and Immunology, Doherty Institute for Infection and Immunity, University of Melbourne, Melbourne, Victoria, Australia.
| |
Collapse
|
6
|
Alexander RG, Venkatakrishnan A, Chanovas J, Ferguson S, Macknik SL, Martinez-Conde S. Why did Rubens add a parrot to Titian's The Fall of Man? A pictorial manipulation of joint attention. J Vis 2024; 24:1. [PMID: 38558160 PMCID: PMC10996941 DOI: 10.1167/jov.24.4.1] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/01/2023] [Accepted: 01/19/2024] [Indexed: 04/04/2024] Open
Abstract
Almost 400 years ago, Rubens copied Titian's The Fall of Man, albeit with important changes. Rubens altered Titian's original composition in numerous ways, including by changing the gaze directions of the depicted characters and adding a striking red parrot to the painting. Here, we quantify the impact of Rubens's choices on the viewer's gaze behavior. We displayed digital copies of Rubens's and Titian's artworks-as well as a version of Rubens's painting with the parrot digitally removed-on a computer screen while recording the eye movements produced by observers during free visual exploration of each image. To assess the effects of Rubens's changes to Titian's composition, we directly compared multiple gaze parameters across the different images. We found that participants gazed at Eve's face more frequently in Rubens's painting than in Titian's. In addition, gaze positions were more tightly focused for the former than for the latter, consistent with different allocations of viewer interest. We also investigated how gaze fixation on Eve's face affected the perceptual visibility of the parrot in Rubens's composition and how the parrot's presence versus its absence impacted gaze dynamics. Taken together, our results demonstrate that Rubens's critical deviations from Titian's painting have powerful effects on viewers' oculomotor behavior.
Collapse
Affiliation(s)
- Robert G Alexander
- Department of Psychology & Counseling, New York Institute of Technology, New York, NY, USA
| | - Ashwin Venkatakrishnan
- Department of Ophthalmology, SUNY Downstate Health Sciences University, Brooklyn, NY, USA
| | - Jordi Chanovas
- Department of Ophthalmology, SUNY Downstate Health Sciences University, Brooklyn, NY, USA
- Graduate Program in Neural and Behavioral Science, SUNY Downstate Health Sciences University, Brooklyn, NY, USA
| | - Sophie Ferguson
- Department of Ophthalmology, SUNY Downstate Health Sciences University, Brooklyn, NY, USA
| | - Stephen L Macknik
- Department of Ophthalmology, SUNY Downstate Health Sciences University, Brooklyn, NY, USA
| | - Susana Martinez-Conde
- Department of Ophthalmology, SUNY Downstate Health Sciences University, Brooklyn, NY, USA
| |
Collapse
|
7
|
Sumner C, Kietzman A, Kadom N, Frigini A, Makary MS, Martin A, McKnight C, Retrouvey M, Spieler B, Griffith B. Medical Malpractice and Diagnostic Radiology: Challenges and Opportunities. Acad Radiol 2024; 31:233-241. [PMID: 37741730 DOI: 10.1016/j.acra.2023.08.015] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/03/2023] [Revised: 08/10/2023] [Accepted: 08/14/2023] [Indexed: 09/25/2023]
Abstract
Medicolegal challenges in radiology are broad and impact both radiologists and patients. Radiologists may be affected directly by malpractice litigation or indirectly due to defensive imaging ordering practices. Patients also could be harmed physically, emotionally, or financially by unnecessary tests or procedures. As technology advances, the incorporation of artificial intelligence into medicine will bring with it new medicolegal challenges and opportunities. This article reviews the current and emerging direct and indirect effects of medical malpractice on radiologists and summarizes evidence-based solutions.
Collapse
Affiliation(s)
- Christina Sumner
- Department of Radiology and Imaging Sciences, Emory University (C.S., N.K.), Atlanta, GA
| | | | - Nadja Kadom
- Department of Radiology and Imaging Sciences, Emory University (C.S., N.K.), Atlanta, GA
| | - Alexandre Frigini
- Department of Radiology, Baylor College of Medicine (A.F.), Houston, TX
| | - Mina S Makary
- Department of Radiology, Ohio State University Wexner Medical Center (M.S.M.), Columbus, OH
| | - Ardenne Martin
- Louisiana State University Health Sciences Center (A.M.), New Orleans, LA
| | - Colin McKnight
- Department of Radiology, Vanderbilt University Medical Center (C.M.), Nashville, TN
| | - Michele Retrouvey
- Department of Radiology, Eastern Virginia Medical School/Medical Center Radiologists (M.R.), Norfolk, VA
| | - Bradley Spieler
- Department of Radiology, University Medical Center, Louisiana State University Health Sciences Center (B.S.), New Orleans, LA
| | - Brent Griffith
- Department of Radiology, Henry Ford Health (B.G.), Detroit, MI.
| |
Collapse
|
8
|
Gefter WB, Hatabu H. Response. Chest 2023; 164:e58. [PMID: 37558338 DOI: 10.1016/j.chest.2023.05.001] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/29/2023] [Accepted: 05/01/2023] [Indexed: 08/11/2023] Open
Affiliation(s)
- Warren B Gefter
- Department of Radiology, Penn Medicine, University of Pennsylvania, Philadelphia, PA
| | - Hiroto Hatabu
- Department of Radiology, Center for Pulmonary Functional Imaging, Brigham and Women's Hospital and Harvard Medical School, Boston, MA.
| |
Collapse
|
9
|
Hena B, Wei Z, Castanedo CI, Maldague X. Deep Learning Neural Network Performance on NDT Digital X-ray Radiography Images: Analyzing the Impact of Image Quality Parameters-An Experimental Study. SENSORS (BASEL, SWITZERLAND) 2023; 23:s23094324. [PMID: 37177528 PMCID: PMC10181732 DOI: 10.3390/s23094324] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 03/31/2023] [Revised: 04/13/2023] [Accepted: 04/25/2023] [Indexed: 05/15/2023]
Abstract
In response to the growing inspection demand exerted by process automation in component manufacturing, non-destructive testing (NDT) continues to explore automated approaches that utilize deep-learning algorithms for defect identification, including within digital X-ray radiography images. This necessitates a thorough understanding of the implication of image quality parameters on the performance of these deep-learning models. This study investigated the influence of two image-quality parameters, namely signal-to-noise ratio (SNR) and contrast-to-noise ratio (CNR), on the performance of a U-net deep-learning semantic segmentation model. Input images were acquired with varying combinations of exposure factors, such as kilovoltage, milli-ampere, and exposure time, which altered the resultant radiographic image quality. The data were sorted into five different datasets according to their measured SNR and CNR values. The deep-learning model was trained five distinct times, utilizing a unique dataset for each training session. Training the model with high CNR values yielded an intersection-over-union (IoU) metric of 0.9594 on test data of the same category but dropped to 0.5875 when tested on lower CNR test data. The result of this study emphasizes the importance of achieving a balance in training dataset according to the investigated quality parameters in order to enhance the performance of deep-learning segmentation models for NDT digital X-ray radiography applications.
Collapse
Affiliation(s)
- Bata Hena
- Department of Electrical and Computer Engineering, Université Laval, Quebec City, QC G1V 0A6, Canada
- Computer Vision and Systems Laboratory, Department of Electrical and Computer Engineering, 1065, Ave de la Médecine, Université Laval, Quebec City, QC G1V 0A6, Canada
| | - Ziang Wei
- Department of Electrical and Computer Engineering, Université Laval, Quebec City, QC G1V 0A6, Canada
- Computer Vision and Systems Laboratory, Department of Electrical and Computer Engineering, 1065, Ave de la Médecine, Université Laval, Quebec City, QC G1V 0A6, Canada
- School of Engineering, University of Applied Sciences in Saarbrücken, 66117 Saarbrücken, Germany
- Fraunhofer Institute for Nondestructive Testing IZFP, 66123 Saarbrücken, Germany
| | - Clemente Ibarra Castanedo
- Department of Electrical and Computer Engineering, Université Laval, Quebec City, QC G1V 0A6, Canada
- Computer Vision and Systems Laboratory, Department of Electrical and Computer Engineering, 1065, Ave de la Médecine, Université Laval, Quebec City, QC G1V 0A6, Canada
| | - Xavier Maldague
- Department of Electrical and Computer Engineering, Université Laval, Quebec City, QC G1V 0A6, Canada
- Computer Vision and Systems Laboratory, Department of Electrical and Computer Engineering, 1065, Ave de la Médecine, Université Laval, Quebec City, QC G1V 0A6, Canada
| |
Collapse
|
10
|
Biddle G, Assadsangabi R, Broadhead K, Hacein-Bey L, Ivanovic V. Diagnostic Errors in Cerebrovascular Pathology: Retrospective Analysis of a Neuroradiology Database at a Large Tertiary Academic Medical Center. AJNR Am J Neuroradiol 2022; 43:1271-1278. [PMID: 35926887 PMCID: PMC9451623 DOI: 10.3174/ajnr.a7596] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/12/2021] [Accepted: 06/16/2022] [Indexed: 01/26/2023]
Abstract
BACKGROUND AND PURPOSE Diagnostic errors affect 2%-8% of neuroradiology studies, resulting in significant potential morbidity and mortality. This retrospective analysis of a large database at a single tertiary academic institution focuses on diagnostic misses in cerebrovascular pathology and suggests error-reduction strategies. MATERIALS AND METHODS CT and MR imaging reports from a consecutive database spanning 2015-2020 were searched for errors of attending physicians in cerebrovascular pathology. Data were collected on missed findings, study types, and interpretation settings. Errors were categorized as ischemic, arterial, venous, hemorrhagic, and "other." RESULTS A total of 245,762 CT and MR imaging neuroradiology examinations were interpreted during the study period. Vascular diagnostic errors were present in 165 reports, with a mean of 49.6 (SD, 23.3) studies on the shifts when an error was made, compared with 34.9 (SD, 19.2) on shifts without detected errors (P < .0001). Seventy percent of examinations occurred in the hospital setting; 93.3% of errors were perceptual; 6.7% were interpretive; and 93.9% (n = 155) were clinically significant (RADPEER 2B or 3B). The distribution of errors was arterial and ischemic each with 33.3%, hemorrhagic with 21.8%, and venous with 7.5%. Most errors involved brain MR imaging (30.3%) followed by head CTA (27.9%) and noncontrast head CT (26.1%). The most common misses were acute/subacute infarcts (25.1%), followed by aneurysms (13.7%) and subdural hematomas (9.7%). CONCLUSIONS Most cerebrovascular diagnostic errors were perceptual and clinically significant, occurred in the emergency/inpatient setting, and were associated with higher-volume shifts. Diagnostic errors could be minimized by adjusting search patterns to ensure vigilance on the sites of the frequently missed pathologies.
Collapse
Affiliation(s)
- G Biddle
- From the Neuroradiology Division (G.B., L.H.-B.), Department of Radiology, University of California Davis School of Medicine, Sacramento, California
| | - R Assadsangabi
- Neuroradiology Division (R.A.), Department of Radiology, University of Southern California, Los Angeles, California
| | - K Broadhead
- Department of Statistics (K.B.), University of California Davis, Davis, California
| | - L Hacein-Bey
- From the Neuroradiology Division (G.B., L.H.-B.), Department of Radiology, University of California Davis School of Medicine, Sacramento, California
| | - V Ivanovic
- Neuroradiology division (V.I.), Department of Radiology, Medical College of Wisconsin, Milwaukee, Wisconsin
| |
Collapse
|
11
|
Phelps AM, Alexander RG, Schmidt J. Negative cues minimize visual search specificity effects. Vision Res 2022; 196:108030. [PMID: 35313163 PMCID: PMC9090971 DOI: 10.1016/j.visres.2022.108030] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/09/2021] [Revised: 03/02/2022] [Accepted: 03/08/2022] [Indexed: 11/28/2022]
Abstract
Prior target knowledge (i.e., positive cues) improves visual search performance. However, there is considerable debate about whether distractor knowledge (i.e., negative cues) can guide search. Some studies suggest the active suppression of negatively cued search items, while others suggest the initial capture of attention by negatively cued items. Prior work has used pictorial or specific text cues but has not explicitly compared them. We build on that work by comparing positive and negative cues presented pictorially and as categorical text labels using photorealistic objects and eye movement measures. Search displays contained a target (cued on positive trials), a lure from the target category (cued on negative trials), and four categorically-unrelated distractors. Search performance with positive cues resulted in stronger attentional guidance and faster object recognition for pictorial relative to categorical cues (i.e., a pictorial advantage, suggesting specific visual details afforded by pictorial cues improved search). However, in most search performance metrics, negative cues mitigate the pictorial advantage. Given that the negatively cued items captured attention, generated target guidance but mitigated the pictorial advantage, these results are partly consistent with both existing theories. Specific visual details provided in positive cues produce a large pictorial advantage in all measures, whereas specific visual details in negative cues only produce a small pictorial advantage for object recognition but not for attentional guidance. This asymmetry in the pictorial advantage suggests that the down-weighting of specific negatively cued visual features is less efficient than the up-weighting of specific positively cued visual features.
Collapse
Affiliation(s)
- Ashley M Phelps
- Department of Psychology, University of Central Florida, Orlando, FL, USA
| | - Robert G Alexander
- Departments of Ophthalmology, Neurology, and Physiology & Pharmacology, SUNY Downstate Health Sciences University, Brooklyn, NY, USA
| | - Joseph Schmidt
- Department of Psychology, University of Central Florida, Orlando, FL, USA.
| |
Collapse
|
12
|
Alexander R, Waite S, Bruno MA, Krupinski EA, Berlin L, Macknik S, Martinez-Conde S. Mandating Limits on Workload, Duty, and Speed in Radiology. Radiology 2022; 304:274-282. [PMID: 35699581 PMCID: PMC9340237 DOI: 10.1148/radiol.212631] [Citation(s) in RCA: 27] [Impact Index Per Article: 13.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/25/2023]
Abstract
Research has not yet quantified the effects of workload or duty hours on the accuracy of radiologists. With the exception of a brief reduction in imaging studies during the 2020 peak of the COVID-19 pandemic, the workload of radiologists in the United States has seen relentless growth in recent years. One concern is that this increased demand could lead to reduced accuracy. Behavioral studies in species ranging from insects to humans have shown that decision speed is inversely correlated to decision accuracy. A potential solution is to institute workload and duty limits to optimize radiologist performance and patient safety. The concern, however, is that any prescribed mandated limits would be arbitrary and thus no more advantageous than allowing radiologists to self-regulate. Specific studies have been proposed to determine whether limits reduce error, and if so, to provide a principled basis for such limits. This could determine the precise susceptibility of individual radiologists to medical error as a function of speed during image viewing, the maximum number of studies that could be read during a work shift, and the appropriate shift duration as a function of time of day. Before principled recommendations for restrictions are made, however, it is important to understand how radiologists function both optimally and at the margins of adequate performance. This study examines the relationship between interpretation speed and error rates in radiology, the potential influence of artificial intelligence on reading speed and error rates, and the possible outcomes of imposed limits on both caseload and duty hours. This review concludes that the scientific evidence needed to make meaningful rules is lacking and notes that regulating workloads without scientific principles can be more harmful than not regulating at all.
Collapse
Affiliation(s)
- Robert Alexander
- From the Departments of Ophthalmology (R.A., S.M., S.M.C.), Radiology (S.W.), Neurology (S.M., S.M.C.), and Physiology & Pharmacology (S.M., S.M.C.), SUNY Downstate Health Sciences University, 450 Clarkson Ave, Brooklyn, NY 11203; Department of Radiology, Penn State Milton S. Hershey Medical Center, Hershey, Pa (M.A.B.); Department of Radiology and Imaging Sciences, Emory University, Atlanta, Ga (E.A.K.); and Department of Radiology, Rush University Medical College and University of Illinois, Chicago, Ill (L.B.)
| | - Stephen Waite
- From the Departments of Ophthalmology (R.A., S.M., S.M.C.), Radiology (S.W.), Neurology (S.M., S.M.C.), and Physiology & Pharmacology (S.M., S.M.C.), SUNY Downstate Health Sciences University, 450 Clarkson Ave, Brooklyn, NY 11203; Department of Radiology, Penn State Milton S. Hershey Medical Center, Hershey, Pa (M.A.B.); Department of Radiology and Imaging Sciences, Emory University, Atlanta, Ga (E.A.K.); and Department of Radiology, Rush University Medical College and University of Illinois, Chicago, Ill (L.B.)
| | - Michael A Bruno
- From the Departments of Ophthalmology (R.A., S.M., S.M.C.), Radiology (S.W.), Neurology (S.M., S.M.C.), and Physiology & Pharmacology (S.M., S.M.C.), SUNY Downstate Health Sciences University, 450 Clarkson Ave, Brooklyn, NY 11203; Department of Radiology, Penn State Milton S. Hershey Medical Center, Hershey, Pa (M.A.B.); Department of Radiology and Imaging Sciences, Emory University, Atlanta, Ga (E.A.K.); and Department of Radiology, Rush University Medical College and University of Illinois, Chicago, Ill (L.B.)
| | - Elizabeth A Krupinski
- From the Departments of Ophthalmology (R.A., S.M., S.M.C.), Radiology (S.W.), Neurology (S.M., S.M.C.), and Physiology & Pharmacology (S.M., S.M.C.), SUNY Downstate Health Sciences University, 450 Clarkson Ave, Brooklyn, NY 11203; Department of Radiology, Penn State Milton S. Hershey Medical Center, Hershey, Pa (M.A.B.); Department of Radiology and Imaging Sciences, Emory University, Atlanta, Ga (E.A.K.); and Department of Radiology, Rush University Medical College and University of Illinois, Chicago, Ill (L.B.)
| | - Leonard Berlin
- From the Departments of Ophthalmology (R.A., S.M., S.M.C.), Radiology (S.W.), Neurology (S.M., S.M.C.), and Physiology & Pharmacology (S.M., S.M.C.), SUNY Downstate Health Sciences University, 450 Clarkson Ave, Brooklyn, NY 11203; Department of Radiology, Penn State Milton S. Hershey Medical Center, Hershey, Pa (M.A.B.); Department of Radiology and Imaging Sciences, Emory University, Atlanta, Ga (E.A.K.); and Department of Radiology, Rush University Medical College and University of Illinois, Chicago, Ill (L.B.)
| | - Stephen Macknik
- From the Departments of Ophthalmology (R.A., S.M., S.M.C.), Radiology (S.W.), Neurology (S.M., S.M.C.), and Physiology & Pharmacology (S.M., S.M.C.), SUNY Downstate Health Sciences University, 450 Clarkson Ave, Brooklyn, NY 11203; Department of Radiology, Penn State Milton S. Hershey Medical Center, Hershey, Pa (M.A.B.); Department of Radiology and Imaging Sciences, Emory University, Atlanta, Ga (E.A.K.); and Department of Radiology, Rush University Medical College and University of Illinois, Chicago, Ill (L.B.)
| | - Susana Martinez-Conde
- From the Departments of Ophthalmology (R.A., S.M., S.M.C.), Radiology (S.W.), Neurology (S.M., S.M.C.), and Physiology & Pharmacology (S.M., S.M.C.), SUNY Downstate Health Sciences University, 450 Clarkson Ave, Brooklyn, NY 11203; Department of Radiology, Penn State Milton S. Hershey Medical Center, Hershey, Pa (M.A.B.); Department of Radiology and Imaging Sciences, Emory University, Atlanta, Ga (E.A.K.); and Department of Radiology, Rush University Medical College and University of Illinois, Chicago, Ill (L.B.)
| |
Collapse
|
13
|
The Role of Intuitive Cognition in Radiologic Decision Making. J Am Coll Radiol 2022; 19:669-676. [DOI: 10.1016/j.jacr.2022.02.027] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/22/2022] [Accepted: 02/19/2022] [Indexed: 11/19/2022]
|
14
|
Banzato T, Wodzinski M, Tauceri F, Donà C, Scavazza F, Müller H, Zotti A. An AI-Based Algorithm for the Automatic Classification of Thoracic Radiographs in Cats. Front Vet Sci 2021; 8:731936. [PMID: 34722699 PMCID: PMC8554083 DOI: 10.3389/fvets.2021.731936] [Citation(s) in RCA: 12] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/28/2021] [Accepted: 09/21/2021] [Indexed: 01/31/2023] Open
Abstract
An artificial intelligence (AI)-based computer-aided detection (CAD) algorithm to detect some of the most common radiographic findings in the feline thorax was developed and tested. The database used for training comprised radiographs acquired at two different institutions. Only correctly exposed and positioned radiographs were included in the database used for training. The presence of several radiographic findings was recorded. Consequenly, the radiographic findings included for training were: no findings, bronchial pattern, pleural effusion, mass, alveolar pattern, pneumothorax, cardiomegaly. Multi-label convolutional neural networks (CNNs) were used to develop the CAD algorithm, and the performance of two different CNN architectures, ResNet 50 and Inception V3, was compared. Both architectures had an area under the receiver operating characteristic curve (AUC) above 0.9 for alveolar pattern, bronchial pattern and pleural effusion, an AUC above 0.8 for no findings and pneumothorax, and an AUC above 0.7 for cardiomegaly. The AUC for mass was low (above 0.5) for both architectures. No significant differences were evident in the diagnostic accuracy of either architecture.
Collapse
Affiliation(s)
- Tommaso Banzato
- Department of Animal Medicine, Production and Health, University of Padua, Legnaro, Italy
| | - Marek Wodzinski
- Department of Measurement and Electronics, AGH University of Science and Technology, Krakow, Poland.,Information Systems Institute, University of Applied Sciences - Western Switzerland (HES-SO Valais), Sierre, Switzerland
| | - Federico Tauceri
- Department of Animal Medicine, Production and Health, University of Padua, Legnaro, Italy
| | - Chiara Donà
- Department of Animal Medicine, Production and Health, University of Padua, Legnaro, Italy
| | - Filippo Scavazza
- Department of Animal Medicine, Production and Health, University of Padua, Legnaro, Italy
| | - Henning Müller
- Information Systems Institute, University of Applied Sciences - Western Switzerland (HES-SO Valais), Sierre, Switzerland
| | - Alessandro Zotti
- Department of Animal Medicine, Production and Health, University of Padua, Legnaro, Italy
| |
Collapse
|
15
|
Kliewer MA, Bagley AR. How to Read an Abdominal CT: Insights from the Visual and Cognitive Sciences Translated for Clinical Practice. Curr Probl Diagn Radiol 2021; 51:639-647. [PMID: 34583872 DOI: 10.1067/j.cpradiol.2021.07.006] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/11/2021] [Revised: 07/01/2021] [Accepted: 07/18/2021] [Indexed: 11/22/2022]
Abstract
When first learning abdominal CT studies, residents are often given little concrete, practical direction. There is, however, a large literature from the visual and cognitive sciences that can provide guidance towards search strategies that maximize efficiency and comprehensiveness. This literature has not penetrated radiology teaching to any great extent. In this article, we will examine the current pedagogy (and why that falls short), why untutored search fails, where misses occur in abdomen/pelvis CT, why these misses occur where they do, how expert radiologists search 3d image stacks, and how novices might expedite the acquisition of expertise.
Collapse
Affiliation(s)
- Mark A Kliewer
- Department of Radiology, University of Wisconsin - Madison, Madison, Wisconsin
| | - Anjuli R Bagley
- Radiology, The University of Colorado - Denver, Department of Radiology, Aurora, CO, USA, University of Colorado Hospital (UCH), Aurora, Colorado
| |
Collapse
|
16
|
Wang Y, Jin C, Yin Z, Wang H, Ji M, Dong M, Liang J. Visual experience modulates whole-brain connectivity dynamics: A resting-state fMRI study using the model of radiologists. Hum Brain Mapp 2021; 42:4538-4554. [PMID: 34156138 PMCID: PMC8410580 DOI: 10.1002/hbm.25563] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/15/2020] [Revised: 04/18/2021] [Accepted: 06/02/2021] [Indexed: 01/01/2023] Open
Abstract
Visual expertise refers to proficiency in visual recognition. It is attributed to accumulated visual experience in a specific domain and manifests in widespread neural activities that extend well beyond the visual cortex to multiple high‐level brain areas. An extensive body of studies has centered on the neural mechanisms underlying a distinctive domain of visual expertise, while few studies elucidated how visual experience modulates resting‐state whole‐brain connectivity dynamics. The current study bridged this gap by modeling the subtle alterations in interregional spontaneous connectivity patterns with a group of superior radiological interns. Functional connectivity analysis was based on functional brain segmentation, which was derived from a data‐driven clustering approach to discriminate subtle changes in connectivity dynamics. Our results showed there was radiographic visual experience accompanied with integration within brain circuits supporting visual processing and decision making, integration across brain circuits supporting high‐order functions, and segregation between high‐order and low‐order brain functions. Also, most of these alterations were significantly correlated with individual nodule identification performance. Our results implied that visual expertise is a controlled, interactive process that develops from reciprocal interactions between the visual system and multiple top‐down factors, including semantic knowledge, top‐down attentional control, and task relevance, which may enhance participants' local brain functional integration to promote their acquisition of specific visual information and modulate the activity of some regions for lower‐order visual feature processing to filter out nonrelevant visual details. The current findings may provide new ideas for understanding the central mechanism underlying the formation of visual expertise.
Collapse
Affiliation(s)
- Yue Wang
- School of Electronic Engineering, Xidian University, Shaanxi, China
| | - Chenwang Jin
- Department of Medical Imaging, First Affiliated Hospital of Medical College, Xi'an Jiaotong University, Shaanxi, China
| | - Zhongliang Yin
- Engineering Research Center of Molecular and Neuro Imaging of Ministry of Education, School of Life Science and Technology, Xidian University, Shaanxi, China
| | - Hongmei Wang
- Department of Medical Imaging, First Affiliated Hospital of Medical College, Xi'an Jiaotong University, Shaanxi, China
| | - Ming Ji
- School of Psychology, Shaanxi Normal University, Shaanxi, China
| | - Minghao Dong
- Engineering Research Center of Molecular and Neuro Imaging of Ministry of Education, School of Life Science and Technology, Xidian University, Shaanxi, China
| | - Jimin Liang
- School of Electronic Engineering, Xidian University, Shaanxi, China
| |
Collapse
|
17
|
Alexander RG, Yazdanie F, Waite S, Chaudhry ZA, Kolla S, Macknik SL, Martinez-Conde S. Visual Illusions in Radiology: Untrue Perceptions in Medical Images and Their Implications for Diagnostic Accuracy. Front Neurosci 2021; 15:629469. [PMID: 34177444 PMCID: PMC8226024 DOI: 10.3389/fnins.2021.629469] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/17/2020] [Accepted: 04/19/2021] [Indexed: 11/13/2022] Open
Abstract
Errors in radiologic interpretation are largely the result of failures of perception. This remains true despite the increasing use of computer-aided detection and diagnosis. We surveyed the literature on visual illusions during the viewing of radiologic images. Misperception of anatomical structures is a potential cause of error that can lead to patient harm if disease is seen when none is present. However, visual illusions can also help enhance the ability of radiologists to detect and characterize abnormalities. Indeed, radiologists have learned to exploit certain perceptual biases in diagnostic findings and as training tools. We propose that further detailed study of radiologic illusions would help clarify the mechanisms underlying radiologic performance and provide additional heuristics to improve radiologist training and reduce medical error.
Collapse
Affiliation(s)
- Robert G Alexander
- Department of Ophthalmology, State University of New York Downstate Health Sciences University, Brooklyn, NY, United States.,Department of Neurology, State University of New York Downstate Health Sciences University, Brooklyn, NY, United States.,Department of Physiology and Pharmacology, State University of New York Downstate Health Sciences University, Brooklyn, NY, United States
| | - Fahd Yazdanie
- Department of Radiology, State University of New York Downstate Health Sciences University, Brooklyn, NY, United States
| | - Stephen Waite
- Department of Radiology, State University of New York Downstate Health Sciences University, Brooklyn, NY, United States
| | - Zeshan A Chaudhry
- Department of Radiology, State University of New York Downstate Health Sciences University, Brooklyn, NY, United States
| | - Srinivas Kolla
- Department of Radiology, State University of New York Downstate Health Sciences University, Brooklyn, NY, United States
| | - Stephen L Macknik
- Department of Ophthalmology, State University of New York Downstate Health Sciences University, Brooklyn, NY, United States.,Department of Neurology, State University of New York Downstate Health Sciences University, Brooklyn, NY, United States.,Department of Physiology and Pharmacology, State University of New York Downstate Health Sciences University, Brooklyn, NY, United States
| | - Susana Martinez-Conde
- Department of Ophthalmology, State University of New York Downstate Health Sciences University, Brooklyn, NY, United States.,Department of Neurology, State University of New York Downstate Health Sciences University, Brooklyn, NY, United States.,Department of Physiology and Pharmacology, State University of New York Downstate Health Sciences University, Brooklyn, NY, United States
| |
Collapse
|
18
|
Diurnal variation of major error rates in the interpretation of abdominal/pelvic CT studies. Abdom Radiol (NY) 2021; 46:1746-1751. [PMID: 33040173 DOI: 10.1007/s00261-020-02807-w] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/21/2020] [Revised: 09/21/2020] [Accepted: 09/30/2020] [Indexed: 10/23/2022]
Abstract
RATIONALE AND OBJECTIVE Variation of visual selective attention through the day has been demonstrated in several arenas of human performance, including radiology. It is uncertain whether this variation translates to an identifiable diurnal pattern of error rates for radiology interpretation. The purpose of this study was to attempt to identify particular days of the week and times of the day when radiologists might be most prone to error. MATERIALS AND METHODS Abdomen/pelvis CT studies containing at least one major error were collected from a 10-year period from the quality assurance (QA) database at our institution. A major error was defined as a missed finding that had altered management in a way potentially detrimental to the patient. The identified studies were categorized by the day of the week and hour of the day that the study was interpreted. Study volume data over this same period was also obtained by day of the week and time of day, so to normalize the data based on case volume. Standard errors of the volume-adjusted error rates were obtained based on the binomial distribution. The null hypothesis of constant error rates over time was tested using a weighted logistic regression model with linear time as predictor. RESULTS A total of 252 major errors were identified. More errors were made on Monday than on any other day of the week (n = 58). Major error rates increased through the mid to late morning (9 am to 12 pm), and then decreased progressively through the afternoon until 4 pm, when a rise in the error rate was seen. This pattern persisted when error rates were normalized by study volume within each hour. Overall tests of time-constancy of error rates by day and hour were statistically significant (both p-values < 0.001). CONCLUSION Our study shows that error rates in abdominal CT do seem to vary with time of day and day of the week. During the workweek, error rates were highest in the late morning and at the close of the workday, and greater on Mondays than other days.
Collapse
|
19
|
Kliewer MA, Hartung M, Green CS. The Search Patterns of Abdominal Imaging Subspecialists for Abdominal Computed Tomography: Toward a Foundational Pattern for New Radiology Residents. J Clin Imaging Sci 2021; 11:1. [PMID: 33500836 PMCID: PMC7827582 DOI: 10.25259/jcis_195_2020] [Citation(s) in RCA: 6] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/20/2020] [Accepted: 12/09/2020] [Indexed: 11/04/2022] Open
Abstract
Objectives: The routine search patterns used by subspecialty abdominal imaging experts to inspect the image volumes of abdominal/pelvic computed tomography (CT) have not been well characterized or rendered in practical or teachable terms. The goal of this study is to describe the search patterns used by experienced subspecialty imagers when reading a normal abdominal CT at a modern picture archiving and communication system workstation, and utilize this information to propose guidelines for residents as they learn to interpret CT during training. Material and Methods: Twenty-two academic subspecialists enacted their routine search pattern on a normal contrast-enhanced abdominal/pelvic CT study under standardized display parameters. Readers were told that the scan was normal and then asked to verbalize where their gaze centered and moved through the axial, coronal, and sagittal image stacks, demonstrating eye position with a cursor as needed. A peer coded the reported eye gaze movements and scrilling behavior. Spearman correlation coefficients were calculated between years of professional experience and the numbers of passes through the lung bases, liver, kidneys, and bowel. Results: All readers followed an initial organ-by-organ approach. Larger organs were examined by drilling, while smaller organs by oscillation or scanning. Search elements were classified as drilling, scanning, oscillation, and scrilling (scan drilling); these categories were parsed as necessary. The greatest variability was found in the examination the body wall and bowel/mesentery. Two modes of scrilling were described, and these classified as roaming and zigzagging. The years of experience of the readers did not correlated to number of passes made through the lung bases, liver, kidneys, or bowel. Conclusion: Subspecialty abdominal radiologists negotiate through the image stacks of an abdominal CT study in broadly similar ways. Collation of the approaches suggests a foundational search pattern for new trainees.
Collapse
Affiliation(s)
- Mark A Kliewer
- Department of Radiology and Ultrasound Imaging, University of Wisconsin School of Medicine and Public Health, Madison, Wisconsin, United States
| | - Michael Hartung
- Department of Radiology and Ultrasound Imaging, University of Wisconsin School of Medicine and Public Health, Madison, Wisconsin, United States
| | - C Shawn Green
- Department of Psychology, University of Wisconsin School of Medicine and Public Health, Madison, Wisconsin, United States
| |
Collapse
|
20
|
Mondal SB, Achilefu S. Virtual and Augmented Reality Technologies in Molecular and Anatomical Imaging. Mol Imaging 2021. [DOI: 10.1016/b978-0-12-816386-3.00066-1] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/16/2022] Open
|
21
|
Li Y, Cao H, Allen CM, Wang X, Erdelez S, Shyu CR. Computational modeling of human reasoning processes for interpretable visual knowledge: a case study with radiographers. Sci Rep 2020; 10:21620. [PMID: 33303770 PMCID: PMC7730148 DOI: 10.1038/s41598-020-77550-9] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/27/2020] [Accepted: 11/10/2020] [Indexed: 11/18/2022] Open
Abstract
Visual reasoning is critical in many complex visual tasks in medicine such as radiology or pathology. It is challenging to explicitly explain reasoning processes due to the dynamic nature of real-time human cognition. A deeper understanding of such reasoning processes is necessary for improving diagnostic accuracy and computational tools. Most computational analysis methods for visual attention utilize black-box algorithms which lack explainability and are therefore limited in understanding the visual reasoning processes. In this paper, we propose a computational method to quantify and dissect visual reasoning. The method characterizes spatial and temporal features and identifies common and contrast visual reasoning patterns to extract significant gaze activities. The visual reasoning patterns are explainable and can be compared among different groups to discover strategy differences. Experiments with radiographers of varied levels of expertise on 10 levels of visual tasks were conducted. Our empirical observations show that the method can capture the temporal and spatial features of human visual attention and distinguish expertise level. The extracted patterns are further examined and interpreted to showcase key differences between expertise levels in the visual reasoning processes. By revealing task-related reasoning processes, this method demonstrates potential for explaining human visual understanding.
Collapse
Affiliation(s)
- Yu Li
- Department of Electrical Engineering and Computer Science, University of Missouri, Columbia, MO, 65211, USA
| | - Hongfei Cao
- Department of Electrical Engineering and Computer Science, University of Missouri, Columbia, MO, 65211, USA
| | - Carla M Allen
- Department of Clinical and Diagnostic Science, University of Missouri, Columbia, MO, 65211, USA
| | - Xin Wang
- Department of Information Science, University of Northern Texas, Denton, TX, 76203, USA
| | - Sanda Erdelez
- School of Library and Information Science, Simmons University, Boston, MA, 02115, USA
| | - Chi-Ren Shyu
- Department of Electrical Engineering and Computer Science, University of Missouri, Columbia, MO, 65211, USA.
- Institute for Data Science and Informatics, University of Missouri, Columbia, MO, 65211, USA.
| |
Collapse
|
22
|
Kliewer MA, Brinkman MR, Hinshaw JL. The Back Alleys and Dark Corners of Abdomen and Pelvis Computed Tomography: The Most Frequent Sites of Missed Findings in the Multiplanar Era. J Clin Imaging Sci 2020; 10:70. [PMID: 33194312 PMCID: PMC7656035 DOI: 10.25259/jcis_184_2020] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/12/2020] [Accepted: 10/18/2020] [Indexed: 11/04/2022] Open
Abstract
Objectives Radiologists reading multiplanar abdominal/pelvic computed tomography (CT) are vulnerable to oversight of specific anatomic areas, leading to perceptual errors (misses). The aims of this study are to identify common sites of major perceptual error at our institution and then to put these in context with earlier studies to produce a comprehensive overview. Material and Methods We reviewed our quality assurance database over an 8-year period for cases of major perceptual error on CT examinations of the abdomen and pelvis. A major perceptual error was defined as a missed finding that had altered management in a way potentially detrimental to the patient. Record was made of patient age, gender, study indication, study priority (stat/routine), and use of IV and/or oral contrast. Anatomic locations were subdivided as lung bases, liver, pancreas, kidneys, spleen, mesentery, peritoneum, retroperitoneum, small bowel, colon, appendix, vasculature, body wall, and bones. Results A total of 216 missed findings were identified in 201 patients. The most common indication for the study was cancer follow-up (71%) followed by infection (11%) and abdominal pain (6%). The most common anatomic regions of error were the liver (15%), peritoneum (10%), body wall (9%), retroperitoneum (8%), and mesentery (6%). Data from other studies were reorganized into congruent categories for comparison. Conclusion This study demonstrates that the most common sites of significant missed findings on multiplanar abdominal/pelvic CT included the mesentery, peritoneum, body wall, bowel, vasculature, and the liver in the arterial phase. Data from other similar studies were reorganized into congruent categories to provide a comprehensive overview.
Collapse
Affiliation(s)
- Mark A Kliewer
- Department of Radiology, University of Wisconsin School of Medicine and Public Health, Madison, Wisconsin
| | - Mikala R Brinkman
- Department of Radiology, Central Illinois Radiological Associates, Peoria, Illinois
| | - J Louis Hinshaw
- Department of Radiology and Urology, University of Wisconsin School of Medicine and Public Health, Madison, Wisconsin, United States
| |
Collapse
|
23
|
Artificial Intelligence and the Trainee Experience in Radiology. J Am Coll Radiol 2020; 17:1388-1393. [DOI: 10.1016/j.jacr.2020.09.028] [Citation(s) in RCA: 9] [Impact Index Per Article: 2.3] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/17/2020] [Revised: 08/26/2020] [Accepted: 09/01/2020] [Indexed: 12/23/2022]
|
24
|
Alexander RG, Waite S, Macknik SL, Martinez-Conde S. What do radiologists look for? Advances and limitations of perceptual learning in radiologic search. J Vis 2020; 20:17. [PMID: 33057623 PMCID: PMC7571277 DOI: 10.1167/jov.20.10.17] [Citation(s) in RCA: 9] [Impact Index Per Article: 2.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/21/2019] [Accepted: 09/14/2020] [Indexed: 12/31/2022] Open
Abstract
Supported by guidance from training during residency programs, radiologists learn clinically relevant visual features by viewing thousands of medical images. Yet the precise visual features that expert radiologists use in their clinical practice remain unknown. Identifying such features would allow the development of perceptual learning training methods targeted to the optimization of radiology training and the reduction of medical error. Here we review attempts to bridge current gaps in understanding with a focus on computational saliency models that characterize and predict gaze behavior in radiologists. There have been great strides toward the accurate prediction of relevant medical information within images, thereby facilitating the development of novel computer-aided detection and diagnostic tools. In some cases, computational models have achieved equivalent sensitivity to that of radiologists, suggesting that we may be close to identifying the underlying visual representations that radiologists use. However, because the relevant bottom-up features vary across task context and imaging modalities, it will also be necessary to identify relevant top-down factors before perceptual expertise in radiology can be fully understood. Progress along these dimensions will improve the tools available for educating new generations of radiologists, and aid in the detection of medically relevant information, ultimately improving patient health.
Collapse
Affiliation(s)
- Robert G Alexander
- Department of Ophthalmology, SUNY Downstate Health Sciences University, Brooklyn, NY, USA
| | - Stephen Waite
- Department of Radiology, SUNY Downstate Health Sciences University, Brooklyn, NY, USA
| | - Stephen L Macknik
- Department of Ophthalmology, SUNY Downstate Health Sciences University, Brooklyn, NY, USA
| | - Susana Martinez-Conde
- Department of Ophthalmology, SUNY Downstate Health Sciences University, Brooklyn, NY, USA
| |
Collapse
|
25
|
Abstract
Humans are perceptual experts and we are constantly refining how we detect and discriminate objects in the world around us, often without any explicit instruction. But instruction can be helpful and sometimes even necessary. New research highlights the importance of instruction for achieving robust long-term retention of learning to identify complex features in natural images such as those used in radiology.
Collapse
Affiliation(s)
- Aaron R Seitz
- Department of Psychology, University of California Riverside, Riverside, CA 92521, USA.
| |
Collapse
|
26
|
Johnston IA, Ji M, Cochrane A, Demko Z, Robbins JB, Stephenson JW, Green CS. Perceptual Learning of Appendicitis Diagnosis in Radiological Images. J Vis 2020; 20:16. [PMID: 32790849 PMCID: PMC7438669 DOI: 10.1167/jov.20.8.16] [Citation(s) in RCA: 6] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/24/2022] Open
Abstract
A sizeable body of work has demonstrated that participants have the capacity to show substantial increases in performance on perceptual tasks given appropriate practice. This has resulted in significant interest in the use of such perceptual learning techniques to positively impact performance in real-world domains where the extraction of perceptual information in the service of guiding decisions is at a premium. Radiological training is one clear example of such a domain. Here we examine a number of basic science questions related to the use of perceptual learning techniques in the context of a radiology-inspired task. On each trial of this task, participants were presented with a single axial slice from a CT image of the abdomen. They were then asked to indicate whether or not the image was consistent with appendicitis. We first demonstrate that, although the task differs in many ways from standard radiological practice, it nonetheless makes use of expert knowledge, as trained radiologists who underwent the task showed high (near ceiling) levels of performance. Then, in a series of four studies we show that (1) performance on this task does improve significantly over a reasonably short period of training (on the scale of a few hours); (2) the learning transfers to previously unseen images and to untrained image orientations; (3) purely correct/incorrect feedback produces weak learning compared to more informative feedback where the spatial position of the appendix is indicated in each image; and (4) there was little benefit seen from purposefully structuring the learning experience by starting with easier images and then moving on to more difficulty images (as compared to simply presenting all images in a random order). The implications for these various findings with respect to the use of perceptual learning techniques as part of radiological training are then discussed.
Collapse
Affiliation(s)
| | - Mohan Ji
- Department of Psychology, University of Wisconsin-Madison, Madison, WI, USA
| | - Aaron Cochrane
- Department of Psychology, University of Wisconsin-Madison, Madison, WI, USA
| | - Zachary Demko
- Department of Psychology, University of Wisconsin-Madison, Madison, WI, USA
| | - Jessica B Robbins
- Department of Radiology, University of Wisconsin School of Medicine and Public Health, Madison, WI, USA
| | - Jason W Stephenson
- Department of Radiology, University of Wisconsin School of Medicine and Public Health, Madison, WI, USA
| | - C Shawn Green
- Department of Psychology, University of Wisconsin-Madison, Madison, WI, USA
| |
Collapse
|
27
|
Jha S, Cook T. Artificial Intelligence in Radiology--The State of the Future. Acad Radiol 2020; 27:1-2. [PMID: 31753720 DOI: 10.1016/j.acra.2019.11.003] [Citation(s) in RCA: 7] [Impact Index Per Article: 1.8] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/10/2019] [Accepted: 11/10/2019] [Indexed: 12/18/2022]
Affiliation(s)
- Saurabh Jha
- Department of Radiology, MRI Learning Center Hospital, University of Pennsylvania, 3400 Spruce Street, Phila, PA 19104, United States.
| | - Tessa Cook
- Department of Radiology, MRI Learning Center Hospital, University of Pennsylvania, 3400 Spruce Street, Phila, PA 19104, United States
| |
Collapse
|