1
|
Darici D, Reissner C, Missler M. Webcam-based eye-tracking to measure visual expertise of medical students during online histology training. GMS JOURNAL FOR MEDICAL EDUCATION 2023; 40:Doc60. [PMID: 37881524 PMCID: PMC10594038 DOI: 10.3205/zma001642] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Figures] [Subscribe] [Scholar Register] [Received: 09/30/2022] [Revised: 06/06/2023] [Accepted: 07/07/2023] [Indexed: 10/27/2023]
Abstract
Objectives Visual expertise is essential for image-based tasks that rely on visual cues, such as in radiology or histology. Studies suggest that eye movements are related to visual expertise and can be measured by near-infrared eye-tracking. With the popularity of device-embedded webcam eye-tracking technology, cost-effective use in educational contexts has recently become amenable. This study investigated the feasibility of such methodology in a curricular online-only histology course during the 2021 summer term. Methods At two timepoints (t1 and t2), third-semester medical students were asked to diagnose a series of histological slides while their eye movements were recorded. Students' eye metrics, performance and behavioral measures were analyzed using variance analyses and multiple regression models. Results First, webcam-eye tracking provided eye movement data with satisfactory quality (mean accuracy=115.7 px±31.1). Second, the eye movement metrics reflected the students' proficiency in finding relevant image sections (fixation count on relevant areas=6.96±1.56 vs. irrelevant areas=4.50±1.25). Third, students' eye movement metrics successfully predicted their performance (R2adj=0.39, p<0.001). Conclusion This study supports the use of webcam-eye-tracking expanding the range of educational tools available in the (digital) classroom. As the students' interest in using the webcam eye-tracking was high, possible areas of implementation will be discussed.
Collapse
Affiliation(s)
- Dogus Darici
- Westfälische-Wilhelms-University, Institute of Anatomy and Neurobiology, Münster, Germany
| | - Carsten Reissner
- Westfälische-Wilhelms-University, Institute of Anatomy and Neurobiology, Münster, Germany
| | - Markus Missler
- Westfälische-Wilhelms-University, Institute of Anatomy and Neurobiology, Münster, Germany
| |
Collapse
|
2
|
Alexander R, Waite S, Bruno MA, Krupinski EA, Berlin L, Macknik S, Martinez-Conde S. Mandating Limits on Workload, Duty, and Speed in Radiology. Radiology 2022; 304:274-282. [PMID: 35699581 PMCID: PMC9340237 DOI: 10.1148/radiol.212631] [Citation(s) in RCA: 27] [Impact Index Per Article: 13.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/25/2023]
Abstract
Research has not yet quantified the effects of workload or duty hours on the accuracy of radiologists. With the exception of a brief reduction in imaging studies during the 2020 peak of the COVID-19 pandemic, the workload of radiologists in the United States has seen relentless growth in recent years. One concern is that this increased demand could lead to reduced accuracy. Behavioral studies in species ranging from insects to humans have shown that decision speed is inversely correlated to decision accuracy. A potential solution is to institute workload and duty limits to optimize radiologist performance and patient safety. The concern, however, is that any prescribed mandated limits would be arbitrary and thus no more advantageous than allowing radiologists to self-regulate. Specific studies have been proposed to determine whether limits reduce error, and if so, to provide a principled basis for such limits. This could determine the precise susceptibility of individual radiologists to medical error as a function of speed during image viewing, the maximum number of studies that could be read during a work shift, and the appropriate shift duration as a function of time of day. Before principled recommendations for restrictions are made, however, it is important to understand how radiologists function both optimally and at the margins of adequate performance. This study examines the relationship between interpretation speed and error rates in radiology, the potential influence of artificial intelligence on reading speed and error rates, and the possible outcomes of imposed limits on both caseload and duty hours. This review concludes that the scientific evidence needed to make meaningful rules is lacking and notes that regulating workloads without scientific principles can be more harmful than not regulating at all.
Collapse
Affiliation(s)
- Robert Alexander
- From the Departments of Ophthalmology (R.A., S.M., S.M.C.), Radiology (S.W.), Neurology (S.M., S.M.C.), and Physiology & Pharmacology (S.M., S.M.C.), SUNY Downstate Health Sciences University, 450 Clarkson Ave, Brooklyn, NY 11203; Department of Radiology, Penn State Milton S. Hershey Medical Center, Hershey, Pa (M.A.B.); Department of Radiology and Imaging Sciences, Emory University, Atlanta, Ga (E.A.K.); and Department of Radiology, Rush University Medical College and University of Illinois, Chicago, Ill (L.B.)
| | - Stephen Waite
- From the Departments of Ophthalmology (R.A., S.M., S.M.C.), Radiology (S.W.), Neurology (S.M., S.M.C.), and Physiology & Pharmacology (S.M., S.M.C.), SUNY Downstate Health Sciences University, 450 Clarkson Ave, Brooklyn, NY 11203; Department of Radiology, Penn State Milton S. Hershey Medical Center, Hershey, Pa (M.A.B.); Department of Radiology and Imaging Sciences, Emory University, Atlanta, Ga (E.A.K.); and Department of Radiology, Rush University Medical College and University of Illinois, Chicago, Ill (L.B.)
| | - Michael A Bruno
- From the Departments of Ophthalmology (R.A., S.M., S.M.C.), Radiology (S.W.), Neurology (S.M., S.M.C.), and Physiology & Pharmacology (S.M., S.M.C.), SUNY Downstate Health Sciences University, 450 Clarkson Ave, Brooklyn, NY 11203; Department of Radiology, Penn State Milton S. Hershey Medical Center, Hershey, Pa (M.A.B.); Department of Radiology and Imaging Sciences, Emory University, Atlanta, Ga (E.A.K.); and Department of Radiology, Rush University Medical College and University of Illinois, Chicago, Ill (L.B.)
| | - Elizabeth A Krupinski
- From the Departments of Ophthalmology (R.A., S.M., S.M.C.), Radiology (S.W.), Neurology (S.M., S.M.C.), and Physiology & Pharmacology (S.M., S.M.C.), SUNY Downstate Health Sciences University, 450 Clarkson Ave, Brooklyn, NY 11203; Department of Radiology, Penn State Milton S. Hershey Medical Center, Hershey, Pa (M.A.B.); Department of Radiology and Imaging Sciences, Emory University, Atlanta, Ga (E.A.K.); and Department of Radiology, Rush University Medical College and University of Illinois, Chicago, Ill (L.B.)
| | - Leonard Berlin
- From the Departments of Ophthalmology (R.A., S.M., S.M.C.), Radiology (S.W.), Neurology (S.M., S.M.C.), and Physiology & Pharmacology (S.M., S.M.C.), SUNY Downstate Health Sciences University, 450 Clarkson Ave, Brooklyn, NY 11203; Department of Radiology, Penn State Milton S. Hershey Medical Center, Hershey, Pa (M.A.B.); Department of Radiology and Imaging Sciences, Emory University, Atlanta, Ga (E.A.K.); and Department of Radiology, Rush University Medical College and University of Illinois, Chicago, Ill (L.B.)
| | - Stephen Macknik
- From the Departments of Ophthalmology (R.A., S.M., S.M.C.), Radiology (S.W.), Neurology (S.M., S.M.C.), and Physiology & Pharmacology (S.M., S.M.C.), SUNY Downstate Health Sciences University, 450 Clarkson Ave, Brooklyn, NY 11203; Department of Radiology, Penn State Milton S. Hershey Medical Center, Hershey, Pa (M.A.B.); Department of Radiology and Imaging Sciences, Emory University, Atlanta, Ga (E.A.K.); and Department of Radiology, Rush University Medical College and University of Illinois, Chicago, Ill (L.B.)
| | - Susana Martinez-Conde
- From the Departments of Ophthalmology (R.A., S.M., S.M.C.), Radiology (S.W.), Neurology (S.M., S.M.C.), and Physiology & Pharmacology (S.M., S.M.C.), SUNY Downstate Health Sciences University, 450 Clarkson Ave, Brooklyn, NY 11203; Department of Radiology, Penn State Milton S. Hershey Medical Center, Hershey, Pa (M.A.B.); Department of Radiology and Imaging Sciences, Emory University, Atlanta, Ga (E.A.K.); and Department of Radiology, Rush University Medical College and University of Illinois, Chicago, Ill (L.B.)
| |
Collapse
|
3
|
Drew T, Lavelle M, Kerr KF, Shucard H, Brunyé TT, Weaver DL, Elmore JG. More scanning, but not zooming, is associated with diagnostic accuracy in evaluating digital breast pathology slides. J Vis 2021; 21:7. [PMID: 34636845 PMCID: PMC8525842 DOI: 10.1167/jov.21.11.7] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/03/2021] [Accepted: 09/15/2021] [Indexed: 12/02/2022] Open
Abstract
Diagnoses of medical images can invite strikingly diverse strategies for image navigation and visual search. In computed tomography screening for lung nodules, distinct strategies, termed scanning and drilling, relate to both radiologists' clinical experience and accuracy in lesion detection. Here, we examined associations between search patterns and accuracy for pathologists (N = 92) interpreting a diverse set of breast biopsy images. While changes in depth in volumetric images reveal new structures through movement in the z-plane, in digital pathology changes in depth are associated with increased magnification. Thus, "drilling" in radiology may be more appropriately termed "zooming" in pathology. We monitored eye-movements and navigation through digital pathology slides to derive metrics of how quickly the pathologists moved through XY (scanning) and Z (zooming) space. Prior research on eye-movements in depth has categorized clinicians as either "scanners" or "drillers." In contrast, we found that there was no reliable association between a clinician's tendency to scan or zoom while examining digital pathology slides. Thus, in the current work we treated scanning and zooming as continuous predictors rather than categorizing as either a "scanner" or "zoomer." In contrast to prior work in volumetric chest images, we found significant associations between accuracy and scanning rate but not zooming rate. These findings suggest fundamental differences in the relative value of information types and review behaviors across two image formats. Our data suggest that pathologists gather critical information by scanning on a given plane of depth, whereas radiologists drill through depth to interrogate critical features.
Collapse
Affiliation(s)
- Trafton Drew
- Department of Psychology, University of Utah, Salt Lake City, UT, USA
| | - Mark Lavelle
- Department of Psychology, University of Utah, Salt Lake City, UT, USA
| | - Kathleen F Kerr
- Department of Biostatistics, University of Washington, Seattle, WA, USA
| | - Hannah Shucard
- Department of Biostatistics, University of Washington, Seattle, WA, USA
| | - Tad T Brunyé
- Department of Psychology, Tufts University, Medford, MA, USA
| | - Donald L Weaver
- Department of Pathology & Laboratory Medicine, University of Vermont, Burlington, VT, USA
| | - Joann G Elmore
- Department of Medicine, David Geffen School of Medicine at UCLA, Los Angeles, CA, USA
| |
Collapse
|
4
|
Williams LH, Carrigan AJ, Mills M, Auffermann WF, Rich AN, Drew T. Characteristics of expert search behavior in volumetric medical image interpretation. J Med Imaging (Bellingham) 2021; 8:041208. [PMID: 34277889 DOI: 10.1117/1.jmi.8.4.041208] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/22/2020] [Accepted: 06/28/2021] [Indexed: 11/14/2022] Open
Abstract
Purpose: Experienced radiologists have enhanced global processing ability relative to novices, allowing experts to rapidly detect medical abnormalities without performing an exhaustive search. However, evidence for global processing models is primarily limited to two-dimensional image interpretation, and it is unclear whether these findings generalize to volumetric images, which are widely used in clinical practice. We examined whether radiologists searching volumetric images use methods consistent with global processing models of expertise. In addition, we investigated whether search strategy (scanning/drilling) differs with experience level. Approach: Fifty radiologists with a wide range of experience evaluated chest computed-tomography scans for lung nodules while their eye movements and scrolling behaviors were tracked. Multiple linear regressions were used to determine: (1) how search behaviors differed with years of experience and the number of chest CTs evaluated per week and (2) which search behaviors predicted better performance. Results: Contrary to global processing models based on 2D images, experience was unrelated to measures of global processing (saccadic amplitude, coverage, time to first fixation, search time, and depth passes) in this task. Drilling behavior was associated with better accuracy than scanning behavior when controlling for observer experience. Greater image coverage was a strong predictor of task accuracy. Conclusions: Global processing ability may play a relatively small role in volumetric image interpretation, where global scene statistics are not available to radiologists in a single glance. Rather, in volumetric images, it may be more important to engage in search strategies that support a more thorough search of the image.
Collapse
Affiliation(s)
- Lauren H Williams
- University of California, San Diego, Department of Psychology, San Diego, California, United States
| | - Ann J Carrigan
- Macquarie University, Department of Psychology, Sydney, New South Wales, Australia.,Macquarie University, Perception in Action Research Centre, Sydney, New South Wales, Australia.,Macquarie University, Centre for Elite Performance, Expertise, and Training, Sydney, New South Wales, Australia
| | - Megan Mills
- University of Utah, School of Medicine, Department of Radiology and Imaging Sciences, Salt Lake City, Utah, United States
| | - William F Auffermann
- University of Utah, School of Medicine, Department of Radiology and Imaging Sciences, Salt Lake City, Utah, United States
| | - Anina N Rich
- Macquarie University, Perception in Action Research Centre, Sydney, New South Wales, Australia.,Macquarie University, Centre for Elite Performance, Expertise, and Training, Sydney, New South Wales, Australia.,Macquarie University, Department of Cognitive Science, Sydney, New South Wales, Australia
| | - Trafton Drew
- University of Utah, Department of Psychology, Salt Lake City, Utah, United States
| |
Collapse
|