1
|
Halligan S, Boone D, Burling D, Helbren E, Mallett S, Plumb A. Doug Altman, medical statistician par excellence: What can radiologists learn from his legacy? Clin Radiol 2024; 79:479-484. [PMID: 38729906 DOI: 10.1016/j.crad.2024.04.004] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/17/2024] [Revised: 04/05/2024] [Accepted: 04/06/2024] [Indexed: 05/12/2024]
Abstract
This narrative review describes our experience of working with Doug Altman, the most highly cited medical statistician in the world. Doug was particularly interested in diagnostics, and imaging studies in particular. We describe how his insights helped improve our own radiological research studies and we provide advice for other researchers hoping to improve their own research practice.
Collapse
Affiliation(s)
- S Halligan
- Centre for Medical Imaging, Division of Medicine, University College London, United Kingdom.
| | - D Boone
- Department of Radiology, University College Hospitals, London, United Kingdom
| | - D Burling
- Department of Radiology, St. Mark's Hospital, London, United Kingdom
| | - E Helbren
- Department of Radiology, Hull & East Yorkshire Hospitals NHS Trust, Hull, United Kingdom
| | - S Mallett
- Centre for Medical Imaging, Division of Medicine, University College London, United Kingdom
| | - A Plumb
- Department of Radiology, University College Hospitals, London, United Kingdom
| |
Collapse
|
2
|
Murphy PM. Visual Image Annotation for Bowel Obstruction: Repeatability and Agreement with Manual Annotation and Neural Networks. J Digit Imaging 2023; 36:2179-2193. [PMID: 37278918 PMCID: PMC10502000 DOI: 10.1007/s10278-023-00825-w] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/04/2023] [Revised: 03/21/2023] [Accepted: 03/29/2023] [Indexed: 06/07/2023] Open
Abstract
Bowel obstruction is a common cause of acute abdominal pain. The development of algorithms for automated detection and characterization of bowel obstruction on CT has been limited by the effort required for manual annotation. Visual image annotation with an eye tracking device may mitigate that limitation. The purpose of this study is to assess the agreement between visual and manual annotations for bowel segmentation and diameter measurement, and to assess agreement with convolutional neural networks (CNNs) trained using that data. Sixty CT scans of 50 patients with bowel obstruction from March to June 2022 were retrospectively included and partitioned into training and test data sets. An eye tracking device was used to record 3-dimensional coordinates within the scans, while a radiologist cast their gaze at the centerline of the bowel, and adjusted the size of a superimposed ROI to approximate the diameter of the bowel. For each scan, 59.4 ± 15.1 segments, 847.9 ± 228.1 gaze locations, and 5.8 ± 1.2 m of bowel were recorded. 2d and 3d CNNs were trained using this data to predict bowel segmentation and diameter maps from the CT scans. For comparisons between two repetitions of visual annotation, CNN predictions, and manual annotations, Dice scores for bowel segmentation ranged from 0.69 ± 0.17 to 0.81 ± 0.04 and intraclass correlations [95% CI] for diameter measurement ranged from 0.672 [0.490-0.782] to 0.940 [0.933-0.947]. Thus, visual image annotation is a promising technique for training CNNs to perform bowel segmentation and diameter measurement in CT scans of patients with bowel obstruction.
Collapse
Affiliation(s)
- Paul M Murphy
- University of California-San Diego, 9500 Gilman Dr, 92093, La Jolla, CA, USA.
- UCSD Radiology, 200 W Arbor Dr, 92103, San Diego, CA, USA.
| |
Collapse
|
3
|
Raza MA, Kiran R, Ghazal S, Kang Z, Salehi S, Cokely E, Jeon J. An Eye Tracking Based Framework for Safety Improvement of Offshore Operations. J Eye Mov Res 2023; 16:10.16910/jemr.16.3.2. [PMID: 38169868 PMCID: PMC10759243 DOI: 10.16910/jemr.16.3.2] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/05/2024] Open
Abstract
Offshore drilling operations consist of complex and high-risk processes. Lack of situational awareness in drilling operations has become an important human factor issue that causes safety accidents. Prolonged work shifts and fatigue are some of the crucial issues that impact performance. Eye tracking technology can be used to distinguish the degree of awareness or alertness of participants that might be related to fatigue or onsite distractions. Oculomotor activity can be used to obtain visual cues that can quantify the drilling operators' situational awareness that might enable us to develop warning alarms to alert the driller. Such systems can help reduce accidents and save non-productive time. In this paper, eye movement char-acteristics were investigated to differentiate the situational awareness between a representa-tive expert and a group of novices using a scenario-based Virtual Reality Drilling Simulator. Significant visual oculomotor activity differences were identified between the expert and the novices that indicate an eye-tracking based system can detect the distraction and alert-ness exhibited by the workers. Results show promise on developing a framework which implements a real-time eye tracking technology in various drilling operations at drilling rigs and Real Time Operation Centers to improve process safety.
Collapse
Affiliation(s)
| | - Raj Kiran
- School of Petroleum and Geological Eng., University of Oklahoma, USA
| | - Saima Ghazal
- Inst. of Applied Psychology, University of the Punjab, Pakistan
| | - Ziho Kang
- Department of Industrial & Systems Eng., University of Oklahoma, USA
| | - Saeed Salehi
- School of Petroleum and Geological Eng., University of Oklahoma, USA
| | - Edward Cokely
- Department of Psychology, University of Oklahoma, USA
| | - Jiwon Jeon
- Department of Industrial & Systems Eng., University of Oklahoma, USA
| |
Collapse
|
4
|
Williams LH, Carrigan AJ, Mills M, Auffermann WF, Rich AN, Drew T. Characteristics of expert search behavior in volumetric medical image interpretation. J Med Imaging (Bellingham) 2021; 8:041208. [PMID: 34277889 DOI: 10.1117/1.jmi.8.4.041208] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/22/2020] [Accepted: 06/28/2021] [Indexed: 11/14/2022] Open
Abstract
Purpose: Experienced radiologists have enhanced global processing ability relative to novices, allowing experts to rapidly detect medical abnormalities without performing an exhaustive search. However, evidence for global processing models is primarily limited to two-dimensional image interpretation, and it is unclear whether these findings generalize to volumetric images, which are widely used in clinical practice. We examined whether radiologists searching volumetric images use methods consistent with global processing models of expertise. In addition, we investigated whether search strategy (scanning/drilling) differs with experience level. Approach: Fifty radiologists with a wide range of experience evaluated chest computed-tomography scans for lung nodules while their eye movements and scrolling behaviors were tracked. Multiple linear regressions were used to determine: (1) how search behaviors differed with years of experience and the number of chest CTs evaluated per week and (2) which search behaviors predicted better performance. Results: Contrary to global processing models based on 2D images, experience was unrelated to measures of global processing (saccadic amplitude, coverage, time to first fixation, search time, and depth passes) in this task. Drilling behavior was associated with better accuracy than scanning behavior when controlling for observer experience. Greater image coverage was a strong predictor of task accuracy. Conclusions: Global processing ability may play a relatively small role in volumetric image interpretation, where global scene statistics are not available to radiologists in a single glance. Rather, in volumetric images, it may be more important to engage in search strategies that support a more thorough search of the image.
Collapse
Affiliation(s)
- Lauren H Williams
- University of California, San Diego, Department of Psychology, San Diego, California, United States
| | - Ann J Carrigan
- Macquarie University, Department of Psychology, Sydney, New South Wales, Australia.,Macquarie University, Perception in Action Research Centre, Sydney, New South Wales, Australia.,Macquarie University, Centre for Elite Performance, Expertise, and Training, Sydney, New South Wales, Australia
| | - Megan Mills
- University of Utah, School of Medicine, Department of Radiology and Imaging Sciences, Salt Lake City, Utah, United States
| | - William F Auffermann
- University of Utah, School of Medicine, Department of Radiology and Imaging Sciences, Salt Lake City, Utah, United States
| | - Anina N Rich
- Macquarie University, Perception in Action Research Centre, Sydney, New South Wales, Australia.,Macquarie University, Centre for Elite Performance, Expertise, and Training, Sydney, New South Wales, Australia.,Macquarie University, Department of Cognitive Science, Sydney, New South Wales, Australia
| | - Trafton Drew
- University of Utah, Department of Psychology, Salt Lake City, Utah, United States
| |
Collapse
|
5
|
Zhang JY, Shen ZH, Wang BP, Liu F, Li J. Influence of 3D laparoscopic surgery on surgeon's visual pattern and mental workload. J Med Eng Technol 2021; 45:375-379. [PMID: 33843431 DOI: 10.1080/03091902.2021.1907466] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/11/2023]
Abstract
Previous studies have found that surgeons perform better in three-dimensional (3D) surgery than in two-dimensional (2D) surgery. However, no comparative studies have revealed the impact of 3D laparoscopic surgery on the surgeon's vision. To explore the effect of laparoscopic surgeons' depth perception during 3D laparoscopic surgery, 10 participants were recruited and performed 4 sets comparative simulated laparoscopic procedures in a virtual simulator, and eye movement signals were acquired, which were used to characteristics the visual differences. Fixation rate and saccade speed were used to characterise the influence of moderating variables for visual characteristics. The results from the data showed significant differences in eye movement behaviour. Compared with 2D laparoscopic surgery, surgeons have more average fixation rate (p-values = 0.001, 0.000, 0.003 and 0.015, respectively) and faster saccade speed (p-values = 0.037, 0.003, 0.073 and 0.105, respectively) in 3D laparoscopic surgery. The results of this study showed that surgeons had more efficient visual search in 3D laparoscopic surgery. At the same time, the results also indicated that surgeon's mental workload in 3D laparoscopic surgery was low. The relevant conclusions of this paper revealed the advantages of 3D laparoscopic surgery through visual efficiency.
Collapse
Affiliation(s)
- Jian-Yang Zhang
- School of Computer and Information Technology, Nanyang Normal University, Nanyang, P. R. China.,Department of Medical Engineering, Union Hospital, Tongji Medical College, Huazhong University of Science and Technology, Wuhan, P. R. China
| | - Zhi-Hao Shen
- School of Computer and Information Technology, Nanyang Normal University, Nanyang, P. R. China
| | - Bao-Ping Wang
- School of Computer and Information Technology, Nanyang Normal University, Nanyang, P. R. China
| | - Feng Liu
- School of Computer and Information Technology, Nanyang Normal University, Nanyang, P. R. China
| | - Juan Li
- School of Civil Engineering and Architecture, Nanyang Normal University, Nanyang, P. R. China
| |
Collapse
|
6
|
Bell CR, Szulewski A, Walker M, McKaigney C, Ross G, Rang L, Newbigging J, Kendall J. Differences in Gaze Fixation Location and Duration Between Resident and Fellowship Sonographers Interpreting a Focused Assessment With Sonography in Trauma. AEM EDUCATION AND TRAINING 2021; 5:28-36. [PMID: 33521488 PMCID: PMC7821074 DOI: 10.1002/aet2.10439] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 09/23/2019] [Revised: 01/22/2020] [Accepted: 02/07/2020] [Indexed: 05/30/2023]
Abstract
OBJECTIVES We quantified the gaze fixation duration of resident and fellowship sonographers interpreting a prerecorded focused assessment with sonography in trauma (FAST). We hypothesized that all sonographers would fixate on each relevant anatomic relationship but that the duration of fixation would differ. METHODS We conducted a cross-sectional study collecting and analyzing the gaze fixations of a convenience sample of current resident and fellowship sonographers. All sonographers viewed a standardized FAST video, and their gaze fixations were recorded using a Tobii X3-120 eye-tracking bar. Gaze fixations over nine anatomic regions of interest (ROIs) were identified. These were assessed for normality and analyzed using the Wilcoxon rank sum test at an alpha of 0.05 and Bonferroni correction p value of <0.0034. The chi-square test and Pearson's correlation were performed to assess statistical association. RESULTS The gaze fixation recordings of 24 resident and eight fellowship sonographers were suitable for analysis. Fourteen of the 24 resident sonographers viewed all ROIs in the FAST, whereas all eight fellowship sonographers viewed each of the nine relevant ROIs. Five ROIs were identified over which at least one resident sonographer did not have a gaze fixation. No statistically significant difference was identified between groups. Resident sonographers gaze fixated over the left upper quadrant (LUQ) splenorenal interface for a median (interquartile range) of 10.64 (9.73-11.60) seconds. The fellowship group viewed the same ROI for 8.43 (6.64-8.95) seconds (p < 0.003). All participants viewed this ROI. No other ROIs had a statistical difference. CONCLUSION Five ROIs were identified that were not visually interrogated by all resident sonographers. Only 14 of 24 resident sonographers visually interrogated every area in the FAST, whereas all fellowship sonographers interrogated every ROI. A statistically significant difference was found in gaze fixation duration between resident and fellowship sonographers in one ROI. Further study is required for gaze fixation assessment to become a tool for the interpretation component of point-of-care ultrasound.
Collapse
Affiliation(s)
- Colin R. Bell
- Department of Emergency MedicineKingston Health Sciences CentreKingstonOntario
- Queen’s University School of MedicineKingstonOntario
| | - Adam Szulewski
- Department of Emergency MedicineKingston Health Sciences CentreKingstonOntario
- Queen’s University School of MedicineKingstonOntario
| | - Melanie Walker
- Department of Emergency MedicineKingston Health Sciences CentreKingstonOntario
| | - Conor McKaigney
- Department of Emergency MedicineSouth Health, Alberta Health Services and Cumming School of Medicine at the University of CalgaryCalgaryAlbertaCanada
| | - Graeme Ross
- Department of Emergency MedicineKingston Health Sciences CentreKingstonOntario
- Queen’s University School of MedicineKingstonOntario
| | - Louise Rang
- Department of Emergency MedicineKingston Health Sciences CentreKingstonOntario
- Queen’s University School of MedicineKingstonOntario
| | - Joseph Newbigging
- Department of Emergency MedicineKingston Health Sciences CentreKingstonOntario
- Queen’s University School of MedicineKingstonOntario
| | - John Kendall
- Department of Emergency MedicineDenver Health Medical Center and University of Colorado Medical SchoolDenverCO
| |
Collapse
|
7
|
Stember JN, Celik H, Krupinski E, Chang PD, Mutasa S, Wood BJ, Lignelli A, Moonis G, Schwartz LH, Jambawalikar S, Bagci U. Eye Tracking for Deep Learning Segmentation Using Convolutional Neural Networks. J Digit Imaging 2020; 32:597-604. [PMID: 31044392 PMCID: PMC6646645 DOI: 10.1007/s10278-019-00220-4] [Citation(s) in RCA: 23] [Impact Index Per Article: 5.8] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/12/2022] Open
Abstract
Deep learning with convolutional neural networks (CNNs) has experienced tremendous growth in multiple healthcare applications and has been shown to have high accuracy in semantic segmentation of medical (e.g., radiology and pathology) images. However, a key barrier in the required training of CNNs is obtaining large-scale and precisely annotated imaging data. We sought to address the lack of annotated data with eye tracking technology. As a proof of principle, our hypothesis was that segmentation masks generated with the help of eye tracking (ET) would be very similar to those rendered by hand annotation (HA). Additionally, our goal was to show that a CNN trained on ET masks would be equivalent to one trained on HA masks, the latter being the current standard approach. Step 1: Screen captures of 19 publicly available radiologic images of assorted structures within various modalities were analyzed. ET and HA masks for all regions of interest (ROIs) were generated from these image datasets. Step 2: Utilizing a similar approach, ET and HA masks for 356 publicly available T1-weighted postcontrast meningioma images were generated. Three hundred six of these image + mask pairs were used to train a CNN with U-net-based architecture. The remaining 50 images were used as the independent test set. Step 1: ET and HA masks for the nonneurological images had an average Dice similarity coefficient (DSC) of 0.86 between each other. Step 2: Meningioma ET and HA masks had an average DSC of 0.85 between each other. After separate training using both approaches, the ET approach performed virtually identically to HA on the test set of 50 images. The former had an area under the curve (AUC) of 0.88, while the latter had AUC of 0.87. ET and HA predictions had trimmed mean DSCs compared to the original HA maps of 0.73 and 0.74, respectively. These trimmed DSCs between ET and HA were found to be statistically equivalent with a p value of 0.015. We have demonstrated that ET can create segmentation masks suitable for deep learning semantic segmentation. Future work will integrate ET to produce masks in a faster, more natural manner that distracts less from typical radiology clinical workflow.
Collapse
Affiliation(s)
- J N Stember
- Department of Radiology, Columbia University Medical Center - NYPH, New York, NY, 10032, USA.
| | - H Celik
- The National Institutes of Health, Clinical Center, Bethesda, MD, 20892, USA
| | - E Krupinski
- Department of Radiology & Imaging Sciences, Emory University, Atlanta, GA, 30322, USA
| | - P D Chang
- Department of Radiology, University of California, Irvine, CA, 92697, USA
| | - S Mutasa
- Department of Radiology, Columbia University Medical Center - NYPH, New York, NY, 10032, USA
| | - B J Wood
- The National Institutes of Health, Clinical Center, Bethesda, MD, 20892, USA
| | - A Lignelli
- Department of Radiology, Columbia University Medical Center - NYPH, New York, NY, 10032, USA
| | - G Moonis
- Department of Radiology, Columbia University Medical Center - NYPH, New York, NY, 10032, USA
| | - L H Schwartz
- Department of Radiology, Columbia University Medical Center - NYPH, New York, NY, 10032, USA
| | - S Jambawalikar
- Department of Radiology, Columbia University Medical Center - NYPH, New York, NY, 10032, USA
| | - U Bagci
- Center for Research in Computer Vision, University of Central Florida, 4328 Scorpius St. HEC 221, Orlando, FL, 32816, USA
| |
Collapse
|
8
|
Kelahan LC, Fong A, Blumenthal J, Kandaswamy S, Ratwani RM, Filice RW. The Radiologist's Gaze: Mapping Three-Dimensional Visual Search in Computed Tomography of the Abdomen and Pelvis. J Digit Imaging 2020; 32:234-240. [PMID: 30291478 DOI: 10.1007/s10278-018-0121-8] [Citation(s) in RCA: 16] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/27/2022] Open
Abstract
A radiologist's search pattern can directly influence patient management. A missed finding is a missed opportunity for intervention. Multiple studies have attempted to describe and quantify search patterns but have mainly focused on chest radiographs and chest CTs. Here, we describe and quantify the visual search patterns of 17 radiologists as they scroll through 6 CTs of the abdomen and pelvis. Search pattern tracings varied among individuals and remained relatively consistent per individual between cases. Attendings and trainees had similar eye metric statistics with respect to time to first fixation (TTFF), number of fixations in the region of interest (ROI), fixation duration in ROI, mean saccadic amplitude, or total number of fixations. Attendings had fewer numbers of fixations per second versus trainees (p < 0.001), suggesting efficiency due to expertise. In those cases that were accurately interpreted, TTFF was shorter (p = 0.04), the number of fixations per second and number of fixations in ROI were higher (p = 0.04, p = 0.02, respectively), and fixation duration in ROI was increased (p = 0.02). We subsequently categorized radiologists as "scanners" or "drillers" by both qualitative and quantitative methods and found no differences in accuracy with most radiologists being categorized as "drillers." This study describes visual search patterns of radiologists in interpretation of CTs of the abdomen and pelvis to better approach future endeavors in determining the effects of manipulations such as fatigue, interruptions, and computer-aided detection.
Collapse
Affiliation(s)
- Linda C Kelahan
- MedStar Georgetown University Hospital, 3800 Reservoir Road NW, Washington, DC, 20007, USA.
- , 300 Pasteur Drive Room H1307, Stanford, CA, USA.
| | - Allan Fong
- MedStar Institute for Innovation, 3007 Tilden St NW, Washington, DC, 20008, USA
| | - Joseph Blumenthal
- MedStar Institute for Innovation, 3007 Tilden St NW, Washington, DC, 20008, USA
| | - Swaminathan Kandaswamy
- University of Massachusetts, 120H Maraton Hall, 160 Governors Dr, Amherst, MA, 01003, USA
| | - Raj M Ratwani
- MedStar Institute for Innovation, 3007 Tilden St NW, Washington, DC, 20008, USA
| | - Ross W Filice
- MedStar Georgetown University Hospital, 3800 Reservoir Road NW, Washington, DC, 20007, USA
| |
Collapse
|
9
|
Williams LH, Drew T. What do we know about volumetric medical image interpretation?: a review of the basic science and medical image perception literatures. Cogn Res Princ Implic 2019; 4:21. [PMID: 31286283 PMCID: PMC6614227 DOI: 10.1186/s41235-019-0171-6] [Citation(s) in RCA: 22] [Impact Index Per Article: 4.4] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/04/2019] [Accepted: 05/19/2019] [Indexed: 11/26/2022] Open
Abstract
Interpretation of volumetric medical images represents a rapidly growing proportion of the workload in radiology. However, relatively little is known about the strategies that best guide search behavior when looking for abnormalities in volumetric images. Although there is extensive literature on two-dimensional medical image perception, it is an open question whether the conclusions drawn from these images can be generalized to volumetric images. Importantly, volumetric images have distinct characteristics (e.g., scrolling through depth, smooth-pursuit eye-movements, motion onset cues, etc.) that should be considered in future research. In this manuscript, we will review the literature on medical image perception and discuss relevant findings from basic science that can be used to generate predictions about expertise in volumetric image interpretation. By better understanding search through volumetric images, we may be able to identify common sources of error, characterize the optimal strategies for searching through depth, or develop new training and assessment techniques for radiology residents.
Collapse
|
10
|
Sharma V, Fong A, Beckman RA, Rao S, Boca SM, McGarvey PB, Ratwani RM, Madhavan S. Eye-Tracking Study to Enhance Usability of Molecular Diagnostics Reports in Cancer Precision Medicine. JCO Precis Oncol 2018; 2:1-11. [PMID: 35135129 DOI: 10.1200/po.17.00296] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/20/2022] Open
Abstract
PURPOSE We conducted usability studies on commercially available molecular diagnostic (MDX) test reports to identify strengths and weaknesses in content and form that drive clinical decision making. Given routine genomic testing in cancer medicine, oncologists must interpret MDX reports as well as evidence concerning clinical utility of biomarkers accurately for treatment or trial selection. This work aims to evaluate effectiveness of MDX reports in facilitating cancer treatment planning. METHODS Fourteen clinicians at an academic tertiary care medical facility, with a wide range of experience in oncology and in the use of molecular testing, participated in this study. Three commercially available, widely used, Clinical Laboratory Improvement Amendments (CLIA)-certified, College of American Pathologists (CAP)-accredited test reports (labeled Laboratories A, B, and C) were used. Eye tracking, surveys, and think-aloud protocols were used to collect usability data for these MDX reports focusing on ease of comprehension and actionability. RESULTS Clinicians found two primary areas in molecular diagnostic reports most useful for patient care: therapy options with benefit or lack of benefit to patients, including enrolling clinical trials; and pathogenic tumor molecular anomalies detected. Therapeutic implications and therapy classes such as US Food and Drug Administration-approved off-label, on-label, clinical trials were critical for decision making. However, all reports had usability and comprehension issues in these areas and could be improved. CONCLUSION Focused usability studies can help drive our understanding of the clinical workflow for use of molecular diagnostic tests in cancer care. This in turn can have major effects on quality of care, outcomes, costs, and patient satisfaction. This study demonstrates the use of specific usability techniques (eye tracking and think-aloud protocols) to help clinical laboratories improve MDX report design in a precision oncology treatment setting.
Collapse
Affiliation(s)
- Vishakha Sharma
- Vishakha Sharma, Robert A. Beckman, Shruti Rao, Simina M. Boca, Peter B. McGarvey, and Subha Madhavan, Georgetown University; Vishakha Sharma, Robert A. Beckman, Simina M. Boca, and Subha Madhavan, Georgetown University Medical Center; Allan Fong and Raj M. Ratwani, National Center for Human Factors in Healthcare, MedStar Health; and Raj M. Ratwani, Georgetown University School of Medicine, Washington, DC
| | - Allan Fong
- Vishakha Sharma, Robert A. Beckman, Shruti Rao, Simina M. Boca, Peter B. McGarvey, and Subha Madhavan, Georgetown University; Vishakha Sharma, Robert A. Beckman, Simina M. Boca, and Subha Madhavan, Georgetown University Medical Center; Allan Fong and Raj M. Ratwani, National Center for Human Factors in Healthcare, MedStar Health; and Raj M. Ratwani, Georgetown University School of Medicine, Washington, DC
| | - Robert A Beckman
- Vishakha Sharma, Robert A. Beckman, Shruti Rao, Simina M. Boca, Peter B. McGarvey, and Subha Madhavan, Georgetown University; Vishakha Sharma, Robert A. Beckman, Simina M. Boca, and Subha Madhavan, Georgetown University Medical Center; Allan Fong and Raj M. Ratwani, National Center for Human Factors in Healthcare, MedStar Health; and Raj M. Ratwani, Georgetown University School of Medicine, Washington, DC
| | - Shruti Rao
- Vishakha Sharma, Robert A. Beckman, Shruti Rao, Simina M. Boca, Peter B. McGarvey, and Subha Madhavan, Georgetown University; Vishakha Sharma, Robert A. Beckman, Simina M. Boca, and Subha Madhavan, Georgetown University Medical Center; Allan Fong and Raj M. Ratwani, National Center for Human Factors in Healthcare, MedStar Health; and Raj M. Ratwani, Georgetown University School of Medicine, Washington, DC
| | - Simina M Boca
- Vishakha Sharma, Robert A. Beckman, Shruti Rao, Simina M. Boca, Peter B. McGarvey, and Subha Madhavan, Georgetown University; Vishakha Sharma, Robert A. Beckman, Simina M. Boca, and Subha Madhavan, Georgetown University Medical Center; Allan Fong and Raj M. Ratwani, National Center for Human Factors in Healthcare, MedStar Health; and Raj M. Ratwani, Georgetown University School of Medicine, Washington, DC
| | - Peter B McGarvey
- Vishakha Sharma, Robert A. Beckman, Shruti Rao, Simina M. Boca, Peter B. McGarvey, and Subha Madhavan, Georgetown University; Vishakha Sharma, Robert A. Beckman, Simina M. Boca, and Subha Madhavan, Georgetown University Medical Center; Allan Fong and Raj M. Ratwani, National Center for Human Factors in Healthcare, MedStar Health; and Raj M. Ratwani, Georgetown University School of Medicine, Washington, DC
| | - Raj M Ratwani
- Vishakha Sharma, Robert A. Beckman, Shruti Rao, Simina M. Boca, Peter B. McGarvey, and Subha Madhavan, Georgetown University; Vishakha Sharma, Robert A. Beckman, Simina M. Boca, and Subha Madhavan, Georgetown University Medical Center; Allan Fong and Raj M. Ratwani, National Center for Human Factors in Healthcare, MedStar Health; and Raj M. Ratwani, Georgetown University School of Medicine, Washington, DC
| | - Subha Madhavan
- Vishakha Sharma, Robert A. Beckman, Shruti Rao, Simina M. Boca, Peter B. McGarvey, and Subha Madhavan, Georgetown University; Vishakha Sharma, Robert A. Beckman, Simina M. Boca, and Subha Madhavan, Georgetown University Medical Center; Allan Fong and Raj M. Ratwani, National Center for Human Factors in Healthcare, MedStar Health; and Raj M. Ratwani, Georgetown University School of Medicine, Washington, DC
| |
Collapse
|
11
|
den Boer L, van der Schaaf MF, Vincken KL, Mol CP, Stuijfzand BG, van der Gijp A. Volumetric image interpretation in radiology: scroll behavior and cognitive processes. ADVANCES IN HEALTH SCIENCES EDUCATION : THEORY AND PRACTICE 2018; 23:783-802. [PMID: 29767400 PMCID: PMC6132416 DOI: 10.1007/s10459-018-9828-z] [Citation(s) in RCA: 14] [Impact Index Per Article: 2.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 08/10/2017] [Accepted: 05/07/2018] [Indexed: 05/12/2023]
Abstract
The interpretation of medical images is a primary task for radiologists. Besides two-dimensional (2D) images, current imaging technologies allow for volumetric display of medical images. Whereas current radiology practice increasingly uses volumetric images, the majority of studies on medical image interpretation is conducted on 2D images. The current study aimed to gain deeper insight into the volumetric image interpretation process by examining this process in twenty radiology trainees who all completed four volumetric image cases. Two types of data were obtained concerning scroll behaviors and think-aloud data. Types of scroll behavior concerned oscillations, half runs, full runs, image manipulations, and interruptions. Think-aloud data were coded by a framework of knowledge and skills in radiology including three cognitive processes: perception, analysis, and synthesis. Relating scroll behavior to cognitive processes showed that oscillations and half runs coincided more often with analysis and synthesis than full runs, whereas full runs coincided more often with perception than oscillations and half runs. Interruptions were characterized by synthesis and image manipulations by perception. In addition, we investigated relations between cognitive processes and found an overall bottom-up way of reasoning with dynamic interactions between cognitive processes, especially between perception and analysis. In sum, our results highlight the dynamic interactions between these processes and the grounding of cognitive processes in scroll behavior. It suggests, that the types of scroll behavior are relevant to describe how radiologists interact with and manipulate volumetric images.
Collapse
Affiliation(s)
- Larissa den Boer
- Utrecht University, Heidelberglaan 1, 3584 CS, Utrecht, The Netherlands.
| | | | - Koen L Vincken
- University Medical Center Utrecht, Utrecht, The Netherlands
| | - Chris P Mol
- University Medical Center Utrecht, Utrecht, The Netherlands
| | | | | |
Collapse
|
12
|
Plumb AA, Phillips P, Spence G, Mallett S, Taylor SA, Halligan S, Fanshawe T. Increasing Navigation Speed at Endoluminal CT Colonography Reduces Colonic Visualization and Polyp Identification. Radiology 2017; 284:413-422. [PMID: 28281908 PMCID: PMC5548448 DOI: 10.1148/radiol.2017162037] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.1] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/22/2022]
Abstract
As navigation speed increases, gaze of the reader becomes more central and polyp identification rates fall. Purpose To investigate the effect of increasing navigation speed on the visual search and decision making during polyp identification for computed tomography (CT) colonography Materials and Methods Institutional review board permission was obtained to use deidentified CT colonography data for this prospective reader study. After obtaining informed consent from the readers, 12 CT colonography fly-through examinations that depicted eight polyps were presented at four different fixed navigation speeds to 23 radiologists. Speeds ranged from 1 cm/sec to 4.5 cm/sec. Gaze position was tracked by using an infrared eye tracker, and readers indicated that they saw a polyp by clicking a mouse. Patterns of searching and decision making by speed were investigated graphically and by multilevel modeling. Results Readers identified polyps correctly in 56 of 77 (72.7%) of viewings at the slowest speed but in only 137 of 225 (60.9%) of viewings at the fastest speed (P = .004). They also identified fewer false-positive features at faster speeds (42 of 115; 36.5%) of videos at slowest speed, 89 of 345 (25.8%) at fastest, P = .02). Gaze location was highly concentrated toward the central quarter of the screen area at faster speeds (mean gaze points at slowest speed vs fastest speed, 86% vs 97%, respectively). Conclusion Faster navigation speed at endoluminal CT colonography led to progressive restriction of visual search patterns. Greater speed also reduced both true-positive and false-positive colorectal polyp identification. © RSNA, 2017 Online supplemental material is available for this article.
Collapse
Affiliation(s)
- Andrew A Plumb
- From the Centre for Medical Imaging, University College London, 3rd Floor East, 250 Euston Rd, London NW1 2PG, England (A.A.P., S.A.T., S.H.); Health and Medical Sciences Group, University of Cumbria, Lancaster, England (P.P.); Department of Primary Care Health Sciences, University of Oxford, Oxford, England (G.S., T.F.); Institute of Applied Health Sciences, University of Birmingham, Birmingham, England (S.M.)
| | - Peter Phillips
- From the Centre for Medical Imaging, University College London, 3rd Floor East, 250 Euston Rd, London NW1 2PG, England (A.A.P., S.A.T., S.H.); Health and Medical Sciences Group, University of Cumbria, Lancaster, England (P.P.); Department of Primary Care Health Sciences, University of Oxford, Oxford, England (G.S., T.F.); Institute of Applied Health Sciences, University of Birmingham, Birmingham, England (S.M.)
| | - Graeme Spence
- From the Centre for Medical Imaging, University College London, 3rd Floor East, 250 Euston Rd, London NW1 2PG, England (A.A.P., S.A.T., S.H.); Health and Medical Sciences Group, University of Cumbria, Lancaster, England (P.P.); Department of Primary Care Health Sciences, University of Oxford, Oxford, England (G.S., T.F.); Institute of Applied Health Sciences, University of Birmingham, Birmingham, England (S.M.)
| | - Susan Mallett
- From the Centre for Medical Imaging, University College London, 3rd Floor East, 250 Euston Rd, London NW1 2PG, England (A.A.P., S.A.T., S.H.); Health and Medical Sciences Group, University of Cumbria, Lancaster, England (P.P.); Department of Primary Care Health Sciences, University of Oxford, Oxford, England (G.S., T.F.); Institute of Applied Health Sciences, University of Birmingham, Birmingham, England (S.M.)
| | - Stuart A Taylor
- From the Centre for Medical Imaging, University College London, 3rd Floor East, 250 Euston Rd, London NW1 2PG, England (A.A.P., S.A.T., S.H.); Health and Medical Sciences Group, University of Cumbria, Lancaster, England (P.P.); Department of Primary Care Health Sciences, University of Oxford, Oxford, England (G.S., T.F.); Institute of Applied Health Sciences, University of Birmingham, Birmingham, England (S.M.)
| | - Steve Halligan
- From the Centre for Medical Imaging, University College London, 3rd Floor East, 250 Euston Rd, London NW1 2PG, England (A.A.P., S.A.T., S.H.); Health and Medical Sciences Group, University of Cumbria, Lancaster, England (P.P.); Department of Primary Care Health Sciences, University of Oxford, Oxford, England (G.S., T.F.); Institute of Applied Health Sciences, University of Birmingham, Birmingham, England (S.M.)
| | - Thomas Fanshawe
- From the Centre for Medical Imaging, University College London, 3rd Floor East, 250 Euston Rd, London NW1 2PG, England (A.A.P., S.A.T., S.H.); Health and Medical Sciences Group, University of Cumbria, Lancaster, England (P.P.); Department of Primary Care Health Sciences, University of Oxford, Oxford, England (G.S., T.F.); Institute of Applied Health Sciences, University of Birmingham, Birmingham, England (S.M.)
| |
Collapse
|
13
|
Bertram R, Kaakinen J, Bensch F, Helle L, Lantto E, Niemi P, Lundbom N. Eye Movements of Radiologists Reflect Expertise in CT Study Interpretation: A Potential Tool to Measure Resident Development. Radiology 2016; 281:805-815. [DOI: 10.1148/radiol.2016151255] [Citation(s) in RCA: 32] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/22/2022]
|
14
|
Krupinski EA, Chung A, Applegate K, DeSimone AK, Tridandapani S. Impact of Patient Photographs on Radiologists' Visual Search of Chest Radiographs. Acad Radiol 2016; 23:953-60. [PMID: 27161208 DOI: 10.1016/j.acra.2016.04.001] [Citation(s) in RCA: 11] [Impact Index Per Article: 1.4] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/02/2016] [Revised: 04/05/2016] [Accepted: 04/08/2016] [Indexed: 11/16/2022]
Abstract
RATIONALE AND OBJECTIVES To increase detection of mislabeled medical imaging studies, evidence shows it may be useful to include patient photographs during interpretation. This study examined how inclusion of photographs impacts visual search. MATERIALS AND METHODS Ten radiologists participated. Average age was 43.00 years and average years Board-certified was 9.70, with 2 residents, 1 general, 2 abdominal, 4 cardiothoracic, and 1 pediatric radiologist. They viewed 21 portable chest radiographs with and without a simultaneously acquired photograph of the patient while visual search was recorded. Their task was to note placement of lines and tubes. RESULTS Presence of the photograph reduced the number of fixations (chest radiograph only mean 98.68; chest with photograph present 80.81; photograph 10.59; p < 0.0001) and total dwell (chest radiograph only mean 30.84 seconds; chest radiograph with photograph present 25.68; photograph 3.93; p < 0.0001) on the chest radiograph as a result of periodically looking at the photograph. Overall viewing time did not increase with addition of the photograph because time not spent on the radiograph was spent on the photograph. On average, readers scanned from the radiograph to the photographs about four times during search. Men and non-cardiothoracic radiologists spent significantly more time scanning all the images, including the photographs. Average preference for having photographs was 6.10 on a 0-10 scale, and neck and chest were preferred as areas to include in the photograph. CONCLUSION Photographs may help with certain image interpretation tasks and may help personalize the reading experience for radiologists without increasing interpretation time.
Collapse
Affiliation(s)
- Elizabeth A Krupinski
- Department of Medical Imaging, University of Arizona, 1609 N Warren Bldg 211 Tucson, AZ 85724; Department of Radiology & Imaging Sciences, Emory University, Atlanta, Georgia.
| | - Alex Chung
- Department of Radiology & Imaging Sciences, Emory University, Atlanta, Georgia
| | - Kimberly Applegate
- Department of Radiology & Imaging Sciences, Emory University, Atlanta, Georgia
| | - Ariadne K DeSimone
- Department of Radiology & Imaging Sciences, Emory University, Atlanta, Georgia
| | - Srini Tridandapani
- Department of Radiology & Imaging Sciences, Emory University, Atlanta, Georgia; School of Electrical and Computer Engineering, Georgia Institute of Technology, Atlanta, Georgia
| |
Collapse
|
15
|
Fanshawe TR, Phillips P, Plumb A, Helbren E, Halligan S, Taylor SA, Gale A, Mallett S. Do prevalence expectations affect patterns of visual search and decision-making in interpreting CT colonography endoluminal videos? Br J Radiol 2016; 89:20150842. [PMID: 26903391 DOI: 10.1259/bjr.20150842] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.4] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/05/2022] Open
Abstract
OBJECTIVE To assess the effect of expected abnormality prevalence on visual search and decision-making in CT colonography (CTC). METHODS 13 radiologists interpreted endoluminal CTC fly-throughs of the same group of 10 patient cases, 3 times each. Abnormality prevalence was fixed (50%), but readers were told, before viewing each group, that prevalence was either 20%, 50% or 80% in the population from which cases were drawn. Infrared visual search recording was used. Readers indicated seeing a polyp by clicking a mouse. Multilevel modelling quantified the effect of expected prevalence on outcomes. RESULTS Differences between expected prevalence were not statistically significant for time to first pursuit of the polyp (median 0.5 s, each prevalence), pursuit rate when no polyp was on screen (median 2.7 s(-1), each prevalence) or number of mouse clicks [mean 0.75/video (20% prevalence), 0.93 (50%), 0.97 (80%)]. There was weak evidence of increased tendency to look outside the central screen area at 80% prevalence and reduction in positive polyp identifications at 20% prevalence. CONCLUSION This study did not find a large effect of prevalence information on most visual search metrics or polyp identification in CTC. Further research is required to quantify effects at lower prevalence and in relation to secondary outcome measures. ADVANCES IN KNOWLEDGE Prevalence effects in evaluating CTC have not previously been assessed. In this study, providing expected prevalence information did not have a large effect on diagnostic decisions or patterns of visual search.
Collapse
Affiliation(s)
- Thomas R Fanshawe
- 1 Nuffield Department of Primary Care Health Sciences, University of Oxford, Oxford, UK
| | - Peter Phillips
- 2 Health and Medical Sciences Group, University of Cumbria, Lancaster, UK
| | - Andrew Plumb
- 3 Centre for Medical Imaging, University College London, London, UK
| | - Emma Helbren
- 3 Centre for Medical Imaging, University College London, London, UK
| | - Steve Halligan
- 3 Centre for Medical Imaging, University College London, London, UK
| | - Stuart A Taylor
- 3 Centre for Medical Imaging, University College London, London, UK
| | - Alastair Gale
- 4 Applied Vision Research Centre, Loughborough University, Loughborough, UK
| | - Susan Mallett
- 5 Public Health, Epidemiology and Biostatistics, Institute of Applied Health Sciences, University of Birmingham, Birmingham, UK
| |
Collapse
|
16
|
Hout MC, Godwin HJ, Fitzsimmons G, Robbins A, Menneer T, Goldinger SD. Using multidimensional scaling to quantify similarity in visual search and beyond. Atten Percept Psychophys 2016; 78:3-20. [PMID: 26494381 PMCID: PMC5523409 DOI: 10.3758/s13414-015-1010-6] [Citation(s) in RCA: 23] [Impact Index Per Article: 2.9] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/08/2022]
Abstract
Visual search is one of the most widely studied topics in vision science, both as an independent topic of interest, and as a tool for studying attention and visual cognition. A wide literature exists that seeks to understand how people find things under varying conditions of difficulty and complexity, and in situations ranging from the mundane (e.g., looking for one's keys) to those with significant societal importance (e.g., baggage or medical screening). A primary determinant of the ease and probability of success during search are the similarity relationships that exist in the search environment, such as the similarity between the background and the target, or the likeness of the non-targets to one another. A sense of similarity is often intuitive, but it is seldom quantified directly. This presents a problem in that similarity relationships are imprecisely specified, limiting the capacity of the researcher to examine adequately their influence. In this article, we present a novel approach to overcoming this problem that combines multi-dimensional scaling (MDS) analyses with behavioral and eye-tracking measurements. We propose a method whereby MDS can be repurposed to successfully quantify the similarity of experimental stimuli, thereby opening up theoretical questions in visual search and attention that cannot currently be addressed. These quantifications, in conjunction with behavioral and oculomotor measures, allow for critical observations about how similarity affects performance, information selection, and information processing. We provide a demonstration and tutorial of the approach, identify documented examples of its use, discuss how complementary computer vision methods could also be adopted, and close with a discussion of potential avenues for future application of this technique.
Collapse
|
17
|
Small Polyps at Endoluminal CT Colonography Are Often Seen But Ignored by Radiologists. AJR Am J Roentgenol 2015; 205:W424-31. [DOI: 10.2214/ajr.14.14093] [Citation(s) in RCA: 4] [Impact Index Per Article: 0.4] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/25/2022]
|
18
|
Venjakob AC, Mello-Thoms CR. Review of prospects and challenges of eye tracking in volumetric imaging. J Med Imaging (Bellingham) 2015; 3:011002. [PMID: 27081663 DOI: 10.1117/1.jmi.3.1.011002] [Citation(s) in RCA: 21] [Impact Index Per Article: 2.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/15/2015] [Accepted: 08/20/2015] [Indexed: 11/14/2022] Open
Abstract
While eye tracking research in conventional radiography has flourished over the past decades, the number of eye tracking studies that looked at multislice images lags behind. A possible reason for the lack of studies in this area might be that the eye tracking methodology used in the context of conventional radiography cannot be applied one-on-one to volumetric imaging material. Challenges associated with eye tracking in volumetric imaging are particularly associated with the selection of stimulus material, the detection of events in the eye tracking data, the calculation of meaningful eye tracking parameters, and the reporting of abnormalities. However, all of these challenges can be addressed in the design of the experiment. If this is done, eye tracking studies using volumetric imaging material offer almost unlimited opportunity for perception research and are highly relevant as the number of volumetric images that are acquired and interpreted is rising.
Collapse
Affiliation(s)
- Antje C Venjakob
- Technische Universität Berlin , Chair of Human-Machine Systems, Department of Psychology and Ergonomics, Marchstraße 23, 10587 Berlin, Germany
| | - Claudia R Mello-Thoms
- University of Sydney, Medical Imaging and Radiation Sciences, Faculty of Health Science, 94 Mallet Street, Level 2, Room 204, Sydney, NSW 2150, Australia; University of Pittsburgh, Department of Biomedical Informatics, 5607 Baum Boulevard, Room 423, Pittsburgh, Pennsylvania 15206-3701, United States
| |
Collapse
|
19
|
Helbren E, Fanshawe TR, Phillips P, Mallett S, Boone D, Gale A, Altman DG, Taylor SA, Manning D, Halligan S. The effect of computer-aided detection markers on visual search and reader performance during concurrent reading of CT colonography. Eur Radiol 2015; 25:1570-8. [PMID: 25577518 DOI: 10.1007/s00330-014-3569-z] [Citation(s) in RCA: 13] [Impact Index Per Article: 1.4] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/22/2014] [Revised: 12/09/2014] [Accepted: 12/15/2014] [Indexed: 11/27/2022]
Abstract
OBJECTIVE We aimed to identify the effect of computer-aided detection (CAD) on visual search and performance in CT Colonography (CTC) of inexperienced and experienced readers. METHODS Fifteen endoluminal CTC examinations were recorded, each with one polyp, and two videos were generated, one with and one without a CAD mark. Forty-two readers (17 experienced, 25 inexperienced) interpreted the videos during infrared visual search recording. CAD markers and polyps were treated as regions of interest in data processing. This multi-reader, multi-case study was analysed using multilevel modelling. RESULTS CAD drew readers' attention to polyps faster, accelerating identification times: median 'time to first pursuit' was 0.48 s (IQR 0.27 to 0.87 s) with CAD, versus 0.58 s (IQR 0.35 to 1.06 s) without. For inexperienced readers, CAD also held visual attention for longer. All visual search metrics used to assess visual gaze behaviour demonstrated statistically significant differences when "with" and "without" CAD were compared. A significant increase in the number of correct polyp identifications across all readers was seen with CAD (74 % without CAD, 87 % with CAD; p < 0.001). CONCLUSIONS CAD significantly alters visual search and polyp identification in readers viewing three-dimensional endoluminal CTC. For polyp and CAD marker pursuit times, CAD generally exerted a larger effect on inexperienced readers. KEY POINTS • Visual gaze is attracted by computer-assisted detection (CAD) marks on polyps • Inexperienced readers' gaze is affected more by CAD than experienced readers. • CAD marks could mean that the unannotated endoluminal surface is relatively neglected. • Correct polyp identification is increased significantly by CAD.
Collapse
Affiliation(s)
- Emma Helbren
- Centre for Medical Imaging, University College London, London, UK
| | | | | | | | | | | | | | | | | | | |
Collapse
|
20
|
Mallett S, Phillips P, Fanshawe TR, Helbren E, Boone D, Gale A, Taylor SA, Manning D, Altman DG, Halligan S. Tracking eye gaze during interpretation of endoluminal three-dimensional CT colonography: visual perception of experienced and inexperienced readers. Radiology 2014; 273:783-92. [PMID: 25028782 DOI: 10.1148/radiol.14132896] [Citation(s) in RCA: 37] [Impact Index Per Article: 3.7] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 02/11/2024]
Abstract
PURPOSE To identify and compare key stages of the visual process in experienced and inexperienced readers and to examine how these processes are used to search a moving three-dimensional ( 3D three-dimensional ) image and their relationship to false-negative errors. MATERIALS AND METHODS Institutional review board research ethics approval was granted to use anonymized computed tomographic (CT) colonographic data from previous studies and to obtain eye-tracking data from volunteers. Sixty-five radiologists (27 experienced, 38 inexperienced) interpreted 23 endoluminal 3D three-dimensional CT colonographic videos. Eye movements were recorded by using eye tracking with a desk-mounted tracker. Readers indicated when they saw a polyp by clicking a computer mouse. Polyp location and boundary on each video frame were quantified and gaze data were related to the polyp boundary for each individual reader and case. Predefined metrics were quantified and used to describe and compare visual search patterns between experienced and inexperienced readers by using multilevel modeling. RESULTS Time to first pursuit was significantly shorter in experienced readers (hazard ratio, 1.22 [95% confidence interval: 1.04, 1.44]; P = .017) but other metrics were not significantly different. Regardless of expertise, metrics such as assessment, identification period, and pursuit times were extended in videos where polyps were visible on screen for longer periods of time. In 97% (760 of 787) of observations, readers correctly pursued polyps. CONCLUSION Experienced readers had shorter time to first eye pursuit, but many other characteristics of eye tracking were similar between experienced and inexperienced readers. Readers pursued polyps in 97% of observations, which indicated that errors during interpretation of 3D three-dimensional CT colonography in this study occurred in either the discovery or the recognition phase, but rarely in the scanning phase of radiologic image inspection.
Collapse
Affiliation(s)
- Susan Mallett
- From the Department of Primary Care Health Sciences, University of Oxford, Radcliffe Observatory Quarter, Woodstock Road, Oxford OX2 6GG, England (S.M., T.R.F.); Health and Medical Sciences Group, University of Cumbria, Lancaster, England (P.P.); Centre for Medical Imaging, University College London, London, England (E.H., S.A.T., S.H., D.B.); Applied Vision Research Centre, Loughborough University, Loughborough, England (A.G.); School of Medicine, Lancaster University, Lancaster, England (D.M.); and Centre for Statistics in Medicine, University of Oxford, Oxford, England (D.G.A.)
| | | | | | | | | | | | | | | | | | | |
Collapse
|