1
|
Vrzáková H, Tapiala J, Iso-Mustajärvi M, Timonen T, Dietz A. Estimating Cognitive Workload Using Task-Related Pupillary Responses in Simulated Drilling in Cochlear Implantation. Laryngoscope 2024. [PMID: 38989899 DOI: 10.1002/lary.31612] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/20/2024] [Revised: 05/31/2024] [Accepted: 06/17/2024] [Indexed: 07/12/2024]
Abstract
OBJECTIVES Training of temporal bone drilling requires more than mastering technical skills with the drill. Skills such as visual imagery, bimanual dexterity, and stress management need to be mastered along with precise knowledge of anatomy. In otorhinolaryngology, these psychomotor skills underlie performance in the drilling of the temporal bone for access to the inner ear in cochlear implant surgery. However, little is known about how psychomotor skills and workload management impact the practitioners' continuous and overall performance. METHODS To understand how the practitioner's workload and performance unfolds over time, we examine task-evoked pupillary responses (TEPR) of 22 medical students who performed transmastoid-posterior tympanotomy (TMPT) and removal of the bony overhang of the round window niche in a 3D-printed model of the temporal bone. We investigate how students' TEPR metrics (Average Pupil Size [APS], Index of Pupil Activity [IPA], and Low/High Index of Pupillary Activity [LHIPA]) and time spent in drilling phases correspond to the performance in key drilling phases. RESULTS All TEPR measures revealed significant differences between key drilling phases that corresponded to the anticipated workload. Enlarging the facial recess lasted significantly longer than other phases. IPA captured significant increase of workload in thinning of the posterior canal wall, while APS revealed increased workload during the drilling of the bony overhang. CONCLUSION Our findings contribute to the contemporary competency-based medical residency programs where objective and continuous monitoring of participants' progress allows to track progress in expertise acquisition. Laryngoscope, 2024.
Collapse
Affiliation(s)
- Hana Vrzáková
- School of Computing, University of Eastern Finland, Joensuu, Finland
| | - Jesse Tapiala
- School of Medicine, Institute of Clinical Medicine, University of Eastern Finland, Kuopio, Finland
| | | | - Tomi Timonen
- Department of Otorhinolaryngology, Kuopio University Hospital, Kuopio, Finland
| | - Aarno Dietz
- Department of Otorhinolaryngology, Kuopio University Hospital, Kuopio, Finland
| |
Collapse
|
2
|
Sugimoto M, Kaneko N, Oyamada M, Tomita A, Sato M. Eye-tracking analysis for situation awareness of incontinence pad changing during older adult nursing training: An observational study. Nurse Educ Pract 2024; 76:103935. [PMID: 38442654 DOI: 10.1016/j.nepr.2024.103935] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/16/2024] [Revised: 01/23/2024] [Accepted: 02/27/2024] [Indexed: 03/07/2024]
Abstract
AIM This study aims to investigate eye tracking in the practical training of incontinence pad change, which is commonly required in older adult nursing. BACKGROUND Some competencies possessed by skilled and experienced personnel are difficult to verbalize into textbooks. However, this is crucial for education, especially nursing practice education. Eye-gaze analysis is one such tool that can aid the efficient transfer of knowledge to students. Therefore, eye-gaze analysis, a novel technology for visualizing situational awareness and decision-making, has recently gained traction in healthcare. DESIGN An observation study METHODS: Ten nursing faculty members and 13 nursing students with prior incontinence pad change experience participated in this study using an older adult simulator. There were two groups of students - S1 with more recent experience in older adult care and incontinence pad changing and S2 with less. Areas of interest (AOIs) during incontinence pad preparation and fitting were determined based on gaze fixation and the time spent fixating on these areas was compared. RESULTS Students took longer than nursing faculty members. When visualizing the eye movements between the AOIs in the network, the faculty nurses and S1 alternated their gaze between the new incontinence pad and the buttocks and between other AOIs. Simultaneously, S2 tended to gaze or stare only at the new incontinence pad. CONCLUSION The presented data may help interpret visual-based situational awareness and establish effective nursing education, especially in acquiring skills that are difficult to verbalize.
Collapse
Affiliation(s)
- Masahiro Sugimoto
- Institute for Advanced Biosciences, Keio University, Tsuruoka 997-0052, Japan; Institute of Medical Sciences, Tokyo Medical University, Tokyo, Shinjuku 160-0022, Japan.
| | - Naomi Kaneko
- Faculty of Nursing, Kanagawa Institute of Technology, Kanagawa 243-0292, Japan.
| | - Michiko Oyamada
- Faculty of Human Care Department, Tohto University, 1-1 Hinode-cho, Numazu City, Shizuoka 410-0032, Japan.
| | - Atsumi Tomita
- Institute of Medical Sciences, Tokyo Medical University, Tokyo, Shinjuku 160-0022, Japan.
| | - Mitsue Sato
- Department of Nursing, Kiryu University, Midori 379-2392, Japan.
| |
Collapse
|
3
|
Brunyé TT, Booth K, Hendel D, Kerr KF, Shucard H, Weaver DL, Elmore JG. Machine learning classification of diagnostic accuracy in pathologists interpreting breast biopsies. J Am Med Inform Assoc 2024; 31:552-562. [PMID: 38031453 PMCID: PMC10873842 DOI: 10.1093/jamia/ocad232] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/22/2023] [Revised: 10/19/2023] [Accepted: 11/20/2023] [Indexed: 12/01/2023] Open
Abstract
OBJECTIVE This study explores the feasibility of using machine learning to predict accurate versus inaccurate diagnoses made by pathologists based on their spatiotemporal viewing behavior when evaluating digital breast biopsy images. MATERIALS AND METHODS The study gathered data from 140 pathologists of varying experience levels who each reviewed a set of 14 digital whole slide images of breast biopsy tissue. Pathologists' viewing behavior, including zooming and panning actions, was recorded during image evaluation. A total of 30 features were extracted from the viewing behavior data, and 4 machine learning algorithms were used to build classifiers for predicting diagnostic accuracy. RESULTS The Random Forest classifier demonstrated the best overall performance, achieving a test accuracy of 0.81 and area under the receiver-operator characteristic curve of 0.86. Features related to attention distribution and focus on critical regions of interest were found to be important predictors of diagnostic accuracy. Further including case-level and pathologist-level information incrementally improved classifier performance. DISCUSSION Results suggest that pathologists' viewing behavior during digital image evaluation can be leveraged to predict diagnostic accuracy, affording automated feedback and decision support systems based on viewing behavior to aid in training and, ultimately, clinical practice. They also carry implications for basic research examining the interplay between perception, thought, and action in diagnostic decision-making. CONCLUSION The classifiers developed herein have potential applications in training and clinical settings to provide timely feedback and support to pathologists during diagnostic decision-making. Further research could explore the generalizability of these findings to other medical domains and varied levels of expertise.
Collapse
Affiliation(s)
- Tad T Brunyé
- Center for Applied Brain and Cognitive Sciences, Tufts University, Medford, MA 02155, United States
- Department of Psychology, Tufts University, Medford, MA 02155, United States
| | - Kelsey Booth
- Center for Applied Brain and Cognitive Sciences, Tufts University, Medford, MA 02155, United States
| | - Dalit Hendel
- Center for Applied Brain and Cognitive Sciences, Tufts University, Medford, MA 02155, United States
| | - Kathleen F Kerr
- Department of Biostatistics, University of Washington, Seattle, WA 98105, United States
| | - Hannah Shucard
- Department of Biostatistics, University of Washington, Seattle, WA 98105, United States
| | - Donald L Weaver
- Department of Pathology and Laboratory Medicine, Larner College of Medicine, University of Vermont and Vermont Cancer Center, Burlington, VT 05405, United States
| | - Joann G Elmore
- Department of Medicine, David Geffen School of Medicine, University of California, Los Angeles, Los Angeles, CA 90095, United States
| |
Collapse
|
4
|
Sugimoto M, Oyamada M, Tomita A, Inada C, Sato M. Assessing the Link between Nurses' Proficiency and Situational Awareness in Neonatal Care Practice Using an Eye Tracker: An Observational Study Using a Simulator. Healthcare (Basel) 2024; 12:157. [PMID: 38255046 PMCID: PMC10815009 DOI: 10.3390/healthcare12020157] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/14/2023] [Revised: 12/28/2023] [Accepted: 01/04/2024] [Indexed: 01/24/2024] Open
Abstract
Nurses are expected to depend on a wide variety of visually available pieces of patient information to understand situations. Thus, we assumed a relationship between nurses' skills and their gaze trajectories. An observational study using a simulator was conducted to analyze gaze during neonatal care practice using eye tracking. We defined the face, thorax, and abdomen of the neonate, the timer, and the pulse oximeter as areas of interest (AOIs). We compared the eye trajectories for respiration and heart rate assessment between 7 experienced and 13 novice nurses. There were no statistically significant differences in the time spent on each AOI for breathing or heart rate confirmation. However, in novice nurses, we observed a significantly higher number of instances of gazing at the thorax and abdomen. The deviation in the number of instances of gazing at the face was also significantly higher among novice nurses. These results indicate that experienced and novice nurses differ in their gaze movements during situational awareness. These objective and quantitative differences in gaze trajectories may help to establish new educational tools for less experienced nurses.
Collapse
Affiliation(s)
- Masahiro Sugimoto
- Institute for Advanced Biosciences, Keio University, Tsuruoka 997-0052, Japan
- Institute of Medical Sciences, Tokyo Medical University, Shinjuku, Tokyo 160-0022, Japan;
| | - Michiko Oyamada
- Faculty of Human Care Department, Tohto University, 1-1 Hinode-cho, Numazu 410-0032, Japan;
- Department of Nursing, Nihon Institute of Medical Science, Iruma 350-0435, Japan
| | - Atsumi Tomita
- Institute of Medical Sciences, Tokyo Medical University, Shinjuku, Tokyo 160-0022, Japan;
| | - Chiharu Inada
- Faculty of Nursing, Japanese Red Cross College of Nursing, 4-1-3 Hiroo, Shibuya, Tokyo 150-0012, Japan;
| | - Mitsue Sato
- Department of Nursing, Kiryu University, Midori 379-2392, Japan;
| |
Collapse
|
5
|
Cioffi GM, Pinilla-Echeverri N, Sheth T, Sibbald MG. Does artificial intelligence enhance physician interpretation of optical coherence tomography: insights from eye tracking. Front Cardiovasc Med 2023; 10:1283338. [PMID: 38144364 PMCID: PMC10739524 DOI: 10.3389/fcvm.2023.1283338] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/25/2023] [Accepted: 11/20/2023] [Indexed: 12/26/2023] Open
Abstract
Background and objectives The adoption of optical coherence tomography (OCT) in percutaneous coronary intervention (PCI) is limited by need for real-time image interpretation expertise. Artificial intelligence (AI)-assisted Ultreon™ 2.0 software could address this barrier. We used eye tracking to understand how these software changes impact viewing efficiency and accuracy. Methods Eighteen interventional cardiologists and fellows at McMaster University, Canada, were included in the study and categorized as experienced or inexperienced based on lifetime OCT use. They were tasked with reviewing OCT images from both Ultreon™ 2.0 and AptiVue™ software platforms while their eye movements were recorded. Key metrics, such as time to first fixation on the area of interest, total task time, dwell time (time spent on the area of interest as a proportion of total task time), and interpretation accuracy, were evaluated using a mixed multivariate model. Results Physicians exhibited improved viewing efficiency with Ultreon™ 2.0, characterized by reduced time to first fixation (Ultreon™ 0.9 s vs. AptiVue™ 1.6 s, p = 0.007), reduced total task time (Ultreon™ 10.2 s vs. AptiVue™ 12.6 s, p = 0.006), and increased dwell time in the area of interest (Ultreon™ 58% vs. AptiVue™ 41%, p < 0.001). These effects were similar for experienced and inexperienced physicians. Accuracy of OCT image interpretation was preserved in both groups, with experienced physicians outperforming inexperienced physicians. Discussion Our study demonstrated that AI-enabled Ultreon™ 2.0 software can streamline the image interpretation process and improve viewing efficiency for both inexperienced and experienced physicians. Enhanced viewing efficiency implies reduced cognitive load potentially reducing the barriers for OCT adoption in PCI decision-making.
Collapse
Affiliation(s)
| | | | | | - Matthew Gary Sibbald
- Division of Cardiology, Hamilton General Hospital, Hamilton Health Sciences, McMaster University, Hamilton, ON, Canada
| |
Collapse
|
6
|
Sharpe BT, Smith MS, Williams SCR, Talbot J, Runswick OR, Smith J. An expert-novice comparison of lifeguard specific vigilance performance. JOURNAL OF SAFETY RESEARCH 2023; 87:416-430. [PMID: 38081714 DOI: 10.1016/j.jsr.2023.08.014] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 02/03/2023] [Revised: 05/22/2023] [Accepted: 08/28/2023] [Indexed: 12/18/2023]
Abstract
INTRODUCTION Lifeguards must maintain alertness and monitor an aquatic space across extended periods. However, lifeguard research has yet to investigate a lifeguard's ability to maintain performance over time and whether this is influenced by years of certified experience or the detection difficulty of a drowning incident. The aim of this study was to examine whether lifeguard experience, drowning duration, bather number, and time on task influences drowning detection performance. METHOD A total of 30 participants took part in nine 60-minute lifeguard specific tasks that included 11 drowning events occurring at five-minute intervals. Each task had manipulated conditions that acted as the independent variables, including bather number and drowning duration. RESULTS The experienced group detected a greater number of drowning events per task, compared to novice and naïve groups. Findings further highlighted that time, bather number, and drowning duration has a substantial influence on lifeguard specific drowning detection performance. PRACTICAL APPLICATIONS It is hoped that the outcome of the study will have applied application in highlighting the critical need for lifeguard organizations to be aware of a lifeguard's capacity to sustain attention, and for researchers to explore methods for minimizing any decrement in vigilance performance.
Collapse
Affiliation(s)
- Benjamin T Sharpe
- Institute of Sport, Nursing and Allied Health, University of Chichester, Chichester, UK; Institute of Psychology, Business and Human Sciences, University of Chichester, Chichester, UK.
| | - Marcus S Smith
- Institute of Sport, Nursing and Allied Health, University of Chichester, Chichester, UK
| | - Steven C R Williams
- Institute of Psychiatry, Psychology and Neuroscience, King's College London, London, UK
| | - Jo Talbot
- Royal Life Saving Society (RLSS), UK
| | - Oliver R Runswick
- Institute of Psychiatry, Psychology and Neuroscience, King's College London, London, UK
| | - Jenny Smith
- Institute of Sport, Nursing and Allied Health, University of Chichester, Chichester, UK
| |
Collapse
|
7
|
Anton NE, Cha JS, Hernandez E, Athanasiadis D, Yang J, Zhou G, Stefanidis D, Yu D. Utilizing Eye Tracking to Assess Medical Student Non-Technical Performance During Scenario-Based Simulation: Results of a Pilot Study. GLOBAL SURGICAL EDUCATION : JOURNAL OF THE ASSOCIATION FOR SURGICAL EDUCATION 2023; 2:49. [PMID: 38414559 PMCID: PMC10896278 DOI: 10.1007/s44186-023-00127-3] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 09/18/2022] [Revised: 03/09/2023] [Accepted: 03/24/2023] [Indexed: 02/29/2024]
Abstract
Background Non-technical skills (NTS) are essential for safe surgical patient management. However, assessing NTS involves observer-based ratings, which can introduce bias. Eye tracking (ET) has been proposed as an effective method to capture NTS. The purpose of the current study was to determine if ET metrics are associated with NTS performance. Methods Participants wore a mobile ET system and participated in two patient care simulations, where they managed a deteriorating patient. The scenarios featured several challenges to leadership, which were evaluated using a 4-point Likert scale. NTS were evaluated by trained raters using the Non-Technical Skills for Surgeons (NOTSS) scale. ET metrics included percentage of fixations and visits on areas of interest. Results Ten medical students participated. Average visit duration on the patient was negatively correlated with participants' communication and leadership. Average visit duration on the patient's intravenous access was negatively correlated with participants' decision making and situation awareness. Conclusions Our preliminary data suggests that visual attention on the patient was negatively associated with NTS and may indicate poor comprehension of the patient's status due to heightened cognitive load. In future work, researchers and educators should consider using ET to objectively evaluate and provide feedback on their NTS.
Collapse
Affiliation(s)
- Nicholas E Anton
- Department of Surgery, Indiana University School of Medicine, Indianapolis, IN
- School of Industrial Engineering, Purdue University, West Lafayette, IN
| | - Jackie S Cha
- Department of Industrial Engineering, Clemson University, Clemson, SC
| | - Edward Hernandez
- Department of Surgery, Indiana University School of Medicine, Indianapolis, IN
| | | | - Jing Yang
- School of Industrial Engineering, Purdue University, West Lafayette, IN
| | - Guoyang Zhou
- School of Industrial Engineering, Purdue University, West Lafayette, IN
| | | | - Denny Yu
- School of Industrial Engineering, Purdue University, West Lafayette, IN
| |
Collapse
|
8
|
Akerman M, Choudhary S, Liebmann JM, Cioffi GA, Chen RWS, Thakoor KA. Extracting decision-making features from the unstructured eye movements of clinicians on glaucoma OCT reports and developing AI models to classify expertise. Front Med (Lausanne) 2023; 10:1251183. [PMID: 37841006 PMCID: PMC10571140 DOI: 10.3389/fmed.2023.1251183] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/03/2023] [Accepted: 09/14/2023] [Indexed: 10/17/2023] Open
Abstract
This study aimed to investigate the eye movement patterns of ophthalmologists with varying expertise levels during the assessment of optical coherence tomography (OCT) reports for glaucoma detection. Objectives included evaluating eye gaze metrics and patterns as a function of ophthalmic education, deriving novel features from eye-tracking, and developing binary classification models for disease detection and expertise differentiation. Thirteen ophthalmology residents, fellows, and clinicians specializing in glaucoma participated in the study. Junior residents had less than 1 year of experience, while senior residents had 2-3 years of experience. The expert group consisted of fellows and faculty with over 3 to 30+ years of experience. Each participant was presented with a set of 20 Topcon OCT reports (10 healthy and 10 glaucomatous) and was asked to determine the presence or absence of glaucoma and rate their confidence of diagnosis. The eye movements of each participant were recorded as they diagnosed the reports using a Pupil Labs Core eye tracker. Expert ophthalmologists exhibited more refined and focused eye fixations, particularly on specific regions of the OCT reports, such as the retinal nerve fiber layer (RNFL) probability map and circumpapillary RNFL b-scan. The binary classification models developed using the derived features demonstrated high accuracy up to 94.0% in differentiating between expert and novice clinicians. The derived features and trained binary classification models hold promise for improving the accuracy of glaucoma detection and distinguishing between expert and novice ophthalmologists. These findings have implications for enhancing ophthalmic education and for the development of effective diagnostic tools.
Collapse
Affiliation(s)
- Michelle Akerman
- Department of Biomedical Engineering, Columbia University, New York, NY, United States
| | - Sanmati Choudhary
- Department of Computer Science, Columbia University, New York, NY, United States
| | - Jeffrey M. Liebmann
- Edward S. Harkness Eye Institute, Department of Ophthalmology, Columbia University Irving Medical Center, New York, NY, United States
| | - George A. Cioffi
- Edward S. Harkness Eye Institute, Department of Ophthalmology, Columbia University Irving Medical Center, New York, NY, United States
| | - Royce W. S. Chen
- Edward S. Harkness Eye Institute, Department of Ophthalmology, Columbia University Irving Medical Center, New York, NY, United States
| | - Kaveri A. Thakoor
- Department of Biomedical Engineering, Columbia University, New York, NY, United States
- Department of Computer Science, Columbia University, New York, NY, United States
- Edward S. Harkness Eye Institute, Department of Ophthalmology, Columbia University Irving Medical Center, New York, NY, United States
| |
Collapse
|
9
|
Anton NE, Zhou G, Hornbeck T, Nagle AM, Norman S, Shroff AD, Yu D. Detailing experienced nurse decision making during acute patient care simulations. APPLIED ERGONOMICS 2023; 109:103988. [PMID: 36801523 DOI: 10.1016/j.apergo.2023.103988] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 09/28/2022] [Revised: 01/22/2023] [Accepted: 01/26/2023] [Indexed: 06/18/2023]
Abstract
INTRODUCTION Nurse decision making (DM) is critical for patient safety. Eye-tracking methods can effectively assess nurse DM. The purpose of this pilot study was to use eye-tracking methods to assess nurse DM during a clinical simulation. MATERIALS AND METHODS Experienced nurses managed a simulated patient manikin who suffered from a stroke mid-simulation. We assessed nurses' gaze patterns prior to and after the stroke. DM in general was assessed by nursing faculty using a clinical judgement rubric, and dichotomously based on recognition of the stroke or not. RESULTS Data from eight experienced nurses was examined. For the nurses who recognized the stroke, visual attention was focused on the vital sign monitor and patient's head, which suggest those locations were consistently examined for correct decision-makers. CONCLUSIONS Dwell time on general AOIs was associated with poorer DM, which may reflect poorer pattern recognition. Eye-tracking metrics may be effective to objectively assess nurse DM.
Collapse
Affiliation(s)
- Nicholas E Anton
- School of Industrial Engineering, Purdue University, West Lafayette, IN, 47906, USA.
| | - Guoyang Zhou
- School of Industrial Engineering, Purdue University, West Lafayette, IN, 47906, USA
| | - Tera Hornbeck
- School of Nursing, Purdue University, West Lafayette, IN, 47907, USA
| | - Amy M Nagle
- School of Nursing, Purdue University, West Lafayette, IN, 47907, USA
| | - Susan Norman
- School of Nursing, Purdue University, West Lafayette, IN, 47907, USA
| | - Anand D Shroff
- School of Industrial Engineering, Purdue University, West Lafayette, IN, 47906, USA
| | - Denny Yu
- School of Industrial Engineering, Purdue University, West Lafayette, IN, 47906, USA.
| |
Collapse
|
10
|
Brunyé TT, Balla A, Drew T, Elmore JG, Kerr KF, Shucard H, Weaver DL. From Image to Diagnosis: Characterizing Sources of Error in Histopathologic Interpretation. Mod Pathol 2023; 36:100162. [PMID: 36948400 DOI: 10.1016/j.modpat.2023.100162] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/04/2022] [Revised: 02/11/2023] [Accepted: 03/07/2023] [Indexed: 03/24/2023]
Abstract
An accurate histopathologic diagnosis on surgical biopsy material is necessary for the clinical management of patients and has important implications for research, clinical trial design/enrollment, and public health education. This study used a mixed methods approach to isolate sources of diagnostic error while residents and attending pathologists interpreted digitized breast biopsy slides. Ninety participants including pathology residents and attendings at major United States medical centers reviewed a set of 14 digitized whole slide images of breast biopsies. Each case had a consensus-defined diagnosis and critical region of interest (cROI) representing the most significant pathology on the slide. Participants were asked to view unmarked digitized slides, draw their own participant region of interest (pROI), describe its features, and render a diagnosis. Participants' review behavior was tracked using case viewer software and an eye-tracking device. Diagnostic accuracy was calculated in comparison to the consensus diagnosis. We measured the frequency of errors emerging during four interpretive phases: 1) detecting the cROI, 2) recognizing its relevance, 3) using the correct terminology to describe findings in the pROI, and 4) making a diagnostic decision. According to eye tracking data, both trainees and attending pathologists were very likely (about 94% of the time) to find the cROI when inspecting a slide. However, trainees were less likely to consider the cROI relevant to their diagnosis. Pathology trainees were more likely (41% of cases) to use incorrect terminology to describe pROI features than attending pathologists (21% of cases). Failure to accurately describe features was the only factor strongly associated with an incorrect diagnosis. Identifying where errors emerge in the interpretive and/or descriptive process and working on building organ-specific feature recognition and verbal fluency in describing those features are critical steps for achieving competency in diagnostic decision making.
Collapse
Affiliation(s)
- Tad T Brunyé
- Center for Applied Brain and Cognitive Sciences, Tufts University, 177 College Ave., Suite 090, Medford, MA 02155; Department of Psychology, Tufts University, 490 Boston Ave., Medford, MA 02155.
| | - Agnes Balla
- Department of Pathology, University of Vermont and Vermont Cancer Center, 89 Beaumont Ave., Burlington, VT 05405
| | - Trafton Drew
- Department of Psychology, University of Utah, 380 S 1530 E Beh S 502, Salt Lake City, UT 84112
| | - Joann G Elmore
- David Geffen School of Medicine, Department of Medicine, University of California, Los Angeles, 885 Tiverton Drive, Los Angeles, CA 90095
| | - Kathleen F Kerr
- Department of Biostatistics, University of Washington, 1705 NE Pacific Street, Seattle, WA 98195
| | - Hannah Shucard
- Department of Biostatistics, University of Washington, 1705 NE Pacific Street, Seattle, WA 98195
| | - Donald L Weaver
- Department of Pathology, University of Vermont and Vermont Cancer Center, 89 Beaumont Ave., Burlington, VT 05405
| |
Collapse
|
11
|
Who and what messages are more suitable for health ads: the combined influence of endorsers and message framing on visual attention and ad effectiveness. ASLIB J INFORM MANAG 2023. [DOI: 10.1108/ajim-03-2022-0108] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 03/13/2023]
Abstract
PurposeThe author examined effects of endorser type and message framing on visual attention and ad effectiveness in health ads, including the moderator of involvement. This paper aims to discuss this issue.Design/methodology/approachAn experiment was conducted with a 2 (celebrity vs. expert) × 2 (positive vs. negative framing) between-subject factorial design. Eye-tracking measured visual attention and a questionnaire measured ad effectiveness and product involvement.FindingsExperimental data from 78 responses showed no vampire effect in the health advertisements. Celebrity endorsement with negative message framing received more attention and had less ad recall than that with positive message framing. Negative and positive message framing attracted the same amount of attention and ad recall in the expert endorsement condition. High involvement participants paid more attention to the ad message with the expert than that with the celebrity, but ad recall was not significantly increased. Low involvement participants exhibited the same attention to the ad message with the expert and with the celebrity, but had greater recall of the ad message with the expert. Visual attention to the endorser was associated with ad attitude but not with ad recall. Ad attitude impacted behavioral intention.Originality/valueStudies examining influences of celebrity and message framing on ad effectiveness have focused on the response to advertising stimuli, not the information process. The author provides empirical evidence of the viewers' information processing of endorsers and health messages, and its relationship with ad effectiveness. The study contributes to the literature by combining endorser and message framing in health ads to promote public health communication from the information processing perspective.
Collapse
|
12
|
Brunyé TT, Drew T, Kerr KF, Shucard H, Powell K, Weaver DL, Elmore JG. Zoom behavior during visual search modulates pupil diameter and reflects adaptive control states. PLoS One 2023; 18:e0282616. [PMID: 36893083 PMCID: PMC9997932 DOI: 10.1371/journal.pone.0282616] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/02/2022] [Accepted: 02/19/2023] [Indexed: 03/10/2023] Open
Abstract
Adaptive gain theory proposes that the dynamic shifts between exploration and exploitation control states are modulated by the locus coeruleus-norepinephrine system and reflected in tonic and phasic pupil diameter. This study tested predictions of this theory in the context of a societally important visual search task: the review and interpretation of digital whole slide images of breast biopsies by physicians (pathologists). As these medical images are searched, pathologists encounter difficult visual features and intermittently zoom in to examine features of interest. We propose that tonic and phasic pupil diameter changes during image review may correspond to perceived difficulty and dynamic shifts between exploration and exploitation control states. To examine this possibility, we monitored visual search behavior and tonic and phasic pupil diameter while pathologists (N = 89) interpreted 14 digital images of breast biopsy tissue (1,246 total images reviewed). After viewing the images, pathologists provided a diagnosis and rated the level of difficulty of the image. Analyses of tonic pupil diameter examined whether pupil dilation was associated with pathologists' difficulty ratings, diagnostic accuracy, and experience level. To examine phasic pupil diameter, we parsed continuous visual search data into discrete zoom-in and zoom-out events, including shifts from low to high magnification (e.g., 1× to 10×) and the reverse. Analyses examined whether zoom-in and zoom-out events were associated with phasic pupil diameter change. Results demonstrated that tonic pupil diameter was associated with image difficulty ratings and zoom level, and phasic pupil diameter showed constriction upon zoom-in events, and dilation immediately preceding a zoom-out event. Results are interpreted in the context of adaptive gain theory, information gain theory, and the monitoring and assessment of physicians' diagnostic interpretive processes.
Collapse
Affiliation(s)
- Tad T. Brunyé
- Center for Applied Brain and Cognitive Sciences, Tufts University, Medford, MA, United States of America
| | - Trafton Drew
- Department of Psychology, University of Utah, Salt Lake City, UT, United States of America
| | - Kathleen F. Kerr
- Department of Biostatistics, University of Washington, Seattle, WA, United States of America
| | - Hannah Shucard
- Department of Biostatistics, University of Washington, Seattle, WA, United States of America
| | - Kate Powell
- Center for Applied Brain and Cognitive Sciences, Tufts University, Medford, MA, United States of America
| | - Donald L. Weaver
- Department of Pathology, University of Vermont and Vermont Cancer Center, Burlington, VT, United States of America
| | - Joann G. Elmore
- David Geffen School of Medicine, Department of Medicine, University of California, Los Angeles, CA, United States of America
| |
Collapse
|
13
|
Sugimoto M, Tomita A, Oyamada M, Sato M. Eye-Tracking-Based Analysis of Situational Awareness of Nurses. Healthcare (Basel) 2022; 10:2131. [PMID: 36360472 PMCID: PMC9690882 DOI: 10.3390/healthcare10112131] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/22/2022] [Revised: 10/17/2022] [Accepted: 10/24/2022] [Indexed: 04/11/2024] Open
Abstract
BACKGROUND Nurses are responsible for comprehensively identifying patient conditions and associated environments. We hypothesize that gaze trajectories of nurses differ based on their experiences, even under the same situation. METHODS An eye-tracking device monitored the gaze trajectories of nurses with various levels of experience, and nursing students during the intravenous injection task on a human patient simulator. RESULTS The areas of interest (AOIs) were identified in the recorded movies, and the gaze durations of AOIs showed different patterns between experienced nurses and nursing students. A state transition diagram visualized the recognition errors of the students and the repeated confirmation of the vital signs of the patient simulator. Clustering analysis of gaze durations also indicated similarity among the participants with similar experiences. CONCLUSIONS As expected, gaze trajectories differed among the participants. The developed gaze transition diagram visualized their differences and helped in interpreting their situational awareness based on visual perception. The demonstrated method can help in establishing an effective nursing education, particularly for learning the skills that are difficult to be verbalized.
Collapse
Affiliation(s)
- Masahiro Sugimoto
- Institute of Medical Sciences, Tokyo Medical University, Shinjuku, Tokyo 160-0022, Japan
- Institute for Advanced Biosciences, Keio University, Tsuruoka 997-0052, Japan
| | - Atsumi Tomita
- Institute of Medical Sciences, Tokyo Medical University, Shinjuku, Tokyo 160-0022, Japan
| | - Michiko Oyamada
- Department of Nursing, Nihon Institute of Medical Science, Moroyama 350-0435, Japan
| | - Mitsue Sato
- Department of Nursing, Kiryu University, Midori 379-2392, Japan
| |
Collapse
|
14
|
Abstract
With the increasing need for eye tracking in head-mounted virtual reality displays, the gaze-based modality has the potential to predict user intention and unlock intuitive new interaction schemes. In the present work, we explore whether gaze-based data and hand-eye coordination data can predict a user’s interaction intention with the digital world, which could be used to develop predictive interfaces. We validate it on the eye-tracking data collected from 10 participants in item selection and teleporting tasks in virtual reality. We demonstrate successful prediction of the onset of item selection and teleporting with an 0.943 F1-Score using a Gradient Boosting Decision Tree, which is the best among the four classifiers compared, while the model size of the Support Vector Machine is the smallest. It is also proven that hand-eye-coordination-related features can improve interaction intention recognition in virtual reality environments.
Collapse
|
15
|
Aust J, Mitrovic A, Pons D. Assessment of the Effect of Cleanliness on the Visual Inspection of Aircraft Engine Blades: An Eye Tracking Study. SENSORS (BASEL, SWITZERLAND) 2021; 21:6135. [PMID: 34577343 PMCID: PMC8473167 DOI: 10.3390/s21186135] [Citation(s) in RCA: 7] [Impact Index Per Article: 2.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 08/18/2021] [Revised: 09/03/2021] [Accepted: 09/07/2021] [Indexed: 01/20/2023]
Abstract
Background-The visual inspection of aircraft parts such as engine blades is crucial to ensure safe aircraft operation. There is a need to understand the reliability of such inspections and the factors that affect the results. In this study, the factor 'cleanliness' was analysed among other factors. Method-Fifty industry practitioners of three expertise levels inspected 24 images of parts with a variety of defects in clean and dirty conditions, resulting in a total of N = 1200 observations. The data were analysed statistically to evaluate the relationships between cleanliness and inspection performance. Eye tracking was applied to understand the search strategies of different levels of expertise for various part conditions. Results-The results show an inspection accuracy of 86.8% and 66.8% for clean and dirty blades, respectively. The statistical analysis showed that cleanliness and defect type influenced the inspection accuracy, while expertise was surprisingly not a significant factor. In contrast, inspection time was affected by expertise along with other factors, including cleanliness, defect type and visual acuity. Eye tracking revealed that inspectors (experts) apply a more structured and systematic search with less fixations and revisits compared to other groups. Conclusions-Cleaning prior to inspection leads to better results. Eye tracking revealed that inspectors used an underlying search strategy characterised by edge detection and differentiation between surface deposits and other types of damage, which contributed to better performance.
Collapse
Affiliation(s)
- Jonas Aust
- Department of Mechanical Engineering, University of Canterbury, Christchurch 8041, New Zealand;
| | - Antonija Mitrovic
- Department of Computer Science and Software Engineering, University of Canterbury, Christchurch 8041, New Zealand;
| | - Dirk Pons
- Department of Mechanical Engineering, University of Canterbury, Christchurch 8041, New Zealand;
| |
Collapse
|
16
|
Visweswaran S, King AJ, Tajgardoon M, Calzoni L, Clermont G, Hochheiser H, Cooper GF. Evaluation of eye tracking for a decision support application. JAMIA Open 2021; 4:ooab059. [PMID: 34350394 PMCID: PMC8327376 DOI: 10.1093/jamiaopen/ooab059] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/18/2020] [Revised: 05/08/2021] [Accepted: 07/01/2021] [Indexed: 11/12/2022] Open
Abstract
Eye tracking is used widely to investigate attention and cognitive processes while performing tasks in electronic medical record (EMR) systems. We explored a novel application of eye tracking to collect training data for a machine learning-based clinical decision support tool that predicts which patient data are likely to be relevant for a clinical task. Specifically, we investigated in a laboratory setting the accuracy of eye tracking compared to manual annotation for inferring which patient data in the EMR are judged to be relevant by physicians. We evaluated several methods for processing gaze points that were recorded using a low-cost eye-tracking device. Our results show that eye tracking achieves accuracy and precision of 69% and 53%, respectively compared to manual annotation and are promising for machine learning. The methods for processing gaze points and scripts that we developed offer a first step in developing novel uses for eye tracking for clinical decision support.
Collapse
Affiliation(s)
- Shyam Visweswaran
- Department of Biomedical Informatics, University of Pittsburgh, Pittsburgh, Pennsylvania, USA.,Intelligent Systems Program, University of Pittsburgh, Pittsburgh, Pennsylvania, USA
| | - Andrew J King
- Department of Biomedical Informatics, University of Pittsburgh, Pittsburgh, Pennsylvania, USA.,Department of Critical Care Medicine, University of Pittsburgh, Pittsburgh, Pennsylvania, USA
| | | | - Luca Calzoni
- Department of Biomedical Informatics, University of Pittsburgh, Pittsburgh, Pennsylvania, USA
| | - Gilles Clermont
- Department of Critical Care Medicine, University of Pittsburgh, Pittsburgh, Pennsylvania, USA
| | - Harry Hochheiser
- Department of Biomedical Informatics, University of Pittsburgh, Pittsburgh, Pennsylvania, USA.,Intelligent Systems Program, University of Pittsburgh, Pittsburgh, Pennsylvania, USA
| | - Gregory F Cooper
- Department of Biomedical Informatics, University of Pittsburgh, Pittsburgh, Pennsylvania, USA.,Intelligent Systems Program, University of Pittsburgh, Pittsburgh, Pennsylvania, USA
| |
Collapse
|
17
|
Wu W, Hall AK, Braund H, Bell CR, Szulewski A. The Development of Visual Expertise in ECG Interpretation: An Eye-Tracking Augmented Re Situ Interview Approach. TEACHING AND LEARNING IN MEDICINE 2021; 33:258-269. [PMID: 33302734 DOI: 10.1080/10401334.2020.1844009] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/12/2023]
Abstract
Phenomenon: Visual expertise in medicine involves a complex interplay between expert visual behavior patterns and higher-level cognitive processes. Previous studies of visual expertise in medicine have centered around traditionally visually intensive disciplines such as radiology and pathology. However, there is limited study of visual expertise in electrocardiogram (ECG) interpretation, a common clinical task that is associated with high error rates. This qualitatively driven multi-methods study aimed to describe differences in cognitive approaches to ECG interpretation between medical students, emergency medicine (EM) residents, and EM attending physicians. Approach: Ten medical students, 10 EM residents, and 10 EM attending physicians were recruited from one tertiary academic center to participate in this study. Participants interpreted 10 ECGs with a screen-based eye-tracking device, then underwent a subjective re situ interview augmented by playback of the participants' own gaze scan-paths via eye-tracking. Interviews were transcribed verbatim and an emergent thematic analysis was performed across participant groups. Diagnostic speed, accuracy, and heat maps of fixation distribution were collected to supplement the qualitative findings. Findings: Qualitative analysis demonstrated differences among the cohorts in three major themes: dual-process reasoning, ability to prioritize, and clinical implications. These qualitative findings were aligned with differences in visual behavior demonstrated by heat maps of fixation distribution across each ECG. More experienced participants completed ECG interpretation significantly faster and more accurately than less experienced participants. Insights: The cognitive processes related to ECG interpretation differed between novices and more experienced providers in EM. Understanding the differences in cognitive approaches to ECG interpretation between these groups may help inform best practices in teaching this ubiquitous diagnostic skill.
Collapse
Affiliation(s)
- William Wu
- School of Medicine, Queen's University, Kingston, Ontario, Canada
| | - Andrew K Hall
- Department of Emergency Medicine, Kingston Health Science Center, Queen's University, Kingston, Ontario, Canada
| | - Heather Braund
- Office of Professional Development and Educational Scholarship, Faculty of Health Sciences, Faculty of Education, Queen's University, Kingston, Ontario, Canada
| | - Colin R Bell
- Department of Emergency Medicine, Kingston Health Science Center, Queen's University, Kingston, Ontario, Canada
| | - Adam Szulewski
- Department of Emergency Medicine, Kingston Health Science Center, Queen's University, Kingston, Ontario, Canada
| |
Collapse
|
18
|
Lee A, Chung H, Cho Y, Kim JL, Choi J, Lee E, Kim B, Cho SJ, Kim SG. Identification of gaze pattern and blind spots by upper gastrointestinal endoscopy using an eye-tracking technique. Surg Endosc 2021; 36:2574-2581. [PMID: 34013392 DOI: 10.1007/s00464-021-08546-3] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/14/2021] [Accepted: 05/04/2021] [Indexed: 11/24/2022]
Abstract
BACKGROUND The lesion detection rate of esophagogastroduodenoscopy (EGD) varies depending on the degree of experience of the endoscopist and anatomical blind spots. This study aimed to identify gaze patterns and blind spots by analyzing the endoscopist's gaze during real-time EGD. METHODS Five endoscopists were enrolled in this study. The endoscopist's eye gaze tracked by an eye tracker was selected from the esophagogastric junction to the second portion of the duodenum without the esophagus during insertion and withdrawal, and then matched with photos. Gaze patterns were visualized as a gaze plot, blind spot detection as a heatmap, observation time (OT), fixation duration (FD), and FD-to-OT ratio. RESULTS The mean OT and FD were 11.10 ± 11.14 min and 8.37 ± 9.95 min, respectively, and the FD-to-OT ratio was 72.5%. A total of 34.3% of the time was spent observing the antrum. When observing the body of the stomach, it took longer to observe the high body in the retroflexion view and the low-to-mid body in the forward view. CONCLUSIONS It is necessary to minimize gaze distraction and observe the posterior wall in the retroflexion view. Our results suggest that eye-tracking techniques may be useful for future endoscopic training and education.
Collapse
Affiliation(s)
- Ayoung Lee
- Division of Gastroenterology, Department of Internal Medicine and Liver Research Institute, Seoul National University College of Medicine, Seoul, Republic of Korea.,Department of Internal Medicine, Ewha Womans University School of Medicine, Seoul, Republic of Korea
| | - Hyunsoo Chung
- Division of Gastroenterology, Department of Internal Medicine and Liver Research Institute, Seoul National University College of Medicine, Seoul, Republic of Korea.
| | - Yejin Cho
- Seoul National University College of Medicine, Seoul, Republic of Korea
| | - Jue Lie Kim
- Division of Gastroenterology, Department of Internal Medicine and Liver Research Institute, Seoul National University College of Medicine, Seoul, Republic of Korea
| | - Jinju Choi
- Division of Gastroenterology, Department of Internal Medicine and Liver Research Institute, Seoul National University College of Medicine, Seoul, Republic of Korea
| | - Eunwoo Lee
- Division of Gastroenterology, Department of Internal Medicine and Liver Research Institute, Seoul National University College of Medicine, Seoul, Republic of Korea
| | - Bokyung Kim
- Division of Gastroenterology, Department of Internal Medicine and Liver Research Institute, Seoul National University College of Medicine, Seoul, Republic of Korea
| | - Soo-Jeong Cho
- Division of Gastroenterology, Department of Internal Medicine and Liver Research Institute, Seoul National University College of Medicine, Seoul, Republic of Korea
| | - Sang Gyun Kim
- Division of Gastroenterology, Department of Internal Medicine and Liver Research Institute, Seoul National University College of Medicine, Seoul, Republic of Korea
| |
Collapse
|
19
|
Rogers BA, Franklin AE. Cognitive load experienced by nurses in simulation-based learning experiences: An integrative review. NURSE EDUCATION TODAY 2021; 99:104815. [PMID: 33640776 DOI: 10.1016/j.nedt.2021.104815] [Citation(s) in RCA: 12] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 05/16/2020] [Revised: 01/11/2021] [Accepted: 02/03/2021] [Indexed: 06/12/2023]
Abstract
BACKGROUND Simulation based learning experiences help nurses gain skills necessary for independent practice. However, increased cognitive load placed on learners in simulation may affect learning outcomes. OBJECTIVES The purpose of this integrative review was to synthesize what is known about nurses' cognitive load in simulation and summarize measurement approaches. DATA SOURCES A search of CINAHL, Medline, ProQuest Nursing and Allied Health, and ERIC databases was limited to peer-reviewed studies published after 2006 in the English language, using the key words nurse, simulation, and cognitive load. REVIEW METHODS Whittemore and Knafl's (2005) integrative review method was used. Studies investigating advanced practice nurses or interprofessional teams were excluded. RESULTS Database and reference lists searches identified a total of 3077 records, and 20 met inclusion criteria. Simulation fidelity, time pressure, dual-tasking, interruptions, task complexity, distractions, and mismatched simulation objectives to learner ability increase nurses' cognitive load. However, past experience, pre-briefing, repeated scenarios, and worked-out modeling optimize cognitive load. Subjective and objective cognitive load measures help researchers understand cognitive load and define its relationship with other variables. CONCLUSIONS Simulation impacts nurses' cognitive load. Varying simulation designs to optimize cognitive load will improve learning outcomes. Future nursing simulation research should utilize well-validated cognitive load measures and measure cognitive load alongside other variables to further understand how cognitive load affects simulation outcomes.
Collapse
Affiliation(s)
- Beth A Rogers
- Texas Christian University, Fort Worth, TX, United States of America.
| | - Ashley E Franklin
- Texas Christian University, Fort Worth, TX, United States of America.
| |
Collapse
|
20
|
Eder TF, Richter J, Scheiter K, Huettig F, Keutel C. Comparing radiographs with signaling improves anomaly detection of dental students: An eye‐tracking study. APPLIED COGNITIVE PSYCHOLOGY 2021. [DOI: 10.1002/acp.3819] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/18/2022]
Affiliation(s)
- Thésése F. Eder
- non‐university research institute Leibniz‐Institut für Wissensmedien Tübingen Germany
| | - Juliane Richter
- non‐university research institute Leibniz‐Institut für Wissensmedien Tübingen Germany
| | - Katharina Scheiter
- non‐university research institute Leibniz‐Institut für Wissensmedien Tübingen Germany
- University of Tübingen Tübingen Germany
| | - Fabian Huettig
- Department of Prosthodontics, Centre for Dentistry, Oral Medicine, and Maxillofacial Surgery, University Hospital Tübingen University of Tübingen Tübingen Germany
| | - Constanze Keutel
- Department of Oral‐ and Maxillofacial Radiology, Centre for Dentistry, Oral Medicine, and Maxillofacial Surgery, University Hospital Tübingen University of Tübingen Tübingen Germany
| |
Collapse
|
21
|
Eder TF, Richter J, Scheiter K, Keutel C, Castner N, Kasneci E, Huettig F. How to support dental students in reading radiographs: effects of a gaze-based compare-and-contrast intervention. ADVANCES IN HEALTH SCIENCES EDUCATION : THEORY AND PRACTICE 2021; 26:159-181. [PMID: 32488458 PMCID: PMC8238744 DOI: 10.1007/s10459-020-09975-w] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.7] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 04/15/2019] [Accepted: 05/20/2020] [Indexed: 06/10/2023]
Abstract
In dental medicine, interpreting radiographs (i.e., orthopantomograms, OPTs) is an error-prone process, even in experts. Effective intervention methods are therefore needed to support students in improving their image reading skills for OPTs. To this end, we developed a compare-and-contrast intervention, which aimed at supporting students in achieving full coverage when visually inspecting OPTs and, consequently, obtaining a better diagnostic performance. The comparison entailed a static eye movement visualization (heat map) on an OPT showing full gaze coverage from a peer-model (other student) and another heat map showing a student's own gaze behavior. The intervention group (N = 38) compared five such heat map combinations, whereas the control group (N = 23) diagnosed five OPTs. Prior to the experimental variation (pre-test) and after it (post-test), students in both conditions searched for anomalies in OPTs while their gaze was recorded. Results showed that students in the intervention group covered more areas of the OPTs and looked less often and for a shorter amount of time at anomalies after the intervention. Furthermore, they fixated on low-prevalence anomalies earlier and high-prevalence anomalies later during the inspection. However, the students in the intervention group did not show any meaningful improvement in detection rate and made more false positive errors compared to the control group. Thus, the intervention guided visual attention but did not improve diagnostic performance substantially. Exploratory analyses indicated that further interventions should teach knowledge about anomalies rather than focusing on full coverage of radiographs.
Collapse
Affiliation(s)
- Thérése F Eder
- Leibniz-Institut für Wissensmedien, Schleichstraße 6, 72076, Tübingen, Germany.
| | - Juliane Richter
- Leibniz-Institut für Wissensmedien, Schleichstraße 6, 72076, Tübingen, Germany
| | - Katharina Scheiter
- Leibniz-Institut für Wissensmedien, Schleichstraße 6, 72076, Tübingen, Germany
- University of Tübingen, Tübingen, Germany
| | - Constanze Keutel
- Department of Oral- and Maxillofacial Radiology, Centre for Dentistry, Oral Medicine, and Maxillofacial Surgery, University Hospital Tübingen, University of Tübingen, Tübingen, Germany
| | - Nora Castner
- Perception Engineering, Department of Computer Science, University of Tübingen, Tübingen, Germany
| | - Enkelejda Kasneci
- Perception Engineering, Department of Computer Science, University of Tübingen, Tübingen, Germany
| | - Fabian Huettig
- Department of Prosthodontics, Centre for Dentistry, Oral Medicine, and Maxillofacial Surgery, University Hospital Tübingen, University of Tübingen, Tübingen, Germany
| |
Collapse
|
22
|
An Immersive Virtual Reality Game for Predicting Risk Taking through the Use of Implicit Measures. APPLIED SCIENCES-BASEL 2021. [DOI: 10.3390/app11020825] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 02/07/2023]
Abstract
Risk taking (RT) measurement constitutes a challenge for researchers and practitioners and has been addressed from different perspectives. Personality traits and temperamental aspects such as sensation seeking and impulsivity influence the individual’s approach to RT, prompting risk-seeking or risk-aversion behaviors. Virtual reality has emerged as a suitable tool for RT measurement, since it enables the exposure of a person to realistic risks, allowing embodied interactions, the application of stealth assessment techniques and physiological real-time measurement. In this article, we present the assessment on decision making in risk environments (AEMIN) tool, as an enhanced version of the spheres and shield maze task, a previous tool developed by the authors. The main aim of this article is to study whether it is possible is to discriminate participants with high versus low scores in the measures of personality, sensation seeking and impulsivity, through their behaviors and physiological responses during playing AEMIN. Applying machine learning methods to the dataset we explored: (a) if through these data it is possible to discriminate between the two populations in each variable; and (b) which parameters better discriminate between the two populations in each variable. The results support the use of AEMIN as an ecological assessment tool to measure RT, since it brings to light behaviors that allow to classify the subjects into high/low risk-related psychological constructs. Regarding physiological measures, galvanic skin response seems to be less salient in prediction models.
Collapse
|
23
|
Monteiro S, Logiudice A, Sibbald M. The unexplored value of “Normal”: A commentary on the lack of normal cases in high-stakes assessment. ARCHIVES OF MEDICINE AND HEALTH SCIENCES 2021. [DOI: 10.4103/amhs.amhs_106_21] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/04/2022] Open
|
24
|
Prokop M, Pilař L, Tichá I. Impact of Think-Aloud on Eye-Tracking: A Comparison of Concurrent and Retrospective Think-Aloud for Research on Decision-Making in the Game Environment. SENSORS (BASEL, SWITZERLAND) 2020; 20:E2750. [PMID: 32408507 PMCID: PMC7294419 DOI: 10.3390/s20102750] [Citation(s) in RCA: 7] [Impact Index Per Article: 1.8] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 03/30/2020] [Revised: 05/08/2020] [Accepted: 05/11/2020] [Indexed: 12/03/2022]
Abstract
Simulations and games bring the possibility to research complex processes of managerial decision-making. However, this modern field requires adequate methodological procedures. Many authors recommend the use of a combination of concurrent think-aloud (CTA) or retrospective think-aloud (RTA) with eye-tracking to investigate cognitive processes such as decision-making. Nevertheless, previous studies have little or no consideration of the possible differential impact of both think-aloud methods on data provided by eye-tracking. Therefore, the main aim of this study is to compare and assess if and how these methods differ in terms of their impact on eye-tracking. The experiment was conducted for this purpose. Participants were 14 managers who played a specific simulation game with CTA use and 17 managers who played the same game with RTA use. The results empirically prove that CTA significantly distorts data provided by eye-tracking, whereas data gathered when RTA is used, provide independent pieces of evidence about the participants' behavior. These findings suggest that RTA is more suitable for combined use with eye-tracking for the purpose of the research of decision-making in the game environment.
Collapse
Affiliation(s)
- Michal Prokop
- Department of Management, Faculty of Economics and Management, Czech University of Life Sciences Prague, 165 21 Prague, Czech Republic; (L.P.); (I.T.)
| | | | | |
Collapse
|
25
|
Hayashi K, Aono S, Fujiwara M, Shiro Y, Ushida T. Difference in eye movements during gait analysis between professionals and trainees. PLoS One 2020; 15:e0232246. [PMID: 32353030 PMCID: PMC7192381 DOI: 10.1371/journal.pone.0232246] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/23/2019] [Accepted: 04/11/2020] [Indexed: 11/18/2022] Open
Abstract
INTRODUCTION Observational gait analysis is a widely used skill in physical therapy. Meanwhile, the skill has not been investigated using objective assessments. The present study investigated the differences in eye movement between professionals and trainees, while observing gait analysis. METHODS The participants included in this study were 26 professional physical therapists and 26 physical therapist trainees. The participants, wearing eye tracker systems, were asked to describe gait abnormalities of a patient as much as possible. The eye movement parameters of interest were fixation count, average fixation duration, and total fixation duration. RESULTS The number of gait abnormalities described was significantly higher in professionals than in trainees, overall and in limbs of the patient. The fixation count was significantly higher in professionals when compared to trainees. Additionally, the average fixation duration and total fixation duration were significantly shorter in professionals. Conversely, in trunks, the number of gait abnormalities and eye movements showed no significant differences between groups. CONCLUSIONS Professionals require shorter fixation durations on areas of interest than trainees, while describing a higher number of gait abnormalities.
Collapse
Affiliation(s)
- Kazuhiro Hayashi
- Multidisciplinary Pain Center, Aichi Medical University, Nagakute, Japan
- Department of Rehabilitation, Aichi Medical University Hospital, Nagakute, Japan
| | - Shuichi Aono
- Multidisciplinary Pain Center, Aichi Medical University, Nagakute, Japan
- Department of Pain Data Management, Aichi Medical University, Nagakute, Japan
| | - Mitsuhiro Fujiwara
- Department of Rehabilitation, Kamiiida Rehabilitation Hospital, Nagoya, Japan
| | - Yukiko Shiro
- Multidisciplinary Pain Center, Aichi Medical University, Nagakute, Japan
- Department of Physical Therapy, Faculty of Rehabilitation Sciences, Nagoya Gakuin University, Nagoya, Japan
| | - Takahiro Ushida
- Multidisciplinary Pain Center, Aichi Medical University, Nagakute, Japan
| |
Collapse
|
26
|
Eysenck MW, Keane MT. Problem solving and expertise. Cogn Psychol 2020. [DOI: 10.4324/9781351058513-16] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/11/2022]
|
27
|
Al-Moteri M, Cooper S, Symmons M, Plummer V. Nurses' cognitive and perceptual bias in the identification of clinical deterioration cues. Aust Crit Care 2019; 33:333-342. [PMID: 31615698 DOI: 10.1016/j.aucc.2019.08.006] [Citation(s) in RCA: 11] [Impact Index Per Article: 2.2] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/01/2018] [Revised: 08/15/2019] [Accepted: 08/29/2019] [Indexed: 11/29/2022] Open
Abstract
BACKGROUND Perception and processing of clinical cues have rarely been investigated in the nursing literature despite their relevance to the early identification and management of clinical deterioration. AIM This study used a hypovolemic shock scenario from the Feedback Incorporating Review and Simulation Techniques to Act on Clinical Trends (FIRST2ACT) virtual simulation program, equipped with an eye tracker, to investigate cue processing during the management of patient deterioration. RESULT The study revealed that attention deviation distorted interpretation of subsequent cues, causing 63% of participants to exhibit a cognitive bias (heightened sensitivity to specific but noncritical cues) and 65% to exhibit at least one episode of nonfixation on clinically relevant cues. Attention deviation and distorted interpretations of clinical cues will have an impact on patient safety. CONCLUSION The findings are likely to have important implications for understanding error and associated training implications.
Collapse
Affiliation(s)
- Modi Al-Moteri
- Nursing Department, Faculty of Applied Medical Sciences, University of Al-Taif, Western Region, Saudi Arabia; School of Nursing and Midwifery, Monash University, Australia.
| | - Simon Cooper
- Emergency Care and Research Development, School of Nursing and Midwifery and Healthcare, Federation University, Victoria, Australia; School of Nursing, University of Hong Kong, Hong Kong, China; School of Nursing and Midwifery, University of Brighton, UK
| | | | - Virginia Plummer
- School of Nursing and Midwifery, Monash University, Australia; Peninsula Health, Frankston, Australia
| |
Collapse
|
28
|
Brunyé TT, Nallamothu BK, Elmore JG. Eye-tracking for assessing medical image interpretation: A pilot feasibility study comparing novice vs expert cardiologists. PERSPECTIVES ON MEDICAL EDUCATION 2019; 8:65-73. [PMID: 30977060 PMCID: PMC6468026 DOI: 10.1007/s40037-019-0505-6] [Citation(s) in RCA: 9] [Impact Index Per Article: 1.8] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Indexed: 06/09/2023]
Abstract
INTRODUCTION As specialized medical professionals such as radiologists, pathologists, and cardiologists gain education and experience, their diagnostic efficiency and accuracy change, and they show altered eye movement patterns during medical image interpretation. Existing research in this area is limited to interpretation of static medical images, such as digitized whole slide biopsies, making it difficult to understand how expertise development might manifest during dynamic image interpretation, such as with angiograms or volumetric scans. METHODS A two-group (novice, expert) comparative pilot study examined the feasibility and utility of tracking and interpreting eye movement patterns while cardiologists viewed video-based coronary angiograms. A non-invasive eye tracking system recorded cardiologists' (n = 8) visual behaviour while they viewed and diagnosed a series of eight angiogram videos. Analyses assessed frame-by-frame video navigation behaviour, eye fixation behaviour, and resulting diagnostic decision making. RESULTS Relative to novices, expert cardiologists demonstrated shorter and less variable video review times, fewer eye fixations and saccadic eye movements, and less time spent paused on individual video frames. Novices showed repeated eye fixations on critical image frames and regions, though these were not predictive of accurate diagnostic decisions. DISCUSSION These preliminary results demonstrate interpretive decision errors among novices, suggesting they identify and process critical diagnostic features, but sometimes fail to accurately interpret those features. Results also showcase the feasibility of tracking and understanding eye movements during video-based coronary angiogram interpretation and suggest that eye tracking may be valuable for informing assessments of competency progression during medical education and training.
Collapse
Affiliation(s)
- Tad T. Brunyé
- Center for Applied Brain & Cognitive Sciences, Tufts University, Medford, MA USA
| | | | - Joann G. Elmore
- Department of Medicine, University of Washington, Seattle, WA USA
| |
Collapse
|
29
|
Brunyé TT, Drew T, Weaver DL, Elmore JG. A review of eye tracking for understanding and improving diagnostic interpretation. COGNITIVE RESEARCH-PRINCIPLES AND IMPLICATIONS 2019; 4:7. [PMID: 30796618 PMCID: PMC6515770 DOI: 10.1186/s41235-019-0159-2] [Citation(s) in RCA: 49] [Impact Index Per Article: 9.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 11/05/2018] [Accepted: 02/01/2019] [Indexed: 12/29/2022]
Abstract
Inspecting digital imaging for primary diagnosis introduces perceptual and cognitive demands for physicians tasked with interpreting visual medical information and arriving at appropriate diagnoses and treatment decisions. The process of medical interpretation and diagnosis involves a complex interplay between visual perception and multiple cognitive processes, including memory retrieval, problem-solving, and decision-making. Eye-tracking technologies are becoming increasingly available in the consumer and research markets and provide novel opportunities to learn more about the interpretive process, including differences between novices and experts, how heuristics and biases shape visual perception and decision-making, and the mechanisms underlying misinterpretation and misdiagnosis. The present review provides an overview of eye-tracking technology, the perceptual and cognitive processes involved in medical interpretation, how eye tracking has been employed to understand medical interpretation and promote medical education and training, and some of the promises and challenges for future applications of this technology.
Collapse
Affiliation(s)
- Tad T Brunyé
- Center for Applied Brain and Cognitive Sciences, Tufts University, 200 Boston Ave., Suite 3000, Medford, MA, 02155, USA.
| | - Trafton Drew
- Department of Psychology, University of Utah, 380 1530 E, Salt Lake City, UT, 84112, USA
| | - Donald L Weaver
- Department of Pathology and University of Vermont Cancer Center, University of Vermont, 111 Colchester Ave., Burlington, VT, 05401, USA
| | - Joann G Elmore
- Department of Medicine, David Geffen School of Medicine at UCLA, University of California at Los Angeles, 10833 Le Conte Ave., Los Angeles, CA, 90095, USA
| |
Collapse
|
30
|
Khosravan N, Celik H, Turkbey B, Jones EC, Wood B, Bagci U. A collaborative computer aided diagnosis (C-CAD) system with eye-tracking, sparse attentional model, and deep learning. Med Image Anal 2019; 51:101-115. [PMID: 30399507 PMCID: PMC6407631 DOI: 10.1016/j.media.2018.10.010] [Citation(s) in RCA: 33] [Impact Index Per Article: 6.6] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/19/2017] [Revised: 07/27/2018] [Accepted: 10/26/2018] [Indexed: 12/19/2022]
Abstract
Computer aided diagnosis (CAD) tools help radiologists to reduce diagnostic errors such as missing tumors and misdiagnosis. Vision researchers have been analyzing behaviors of radiologists during screening to understand how and why they miss tumors or misdiagnose. In this regard, eye-trackers have been instrumental in understanding visual search processes of radiologists. However, most relevant studies in this aspect are not compatible with realistic radiology reading rooms. In this study, we aim to develop a paradigm shifting CAD system, called collaborative CAD (C-CAD), that unifies CAD and eye-tracking systems in realistic radiology room settings. We first developed an eye-tracking interface providing radiologists with a real radiology reading room experience. Second, we propose a novel algorithm that unifies eye-tracking data and a CAD system. Specifically, we present a new graph based clustering and sparsification algorithm to transform eye-tracking data (gaze) into a graph model to interpret gaze patterns quantitatively and qualitatively. The proposed C-CAD collaborates with radiologists via eye-tracking technology and helps them to improve their diagnostic decisions. The C-CAD uses radiologists' search efficiency by processing their gaze patterns. Furthermore, the C-CAD incorporates a deep learning algorithm in a newly designed multi-task learning platform to segment and diagnose suspicious areas simultaneously. The proposed C-CAD system has been tested in a lung cancer screening experiment with multiple radiologists, reading low dose chest CTs. Promising results support the efficiency, accuracy and applicability of the proposed C-CAD system in a real radiology room setting. We have also shown that our framework is generalizable to more complex applications such as prostate cancer screening with multi-parametric magnetic resonance imaging (mp-MRI).
Collapse
Affiliation(s)
- Naji Khosravan
- Center for Research in Computer Vision, University of Central Florida, FL, United States
| | - Haydar Celik
- Clinical Center, National Institutes of Health, Bethesda, MD, United States
| | - Baris Turkbey
- Clinical Center, National Institutes of Health, Bethesda, MD, United States
| | - Elizabeth C Jones
- Clinical Center, National Institutes of Health, Bethesda, MD, United States
| | - Bradford Wood
- Clinical Center, National Institutes of Health, Bethesda, MD, United States
| | - Ulas Bagci
- Center for Research in Computer Vision, University of Central Florida, FL, United States.
| |
Collapse
|
31
|
Al-Moteri MO, Symmons M, Cooper S, Plummer V. Inattentional blindness and pattern-matching failure: The case of failure to recognize clinical cues. APPLIED ERGONOMICS 2018; 73:174-182. [PMID: 30098633 DOI: 10.1016/j.apergo.2018.07.001] [Citation(s) in RCA: 10] [Impact Index Per Article: 1.7] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 08/08/2017] [Revised: 06/24/2018] [Accepted: 07/05/2018] [Indexed: 06/08/2023]
Abstract
Eye-tracking methodology was used to investigate lapses in the appropriate treatment of ward patients due to not noticing critical cues of deterioration. Forty nursing participants with different levels of experience participated in an interactive screen-based simulation of hypovolemic shock. The results show that 65% of the participants exhibited at least one episode of non-fixation on clinically relevant, fully visible cues that were in plain sight. Thirty-five percent of participants dwelt for sufficient time (>200 ms) on important cues for perception to take place, but no action followed, indicating they had pattern-matching failure. When participants fail to notice what, they should notice in patient status until it is too late, this can have serious consequences. Much work needs to be done, since these human perceptual limitations can affect patient safety in general wards.
Collapse
Affiliation(s)
- Modi Owied Al-Moteri
- Nursing Department, Faculty of Applied Medical Sciences, University of Al-Taif, Western Region, Saudi Arabia; School of Nursing and Midwifery, Monash University, Frankston, Victoria, Australia.
| | - Mark Symmons
- Australian Catholic University, Melbourne, Australia
| | - Simon Cooper
- School of Nursing and Health Professions, Federation University Australia, Room 2W-249, Gippsland Campus, Churchill, Victoria, Australia
| | - Virginia Plummer
- School of Nursing and Midwifery, Monash University, Frankston, Victoria, Australia; Peninsula Health, Frankston, Victoria, Australia
| |
Collapse
|