1
|
Salava A, Salmela V. Diagnostic errors during perceptual learning in dermatology: a prospective cohort study of Finnish undergraduate students. Clin Exp Dermatol 2024; 49:866-874. [PMID: 38391032 DOI: 10.1093/ced/llae063] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/23/2024] [Revised: 02/14/2024] [Accepted: 02/19/2024] [Indexed: 02/24/2024]
Abstract
BACKGROUND Perceptual learning modules (PLMs) have been shown to significantly improve learning outcomes in teaching dermatology. OBJECTIVES To investigate the quantity and quality of diagnostic errors made during undergraduate PLMs and their potential implications. METHODS The study data were acquired from 8 successive dermatology courses (2021-23) from 142 undergraduate medical students. Digital PLMs were held before, during and after the courses. We investigated the number and distribution of diagnostic errors, differences between specific skin conditions and classified the errors based on type. RESULTS Diagnostic errors were not randomly distributed. Some skin conditions were almost always correctly identified, whereas a significant number of errors were made for other diagnoses. Errors were classified into one of three groups: mostly systematic errors of relevant differential diagnoses ('similarity' errors); partly systematic errors ('mixed' errors); and 'random' errors. While a significant learning effect during the repeated measures was found in accuracy (P < 0.001, η²P = 0.64), confidence (P < 0.001, η²P = 0.60) and fluency (P < 0.001, η²P = 0.16), the three categories differed in all outcome measures (all P < 0.001, all η²P > 0.47). Visual learning was more difficult for diagnoses in the similarity category (all P < 0.001, all η²P > 0.12) than for those in the mixed and random categories. CONCLUSIONS Error analysis of PLMs provided relevant information about learning efficacy and progression, and systematic errors in tasks and more difficult-to-learn conditions. This information could be used in the development of adaptive, individual error-based PLMs to improve learning outcomes, both in dermatology and medical education in general.
Collapse
Affiliation(s)
- Alexander Salava
- Department of Dermatology, Venereology and Allergology, University Hospital Helsinki and University of Helsinki, Helsinki, Finland
| | - Viljami Salmela
- Department of Psychology and Logopedics, University of Helsinki, Helsinki, Finland
| |
Collapse
|
2
|
Specian Junior FC, Litchfield D, Sandars J, Cecilio-Fernandes D. Use of eye tracking in medical education. MEDICAL TEACHER 2024:1-8. [PMID: 38382474 DOI: 10.1080/0142159x.2024.2316863] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Received: 08/08/2023] [Accepted: 02/06/2024] [Indexed: 02/23/2024]
Abstract
Eye tracking has become increasingly applied in medical education research for studying the cognitive processes that occur during the performance of a task, such as image interpretation and surgical skills development. However, analysis and interpretation of the large amount of data obtained by eye tracking can be confusing. In this article, our intention is to clarify the analysis and interpretation of the data obtained from eye tracking. Understanding the relationship between eye tracking metrics (such as gaze, pupil and blink rate) and cognitive processes (such as visual attention, perception, memory and cognitive workload) is essential. The importance of calibration and how the limitations of eye tracking can be overcome is also highlighted.
Collapse
Affiliation(s)
| | | | - John Sandars
- Health Research Institute, Edge Hill University, Ormskirk, UK
| | - Dario Cecilio-Fernandes
- Department of Medical Psychology and Psychiatry, School of Medical Sciences, University of Campinas, Campinas, São Paulo, Brazil
| |
Collapse
|
3
|
Beeler N, Ziegler E, Navarini AA, Kapur M. Factors related to the performance of laypersons diagnosing pigmented skin cancer: an explorative study. Sci Rep 2023; 13:22790. [PMID: 38123698 PMCID: PMC10733329 DOI: 10.1038/s41598-023-50152-x] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/24/2023] [Accepted: 12/15/2023] [Indexed: 12/23/2023] Open
Abstract
It is important but challenging for prospective health professionals to learn the visual distinction between potentially harmful and harmless skin lesions, such as malignant melanomas and benign nevi. Knowledge about factors related to diagnostic performance is sparse but a prerequisite for designing and evaluating evidence-based educational interventions. Hence, this study explored how the characteristics of 240 skin lesions, the number of classified lesions and the response times of 137 laypeople were related to performance in diagnosing pigmented skin cancer. Our results showed large differences between the lesions, as some were classified correctly by more than 90% and others by less than 10% of the participants. A t-test showed that for melanomas, the correct diagnosis was provided significantly more often than for nevi. Furthermore, we found a significant Pearson correlation between the number of solved tasks and performance in the first 50 diagnostic tasks. Finally, t-tests for investigating the response times revealed that compared to true decisions, participants spent longer on false-negative but not on false-positive decisions. These results provide novel knowledge about performance-related factors that can be useful when designing diagnostic tests and learning interventions for melanoma detection.
Collapse
Affiliation(s)
- Nadja Beeler
- Professorship for Learning Sciences and Higher Education, ETH Zurich, Clausiusstrasse 59, 8092, Zurich, Switzerland.
| | - Esther Ziegler
- Professorship for Learning Sciences and Higher Education, ETH Zurich, Clausiusstrasse 59, 8092, Zurich, Switzerland
| | - Alexander A Navarini
- Department of Dermatology, University Hospital Basel, Burgfelderstrasse 101, 4055, Basel, Switzerland
| | - Manu Kapur
- Professorship for Learning Sciences and Higher Education, ETH Zurich, Clausiusstrasse 59, 8092, Zurich, Switzerland
| |
Collapse
|
4
|
Darici D, Reissner C, Missler M. Webcam-based eye-tracking to measure visual expertise of medical students during online histology training. GMS JOURNAL FOR MEDICAL EDUCATION 2023; 40:Doc60. [PMID: 37881524 PMCID: PMC10594038 DOI: 10.3205/zma001642] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Figures] [Subscribe] [Scholar Register] [Received: 09/30/2022] [Revised: 06/06/2023] [Accepted: 07/07/2023] [Indexed: 10/27/2023]
Abstract
Objectives Visual expertise is essential for image-based tasks that rely on visual cues, such as in radiology or histology. Studies suggest that eye movements are related to visual expertise and can be measured by near-infrared eye-tracking. With the popularity of device-embedded webcam eye-tracking technology, cost-effective use in educational contexts has recently become amenable. This study investigated the feasibility of such methodology in a curricular online-only histology course during the 2021 summer term. Methods At two timepoints (t1 and t2), third-semester medical students were asked to diagnose a series of histological slides while their eye movements were recorded. Students' eye metrics, performance and behavioral measures were analyzed using variance analyses and multiple regression models. Results First, webcam-eye tracking provided eye movement data with satisfactory quality (mean accuracy=115.7 px±31.1). Second, the eye movement metrics reflected the students' proficiency in finding relevant image sections (fixation count on relevant areas=6.96±1.56 vs. irrelevant areas=4.50±1.25). Third, students' eye movement metrics successfully predicted their performance (R2adj=0.39, p<0.001). Conclusion This study supports the use of webcam-eye-tracking expanding the range of educational tools available in the (digital) classroom. As the students' interest in using the webcam eye-tracking was high, possible areas of implementation will be discussed.
Collapse
Affiliation(s)
- Dogus Darici
- Westfälische-Wilhelms-University, Institute of Anatomy and Neurobiology, Münster, Germany
| | - Carsten Reissner
- Westfälische-Wilhelms-University, Institute of Anatomy and Neurobiology, Münster, Germany
| | - Markus Missler
- Westfälische-Wilhelms-University, Institute of Anatomy and Neurobiology, Münster, Germany
| |
Collapse
|
5
|
Schroeder JA, Cao Q, Sotirchos VS, Gillman JA, Anderson T, Pilati S, Dubroff JG, Farwell M, Kozlov A, Korhonen K, Pryma DA, Pantel AR. Perfusion-only imaging in pregnant women: A comparative reader study with implications for practice patterns. Medicine (Baltimore) 2022; 101:e30800. [PMID: 36181041 PMCID: PMC9524957 DOI: 10.1097/md.0000000000030800] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 01/05/2023] Open
Abstract
This study seeks to understand the value of ventilation imaging in pregnant patients imaged for suspected pulmonary embolism (PE). Ventilation-perfusion (VQ) scans in this high-risk population were compared to ventilation-only scans. We hypothesize that in this relatively healthy population, the exclusion of ventilation scans will not impact the rate of scans interpreted as positive. This retrospective blinded comparative reader study on collated VQ scans performed on pregnant patients in the course of routine clinical care in a > 5 year period (03/2012 to 07/2017). Each set of VQ and perfusion only (Q) studies were reviewed by 8 readers (4 nuclear radiology fellows and 4 nuclear medicine faculty) in random order; the Q scans simply omitted the ventilation images. Readers recorded each study as PE, no PE, or non-diagnostic (prospective investigative study of acute PE diagnosis classifications). Logistic mixed effects models were used to test the association between scan type (VQ vs Q). 203 pairs of studies in 197 patients were included (6 patients had 2 scans). Subjects ranged from 14 to 45 years of age, with a median 28 years. A significant association between scan type and positive/negative classification. Q-scans received more positive classifications than VQ-scans (median of 7.6% vs 6.7%). No association was seen between scan type and positive/indeterminate classification, nor between scan type and negative/indeterminate classification. The exclusion of ventilation images in VQ-scans was associated with a higher rate of positive studies, but this difference was small (<1%). Given the overwhelmingly normal percentage of Q-exams (>90% in our study), and the benefits of omitting ventilation imaging, perfusion-only imaging should be considered a reasonable option for imaging the pregnant patient to exclude PE.
Collapse
Affiliation(s)
- Jennifer A. Schroeder
- Department of Radiology, Nuclear Medicine, Atrium Health Wake Forest Baptist Medical Center, Winston-Salem, NC, USA
- * Correspondence: Jennifer A. Schroeder, Department of Radiology, Nuclear Medicine, Atrium Health Wake Forest Baptist Medical Center, Winston-Salem, NC, USA (e-mail: )
| | - Quy Cao
- Department of Biostatistics, Epidemiology, & Informatics, University of Pennsylvania, Philadelphia, PA, USA
| | - Vlasios S. Sotirchos
- Interventional Radiology, Memorial Sloan Kettering Cancer Center, New York, NY, USA
| | - Jennifer A. Gillman
- Department of Radiology, Mid-Atlantic Permanente Medical Group, Rockville, MD, USA
| | - Thomas Anderson
- Department of Radiology, University of New Mexico, Albuquerque, NM, USA
| | - Stamatoula Pilati
- Department of Radiology and Nuclear Medicine, Cook County Health, Chicago, IL, USA
| | - Jacob G. Dubroff
- Department of Radiology, Nuclear Medicine, University of Pennsylvania, Philadelphia, PA, USA
| | - Michael Farwell
- Department of Radiology, Nuclear Medicine, University of Pennsylvania, Philadelphia, PA, USA
| | - Andrew Kozlov
- Radiology Associates of Florida & the University of South Florida, Morsani College of Medicine, Philadelphia, PA, USA
| | | | - Daniel A. Pryma
- Department of Radiology, Nuclear Medicine, University of Pennsylvania, Philadelphia, PA, USA
| | - Austin R. Pantel
- Department of Radiology, Nuclear Medicine, University of Pennsylvania, Philadelphia, PA, USA
| |
Collapse
|
6
|
Bell CR, Szulewski A, Walker M, McKaigney C, Ross G, Rang L, Newbigging J, Kendall J. Differences in Gaze Fixation Location and Duration Between Resident and Fellowship Sonographers Interpreting a Focused Assessment With Sonography in Trauma. AEM EDUCATION AND TRAINING 2021; 5:28-36. [PMID: 33521488 PMCID: PMC7821074 DOI: 10.1002/aet2.10439] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 09/23/2019] [Revised: 01/22/2020] [Accepted: 02/07/2020] [Indexed: 05/30/2023]
Abstract
OBJECTIVES We quantified the gaze fixation duration of resident and fellowship sonographers interpreting a prerecorded focused assessment with sonography in trauma (FAST). We hypothesized that all sonographers would fixate on each relevant anatomic relationship but that the duration of fixation would differ. METHODS We conducted a cross-sectional study collecting and analyzing the gaze fixations of a convenience sample of current resident and fellowship sonographers. All sonographers viewed a standardized FAST video, and their gaze fixations were recorded using a Tobii X3-120 eye-tracking bar. Gaze fixations over nine anatomic regions of interest (ROIs) were identified. These were assessed for normality and analyzed using the Wilcoxon rank sum test at an alpha of 0.05 and Bonferroni correction p value of <0.0034. The chi-square test and Pearson's correlation were performed to assess statistical association. RESULTS The gaze fixation recordings of 24 resident and eight fellowship sonographers were suitable for analysis. Fourteen of the 24 resident sonographers viewed all ROIs in the FAST, whereas all eight fellowship sonographers viewed each of the nine relevant ROIs. Five ROIs were identified over which at least one resident sonographer did not have a gaze fixation. No statistically significant difference was identified between groups. Resident sonographers gaze fixated over the left upper quadrant (LUQ) splenorenal interface for a median (interquartile range) of 10.64 (9.73-11.60) seconds. The fellowship group viewed the same ROI for 8.43 (6.64-8.95) seconds (p < 0.003). All participants viewed this ROI. No other ROIs had a statistical difference. CONCLUSION Five ROIs were identified that were not visually interrogated by all resident sonographers. Only 14 of 24 resident sonographers visually interrogated every area in the FAST, whereas all fellowship sonographers interrogated every ROI. A statistically significant difference was found in gaze fixation duration between resident and fellowship sonographers in one ROI. Further study is required for gaze fixation assessment to become a tool for the interpretation component of point-of-care ultrasound.
Collapse
Affiliation(s)
- Colin R. Bell
- Department of Emergency MedicineKingston Health Sciences CentreKingstonOntario
- Queen’s University School of MedicineKingstonOntario
| | - Adam Szulewski
- Department of Emergency MedicineKingston Health Sciences CentreKingstonOntario
- Queen’s University School of MedicineKingstonOntario
| | - Melanie Walker
- Department of Emergency MedicineKingston Health Sciences CentreKingstonOntario
| | - Conor McKaigney
- Department of Emergency MedicineSouth Health, Alberta Health Services and Cumming School of Medicine at the University of CalgaryCalgaryAlbertaCanada
| | - Graeme Ross
- Department of Emergency MedicineKingston Health Sciences CentreKingstonOntario
- Queen’s University School of MedicineKingstonOntario
| | - Louise Rang
- Department of Emergency MedicineKingston Health Sciences CentreKingstonOntario
- Queen’s University School of MedicineKingstonOntario
| | - Joseph Newbigging
- Department of Emergency MedicineKingston Health Sciences CentreKingstonOntario
- Queen’s University School of MedicineKingstonOntario
| | - John Kendall
- Department of Emergency MedicineDenver Health Medical Center and University of Colorado Medical SchoolDenverCO
| |
Collapse
|
7
|
Richter J, Scheiter K, Eder TF, Huettig F, Keutel C. How massed practice improves visual expertise in reading panoramic radiographs in dental students: An eye tracking study. PLoS One 2020; 15:e0243060. [PMID: 33270704 PMCID: PMC7714201 DOI: 10.1371/journal.pone.0243060] [Citation(s) in RCA: 8] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/02/2020] [Accepted: 11/15/2020] [Indexed: 02/06/2023] Open
Abstract
The interpretation of medical images is an error-prone process that may yield severe consequences for patients. In dental medicine panoramic radiography (OPT) is a frequently used diagnostic procedure. OPTs typically contain multiple, diverse anomalies within one image making the diagnostic process very demanding, rendering students’ development of visual expertise a complex task. Radiograph interpretation is typically taught through massed practice; however, it is not known how effective this approach is nor how it changes students’ visual inspection of radiographs. Therefore, this study investigated how massed practice–an instructional method that entails massed learning of one type of material–affects processing of OPTs and the development of diagnostic performance. From 2017 to 2018, 47 dental students in their first clinical semester diagnosed 10 OPTs before and after their regular massed practice training, which is embedded in their curriculum. The OPTs contained between 3 to 26 to-be-identified anomalies. During massed practice they diagnosed 100 dental radiographs without receiving corrective feedback. The authors recorded students’ eye movements and assessed the number of correctly identified and falsely marked low- and high prevalence anomalies before and after massed practice. Massed practice had a positive effect on detecting anomalies especially with low prevalence (p < .001). After massed practice students covered a larger proportion of the OPTs (p < .001), which was positively related to the detection of low-prevalence anomalies (p = .04). Students also focused longer, more frequently, and earlier on low-prevalence anomalies after massed practice (ps < .001). While massed practice improved visual expertise in dental students with limited prior knowledge, there is still substantial room for improvement. The results suggest integrating massed practice with more deliberate practice, where, for example, corrective feedback is provided, and support is adapted to students’ needs.
Collapse
Affiliation(s)
- Juliane Richter
- Leibniz-Institut für Wissensmedien, Tübingen, Germany
- * E-mail:
| | - Katharina Scheiter
- Leibniz-Institut für Wissensmedien, Tübingen, Germany
- Eberhard Karls University of Tübingen, Tübingen, Germany
| | | | - Fabian Huettig
- Department of Prosthodontics, University Hospital for Dentistry, Oral Medicine, and Maxillofacial Surgery at the University Hospital Tübingen, Eberhard Karls University of Tübingen, Tübingen, Germany
| | - Constanze Keutel
- Radiology Department of the University Hospital for Dentistry, Oral Medicine, and Maxillofacial Surgery at the University Hospital Tübingen, Eberhard Karls University of Tübingen, Tübingen, Germany
| |
Collapse
|
8
|
Yoon JS, Boutis K, Pecaric MR, Fefferman NR, Ericsson KA, Pusic MV. A think-aloud study to inform the design of radiograph interpretation practice. ADVANCES IN HEALTH SCIENCES EDUCATION : THEORY AND PRACTICE 2020; 25:877-903. [PMID: 32140874 PMCID: PMC7471179 DOI: 10.1007/s10459-020-09963-0] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 11/08/2018] [Accepted: 02/17/2020] [Indexed: 06/10/2023]
Abstract
Models for diagnostic reasoning in radiology have been based on the observed behaviors of experienced radiologists but have not directly focused on the thought processes of novices as they improve their accuracy of image interpretation. By collecting think-aloud verbal reports, the current study was designed to investigate differences in specific thought processes between medical students (novices) as they learn and radiologists (experts), so that we can better design future instructional environments. Seven medical students and four physicians with radiology training were asked to interpret and diagnose pediatric elbow radiographs where fracture is suspected. After reporting their diagnosis of a case, they were given immediate feedback. Participants were asked to verbalize their thoughts while completing the diagnosis and while they reflected on the provided feedback. The protocol analysis of their verbalizations showed that participants used some combination of four processes to interpret the case: gestalt interpretation, purposeful search, rule application, and reasoning from a prior case. All types of processes except reasoning from a prior case were applied significantly more frequently by experts. Further, gestalt interpretation was used with higher frequency in abnormal cases while purposeful search was used more often for normal cases. Our assessment of processes could help guide the design of instructional environments with well-curated image banks and analytics to facilitate the novice's journey to expertise in image interpretation.
Collapse
Affiliation(s)
- Jong-Sung Yoon
- Department of Psychology, University of South Dakota, Vermillion, SD, USA
| | - Kathy Boutis
- Dept. of Pediatrics, The Hospital for Sick Children, and University of Toronto, Toronto, Canada
| | | | - Nancy R Fefferman
- Department of Radiology, New York University School of Medicine, New York, USA
| | - K Anders Ericsson
- Department of Psychology, Florida State University, Tallahassee, FL, USA
| | - Martin V Pusic
- Department of Emergency Medicine, New York University School of Medicine, New York, USA.
- Division of Learning Analytics, Institute for Innovation in Medical Education, 550 First Avenue, MSB G109, New York, NY, 10016, USA.
| |
Collapse
|
9
|
Darras KE, Spouge RJ, de Bruin ABH, Sedlic A, Hague C, Forster BB. Undergraduate Radiology Education During the COVID-19 Pandemic: A Review of Teaching and Learning Strategies [Formula: see text]. Can Assoc Radiol J 2020; 72:194-200. [PMID: 32749165 DOI: 10.1177/0846537120944821] [Citation(s) in RCA: 41] [Impact Index Per Article: 10.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/25/2022] Open
Abstract
The Coronavirus disease 2019 (COVID-19) pandemic has altered how medical education is delivered, worldwide. Didactic sessions have transitioned to electronic/online platforms and clinical teaching opportunities are limited. These changes will affect how radiology is taught to medical students at both the pre-clerkship (ie, year 1 and 2) and clinical (ie, year 3 and 4) levels. In the pre-clerkship learning environment, medical students are typically exposed to radiology through didactic lectures, integrated anatomy laboratories, case-based learning, and ultrasound clinical skills sessions. In the clinical learning environment, medical students primarily shadow radiologists and radiology residents and attend radiology resident teaching sessions. These formats of radiology education, which have been the tenets of the specialty, pose significant challenges during the pandemic. This article reviews how undergraduate radiology education is affected by COVID-19 and explores solutions for teaching and learning based on e-learning and blended learning theory.
Collapse
Affiliation(s)
- Kathryn E Darras
- Faculty of Medicine, Department of Radiology, 8166University of British Columbia, Vancouver, British Columbia, Canada
| | - Rebecca J Spouge
- Faculty of Medicine, Department of Radiology, 8166University of British Columbia, Vancouver, British Columbia, Canada
| | - Anique B H de Bruin
- School of Health Professions Education, 5211Maastricht University, Maastricht, the Netherlands
| | - Anto Sedlic
- Faculty of Medicine, Department of Radiology, 8166University of British Columbia, Vancouver, British Columbia, Canada
| | - Cameron Hague
- Faculty of Medicine, Department of Radiology, 8166University of British Columbia, Vancouver, British Columbia, Canada
| | - Bruce B Forster
- Faculty of Medicine, Department of Radiology, 8166University of British Columbia, Vancouver, British Columbia, Canada
| |
Collapse
|
10
|
Han H, Williams R, Hingle S, Klamen DL, Rull GM, Clark T, Daniels J. Medical Students' Progress in Detecting and Interpreting Visual and Auditory Clinical Findings. TEACHING AND LEARNING IN MEDICINE 2020; 32:380-388. [PMID: 32281403 DOI: 10.1080/10401334.2020.1749636] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/11/2023]
Abstract
Phenomenon: Detection of visual and auditory clinical findings is part of medical students' core clinical performance abilities that a medical education curriculum should teach, assess, and remediate. However, there is a limited understanding of how students develop these skills. While training physical exam technical skills has received significant attention and emphasis, teaching and assessing medical students' ability to detect and interpret visual and auditory clinical findings skills has been less systematic. Therefore, the purpose of this study is to investigate how medical students' visual and auditory clinical findings skills progress and develop over their four years of undergraduate medical education. This study will provide educators insights that can guide curriculum refinements that lead to improving students' abilities in this area. Approach: A computer-based progress exam was created to measure the longitudinal development of students' abilities to detect and interpret visual and auditory findings. After pilot testing, sixty test items were developed in collaboration with six clinical faculty members and two medical education researchers. The exam includes detection and description of ECG, x-ray, heart sounds, breath sounds, skin lesions, and movement findings. The exam was administered to students at the beginning of each training year since 2014. Additionally, the exam was administered to the Class of 2017 prior to their graduation. Measurement validity and reliability tests were conducted. Descriptive statistics and ANOVA were used to determine progress. Findings: More than 98% of students in four years of training completed the exam each year. The exam instrument had high reliabilities and demonstrated acceptable concurrent validity when compared with other academic performance data. Findings showed that students' visual and auditory clinical findings skills increased each training year until their fourth year. There was no performance improvement between incoming Year 4 students and graduating Year 4 students. While group means increased, class performance did not become more homogeneous across four years. Longitudinal data showed the same performance patterns as the cross-sectional data. Performance of the bottom quartile of graduating fourth-year students was not significantly higher than the performance of the top quartile of incoming first-year students who had not had formal medical training. Insights: A longitudinal study to follow learners' performance in detecting and interpreting visual and auditory clinical findings can provide meaningful insights regarding the effects of medical training programs on performance growth. The present study suggests that our medical curriculum is not effective in bringing all students to a higher level of performance in detecting and interpreting visual and auditory clinical findings. This study calls for further investigation how medical students can develop visual and auditory detection and interpretation skills in undergraduate medical education. There is a need for planned curriculum and assessment of medical students' skills in detecting and interpreting visual and auditory clinical findings.
Collapse
Affiliation(s)
- Heeyoung Han
- Department of Medicine Education, Southern Illinois University School of Medicine, Springfield, Illinois, USA
| | - Reed Williams
- J. Roland Folse Professor of Surgical Education, Southern Illinois University School of Medicine, Springfield, Illinois, USA
| | - Susan Hingle
- Department of Internal Medicine, Southern Illinois University School of Medicine, Springfield, Illinois, USA
| | - Debra L Klamen
- Department of Medicine Education, Southern Illinois University School of Medicine, Springfield, Illinois, USA
| | - Gary M Rull
- Department of Internal Medicine, Southern Illinois University School of Medicine, Springfield, Illinois, USA
| | - Ted Clark
- Department of Emergency Medicine, Southern Illinois University School of Medicine, Springfield, Illinois, USA
| | - James Daniels
- Department of Family and Community Medicine, Quincy, Illinois, USA
| |
Collapse
|
11
|
Król M, Król M. Learning From Peers' Eye Movements in the Absence of Expert Guidance: A Proof of Concept Using Laboratory Stock Trading, Eye Tracking, and Machine Learning. Cogn Sci 2020; 43:e12716. [PMID: 30803012 DOI: 10.1111/cogs.12716] [Citation(s) in RCA: 9] [Impact Index Per Article: 2.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/25/2018] [Revised: 01/15/2019] [Accepted: 01/18/2019] [Indexed: 11/29/2022]
Abstract
Existing research shows that people can improve their decision skills by learning what experts paid attention to when faced with the same problem. However, in domains like financial education, effective instruction requires frequent, personalized feedback given at the point of decision, which makes it time-consuming for experts to provide and thus, prohibitively costly. We address this by demonstrating an automated feedback mechanism that allows amateur decision-makers to learn what information to attend to from one another, rather than from an expert. In the first experiment, eye movements of N = 100 subjects were recorded while they repeatedly performed a standard behavioral finance investment task. Consistent with previous studies, we found that a significant proportion of subjects were affected by decision bias. In the second experiment, a different group of N = 100 subjects faced the same task but, after each choice, they received individual, machine learning-generated feedback on whether their pre-decision eye movements resembled those made by Experiment 1 subjects prior to good decisions. As a result, Experiment 2 subjects learned to analyze information similarly to their successful peers, which in turn reduced their decision bias. Furthermore, subjects with low Cognitive Reflection Test scores gained more from the proposed form of process feedback than from standard behavioral feedback based on decision outcomes.
Collapse
Affiliation(s)
- Michał Król
- Wrocław Faculty of Psychology, SWPS University of Social Sciences and Humanities in Wrocław
| | - Magdalena Król
- Department of Economics, School of Social Sciences, University of Manchester
| |
Collapse
|
12
|
Jeker L, Birrenbach T, Walter R, Hari R. [Peer Teaching in Ultrasound Education - A Narrative Review]. PRAXIS 2020; 109:631-635. [PMID: 32517595 DOI: 10.1024/1661-8157/a003498] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/11/2023]
Abstract
Peer Teaching in Ultrasound Education - A Narrative Review Abstract. Learning about ultrasound is becoming an increasingly important component of the undergraduate medical curriculum. However, teaching about this is very time-consuming, which is why many universities use peer teaching in ultrasound training. Peer teaching has various advantages that go beyond reducing specialists' teaching time: students may learn more from their peers because their cognitive schemata are more congruent and they are more likely to be open about their learning deficits. In addition, the process of teaching leads to increased learning and motivation in the peer teachers themselves. Most studies that compare peer teaching with teaching from medically qualified tutors show comparable learning effects, with different advantages for the two settings.
Collapse
Affiliation(s)
- Lia Jeker
- Berner Institut für Hausarztmedizin, Universität Bern, Bern
| | - Tanja Birrenbach
- Universitäres Notfallzentrum, Inselspital, Universitätsspital, Universität Bern, Bern
| | - Robin Walter
- Berner Institut für Hausarztmedizin, Universität Bern, Bern
| | - Roman Hari
- Berner Institut für Hausarztmedizin, Universität Bern, Bern
- Department of Educational Development and Research, Maastricht University, Maastricht, Niederlande
| |
Collapse
|
13
|
van Geel K, Kok EM, Krol JP, Houben IPL, Thibault FE, Pijnappel RM, van Merriënboer JJG, Lobbes MBI. Reversal of the hanging protocol of Contrast Enhanced Mammography leads to similar diagnostic performance yet decreased reading times. Eur J Radiol 2019; 117:62-68. [PMID: 31307654 DOI: 10.1016/j.ejrad.2019.05.013] [Citation(s) in RCA: 4] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/03/2019] [Revised: 05/11/2019] [Accepted: 05/14/2019] [Indexed: 10/26/2022]
Abstract
OBJECTIVES Contrast-enhanced mammography (CEM) was found superior to Full-Field Digital Mammography (FFDM) for breast cancer detection. Current hanging protocols show low-energy (LE, similar to FFDM) images first, followed by recombined (RC) images. However, evidence regarding which hanging protocol leads to the most efficient reading process and highest diagnostic performance is lacking. This study investigates the effects of hanging-protocol ordering on the reading process and diagnostic performance of breast radiologists using eye-tracking methodology. Furthermore, it investigates differences in reading processes and diagnostic performance between LE, RC and FFDM images. MATERIALS AND METHODS Twenty-seven breast radiologists were randomized into three reading groups: LE-RC (commonly used hangings), RC-LE (reversed hangings) and FFDM. Thirty cases (nine malignant) were used. Fixation count, net dwell time and time-to-first fixation on malignancies as visual search measures were registered by the eye-tracker. Reading time per image was measured. Participants clicked on suspicious lesions to determine sensitivity and specificity. Area-under-the-ROC-curve (AUC) values were calculated. RESULTS RC-LE scored identical on visual search measures, t(16)= -1.45, p = .17 or higher-p values, decreased reading time with 31%, t(16)= -2.20, p = .04, while scoring similar diagnostic performance compared to LE-RC, t(13.2) = -1.39, p - .20 or higher p-values. The reading process was more efficient on RC compared to LE. Diagnostic performance of CEM was superior to FFDM; F (2,26) = 16.1, p < .001. Average reading time did not differ between the three groups, F (2,25) = 3.15, p = .06. CONCLUSION The reversed CEM hanging protocol (RC-LE) scored similar on diagnostic performance compared to LE-RC, while reading time was a third faster. Abnormalities were interpreted quicker on RC images. A RC-LE hanging protocol is therefore recommended for clinical practice and training. Diagnostic performance of CEM was (again) superior to FFDM.
Collapse
Affiliation(s)
- Koos van Geel
- Department of Radiology, Maastricht University Medical Center, PO box 5800, 6202, AZ, Maastricht, the Netherlands.
| | - Ellen M Kok
- Department of Education, Utrecht University, Heidelberglaan 1, Room E3.34, 3584, CS, Utrecht, the Netherlands
| | - Jorian P Krol
- Department of Radiology, Maastricht University Medical Center, PO box 5800, 6202, AZ, Maastricht, the Netherlands
| | - Ivo P L Houben
- Department of Radiology, Maastricht University Medical Center, PO box 5800, 6202, AZ, Maastricht, the Netherlands; Maastricht University, School for Oncology and Developmental Biology (GROW), Universiteitssingel 40, 6229, ER, Maastricht, the Netherlands
| | - Fabienne E Thibault
- Department of Radiology, Institut Curie, 26 rue d'Ulm, 75248, Paris Cedex 05, France
| | - Ruud M Pijnappel
- Department of Radiology, Universitair Medisch Centrum Utrecht, Heidelberglaan 100, 3584, CX, Utrecht, the Netherlands
| | - Jeroen J G van Merriënboer
- Maastricht University, School of Health Professions Education, Maastricht University, PO Box 616, 6200, MD, Maastricht, the Netherlands
| | - Marc B I Lobbes
- Department of Radiology, Maastricht University Medical Center, PO box 5800, 6202, AZ, Maastricht, the Netherlands; Maastricht University, School for Oncology and Developmental Biology (GROW), Universiteitssingel 40, 6229, ER, Maastricht, the Netherlands
| |
Collapse
|
14
|
Boutis K, Pecaric M, Carrière B, Stimec J, Willan A, Chan J, Pusic M. The effect of testing and feedback on the forgetting curves for radiograph interpretation skills. MEDICAL TEACHER 2019; 41:756-764. [PMID: 31046500 DOI: 10.1080/0142159x.2019.1570098] [Citation(s) in RCA: 14] [Impact Index Per Article: 2.8] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/09/2023]
Abstract
Objectives: Forgetting curves plot skill decay over time. After exposure to a simulation-based radiograph interpretation learning system, we determined the rate of learning decay and how this was impacted by testing (with and without feedback). Further, we examined the association of initial learning parameters on the forgetting curve. Methods: This was a multicenter, four-arm randomized control trial. Medical trainees completed 80 elbow radiographs and a 20-case post-test. Group 1 had no testing until 12 months; Groups 2-4 had testing every 2 months until 12 months. At 6 months, Group 3 testing was feedback-enhanced, while Group 4 had feedback-enhanced testing at 2, 6, and 10 months. Results: There were 106 participants (n = 42 Group 1; n = 22 Groups 2 and 3; n = 20 Group 4). Group 1 showed an -8.1% learning decay at 12-months relative to other groups. In Groups 2, 3, and 4, there was no significant learning decay (+0.8%), and there were no differences in skill decay between these groups. Initial score and learning curve slope were predictive of retained skill. Conclusions: Learning decay was mitigated by exposure to 20 test cases (with and without feedback) every two months. Initial learning parameters predicted learning retention and may inform refresher education scheduling.
Collapse
Affiliation(s)
- Kathy Boutis
- a Department of Pediatrics, Division of Emergency Medicine , Hospital for Sick Children and the University of Toronto , Toronto , Canada
| | | | - Benoit Carrière
- c Department of Pediatrics, Division of Emergency Medicine , CHU Sainte-Justine and Université de Montréal , Montreal , Canada
| | - Jennifer Stimec
- d Department of Diagnostic Imaging , Hospital for Sick Children and the University of Toronto , Toronto , Canada
| | - Andrew Willan
- e Hospital for Sick Children , Research Institute, University of Toronto , Toronto , Canada
- f Dalla Lana School of Public Health, University of Toronto , Toronto , Canada
| | - Julie Chan
- g Department of Surgery, Division of Orthopedic Surgery , Queen's University , Kingston , Canada
| | - Martin Pusic
- h Department of Emergency Medicine, Division of Learning Analytics , NYU School of Medicine , New York , NY , USA
| |
Collapse
|
15
|
Waite S, Grigorian A, Alexander RG, Macknik SL, Carrasco M, Heeger DJ, Martinez-Conde S. Analysis of Perceptual Expertise in Radiology - Current Knowledge and a New Perspective. Front Hum Neurosci 2019; 13:213. [PMID: 31293407 PMCID: PMC6603246 DOI: 10.3389/fnhum.2019.00213] [Citation(s) in RCA: 49] [Impact Index Per Article: 9.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/18/2019] [Accepted: 06/07/2019] [Indexed: 12/14/2022] Open
Abstract
Radiologists rely principally on visual inspection to detect, describe, and classify findings in medical images. As most interpretive errors in radiology are perceptual in nature, understanding the path to radiologic expertise during image analysis is essential to educate future generations of radiologists. We review the perceptual tasks and challenges in radiologic diagnosis, discuss models of radiologic image perception, consider the application of perceptual learning methods in medical training, and suggest a new approach to understanding perceptional expertise. Specific principled enhancements to educational practices in radiology promise to deepen perceptual expertise among radiologists with the goal of improving training and reducing medical error.
Collapse
Affiliation(s)
- Stephen Waite
- Department of Radiology, SUNY Downstate Medical Center, Brooklyn, NY, United States
| | - Arkadij Grigorian
- Department of Radiology, SUNY Downstate Medical Center, Brooklyn, NY, United States
| | - Robert G. Alexander
- Department of Ophthalmology, SUNY Downstate Medical Center, Brooklyn, NY, United States
- Department of Neurology, SUNY Downstate Medical Center, Brooklyn, NY, United States
- Department of Physiology/Pharmacology, SUNY Downstate Medical Center, Brooklyn, NY, United States
| | - Stephen L. Macknik
- Department of Ophthalmology, SUNY Downstate Medical Center, Brooklyn, NY, United States
- Department of Neurology, SUNY Downstate Medical Center, Brooklyn, NY, United States
- Department of Physiology/Pharmacology, SUNY Downstate Medical Center, Brooklyn, NY, United States
| | - Marisa Carrasco
- Department of Psychology and Center for Neural Science, New York University, New York, NY, United States
| | - David J. Heeger
- Department of Psychology and Center for Neural Science, New York University, New York, NY, United States
| | - Susana Martinez-Conde
- Department of Ophthalmology, SUNY Downstate Medical Center, Brooklyn, NY, United States
- Department of Neurology, SUNY Downstate Medical Center, Brooklyn, NY, United States
- Department of Physiology/Pharmacology, SUNY Downstate Medical Center, Brooklyn, NY, United States
| |
Collapse
|
16
|
Szulewski A, Braund H, Egan R, Gegenfurtner A, Hall AK, Howes D, Dagnone D, van Merrienboer JJG. Starting to Think Like an Expert: An Analysis of Resident Cognitive Processes During Simulation-Based Resuscitation Examinations. Ann Emerg Med 2019; 74:647-659. [PMID: 31080034 DOI: 10.1016/j.annemergmed.2019.04.002] [Citation(s) in RCA: 12] [Impact Index Per Article: 2.4] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/02/2019] [Revised: 03/25/2019] [Accepted: 04/03/2019] [Indexed: 11/15/2022]
Abstract
STUDY OBJECTIVE Simulation is commonly used to teach crisis resource management skills and assess them in emergency medicine residents. However, our understanding of the cognitive processes underlying crisis resource management skills is limited because these processes are difficult to assess and describe. The objective of this study is to uncover and characterize the cognitive processes underlying crisis resource management skills and to describe how these processes vary between residents according to performance in a simulation-based examination. METHODS Twenty-two of 24 eligible emergency medicine trainees from 1 tertiary academic center completed 1 or 2 resuscitation-based examinations in the simulation laboratory. Resident performance was assessed by a blinded expert using an entrustment-based scoring tool. Participants wore eye-tracking glasses that generated first-person video that was used to augment subsequent interviews led by an emergency medicine faculty member. Interviews were audio recorded and then transcribed. An emergent thematic analysis was completed with a codebook that was developed by 4 research assistants, with subsequent analyses conducted by the lead research assistant with input from emergency medicine faculty. Themes from high- and low-performing residents were subsequently qualitatively compared. RESULTS Higher-performing residents were better able to anticipate, selectively attend to relevant information, and manage cognitive demands, and took a concurrent (as opposed to linear) approach to managing the simulated patient. CONCLUSION The results provide new insights into residents' cognitive processes while managing simulated patients in an examination environment and how these processes vary with performance. More work is needed to determine how best to apply these findings to improve crisis resource management education.
Collapse
Affiliation(s)
- Adam Szulewski
- Department of Emergency Medicine, Queen's University, Kingston, Ontario, Canada.
| | - Heather Braund
- Faculty of Education and Office of Professional Development and Educational Scholarship, Faculty of Health Sciences, Queen's University, Kingston, Ontario, Canada
| | - Rylan Egan
- Health Quality Programs, Queen's University, Kingston, Ontario, Canada
| | - Andreas Gegenfurtner
- Institut für Qualität und Weiterbildung, Technische Hochschule Deggendorf, Deggendorf, Germany
| | - Andrew K Hall
- Department of Emergency Medicine, Queen's University, Kingston, Ontario, Canada
| | - Daniel Howes
- Department of Emergency Medicine, Queen's University, Kingston, Ontario, Canada; Department of Critical Care Medicine, Queen's University, Kingston, Ontario, Canada
| | - Damon Dagnone
- Department of Emergency Medicine, Queen's University, Kingston, Ontario, Canada
| | - Jeroen J G van Merrienboer
- School of Health Professions Education, Faculty of Health, Medicine and Life Sciences, Maastricht University, Maastricht, the Netherlands
| |
Collapse
|
17
|
Simple eye movement metrics can predict future decision making performance: The case of financial choices. JUDGMENT AND DECISION MAKING 2019. [DOI: 10.1017/s1930297500004290] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 01/04/2023]
Abstract
AbstractDecisions are often delegated to experts chosen based on their past performance record which may be subject to noise. For instance, a person with little skill could still make a lucky decision that proves correct ex-post, while a skilled expert could make the best possible use of available information to reach a decision that, with hindsight, turns out incorrect. We aimed to show that one could assess decision skills more accurately when analyzing not only the observed decisions, but also the decision-making process. Incorporating eye-tracking into an established behavioral finance experimental framework, we found that making an eye transition between pieces of information that previous research associated with bias makes one less likely to make good financial decisions in future trials. Thus, even the simplest, easy to obtain eye metrics could allow us to more accurately judge if a person’s performance is a reflection of skill, or down to luck and unlikely to be reproduced in the future.
Collapse
|
18
|
Geel KV, Kok EM, Aldekhayel AD, Robben SGF, van Merriënboer JJG. Chest X-ray evaluation training: impact of normal and abnormal image ratio and instructional sequence. MEDICAL EDUCATION 2019; 53:153-164. [PMID: 30474292 PMCID: PMC6587445 DOI: 10.1111/medu.13756] [Citation(s) in RCA: 8] [Impact Index Per Article: 1.6] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 03/27/2018] [Revised: 09/07/2018] [Accepted: 09/13/2018] [Indexed: 06/09/2023]
Abstract
CONTEXT Medical image perception training generally focuses on abnormalities, whereas normal images are more prevalent in medical practice. Furthermore, instructional sequences that let students practice prior to expert instruction (inductive) may lead to improved performance compared with methods that give students expert instruction before practice (deductive). This study investigates the effects of the proportion of normal images and practice-instruction order on learning to interpret medical images. It is hypothesised that manipulation of the proportion of normal images will lead to a sensitivity-specificity trade-off and that students in practice-first (inductive) conditons need more time per practice case but will correctly identify more test cases. METHODS Third-year medical students (n = 103) learned radiograph interpretation by practising cases with, respectively, 30% or 70% normal radiographs prior to expert instruction (practice-first order) or after expert instruction (instruction-first order). After training, students performed a test (60% normal) and sensitivity (% of correctly identified abnormal radiographs), specificity (% of correctly identified normal radiographs), diagnostic performance (% of correct diagnoses) and case duration were measured. RESULTS The conditions with 30% of normal images scored higher on sensitivity but the conditions with 70% of normal images scored higher on specificity, indicating a sensitivity and specificity trade-off. Those who participated in inductive conditions took less time per practice case but more per test case. They had similar test sensitivity, but scored lower on test specificity. CONCLUSIONS The proportion of normal images impacted the sensitivity-specificity trade-off. This trade-off should be an important consideration for the alignment of training with future practice. Furthermore, the deductive conditions unexpectedly scored higher on specificity when participants took less time per case. An inductive approach did not lead to higher diagnostic performance, possibly because participants might already have relevant prior knowledge. Deductive approaches are therefore advised for the training of advanced learners.
Collapse
Affiliation(s)
- Koos van Geel
- Department of Radiology, Maastricht University Medical Center, Maastricht, the Netherlands
| | - Ellen M Kok
- Department of Education, Utrecht University, Utrecht, the Netherlands
| | - Abdullah D Aldekhayel
- Department of Radiology, Maastricht University Medical Center, Maastricht, the Netherlands
| | - Simon G F Robben
- Department of Radiology, Maastricht University Medical Center, Maastricht, the Netherlands
| | - Jeroen J G van Merriënboer
- School of Health Professions Education, Department of Educational Research and Development, Maastricht University, Maastricht, the Netherlands
| |
Collapse
|
19
|
Eva KW. Medical education research approaches. MEDICAL EDUCATION 2018; 52:1100-1102. [PMID: 30345681 DOI: 10.1111/medu.13739] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/08/2023]
Affiliation(s)
- Kevin W Eva
- Centre for Health Education Scholarship, University of British Columbia, Vancouver, British Columbia, Canada
| |
Collapse
|
20
|
Baim AD. Getting the picture: visual interpretation in ophthalmology residency training. MEDICAL EDUCATION 2018; 52:816-825. [PMID: 29785772 DOI: 10.1111/medu.13590] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 08/18/2017] [Revised: 10/16/2017] [Accepted: 02/19/2018] [Indexed: 06/08/2023]
Abstract
OBJECTIVES Visual interpretation is essential in many fields of health care. Although diagnostic competency can be measured as an educational outcome, few accounts have addressed tacit aspects of visual interpretation in clinical training; these include the disciplining of the trainee's attentions and the trainee's acculturation into expected styles of communicating visual interpretations to others. This paper describes values and dispositions that are taught to ophthalmology trainees as they learn to reason through visual information, and explores how these qualities are evaluated during residency training. METHODS The project was based on 6 months of ethnographic participant observation and interviews in an ophthalmology residency programme. Observational notes and interview transcripts pertaining to visual interpretation were isolated for qualitative analysis in the tradition of sociocultural anthropology, guided by literature on communication in medical education and the socialisation of health professionals. RESULTS Residents and faculty members identified visual interpretation as one of the most challenging skills expected of ophthalmology trainees. They expressed a belief that 'systematic' approaches, where visual information is parsed in a stepwise fashion, reduce the chance of trainees overlooking or misinterpreting key diagnostic features. This sensory discipline was represented in narrative form when faculty members asked residents to interpret images aloud, as residents were expected to follow prescribed sequences for describing the content of images before commenting on possible diagnoses. CONCLUSIONS Sensory processing is ordinarily opaque to outside observers, but the ritual of describing images in highly regimented narratives allows residents to demonstrate how they gather and reason through visual information. The form of these narratives reflects values that residents are expected to embody during their training, such as being thorough and methodical; it may also serve a pedagogical function by entrenching those values. Further research is needed to characterise how the performance of speech genres shapes the interpretive skills of medical trainees.
Collapse
Affiliation(s)
- Adam D Baim
- Center for Health and the Social Sciences, The University of Chicago, Chicago, Illinois, USA
| |
Collapse
|
21
|
A new way to look at simulation-based assessment: the relationship between gaze-tracking and exam performance. CAN J EMERG MED 2018; 21:129-137. [DOI: 10.1017/cem.2018.391] [Citation(s) in RCA: 10] [Impact Index Per Article: 1.7] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/07/2022]
Abstract
AbstractObjectiveA key task of the team leader in a medical emergency is effective information gathering. Studying information gathering patterns is readily accomplished with the use of gaze-tracking glasses. This technology was used to generate hypotheses about the relationship between performance scores and expert-hypothesized visual areas of interest in residents across scenarios in simulated medical resuscitation examinations.MethodsEmergency medicine residents wore gaze-tracking glasses during two simulation-based examinations (n=29 and 13 respectively). Blinded experts assessed video-recorded performances using a simulation performance assessment tool that has validity evidence in this context. The relationships between gaze patterns and performance scores were analyzed and potential hypotheses generated. Four scenarios were assessed in this study: diabetic ketoacidosis, bradycardia secondary to beta-blocker overdose, ruptured abdominal aortic aneurysm and metabolic acidosis caused by antifreeze ingestion.ResultsSpecific gaze patterns were correlated with objective performance. High performers were more likely to fixate on task-relevant stimuli and appropriately ignore task-irrelevant stimuli compared with lower performers. For example, shorter latency to fixation on the vital signs in a case of diabetic ketoacidosis was positively correlated with performance (r=0.70, p<0.05). Conversely, total time spent fixating on lab values in a case of ruptured abdominal aortic aneurysm was negatively correlated with performance (r= −0.50, p<0.05).ConclusionsThere are differences between the visual patterns of high and low-performing residents. These findings may allow for better characterization of expertise development in resuscitation medicine and provide a framework for future study of visual behaviours in resuscitation cases.
Collapse
|
22
|
White MR, Braund H, Howes D, Egan R, Gegenfurtner A, van Merrienboer JJG, Szulewski A. Getting Inside the Expert's Head: An Analysis of Physician Cognitive Processes During Trauma Resuscitations. Ann Emerg Med 2018; 72:289-298. [PMID: 29699720 DOI: 10.1016/j.annemergmed.2018.03.005] [Citation(s) in RCA: 24] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/01/2017] [Revised: 01/31/2018] [Accepted: 02/28/2018] [Indexed: 11/29/2022]
Abstract
STUDY OBJECTIVE Crisis resource management skills are integral to leading the resuscitation of a critically ill patient. Despite their importance, crisis resource management skills (and their associated cognitive processes) have traditionally been difficult to study in the real world. The objective of this study was to derive key cognitive processes underpinning expert performance in resuscitation medicine, using a new eye-tracking-based video capture method during clinical cases. METHODS During an 18-month period, a sample of 10 trauma resuscitations led by 4 expert trauma team leaders was analyzed. The physician team leaders were outfitted with mobile eye-tracking glasses for each case. After each resuscitation, participants were debriefed with a modified cognitive task analysis, based on a cued-recall protocol, augmented by viewing their own first-person perspective eye-tracking video from the clinical encounter. RESULTS Eye-tracking technology was successfully applied as a tool to aid in the qualitative analysis of expert performance in a clinical setting. All participants stated that using these methods helped uncover previously unconscious aspects of their cognition. Overall, 5 major themes were derived from the interviews: logistic awareness, managing uncertainty, visual fixation behaviors, selective attendance to information, and anticipatory behaviors. CONCLUSION The novel approach of cognitive task analysis augmented by eye tracking allowed the derivation of 5 unique cognitive processes underpinning expert performance in leading a resuscitation. An understanding of these cognitive processes has the potential to enhance educational methods and to create new assessment modalities of these previously tacit aspects of expertise in this field.
Collapse
Affiliation(s)
- Matthew R White
- Department of Emergency Medicine, Queen's University, Kingston, Ontario, Canada; Department of Critical Care Medicine, Queen's University, Kingston, Ontario, Canada
| | - Heather Braund
- Department of Education, Queen's University, Kingston, Ontario, Canada; Office of Professional Development and Educational Scholarship, Queen's University, Kingston, Ontario, Canada
| | - Daniel Howes
- Department of Emergency Medicine, Queen's University, Kingston, Ontario, Canada; Department of Critical Care Medicine, Queen's University, Kingston, Ontario, Canada
| | - Rylan Egan
- Office of Professional Development and Educational Scholarship, Queen's University, Kingston, Ontario, Canada
| | - Andreas Gegenfurtner
- Institut für Qualität und Weiterbildung, Technische Hochschule Deggendorf, Deggendorf, Germany
| | - Jeroen J G van Merrienboer
- School of Health Professions Education, Faculty of Health, Medicine and Life Sciences, Maastricht University, Maastricht, the Netherlands
| | - Adam Szulewski
- Department of Emergency Medicine, Queen's University, Kingston, Ontario, Canada.
| |
Collapse
|
23
|
Gauthier I. Domain-Specific and Domain-General Individual Differences in Visual Object Recognition. CURRENT DIRECTIONS IN PSYCHOLOGICAL SCIENCE 2018. [DOI: 10.1177/0963721417737151] [Citation(s) in RCA: 15] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/21/2023]
|
24
|
Szulewski A, Gegenfurtner A, Howes DW, Sivilotti MLA, van Merriënboer JJG. Measuring physician cognitive load: validity evidence for a physiologic and a psychometric tool. ADVANCES IN HEALTH SCIENCES EDUCATION : THEORY AND PRACTICE 2017; 22:951-968. [PMID: 27787677 DOI: 10.1007/s10459-016-9725-2] [Citation(s) in RCA: 32] [Impact Index Per Article: 4.6] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 02/25/2016] [Accepted: 10/19/2016] [Indexed: 05/14/2023]
Abstract
In general, researchers attempt to quantify cognitive load using physiologic and psychometric measures. Although the construct measured by both of these metrics is thought to represent overall cognitive load, there is a paucity of studies that compares these techniques to one another. The authors compared data obtained from one physiologic tool (pupillometry) to one psychometric tool (Paas scale) to explore whether they actually measured the construct of cognitive load as purported. Thirty-two participants with a range of resuscitation medicine experience and expertise completed resuscitation-medicine based multiple-choice-questions as well as arithmetic questions. Cognitive load, as measured by both tools, was found to be higher for the more difficult questions as well as for questions that were answered incorrectly (p < 0.001). The group with the least medical experience had higher cognitive load than both the intermediate and experienced groups when answering domain-specific questions (p = 0.023 and p = 0.003 respectively for the physiologic tool; p = 0.006 and p < 0.001 respectively for the psychometric tool). There was a strong positive correlation (Spearman's ρ = 0.827, p < 0.001 for arithmetic questions; Spearman's ρ = 0.606, p < 0.001 for medical questions) between the two cognitive load measurement tools. These findings support the validity argument that both physiologic and psychometric metrics measure the construct of cognitive load.
Collapse
Affiliation(s)
- Adam Szulewski
- Department of Emergency Medicine, Queen's University, Kingston General Hospital, 76 Stuart Street, Kingston, ON, K7L 2V7, Canada
| | - Andreas Gegenfurtner
- Institut für Qualität und Weiterbildung, Technische Hochschule Deggendorf, Edlmairstraße 9, 94469, Deggendorf, Germany
| | - Daniel W Howes
- Departments of Emergency Medicine and Critical Care, Queen's University, Kingston General Hospital, 76 Stuart Street, Kingston, ON, K7L 2V7, Canada.
| | - Marco L A Sivilotti
- Departments of Emergency Medicine and Biomedical and Molecular Sciences, Kingston General Hospital, 76 Stuart Street, Kingston, ON, K7L 2V7, Canada
| | - Jeroen J G van Merriënboer
- Department of Educational Development and Research, Faculty of Health, Medicine and Life Sciences, Maastricht University, Universiteitssingel 60, Room N.5.06, 6200 MD, Maastricht, The Netherlands
| |
Collapse
|
25
|
Sheridan H, Reingold EM. The Holistic Processing Account of Visual Expertise in Medical Image Perception: A Review. Front Psychol 2017; 8:1620. [PMID: 29033865 PMCID: PMC5627012 DOI: 10.3389/fpsyg.2017.01620] [Citation(s) in RCA: 48] [Impact Index Per Article: 6.9] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/05/2017] [Accepted: 09/04/2017] [Indexed: 12/11/2022] Open
Abstract
In the field of medical image perception, the holistic processing perspective contends that experts can rapidly extract global information about the image, which can be used to guide their subsequent search of the image (Swensson, 1980; Nodine and Kundel, 1987; Kundel et al., 2007). In this review, we discuss the empirical evidence supporting three different predictions that can be derived from the holistic processing perspective: Expertise in medical image perception is domain-specific, experts use parafoveal and/or peripheral vision to process large regions of the image in parallel, and experts benefit from a rapid initial glimpse of an image. In addition, we discuss a pivotal recent study (Litchfield and Donovan, 2016) that seems to contradict the assumption that experts benefit from a rapid initial glimpse of the image. To reconcile this finding with the existing literature, we suggest that global processing may serve multiple functions that extend beyond the initial glimpse of the image. Finally, we discuss future research directions, and we highlight the connections between the holistic processing account and similar theoretical perspectives and findings from other domains of visual expertise.
Collapse
Affiliation(s)
- Heather Sheridan
- Department of Psychology, University at Albany, State University of New York, Albany, NY, United States
| | - Eyal M. Reingold
- Department of Psychology, University of Toronto, Mississauga, ON, Canada
| |
Collapse
|
26
|
Eva KW. What's in a name? Definitional clarity and its unintended consequences. MEDICAL EDUCATION 2017; 51:1-2. [PMID: 27981659 DOI: 10.1111/medu.13233] [Citation(s) in RCA: 15] [Impact Index Per Article: 2.1] [Reference Citation Analysis] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/25/2023]
|