Looking in the Same Manner but Seeing it Differently: Bottom-up and Expertise Effects in Radiology

Expert gaze as a usability indicator of medical AI decision support systems: a preliminary study

Given the current state of medical artificial intelligence (AI) and perceptions towards it, collaborative systems are becoming the preferred choice for clinical workflows. This work aims to address expert interaction with medical AI support systems to gain insight towards how these systems can be better designed with the user in mind. As eye tracking metrics have been shown to be robust indicators of usability, we employ them for evaluating the usability and user interaction with medical AI support systems. We use expert gaze to assess experts' interaction with an AI software for caries detection in bitewing x-ray images. We compared standard viewing of bitewing images without AI support versus viewing where AI support could be freely toggled on and off. We found that experts turned the AI on for roughly 25% of the total inspection task, and generally turned it on halfway through the course of the inspection. Gaze behavior showed that when supported by AI, more attention was dedicated to user interface elements related to the AI support, with more frequent transitions from the image itself to these elements. When considering that expert visual strategy is already optimized for fast and effective image inspection, such interruptions in attention can lead to increased time needed for the overall assessment. Gaze analysis provided valuable insights into an AI's usability for medical image inspection. Further analyses of these tools and how to delineate metrical measures of usability should be developed.

The effects of gaze-display feedback on medical students' self-monitoring and learning in radiology

Self-monitoring is essential for effectively regulating learning, but difficult in visual diagnostic tasks such as radiograph interpretation. Eye-tracking technology can visualize viewing behavior in gaze displays, thereby providing information about visual search and decision-making. We hypothesized that individually adaptive gaze-display feedback improves posttest performance and self-monitoring of medical students who learn to detect nodules in radiographs. We investigated the effects of: (1) Search displays, showing which part of the image was searched by the participant; and (2) Decision displays, showing which parts of the image received prolonged attention in 78 medical students. After a pretest and instruction, participants practiced identifying nodules in 16 cases under search-display, decision-display, or no feedback conditions (n = 26 per condition). A 10-case posttest, without feedback, was administered to assess learning outcomes. After each case, participants provided self-monitoring and confidence judgments. Afterward, participants reported on self-efficacy, perceived competence, feedback use, and perceived usefulness of the feedback. Bayesian analyses showed no benefits of gaze displays for post-test performance, monitoring accuracy (absolute difference between participants' estimated and their actual test performance), completeness of viewing behavior, self-efficacy, and perceived competence. Participants receiving search-displays reported greater feedback utilization than participants receiving decision-displays, and also found the feedback more useful when the gaze data displayed was precise and accurate. As the completeness of search was not related to posttest performance, search displays might not have been sufficiently informative to improve self-monitoring. Information from decision displays was rarely used to inform self-monitoring. Further research should address if and when gaze displays can support learning.

Gaze patterns of dentists while evaluating bitewing radiographs

OBJECTIVES

Understanding dentists' gaze patterns on radiographs may allow to unravel sources of their limited accuracy and develop strategies to mitigate them. We conducted an eye tracking experiment to characterize dentists' scanpaths and thus their gaze patterns when assessing bitewing radiographs to detect primary proximal carious lesions.

METHODS

22 dentists assessed a median of nine bitewing images each, resulting in 170 datasets after excluding data with poor quality of gaze recording. Fixation was defined as an area of attentional focus related to visual stimuli. We calculated time to first fixation, fixation count, average fixation duration, and fixation frequency. Analyses were performed for the entire image and stratified by (1) presence of carious lesions and/or restorations and (2) lesion depth (E1/2: outer/inner enamel; D1-3: outer-inner third of dentin). We also examined the transitional nature of the dentists' gaze.

RESULTS

Dentists had more fixations on teeth with lesions and/or restorations (median=138 [interquartile range=87, 204]) than teeth without them (32 [15, 66]), p<0.001. Notably, teeth with lesions had longer fixation durations (407 milliseconds [242, 591]) than those with restorations (289 milliseconds [216, 337]), p<0.001. Time to first fixation was longer for teeth with E1 lesions (17,128 milliseconds [8813, 21,540]) than lesions of other depths (p = 0.049). The highest number of fixations were on teeth with D2 lesions (43 [20, 51]) and lowest on teeth with E1 lesions (5 [1, 37]), p<0.001. Generally, a systematic tooth-by-tooth gaze pattern was observed.

CONCLUSIONS

As hypothesized, while visually inspecting bitewing radiographic images, dentists employed a heightened focus on certain image features/areas, relevant to the assigned task. Also, they generally examined the entire image in a systematic tooth-by-tooth pattern.

Comparing the Diagnostic Performance of ECG Gated versus Non-Gated CT Angiography in Ascending Aortic Dissection: A GRRAS Study

Rationale and Objective: Thoracic CT angiography (CTA) for ascending aortic dissection, a life-threatening emergency, is performed routinely without Electrocardiographic (ECG) gating, therefore allowing the apparition of a pulsation artefact. We aimed to evaluate and compare the diagnostic performance, the inter and intra-reporter agreement of ECG gated CTA and non-ECG gated CTA for detecting ascending aortic dissection, considering their training level. Our hypothesis is that ECG gated CTA has superior diagnostic accuracy for ascending aortic dissection compared to non-gated CTA. Materials and Methods: We collected data using 24 questions survey using clinically validated CT examinations. Sixty-six respondents (medical students, radiology residents, and consultants) blinded to the actual diagnosis independently evaluated the images pertaining to the presence of ascending aortic dissection. The reference standard was represented by clinical and imaging diagnosis. Inter-rater and inter-group concordance was evaluated; the agreement with reference tests was calculated and assessed as a function of reporters’ training level. Results: Reporters’ ascending aortic dissection assessment showed a better correlation with the reference standard in the ECG gated CTA. The inter-rater correlation was higher in the ECG gated CTA compared to non-ECG gated CTA. Observers’ confidence for diagnosing ascending aortic dissection was higher in the ECG gated CTA. Statistically significant differences (p < 0.05) were found between different training levels when assessing non-ECG gated examinations. Conclusions: ECG gated CTA shows a higher diagnostic performance for ascending aortic dissection than non-ECG gated CTA, regardless of the reporters’ training level.

Intuitive and deliberative approaches for diagnosing 'well' versus 'unwell': evidence from eye tracking, and potential implications for training

Rapidly assessing how ill a patient is based on their immediate presentation-colloquially termed 'eyeballing' in practice-serves a vital role in acute care settings. Yet surprisingly little is known about how this diagnostic skill is learned or how it should be taught. Some authors have pointed to a dual-process model, suggesting that assessments of illness severity are driven by two distinct types of processing: an intuitive, fast, pattern recognition-like process (Type 1) that depends on many prior patient encounters and outcomes being stored in memory; and a deliberate, slow, analytic process (Type 2) characterized by additional data gathering, data scrutiny, or recollection of rules. But prior studies have supported a dual-process model for the assessment of illness severity only insofar as experienced clinicians chiefly displayed what was presumed to be Type 1 processing. Here we further explored a dual-process model by examining whether less experienced clinicians displayed both types of processing when assessing illness severity across a series of cases. Consistent with the model, a dissociation between Type 1 and Type 2 processing was observed through resident reports of deliberation, response times, and three eye tracking metrics associated with diagnostic expertise. We conclude by discussing potential implications for the training of this enigmatic diagnostic skill.

Cognitive load and processes during chest radiograph interpretation in the emergency department across the spectrum of expertise

BACKGROUND

In the emergency department (ED), chest radiographs (CXRs) provide essential information for clinical diagnostic reasoning. Errors in interpretation by emergency physicians can lead to negative patient outcomes. To aid in teaching this important skill, an understanding of cognitive processes and cognitive load (CL) in CXR interpretation in emergency medicine (EM) personnel is warranted.

METHODS

This study adopted a concurrent mixed-methods research design. Participant groups included medical students (M), junior (J) and senior (S) EM residents, and attending emergency physicians (P) in the ED at an academic hospital. To elucidate cognitive processes, a real-time cognitive task analysis during CXR interpretation was performed. Interviews were audio recorded, transcribed verbatim, and analyzed thematically. The interview was followed by a questionnaire, where participants rated their CL, stress, and confidence level.

RESULTS

Levels of CL (M vs. S and M vs. P, p = 0.002; J vs. S, p = 0.004; J vs. P, p = 0.005) and stress (J vs. P, p = 0.002) decreased, while confidence levels increased (M vs. S, p = 0.006; J vs. S, p ≤ 0.001; J vs. P, p = 0.003) as experience level increased. Qualitative analysis of interviews revealed four themes: checking behavior, information reduction, pattern recognition versus systematic viewing, and recognizing scope of practice. Experts commonly utilized checking behavior (e.g., comparison to prior radiographs) and deprioritized task irrelevant data. Experts used a general overview technique as their initial approach as opposed to a systematic viewing approach, and they more readily recognized an EM physicians' scope of practice in this task.

CONCLUSION

This study characterized differences in cognition that led to increased CL, stress, and lower level of confidence in EM learners during CXR interpretation and provided insight into expertise development in this important skill.

Expertise development in volumetric image interpretation of radiology residents: what do longitudinal scroll data reveal?

The current study used theories on expertise development (the holistic model of image perception and the information reduction hypothesis) as a starting point to identify and explore potentially relevant process measures to monitor and evaluate expertise development in radiology residency training. It is the first to examine expertise development in volumetric image interpretation (i.e., CT scans) within radiology residents using scroll data collected longitudinally over five years of residency training. Consistent with the holistic model of image perception, the percentage of time spent on full runs, i.e. scrolling through more than 50% of the CT-scan slices (global search), decreased within residents over residency training years. Furthermore, the percentage of time spent on question-relevant areas in the CT scans increased within residents over residency training years, consistent with the information reduction hypothesis. Second, we examined if scroll patterns can predict diagnostic accuracy. The percentage of time spent on full runs and the percentage of time spent on question-relevant areas did not predict diagnostic accuracy. Thus, although scroll patterns over training years are consistent with visual expertise theories, they could not be used as predictors of diagnostic accuracy in the current study. Therefore, the relation between scroll patterns and performance needs to be further examined, before process measures can be used to monitor and evaluate expertise development in radiology residency training.

How massed practice improves visual expertise in reading panoramic radiographs in dental students: An eye tracking study

The interpretation of medical images is an error-prone process that may yield severe consequences for patients. In dental medicine panoramic radiography (OPT) is a frequently used diagnostic procedure. OPTs typically contain multiple, diverse anomalies within one image making the diagnostic process very demanding, rendering students’ development of visual expertise a complex task. Radiograph interpretation is typically taught through massed practice; however, it is not known how effective this approach is nor how it changes students’ visual inspection of radiographs. Therefore, this study investigated how massed practice–an instructional method that entails massed learning of one type of material–affects processing of OPTs and the development of diagnostic performance. From 2017 to 2018, 47 dental students in their first clinical semester diagnosed 10 OPTs before and after their regular massed practice training, which is embedded in their curriculum. The OPTs contained between 3 to 26 to-be-identified anomalies. During massed practice they diagnosed 100 dental radiographs without receiving corrective feedback. The authors recorded students’ eye movements and assessed the number of correctly identified and falsely marked low- and high prevalence anomalies before and after massed practice. Massed practice had a positive effect on detecting anomalies especially with low prevalence (p < .001). After massed practice students covered a larger proportion of the OPTs (p < .001), which was positively related to the detection of low-prevalence anomalies (p = .04). Students also focused longer, more frequently, and earlier on low-prevalence anomalies after massed practice (p_s < .001). While massed practice improved visual expertise in dental students with limited prior knowledge, there is still substantial room for improvement. The results suggest integrating massed practice with more deliberate practice, where, for example, corrective feedback is provided, and support is adapted to students’ needs.

Focal lung pathology detection in radiology: Is there an effect of experience on visual search behavior?

In radiology, 60% to 80% of diagnostic errors are perceptual. The use of more efficient visual search behaviors is expected to reduce these errors. We collected eye-tracking data from participants with different levels of experience when interpreting chest X-rays during the completion of a pathology-detection task. Eye-tracking measures were assessed in the context of three existing visual search theories from the literature to understand the association between visual search behavior and underlying processes: the long-term working memory theory, the information-reduction hypothesis, and the holistic model of image perception. The most experienced participants (radiology residents) showed the highest level of performance, albeit their visual search behaviors did not differ from the intermediate group. This suggests that radiology residents better processed the represented information on the X-ray, using a visual search strategy similar to the intermediate group. Since similar visual search resulted in more information extraction in the radiology residents compared with the intermediates, we suggest that this result might support the long-term working memory theory. Furthermore, compared with novices, intermediates and radiology residents fixated longer on areas that were more important to avoid missing any pathology, which possibly confirms the information-reduction hypothesis. Finally, the larger distances between fixations observed in more experienced participants could support the holistic model of image perception. In addition, measures of generic skills were related to a lower time cost for switching between global and local information processing. Our findings suggest that the three theories may be complementary in chest X-ray interpretation. Therefore, a unified theory explaining perceptual-cognitive superiority in radiology is considered.

Pupil diameter differentiates expertise in dental radiography visual search

Expert behavior is characterized by rapid information processing abilities, dependent on more structured schemata in long-term memory designated for their domain-specific tasks. From this understanding, expertise can effectively reduce cognitive load on a domain-specific task. However, certain tasks could still evoke different gradations of load even for an expert, e.g., when having to detect subtle anomalies in dental radiographs. Our aim was to measure pupil diameter response to anomalies of varying levels of difficulty in expert and student dentists’ visual examination of panoramic radiographs. We found that students’ pupil diameter dilated significantly from baseline compared to experts, but anomaly difficulty had no effect on pupillary response. In contrast, experts’ pupil diameter responded to varying levels of anomaly difficulty, where more difficult anomalies evoked greater pupil dilation from baseline. Experts thus showed proportional pupillary response indicative of increasing cognitive load with increasingly difficult anomalies, whereas students showed pupillary response indicative of higher cognitive load for all anomalies when compared to experts.

[Quality of findings, interpretation, and self-assessment of readers of chest x-rays acquired in one plane depending on the use of an anatomical viewing scheme and reader expertise]

PURPOSE

To objectify effects of an anatomical viewing scheme on the respective correctness of (a) findings, (b) interpretations, and (c) self-assessments of readers in chest radiographs acquired in one plane and the assessment of other influencing factors.

MATERIALS AND METHODS

In all, 20 radiologists with 3-60 months of full-time radiography experience evaluated 12 chest radiographs of varying difficulty: once with and once without using an anatomical viewing scheme with at least 1 month in between (n = 480). In consensus of 3 radiological experts (a) and (b) were determined by means of a current computed tomography. The self-assessment (c) of readers was queried.

RESULTS

(a) Findings were either missed or not described in 21%. Another 20% were recognized, but incorrectly described, (b) 62% of interpretations and 31% of derived clinical consequences were wrong and (c) in 39% of items the readers overestimated themselves. Experts were faster and better than novices, but for the scheme usage no further significant differences were detected (p > 0.5, respectively). The most pronounced effect was found in comparison with the routine report produced by the joint evaluation of novices and experts being clearly superior even to the expert study results (a), (b) and (c) alone (p < 0.001, respectively).

CONCLUSION

Reporting of chest X‑rays acquired in one plane was often incomplete or even wrong, and the evaluators overestimated themselves, which was not influenced by the use of the anatomical viewing scheme. Since errors between the evaluators sometimes differed greatly, duplicate evaluation of the radiographs by two different radiologists, which is already the case in many training facilities, may possibly be advisable in general.

A Review of Perceptual Expertise in Radiology-How it develops, How we can test it, and Why humans still matter in the era of Artificial Intelligence

As the first step in image interpretation is detection, an error in perception can prematurely end the diagnostic process leading to missed diagnoses. Because perceptual errors of this sort-"failure to detect"-are the most common interpretive error (and cause of litigation) in radiology, understanding the nature of perceptual expertise is essential in decreasing radiology's long-standing error rates. In this article, we review what constitutes a perceptual error, the existing models of radiologic image perception, the development of perceptual expertise and how it can be tested, perceptual learning methods in training radiologists, and why understanding perceptual expertise is still relevant in the era of artificial intelligence. Adding targeted interventions, such as perceptual learning, to existing teaching practices, has the potential to enhance expertise and reduce medical error.

An Expert Derived Feedforward Histology Module Improves Pattern Recognition Efficiency in Novice Students

Histology is a visually oriented, foundational anatomical sciences subject in professional health curricula that has seen a dramatic reduction in educational contact hours and an increase in content migration to a digital platform. While the digital migration of histology laboratories has transformed histology education, few studies have shown the impact of this change on visual literacy development, a critical competency in histology. The objective of this study was to assess whether providing a video clip of an expert's gaze while completing leukocyte identification tasks would increase the efficiency and performance of novices completing similar identification tasks. In a randomized study, one group of novices (n = 9) was provided with training materials that included expert eye gaze, while the other group (n = 12) was provided training materials with identical content, but without the expert eye gaze. Eye movement parameters including fixation rate and total scan path distance, and performance measures including time-to-task-completion and accuracy, were collected during an identification task assessment. Compared to the control group, the average fixation duration was 13.2% higher (P < 0.02) and scan path distance was 35.0% shorter in the experimental group (P = 0.14). Analysis of task performance measures revealed no significant difference between the groups. These preliminary results suggest a more efficient search performed by the experimental group, indicating the potential efficacy of training using an expert's gaze to enhance visual literacy development. With further investigation, such feedforward enhanced training methods could be utilized for histology and other visually oriented subjects.

What do we know about volumetric medical image interpretation?: a review of the basic science and medical image perception literatures

Interpretation of volumetric medical images represents a rapidly growing proportion of the workload in radiology. However, relatively little is known about the strategies that best guide search behavior when looking for abnormalities in volumetric images. Although there is extensive literature on two-dimensional medical image perception, it is an open question whether the conclusions drawn from these images can be generalized to volumetric images. Importantly, volumetric images have distinct characteristics (e.g., scrolling through depth, smooth-pursuit eye-movements, motion onset cues, etc.) that should be considered in future research. In this manuscript, we will review the literature on medical image perception and discuss relevant findings from basic science that can be used to generate predictions about expertise in volumetric image interpretation. By better understanding search through volumetric images, we may be able to identify common sources of error, characterize the optimal strategies for searching through depth, or develop new training and assessment techniques for radiology residents.

Analysis of Perceptual Expertise in Radiology - Current Knowledge and a New Perspective

Radiologists rely principally on visual inspection to detect, describe, and classify findings in medical images. As most interpretive errors in radiology are perceptual in nature, understanding the path to radiologic expertise during image analysis is essential to educate future generations of radiologists. We review the perceptual tasks and challenges in radiologic diagnosis, discuss models of radiologic image perception, consider the application of perceptual learning methods in medical training, and suggest a new approach to understanding perceptional expertise. Specific principled enhancements to educational practices in radiology promise to deepen perceptual expertise among radiologists with the goal of improving training and reducing medical error.

Eye tracking: the silver bullet of competency assessment in medical image interpretation?

None

Volumetric image interpretation in radiology: scroll behavior and cognitive processes

The interpretation of medical images is a primary task for radiologists. Besides two-dimensional (2D) images, current imaging technologies allow for volumetric display of medical images. Whereas current radiology practice increasingly uses volumetric images, the majority of studies on medical image interpretation is conducted on 2D images. The current study aimed to gain deeper insight into the volumetric image interpretation process by examining this process in twenty radiology trainees who all completed four volumetric image cases. Two types of data were obtained concerning scroll behaviors and think-aloud data. Types of scroll behavior concerned oscillations, half runs, full runs, image manipulations, and interruptions. Think-aloud data were coded by a framework of knowledge and skills in radiology including three cognitive processes: perception, analysis, and synthesis. Relating scroll behavior to cognitive processes showed that oscillations and half runs coincided more often with analysis and synthesis than full runs, whereas full runs coincided more often with perception than oscillations and half runs. Interruptions were characterized by synthesis and image manipulations by perception. In addition, we investigated relations between cognitive processes and found an overall bottom-up way of reasoning with dynamic interactions between cognitive processes, especially between perception and analysis. In sum, our results highlight the dynamic interactions between these processes and the grounding of cognitive processes in scroll behavior. It suggests, that the types of scroll behavior are relevant to describe how radiologists interact with and manipulate volumetric images.

A new way to look at simulation-based assessment: the relationship between gaze-tracking and exam performance

Objective

A key task of the team leader in a medical emergency is effective information gathering. Studying information gathering patterns is readily accomplished with the use of gaze-tracking glasses. This technology was used to generate hypotheses about the relationship between performance scores and expert-hypothesized visual areas of interest in residents across scenarios in simulated medical resuscitation examinations.

Methods

Emergency medicine residents wore gaze-tracking glasses during two simulation-based examinations (n=29 and 13 respectively). Blinded experts assessed video-recorded performances using a simulation performance assessment tool that has validity evidence in this context. The relationships between gaze patterns and performance scores were analyzed and potential hypotheses generated. Four scenarios were assessed in this study: diabetic ketoacidosis, bradycardia secondary to beta-blocker overdose, ruptured abdominal aortic aneurysm and metabolic acidosis caused by antifreeze ingestion.

Results

Specific gaze patterns were correlated with objective performance. High performers were more likely to fixate on task-relevant stimuli and appropriately ignore task-irrelevant stimuli compared with lower performers. For example, shorter latency to fixation on the vital signs in a case of diabetic ketoacidosis was positively correlated with performance (r=0.70, p<0.05). Conversely, total time spent fixating on lab values in a case of ruptured abdominal aortic aneurysm was negatively correlated with performance (r= −0.50, p<0.05).

Conclusions

There are differences between the visual patterns of high and low-performing residents. These findings may allow for better characterization of expertise development in resuscitation medicine and provide a framework for future study of visual behaviours in resuscitation cases.

Measuring physician cognitive load: validity evidence for a physiologic and a psychometric tool

In general, researchers attempt to quantify cognitive load using physiologic and psychometric measures. Although the construct measured by both of these metrics is thought to represent overall cognitive load, there is a paucity of studies that compares these techniques to one another. The authors compared data obtained from one physiologic tool (pupillometry) to one psychometric tool (Paas scale) to explore whether they actually measured the construct of cognitive load as purported. Thirty-two participants with a range of resuscitation medicine experience and expertise completed resuscitation-medicine based multiple-choice-questions as well as arithmetic questions. Cognitive load, as measured by both tools, was found to be higher for the more difficult questions as well as for questions that were answered incorrectly (p < 0.001). The group with the least medical experience had higher cognitive load than both the intermediate and experienced groups when answering domain-specific questions (p = 0.023 and p = 0.003 respectively for the physiologic tool; p = 0.006 and p < 0.001 respectively for the psychometric tool). There was a strong positive correlation (Spearman's ρ = 0.827, p < 0.001 for arithmetic questions; Spearman's ρ = 0.606, p < 0.001 for medical questions) between the two cognitive load measurement tools. These findings support the validity argument that both physiologic and psychometric metrics measure the construct of cognitive load.

The Holistic Processing Account of Visual Expertise in Medical Image Perception: A Review

In the field of medical image perception, the holistic processing perspective contends that experts can rapidly extract global information about the image, which can be used to guide their subsequent search of the image (Swensson, 1980; Nodine and Kundel, 1987; Kundel et al., 2007). In this review, we discuss the empirical evidence supporting three different predictions that can be derived from the holistic processing perspective: Expertise in medical image perception is domain-specific, experts use parafoveal and/or peripheral vision to process large regions of the image in parallel, and experts benefit from a rapid initial glimpse of an image. In addition, we discuss a pivotal recent study (Litchfield and Donovan, 2016) that seems to contradict the assumption that experts benefit from a rapid initial glimpse of the image. To reconcile this finding with the existing literature, we suggest that global processing may serve multiple functions that extend beyond the initial glimpse of the image. Finally, we discuss future research directions, and we highlight the connections between the holistic processing account and similar theoretical perspectives and findings from other domains of visual expertise.

How visual search relates to visual diagnostic performance: a narrative systematic review of eye-tracking research in radiology

Eye tracking research has been conducted for decades to gain understanding of visual diagnosis such as in radiology. For educational purposes, it is important to identify visual search patterns that are related to high perceptual performance and to identify effective teaching strategies. This review of eye-tracking literature in the radiology domain aims to identify visual search patterns associated with high perceptual performance. Databases PubMed, EMBASE, ERIC, PsycINFO, Scopus and Web of Science were searched using 'visual perception' OR 'eye tracking' AND 'radiology' and synonyms. Two authors independently screened search results and included eye tracking studies concerning visual skills in radiology published between January 1, 1994 and July 31, 2015. Two authors independently assessed study quality with the Medical Education Research Study Quality Instrument, and extracted study data with respect to design, participant and task characteristics, and variables. A thematic analysis was conducted to extract and arrange study results, and a textual narrative synthesis was applied for data integration and interpretation. The search resulted in 22 relevant full-text articles. Thematic analysis resulted in six themes that informed the relation between visual search and level of expertise: (1) time on task, (2) eye movement characteristics of experts, (3) differences in visual attention, (4) visual search patterns, (5) search patterns in cross sectional stack imaging, and (6) teaching visual search strategies. Expert search was found to be characterized by a global-focal search pattern, which represents an initial global impression, followed by a detailed, focal search-to-find mode. Specific task-related search patterns, like drilling through CT scans and systematic search in chest X-rays, were found to be related to high expert levels. One study investigated teaching of visual search strategies, and did not find a significant effect on perceptual performance. Eye tracking literature in radiology indicates several search patterns are related to high levels of expertise, but teaching novices to search as an expert may not be effective. Experimental research is needed to find out which search strategies can improve image perception in learners.

The Effect of Teaching Search Strategies on Perceptual Performance

RATIONALE AND OBJECTIVES

Radiology expertise is dependent on the use of efficient search strategies. The aim of this study is to investigate the effect of teaching search strategies on trainee's accuracy in detecting lung nodules at computed tomography.

MATERIALS AND METHODS

Two search strategies, "scanning" and "drilling," were tested with a randomized crossover design. Nineteen junior radiology residents were randomized into two groups. Both groups first completed a baseline lung nodule detection test allowing a free search strategy, followed by a test after scanning instruction and drilling instruction or vice versa. True positive (TP) and false positive (FP) scores and scroll behavior were registered. A mixed-design analysis of variance was applied to compare the three search conditions.

RESULTS

Search strategy instruction had a significant effect on scroll behavior, F(1.3) = 54.2, P < 0.001; TP score, F(2) = 16.1, P < 0.001; and FP score, F(1.3) = 15.3, P < 0.001. Scanning instruction resulted in significantly lower TP scores than drilling instruction (M = 10.7, SD = 5.0 versus M = 16.3, SD = 5.3), t(18) = 4.78, P < 0.001; or free search (M = 15.3, SD = 4.6), t(18) = 4.44, P < 0.001. TP scores for drilling did not significantly differ from free search. FP scores for drilling (M = 7.3, SD = 5.6) were significantly lower than for free search (M = 12.5, SD = 7.8), t(18) = 4.86, P < 0.001.

CONCLUSIONS

Teaching a drilling strategy is preferable to teaching a scanning strategy for finding lung nodules.

RecutClub.com: An Open Source, Whole Slide Image-based Pathology Education System

BACKGROUND

Our institution's pathology unknown conferences provide educational cases for our residents. However, the cases have not been previously available digitally, have not been collated for postconference review, and were not accessible to a wider audience. Our objective was to create an inexpensive whole slide image (WSI) education suite to address these limitations and improve the education of pathology trainees.

MATERIALS AND METHODS

We surveyed residents regarding their preference between four unique WSI systems. We then scanned weekly unknown conference cases and study set cases and uploaded them to our custom built WSI viewer located at RecutClub.com. We measured site utilization and conference participation.

RESULTS

Residents preferred our OpenLayers WSI implementation to Ventana Virtuoso, Google Maps API, and OpenSlide. Over 16 months, we uploaded 1366 cases from 77 conferences and ten study sets, occupying 793.5 GB of cloud storage. Based on resident evaluations, the interface was easy to use and demonstrated minimal latency. Residents are able to review cases from home and from their mobile devices. Worldwide, 955 unique IP addresses from 52 countries have viewed cases in our site.

CONCLUSIONS

We implemented a low-cost, publicly available repository of WSI slides for resident education. Our trainees are very satisfied with the freedom to preview either the glass slides or WSI and review the WSI postconference. Both local users and worldwide users actively and repeatedly view cases in our study set.

Before your very eyes: the value and limitations of eye tracking in medical education

CONTEXT

Medicine is a highly visual discipline. Physicians from many specialties constantly use visual information in diagnosis and treatment. However, they are often unable to explain how they use this information. Consequently, it is unclear how to train medical students in this visual processing. Eye tracking is a research technique that may offer answers to these open questions, as it enables researchers to investigate such visual processes directly by measuring eye movements. This may help researchers understand the processes that support or hinder a particular learning outcome.

AIM

In this article, we clarify the value and limitations of eye tracking for medical education researchers. For example, eye tracking can clarify how experience with medical images mediates diagnostic performance and how students engage with learning materials. Furthermore, eye tracking can also be used directly for training purposes by displaying eye movements of experts in medical images.

CONCLUSIONS

Eye movements reflect cognitive processes, but cognitive processes cannot be directly inferred from eye-tracking data. In order to interpret eye-tracking data properly, theoretical models must always be the basis for designing experiments as well as for analysing and interpreting eye-tracking data. The interpretation of eye-tracking data is further supported by sound experimental design and methodological triangulation.

Systematic viewing in radiology: seeing more, missing less?

To prevent radiologists from overlooking lesions, radiology textbooks recommend "systematic viewing," a technique whereby anatomical areas are inspected in a fixed order. This would ensure complete inspection (full coverage) of the image and, in turn, improve diagnostic performance. To test this assumption, two experiments were performed. Both experiments investigated the relationship between systematic viewing, coverage, and diagnostic performance. Additionally, the first investigated whether systematic viewing increases with expertise; the second investigated whether novices benefit from full-coverage or systematic viewing training. In Experiment 1, 11 students, ten residents, and nine radiologists inspected five chest radiographs. Experiment 2 had 75 students undergo a training in either systematic, full-coverage (without being systematic) or non-systematic viewing. Eye movements and diagnostic performance were measured throughout both experiments. In Experiment 1, no significant correlations were found between systematic viewing and coverage, r = -.10, p = .62, and coverage and performance, r = -.06, p = .74. Experts were significantly more systematic than students F2,25 = 4.35, p = .02. In Experiment 2, significant correlations were found between systematic viewing and coverage, r = -.35, p < .01, but not between coverage and performance, r = .13, p = .31. Participants in the full-coverage training performed worse compared with both other groups, which did not differ between them, F2,71 = 3.95, p = .02. In conclusion, the data question the assumption that systematic viewing leads to increased coverage, and, consequently, to improved performance. Experts inspected cases more systematically, but students did not benefit from systematic viewing training.

Case Comparisons: An Efficient Way of Learning Radiology

RATIONALE AND OBJECTIVES

Radiologists commonly use comparison films to improve their differential diagnosis. Educational literature suggests that this technique might also be used to bolster the process of learning to interpret radiographs. We investigated the effectiveness of three comparison techniques in medical students, whom we invited to compare cases of the same disease (same-disease comparison), cases of different diseases (different-disease comparison), disease images with normal images (disease/normal comparison), and identical images (no comparison/control condition). Furthermore, we used eye-tracking technology to investigate which elements of the two cases were compared by the students.

MATERIALS AND METHODS

We randomly assigned 84 medical students to one of four conditions and had them study different diseases on chest radiographs, while their eye movements were being measured. Thereafter, participants took two tests that measured diagnostic performance and their ability to locate diseases, respectively.

RESULTS

Students studied most efficiently in the same-disease and different-disease comparison conditions: test 1, F(3, 68) = 3.31, P = .025, ηp(2) = 0.128; test 2, F(3, 65) = 2.88, P = .043, ηp(2) = 0.117. We found that comparisons were effected in 91% of all trials (except for the control condition). Comparisons between normal anatomy were particularly common (45.8%) in all conditions.

CONCLUSIONS

Comparing cases can be an efficient way of learning to interpret radiographs, especially when the comparison technique used is specifically tailored to the learning goal. Eye tracking provided insight into the comparison process, by showing that few comparisons were made between abnormalities, for example.

Combining bimodal presentation schemes and buzz groups improves clinical reasoning and learning at morning report

Morning reports offer opportunities for intensive work-based learning. In this controlled study, we measured learning processes and outcomes with the report of paediatric emergency room patients. Twelve specialists and 12 residents were randomised into four groups and discussed the same two paediatric cases. The groups differed in their presentation modality (verbal only vs. verbal + text) and the use of buzz groups (with vs. without). The verbal interactions were analysed for clinical reasoning processes. Perceptions of learning and judgment of learning were reported in a questionnaire. Diagnostic accuracy was assessed by a 20-item multiple-choice test. Combined bimodal presentation and buzz groups increased the odds ratio of clinical reasoning to occur in the discussion of cases by a factor of 1.90 (p = 0.013), indicating superior reasoning for buzz groups working with bimodal materials. For specialists, a positive effect of bimodal presentation was found on perceptions of learning (p < 0.05), and for residents, a positive effect of buzz groups was found on judgment of learning (p < 0.005). A positive effect of bimodal presentation on diagnostic accuracy was noted in the specialists (p < 0.05). Combined bimodal presentation and buzz group discussion of emergency cases improves clinicians' clinical reasoning and learning.

Holistic processing improves change detection but impairs change identification

It has been just over a century since Gestalt psychologists described the factors that contribute to the holistic processing of visually presented stimuli. Recent research indicates that holistic processing may come at a cost; specifically, the perception of holistic forms may reduce the visibility of constituent parts. In the present experiment, we examined change detection and change identification accuracy with Kanizsa rectangle patterns that were arranged to either form a Gestalt whole or not. Results from an experiment with 62 participants support this trade-off in processing holistic forms. Holistic processing improved the detection of change but obstructed its identification. Results are discussed in terms of both their theoretical significance and their application in areas ranging from baggage screening and the detection of changes in radiological images to the systems that are used to generate composite images of perpetrators on the basis of eyewitness reports.