Thin-section CT of the lungs: Eye-tracking analysis of the visual approach to reading tiled and stacked display formats

How to Read an Abdominal CT: Insights from the Visual and Cognitive Sciences Translated for Clinical Practice

When first learning abdominal CT studies, residents are often given little concrete, practical direction. There is, however, a large literature from the visual and cognitive sciences that can provide guidance towards search strategies that maximize efficiency and comprehensiveness. This literature has not penetrated radiology teaching to any great extent. In this article, we will examine the current pedagogy (and why that falls short), why untutored search fails, where misses occur in abdomen/pelvis CT, why these misses occur where they do, how expert radiologists search 3d image stacks, and how novices might expedite the acquisition of expertise.

The Radiologist's Gaze: Mapping Three-Dimensional Visual Search in Computed Tomography of the Abdomen and Pelvis

A radiologist's search pattern can directly influence patient management. A missed finding is a missed opportunity for intervention. Multiple studies have attempted to describe and quantify search patterns but have mainly focused on chest radiographs and chest CTs. Here, we describe and quantify the visual search patterns of 17 radiologists as they scroll through 6 CTs of the abdomen and pelvis. Search pattern tracings varied among individuals and remained relatively consistent per individual between cases. Attendings and trainees had similar eye metric statistics with respect to time to first fixation (TTFF), number of fixations in the region of interest (ROI), fixation duration in ROI, mean saccadic amplitude, or total number of fixations. Attendings had fewer numbers of fixations per second versus trainees (p < 0.001), suggesting efficiency due to expertise. In those cases that were accurately interpreted, TTFF was shorter (p = 0.04), the number of fixations per second and number of fixations in ROI were higher (p = 0.04, p = 0.02, respectively), and fixation duration in ROI was increased (p = 0.02). We subsequently categorized radiologists as "scanners" or "drillers" by both qualitative and quantitative methods and found no differences in accuracy with most radiologists being categorized as "drillers." This study describes visual search patterns of radiologists in interpretation of CTs of the abdomen and pelvis to better approach future endeavors in determining the effects of manipulations such as fatigue, interruptions, and computer-aided detection.

Scan, dwell, decide: Strategies for detecting abnormalities in diabetic retinopathy

Diabetic retinopathy (DR) is a disease which is widely diagnosed using (colour fundus) images. Efficiency and accuracy are critical in diagnosing DR as lack of timely intervention can lead to irreversible visual impairment. In this paper, we examine strategies for scrutinizing images which affect diagnostic performance of medical practitioners via an eye-tracking study. A total of 56 subjects with 0 to 18 years of experience participated in the study. Every subject was asked to detect DR from 40 images. The findings indicate that practitioners use mainly two types of strategies characterized by either higher dwell duration or longer track length. The main findings of the study are that higher dwell-based strategy led to higher average accuracy (> 85%) in diagnosis, irrespective of the expertise of practitioner; whereas, the average obtained accuracy with a long-track length-based strategy was dependent on the expertise of the practitioner. In the second part of the paper, we use the experimental findings to recommend a scanning strategy for fast and accurate diagnosis of DR that can be potentially used by image readers. This is derived by combining the eye-tracking gaze maps of medical experts in a novel manner based on a set of rules. This strategy requires scrutiny of images in a manner which is consistent with spatial preferences found in human perception in general and in the domain of fundus images in particular. The Levenshtein distance-based assessment of gaze patterns also establish the effectiveness of the derived scanning pattern and is thus recommended for image readers.

Comparing search patterns in digital breast tomosynthesis and full-field digital mammography: an eye tracking study

As a promising imaging modality, digital breast tomosynthesis (DBT) leads to better diagnostic performance than traditional full-field digital mammograms (FFDM) alone. DBT allows different planes of the breast to be visualized, reducing occlusion from overlapping tissue. Although DBT is gaining popularity, best practices for search strategies in this medium are unclear. Eye tracking allowed us to describe search patterns adopted by radiologists searching DBT and FFDM images. Eleven radiologists examined eight DBT and FFDM cases. Observers marked suspicious masses with mouse clicks. Eye position was recorded at 1000 Hz and was coregistered with slice/depth plane as the radiologist scrolled through the DBT images, allowing a 3-D representation of eye position. Hit rate for masses was higher for tomography cases than 2-D cases and DBT led to lower false positive rates. However, search duration was much longer for DBT cases than FFDM. DBT was associated with longer fixations but similar saccadic amplitude compared with FFDM. When comparing radiologists' eye movements to a previous study, which tracked eye movements as radiologists read chest CT, we found DBT viewers did not align with previously identified "driller" or "scanner" strategies, although their search strategy most closely aligns with a type of vigorous drilling strategy.

GazeDx: Interactive Visual Analytics Framework for Comparative Gaze Analysis with Volumetric Medical Images

We present an interactive visual analytics framework, GazeDx (abbr. of GazeDiagnosis), for the comparative analysis of gaze data from multiple readers examining volumetric images while integrating important contextual information with the gaze data. Gaze pattern comparison is essential to understanding how radiologists examine medical images, and to identifying factors influencing the examination. Most prior work depended upon comparisons with manually juxtaposed static images of gaze tracking results. Comparative gaze analysis with volumetric images is more challenging due to the additional cognitive load on 3D perception. A recent study proposed a visualization design based on direct volume rendering (DVR) for visualizing gaze patterns in volumetric images; however, effective and comprehensive gaze pattern comparison is still challenging due to a lack of interactive visualization tools for comparative gaze analysis. We take the challenge with GazeDx while integrating crucial contextual information such as pupil size and windowing into the analysis process for more in-depth and ecologically valid findings. Among the interactive visualization components in GazeDx, a context-embedded interactive scatterplot is especially designed to help users examine abstract gaze data in diverse contexts by embedding medical imaging representations well known to radiologists in it. We present the results from two case studies with two experienced radiologists, where they compared the gaze patterns of 14 radiologists reading two patients' volumetric CT images.

Temporal Characteristics of Radiologists' and Novices' Lesion Detection in Viewing Medical Images Presented Rapidly and Sequentially

Although viewing multiple stacks of medical images presented on a display is a relatively new but useful medical task, little is known about this task. Particularly, it is unclear how radiologists search for lesions in this type of image reading. When viewing cluttered and dynamic displays, continuous motion itself does not capture attention. Thus, it is effective for the target detection that observers' attention is captured by the onset signal of a suddenly appearing target among the continuously moving distractors (i.e., a passive viewing strategy). This can be applied to stack viewing tasks, because lesions often show up as transient signals in medical images which are sequentially presented simulating a dynamic and smoothly transforming image progression of organs. However, it is unclear whether observers can detect a target when the target appears at the beginning of a sequential presentation where the global apparent motion onset signal (i.e., signal of the initiation of the apparent motion by sequential presentation) occurs. We investigated the ability of radiologists to detect lesions during such tasks by comparing the performances of radiologists and novices. Results show that overall performance of radiologists is better than novices. Furthermore, the temporal locations of lesions in CT image sequences, i.e., when a lesion appears in an image sequence, does not affect the performance of radiologists, whereas it does affect the performance of novices. Results indicate that novices have greater difficulty in detecting a lesion appearing early than late in the image sequence. We suggest that radiologists have other mechanisms to detect lesions in medical images with little attention which novices do not have. This ability is critically important when viewing rapid sequential presentations of multiple CT images, such as stack viewing tasks.

Review of prospects and challenges of eye tracking in volumetric imaging

While eye tracking research in conventional radiography has flourished over the past decades, the number of eye tracking studies that looked at multislice images lags behind. A possible reason for the lack of studies in this area might be that the eye tracking methodology used in the context of conventional radiography cannot be applied one-on-one to volumetric imaging material. Challenges associated with eye tracking in volumetric imaging are particularly associated with the selection of stimulus material, the detection of events in the eye tracking data, the calculation of meaningful eye tracking parameters, and the reporting of abnormalities. However, all of these challenges can be addressed in the design of the experiment. If this is done, eye tracking studies using volumetric imaging material offer almost unlimited opportunity for perception research and are highly relevant as the number of volumetric images that are acquired and interpreted is rising.

Volumetric and two-dimensional image interpretation show different cognitive processes in learners

RATIONALE AND OBJECTIVES

In current practice, radiologists interpret digital images, including a substantial amount of volumetric images. We hypothesized that interpretation of a stack of a volumetric data set demands different skills than interpretation of two-dimensional (2D) cross-sectional images. This study aimed to investigate and compare knowledge and skills used for interpretation of volumetric versus 2D images.

MATERIALS AND METHODS

Twenty radiology clerks were asked to think out loud while reading four or five volumetric computed tomography (CT) images in stack mode and four or five 2D CT images. Cases were presented in a digital testing program allowing stack viewing of volumetric data sets and changing views and window settings. Thoughts verbalized by the participants were registered and coded by a framework of knowledge and skills concerning three components: perception, analysis, and synthesis. The components were subdivided into 16 discrete knowledge and skill elements. A within-subject analysis was performed to compare cognitive processes during volumetric image readings versus 2D cross-sectional image readings.

RESULTS

Most utterances contained knowledge and skills concerning perception (46%). A smaller part involved synthesis (31%) and analysis (23%). More utterances regarded perception in volumetric image interpretation than in 2D image interpretation (Median 48% vs 35%; z = -3.9; P < .001). Synthesis was less prominent in volumetric than in 2D image interpretation (Median 28% vs 42%; z = -3.9; P < .001). No differences were found in analysis utterances.

CONCLUSIONS

Cognitive processes in volumetric and 2D cross-sectional image interpretation differ substantially. Volumetric image interpretation draws predominantly on perceptual processes, whereas 2D image interpretation is mainly characterized by synthesis. The results encourage the use of volumetric images for teaching and testing perceptual skills.

A passing glance? Differences in eye tracking and gaze patterns between trainees and experts reading plain film bunion radiographs

Eye tracking and gaze pattern studies have been used to evaluate human behavior for decades. This is because of its ability to reveal conscious and subconscious behaviors when subjects are tasked with observation, decision making, and surgical performance. Many have popularized the use of this technology for radiographic assessment while evaluating radiologist behaviors, but little has been described for surgeon behavior patterns when evaluating preoperative deformities by radiograph. Because the radiographic assessment strongly influences surgical selection, the present study was designed to evaluate the differences between groups of novice and experienced surgeons' gaze patterns when tasked to describe hallux valgus deformities. The subjects were asked to rate the deformity as "none," "mild," "moderate," or "severe." Using an externally mounted eye tracking system, our study assessed saccades, fixations, overall time spent per radiograph, and the subjects' chosen bunion rating. Both the novice and advanced groups of foot and ankle surgeons were tasked to evaluate 25 total anteroposterior radiographs from patients who presented with a primary complaint of bunion pain. These patients were chosen at random, such that all participating surgeons had no previous patient familiarization. Statistically significant differences were observed with regard to the activity and rating of the moderate bunion films. The experience of surgeons does appear to modify gaze behavior with respect to time and attention, such that less overall time spent per image is needed by the advanced group, with improved efficiency. Future academic curriculum and training techniques could be developed to reflect these potential technical differences in search behavior, diagnostic technique, and surgical selection strategy.

Characterizing search, recognition, and decision in the detection of lung nodules on CT scans: elucidation with eye tracking

PURPOSE

To determine the effectiveness of radiologists' search, recognition, and acceptance of lung nodules on computed tomographic (CT) images by using eye tracking.

MATERIALS AND METHODS

This study was performed with a protocol approved by the institutional review board. All study subjects provided informed consent, and all private health information was protected in accordance with HIPAA. A remote eye tracker was used to record time-varying gaze paths while 13 radiologists interpreted 40 lung CT images with an average of 3.9 synthetic nodules (5-mm diameter) embedded randomly in the lung parenchyma. The radiologists' gaze volumes ( GV gaze volume s) were defined as the portion of the lung parenchyma within 50 pixels (approximately 3 cm) of all gaze points. The fraction of the total lung volume encompassed within the GV gaze volume s, the fraction of lung nodules encompassed within each GV gaze volume (search effectiveness), the fraction of lung nodules within the GV gaze volume detected by the reader (recognition-acceptance effectiveness), and overall sensitivity of lung nodule detection were measured.

RESULTS

Detected nodules were within 50 pixels of the nearest gaze point for 990 of 992 correct detections. On average, radiologists searched 26.7% of the lung parenchyma in 3 minutes and 16 seconds and encompassed between 86 and 143 of 157 nodules within their GV gaze volume s. Once encompassed within their GV gaze volume , the average sensitivity of nodule recognition and acceptance ranged from 47 of 100 nodules to 103 of 124 nodules (sensitivity, 0.47-0.82). Overall sensitivity ranged from 47 to 114 of 157 nodules (sensitivity, 0.30-0.73) and showed moderate correlation (r = 0.62, P = .02) with the fraction of lung volume searched.

CONCLUSION

Relationships between reader search, recognition and acceptance, and overall lung nodule detection rate can be studied with eye tracking. Radiologists appear to actively search less than half of the lung parenchyma, with substantial interreader variation in volume searched, fraction of nodules included within the search volume, sensitivity for nodules within the search volume, and overall detection rate.

Towards a framework for analysis of eye-tracking studies in the three dimensional environment: a study of visual search by experienced readers of endoluminal CT colonography

OBJECTIVE

Eye tracking in three dimensions is novel, but established descriptors derived from two-dimensional (2D) studies are not transferable. We aimed to develop metrics suitable for statistical comparison of eye-tracking data obtained from readers of three-dimensional (3D) "virtual" medical imaging, using CT colonography (CTC) as a typical example.

METHODS

Ten experienced radiologists were eye tracked while observing eight 3D endoluminal CTC videos. Subsequently, we developed metrics that described their visual search patterns based on concepts derived from 2D gaze studies. Statistical methods were developed to allow analysis of the metrics.

RESULTS

Eye tracking was possible for all readers. Visual dwell on the moving region of interest (ROI) was defined as pursuit of the moving object across multiple frames. Using this concept of pursuit, five categories of metrics were defined that allowed characterization of reader gaze behaviour. These were time to first pursuit, identification and assessment time, pursuit duration, ROI size and pursuit frequency. Additional subcategories allowed us to further characterize visual search between readers in the test population.

CONCLUSION

We propose metrics for the characterization of visual search of 3D moving medical images. These metrics can be used to compare readers' visual search patterns and provide a reproducible framework for the analysis of gaze tracking in the 3D environment.

ADVANCES IN KNOWLEDGE

This article describes a novel set of metrics that can be used to describe gaze behaviour when eye tracking readers during interpretation of 3D medical images. These metrics build on those established for 2D eye tracking and are applicable to increasingly common 3D medical image displays.

Scanners and drillers: characterizing expert visual search through volumetric images

Modern imaging methods like computed tomography (CT) generate 3-D volumes of image data. How do radiologists search through such images? Are certain strategies more efficient? Although there is a large literature devoted to understanding search in 2-D, relatively little is known about search in volumetric space. In recent years, with the ever-increasing popularity of volumetric medical imaging, this question has taken on increased importance as we try to understand, and ultimately reduce, errors in diagnostic radiology. In the current study, we asked 24 radiologists to search chest CTs for lung nodules that could indicate lung cancer. To search, radiologists scrolled up and down through a "stack" of 2-D chest CT "slices." At each moment, we tracked eye movements in the 2-D image plane and coregistered eye position with the current slice. We used these data to create a 3-D representation of the eye movements through the image volume. Radiologists tended to follow one of two dominant search strategies: "drilling" and "scanning." Drillers restrict eye movements to a small region of the lung while quickly scrolling through depth. Scanners move more slowly through depth and search an entire level of the lung before moving on to the next level in depth. Driller performance was superior to the scanners on a variety of metrics, including lung nodule detection rate, percentage of the lung covered, and the percentage of search errors where a nodule was never fixated.

Investigation of viewing procedures for interpretation of breast tomosynthesis image volumes: a detection-task study with eye tracking

Objectives

To evaluate the efficiency of different methods of reading breast tomosynthesis (BT) image volumes.

Methods

All viewing procedures consisted of free scroll volume browsing and three were combined with initial cine loops at three different frame rates (9, 14 and 25 fps). The presentation modes consisted of vertically and horizontally orientated BT image volumes. Fifty-five normal BT image volumes in mediolateral oblique view were collected. In these, simulated lesions were inserted, creating four unique image sets, one for each viewing procedure. Four observers interpreted the cases in a free-response task. Time efficiency, visual attention and search were investigated using eye tracking.

Results

Horizontally orientated BT image volumes were read faster than vertically when using free scroll browsing only and when combined with fast cine loop. Cine loops at slow frame rates were ruled out as inefficient.

Conclusions

In general, horizontally oriented BT image volumes were read more efficiently. All viewing procedures except for slow frame rates were promising when assuming equivalent detection performance.

Key Points

• Breast tomosynthesis is increasingly used for breast cancer detection

• There is a benefit in reading breast tomosynthesis image volumes presented horizontally

• Align image content to visual field, especially for dynamic 3D images

• Reading at slow frame rates was considered inefficient

Where do neurologists look when viewing brain CT images? An eye-tracking study involving stroke cases

The aim of this study was to investigate where neurologists look when they view brain computed tomography (CT) images and to evaluate how they deploy their visual attention by comparing their gaze distribution with saliency maps. Brain CT images showing cerebrovascular accidents were presented to 12 neurologists and 12 control subjects. The subjects' ocular fixation positions were recorded using an eye-tracking device (Eyelink 1000). Heat maps were created based on the eye-fixation patterns of each group and compared between the two groups. The heat maps revealed that the areas on which control subjects frequently fixated often coincided with areas identified as outstanding in saliency maps, while the areas on which neurologists frequently fixated often did not. Dwell time in regions of interest (ROI) was likewise compared between the two groups, revealing that, although dwell time on large lesions was not different between the two groups, dwell time in clinically important areas with low salience was longer in neurologists than in controls. Therefore it appears that neurologists intentionally scan clinically important areas when reading brain CT images showing cerebrovascular accidents. Both neurologists and control subjects used the "bottom-up salience" form of visual attention, although the neurologists more effectively used the "top-down instruction" form.

Evaluating interaction techniques for stack mode viewing

Three interaction techniques were evaluated for scrolling stack mode displays of volumetric data. Two used a scroll-wheel mouse: one used only the wheel, while another used a "click and drag" technique for fast scrolling, leaving the wheel for fine adjustments. The third technique used a Shuttle Xpress jog wheel. In a within-subjects design, nine radiologists searched stacked images for simulated hyper-intense regions on brain, knee, and thigh MR studies. Dependent measures were speed, accuracy, navigation path, and user preference. The radiologists considered the task realistic. They had high inter-subject variability in completion times, far larger than the differences between techniques. Most radiologists (eight out of nine) preferred familiar mouse-based techniques. Most participants scanned the data in two passes, first locating anomalies, then scanning for omissions. Participants spent a mean 10.4 s/trial exploring anomalies, with only mild variation between participants. Their rates of forward navigation searching for anomalies varied much more. Interaction technique significantly affected forward navigation rate (scroll wheel 5.4 slices/s, click and drag 9.4, and jog wheel 6.9). It is not clear what constrained the slowest navigators. The fastest navigator used a unique strategy of moving quickly just beyond an anomaly, then backing up. Eight naïve students performed a similar protocol. Their times and variability were similar to the radiologists, but more (three out of eight) students preferred the jog wheel. It may be worthwhile to introduce techniques such as the jog wheel to radiologists during training, and several techniques might be provided on workstations, allowing individuals to choose their preferred method.