Searching for lung nodules. The guidance of visual scanning

Use of eye tracking in medical education

Eye tracking has become increasingly applied in medical education research for studying the cognitive processes that occur during the performance of a task, such as image interpretation and surgical skills development. However, analysis and interpretation of the large amount of data obtained by eye tracking can be confusing. In this article, our intention is to clarify the analysis and interpretation of the data obtained from eye tracking. Understanding the relationship between eye tracking metrics (such as gaze, pupil and blink rate) and cognitive processes (such as visual attention, perception, memory and cognitive workload) is essential. The importance of calibration and how the limitations of eye tracking can be overcome is also highlighted.

The cognitive and perceptual processes that affect observer performance in lung cancer detection: a scoping review

INTRODUCTION

Early detection of malignant pulmonary nodules through screening has been shown to reduce lung cancer-related mortality by 20%. However, perceptual and cognitive factors that affect nodule detection are poorly understood. This review examines the cognitive and visual processes of various observers, with a particular focus on radiologists, during lung nodule detection.

METHODS

Four databases (Medline, Embase, Scopus and PubMed) were searched to extract studies on eye-tracking in pulmonary nodule detection. Studies were included if they used eye-tracking to assess the search and detection of lung nodules in computed tomography or 2D radiographic imaging. Data were charted according to identified themes and synthesised using a thematic narrative approach.

RESULTS

The literature search yielded 25 articles and five themes were discovered: 1 - functional visual field and satisfaction of search, 2 - expert search patterns, 3 - error classification through dwell time, 4 - the impact of the viewing environment and 5 - the effect of prevalence expectation on search. Functional visual field reduced to 2.7° in 3D imaging compared to 5° in 2D radiographs. Although greater visual coverage improved nodule detection, incomplete search was not responsible for missed nodules. Most radiological errors during lung nodule detection were decision-making errors (30%-45%). Dwell times associated with false-positive (FP) decisions informed feedback systems to improve diagnosis. Interruptions did not influence diagnostic performance; however, it increased viewing time by 8% and produced a 23.1% search continuation accuracy. Comparative scanning was found to increase the detection of low contrast nodules. Prevalence expectation did not directly affect diagnostic accuracy; however, decision-making time increased by 2.32 seconds with high prevalence expectations.

CONCLUSION

Visual and cognitive factors influence pulmonary nodule detection. Insights gained from eye-tracking can inform advancements in lung screening. Further exploration of eye-tracking in lung screening, particularly with low-dose computed tomography (LDCT), will benefit the future of lung cancer screening.

Can a Machine Learn from Radiologists' Visual Search Behaviour and Their Interpretation of Mammograms-a Deep-Learning Study

Visual search behaviour and the interpretation of mammograms have been studied for errors in breast cancer detection. We aim to ascertain whether machine-learning models can learn about radiologists' attentional level and the interpretation of mammograms. We seek to determine whether these models are practical and feasible for use in training and teaching programmes. Eight radiologists of varying experience levels in reading mammograms reviewed 120 two-view digital mammography cases (59 cancers). Their search behaviour and decisions were captured using a head-mounted eye-tracking device and software allowing them to record their decisions. This information from radiologists was used to build an ensembled machine-learning model using top-down hierarchical deep convolution neural network. Separately, a model to determine type of missed cancer (search, perception or decision-making) was also built. Analysis and comparison of variants of these models using different convolution networks with and without transfer learning were also performed. Our ensembled deep-learning network architecture can be trained to learn about radiologists' attentional level and decisions. High accuracy (95%, p value ≅ 0 [better than dumb/random model]) and high agreement between true and predicted values (kappa = 0.83) in such modelling can be achieved. Transfer learning techniques improve by < 10% with the performance of this model. We also show that spatial convolution neural networks are insufficient in determining the type of missed cancers. Ensembled hierarchical deep convolution machine-learning models are plausible in modelling radiologists' attentional level and their interpretation of mammograms. However, deep convolution networks fail to characterise the type of false-negative decisions.

A review of factors influencing radiologists' visual search behaviour

This narrative literature review aims to identify the various factors that have significant impact on radiologists' visual search patterns. Identifying the factors that influences readers' visual search behaviour helps to understand their perception and interpretation of medical images, which in turn could lead to the development and implementation of effective strategies that could aid in improving the ability to detect abnormalities. Databases including PubMed, MedLine, Web of Science and ScienceDirect were searched using terms 'visual search', 'eye-tracking', 'radiology OR radiography', 'mammogram OR mammography' published since the early 1960s until June 30, 2016. Some of the factors that have been identified to significantly influence radiologists' visual search patterns were (i) readers' expertise, (ii) Satisfaction of Search, (iii) readers' visual fatigue, (iv) readers' confidence in reporting abnormalities, (v) training received and (vi) readers' prior knowledge. Readers' level of expertise was the factor that has been identified to have the most significant impact on their visual search pattern. Eye-tracking studies have shown the differences in visual search patterns of readers with different levels of experience and not so surprisingly, more experienced readers have shown effective visual search strategies. Readers' expertise has also been found to have significant impact in their confidence in reporting abnormalities and their ability to discriminate lesions from background structures in medical images.

Modeling visual search behavior of breast radiologists using a deep convolution neural network

Visual search, the process of detecting and identifying objects using eye movements (saccades) and foveal vision, has been studied for identification of root causes of errors in the interpretation of mammograms. The aim of this study is to model visual search behavior of radiologists and their interpretation of mammograms using deep machine learning approaches. Our model is based on a deep convolutional neural network, a biologically inspired multilayer perceptron that simulates the visual cortex and is reinforced with transfer learning techniques. Eye-tracking data were obtained from eight radiologists (of varying experience levels in reading mammograms) reviewing 120 two-view digital mammography cases (59 cancers), and it has been used to train the model, which was pretrained with the ImageNet dataset for transfer learning. Areas of the mammogram that received direct (foveally fixated), indirect (peripherally fixated), or no (never fixated) visual attention were extracted from radiologists' visual search maps (obtained by a head mounted eye-tracking device). These areas along with the radiologists' assessment (including confidence in the assessment) of the presence of suspected malignancy were used to model: (1) radiologists' decision, (2) radiologists' confidence in such decisions, and (3) the attentional level (i.e., foveal, peripheral, or none) in an area of the mammogram. Our results indicate high accuracy and low misclassification in modeling such behaviors.

Eye-tracking technology in medical education: A systematic review

BACKGROUND

Eye-tracking technology is an established research tool within allied industries such as advertising, psychology and aerospace. This review aims to consolidate literature describing the evidence for use of eye-tracking as an adjunct to traditional teaching methods in medical education.

METHODS

A systematic literature review was conducted in line with STORIES guidelines. A search of EMBASE, OVID MEDLINE, PsycINFO, TRIP database, and Science Direct was conducted until January 2017. Studies describing the use of eye-tracking in the training, assessment, and feedback of clinicians were included in the review.

RESULTS

Thirty-three studies were included in the final qualitative synthesis. Three studies were based on the use of gaze training, three studies on the changes in gaze behavior during the learning curve, 17 studies on clinical assessment and six studies focused on the use of eye-tracking methodology as a feedback tool. The studies demonstrated feasibility and validity in the use of eye-tracking as a training and assessment method.

CONCLUSIONS

Overall, eye-tracking methodology has contributed significantly to the training, assessment, and feedback practices used in the clinical setting. The technology provides reliable quantitative data, which can be interpreted to give an indication of clinical skill, provide training solutions and aid in feedback and reflection. This review provides a detailed summary of evidence relating to eye-tracking methodology and its uses as a training method, changes in visual gaze behavior during the learning curve, eye-tracking methodology for proficiency assessment and its uses as a feedback tool.

The Holistic Processing Account of Visual Expertise in Medical Image Perception: A Review

In the field of medical image perception, the holistic processing perspective contends that experts can rapidly extract global information about the image, which can be used to guide their subsequent search of the image (Swensson, 1980; Nodine and Kundel, 1987; Kundel et al., 2007). In this review, we discuss the empirical evidence supporting three different predictions that can be derived from the holistic processing perspective: Expertise in medical image perception is domain-specific, experts use parafoveal and/or peripheral vision to process large regions of the image in parallel, and experts benefit from a rapid initial glimpse of an image. In addition, we discuss a pivotal recent study (Litchfield and Donovan, 2016) that seems to contradict the assumption that experts benefit from a rapid initial glimpse of the image. To reconcile this finding with the existing literature, we suggest that global processing may serve multiple functions that extend beyond the initial glimpse of the image. Finally, we discuss future research directions, and we highlight the connections between the holistic processing account and similar theoretical perspectives and findings from other domains of visual expertise.

Fixated and Not Fixated Regions of Mammograms: A Higher-Order Statistical Analysis of Visual Search Behavior

RATIONALE AND OBJECTIVES

Visual search is an inhomogeneous yet efficient sampling process accomplished by the saccades and the central (foveal) vision. Areas that attract the central vision have been studied for errors in interpretation of medical imaging. In this study, we extend existing visual search studies to understand what characterizes areas that receive direct visual attention and elicit a mark by the radiologist (True and False Positive decisions) from those that elicit a mark but were captured by the peripheral vision. We also investigate if there are any differences between these areas and those that are never fixated by radiologists.

MATERIALS AND METHODS

Eight radiologists participated in this fully crossed multi-reader multi-case visual search study of digital mammography (DM) involving 120 two-view cases (59 cancers). From these DM images, 3 types of areas, namely Fixated Clusters (FC), Marked Peripherally Fixated Clusters (MPFC) and Never Fixated Clusters (NFC), were extracted and analysed using statistical information theory (in the form of third and fourth-order cumulants and polyspectrum [specifically bispectrum and trispectrum]) in addition to traditional second-order statistics (in the form of power spectrum) and other nonspectral features to characterize these types of areas.

RESULTS

Our results suggest that energy profiles of FC, MPFC, and NFC areas are distinct. We found evidence that energy profiles and dwell time of these areas influence radiologists' decisions (and confidence in such decisions). We also noted that foveated areas are selected on the basis of being most informative.

CONCLUSION

We show that properties of these areas influence radiologists' decisions and their confidence in the decisions made.

Comparative study of computational visual attention models on two-dimensional medical images

Computational modeling of visual attention is an active area of research. These models have been successfully employed in applications such as robotics. However, most computational models of visual attention are developed in the context of natural scenes, and their role with medical images is not well investigated. As radiologists interpret a large number of clinical images in a limited time, an efficient strategy to deploy their visual attention is necessary. Visual saliency maps, highlighting image regions that differ dramatically from their surroundings, are expected to be predictive of where radiologists fixate their gaze. We compared 16 state-of-art saliency models over three medical imaging modalities. The estimated saliency maps were evaluated against radiologists' eye movements. The results show that the models achieved competitive accuracy using three metrics, but the rank order of the models varied significantly across the three modalities. Moreover, the model ranks on the medical images were all considerably different from the model ranks on the benchmark MIT300 dataset of natural images. Thus, modality-specific tuning of saliency models is necessary to make them valuable for applications in fields such as medical image compression and radiology education.

Temporal Characteristics of Radiologists' and Novices' Lesion Detection in Viewing Medical Images Presented Rapidly and Sequentially

Although viewing multiple stacks of medical images presented on a display is a relatively new but useful medical task, little is known about this task. Particularly, it is unclear how radiologists search for lesions in this type of image reading. When viewing cluttered and dynamic displays, continuous motion itself does not capture attention. Thus, it is effective for the target detection that observers' attention is captured by the onset signal of a suddenly appearing target among the continuously moving distractors (i.e., a passive viewing strategy). This can be applied to stack viewing tasks, because lesions often show up as transient signals in medical images which are sequentially presented simulating a dynamic and smoothly transforming image progression of organs. However, it is unclear whether observers can detect a target when the target appears at the beginning of a sequential presentation where the global apparent motion onset signal (i.e., signal of the initiation of the apparent motion by sequential presentation) occurs. We investigated the ability of radiologists to detect lesions during such tasks by comparing the performances of radiologists and novices. Results show that overall performance of radiologists is better than novices. Furthermore, the temporal locations of lesions in CT image sequences, i.e., when a lesion appears in an image sequence, does not affect the performance of radiologists, whereas it does affect the performance of novices. Results indicate that novices have greater difficulty in detecting a lesion appearing early than late in the image sequence. We suggest that radiologists have other mechanisms to detect lesions in medical images with little attention which novices do not have. This ability is critically important when viewing rapid sequential presentations of multiple CT images, such as stack viewing tasks.

Eye-tracking of nodule detection in lung CT volumetric data

PURPOSE

Signal detection on 3D medical images depends on many factors, such as foveal and peripheral vision, the type of signal, and background complexity, and the speed at which the frames are displayed. In this paper, the authors focus on the speed with which radiologists and naïve observers search through medical images. Prior to the study, the authors asked the radiologists to estimate the speed at which they scrolled through CT sets. They gave a subjective estimate of 5 frames per second (fps). The aim of this paper is to measure and analyze the speed with which humans scroll through image stacks, showing a method to visually display the behavior of observers as the search is made as well as measuring the accuracy of the decisions. This information will be useful in the development of model observers, mathematical algorithms that can be used to evaluate diagnostic imaging systems.

METHODS

The authors performed a series of 3D 4-alternative forced-choice lung nodule detection tasks on volumetric stacks of chest CT images iteratively reconstructed in lung algorithm. The strategy used by three radiologists and three naïve observers was assessed using an eye-tracker in order to establish where their gaze was fixed during the experiment and to verify that when a decision was made, a correct answer was not due only to chance. In a first set of experiments, the observers were restricted to read the images at three fixed speeds of image scrolling and were allowed to see each alternative once. In the second set of experiments, the subjects were allowed to scroll through the image stacks at will with no time or gaze limits. In both static-speed and free-scrolling conditions, the four image stacks were displayed simultaneously. All trials were shown at two different image contrasts.

RESULTS

The authors were able to determine a histogram of scrolling speeds in frames per second. The scrolling speed of the naïve observers and the radiologists at the moment the signal was detected was measured at 25-30 fps. For the task chosen, the performance of the observers was not affected by the contrast or experience of the observer. However, the naïve observers exhibited a different pattern of scrolling than the radiologists, which included a tendency toward higher number of direction changes and number of slices viewed.

CONCLUSIONS

The authors have determined a distribution of speeds for volumetric detection tasks. The speed at detection was higher than that subjectively estimated by the radiologists before the experiment. The speed information that was measured will be useful in the development of 3D model observers, especially anthropomorphic model observers which try to mimic human behavior.

Computational assessment of visual search strategies in volumetric medical images

When searching through volumetric images [e.g., computed tomography (CT)], radiologists appear to use two different search strategies: "drilling" (restrict eye movements to a small region of the image while quickly scrolling through slices), or "scanning" (search over large areas at a given depth before moving on to the next slice). To computationally identify the type of image information that is used in these two strategies, 23 naïve observers were instructed with either "drilling" or "scanning" when searching for target T's in 20 volumes of faux lung CTs. We computed saliency maps using both classical two-dimensional (2-D) saliency, and a three-dimensional (3-D) dynamic saliency that captures the characteristics of scrolling through slices. Comparing observers' gaze distributions with the saliency maps showed that search strategy alters the type of saliency that attracts fixations. Drillers' fixations aligned better with dynamic saliency and scanners with 2-D saliency. The computed saliency was greater for detected targets than for missed targets. Similar results were observed in data from 19 radiologists who searched five stacks of clinical chest CTs for lung nodules. Dynamic saliency may be superior to the 2-D saliency for detecting targets embedded in volumetric images, and thus "drilling" may be more efficient than "scanning."

Lung nodule and cancer detection in computed tomography screening

Fundamental to the diagnosis of lung cancer in computed tomography (CT) scans is the detection and interpretation of lung nodules. As the capabilities of CT scanners have advanced, higher levels of spatial resolution reveal tinier lung abnormalities. Not all detected lung nodules should be reported; however, radiologists strive to detect all nodules that might have relevance to cancer diagnosis. Although medium to large lung nodules are detected consistently, interreader agreement and reader sensitivity for lung nodule detection diminish substantially as the nodule size falls below 8 to 10 mm. The difficulty in establishing an absolute reference standard presents a challenge to the reliability of studies performed to evaluate lung nodule detection. In the interest of improving detection performance, investigators are using eye tracking to analyze the effectiveness with which radiologists search CT scans relative to their ability to recognize nodules within their search path in order to determine whether strategies might exist to improve performance across readers. Beyond the viewing of transverse CT reconstructions, image processing techniques such as thin-slab maximum-intensity projections are used to substantially improve reader performance. Finally, the development of computer-aided detection has continued to evolve with the expectation that one day it will serve routinely as a tireless partner to the radiologist to enhance detection performance without significant prolongation of the interpretive process. This review provides an introduction to the current understanding of these varied issues as we enter the era of widespread lung cancer screening.

Review of prospects and challenges of eye tracking in volumetric imaging

While eye tracking research in conventional radiography has flourished over the past decades, the number of eye tracking studies that looked at multislice images lags behind. A possible reason for the lack of studies in this area might be that the eye tracking methodology used in the context of conventional radiography cannot be applied one-on-one to volumetric imaging material. Challenges associated with eye tracking in volumetric imaging are particularly associated with the selection of stimulus material, the detection of events in the eye tracking data, the calculation of meaningful eye tracking parameters, and the reporting of abnormalities. However, all of these challenges can be addressed in the design of the experiment. If this is done, eye tracking studies using volumetric imaging material offer almost unlimited opportunity for perception research and are highly relevant as the number of volumetric images that are acquired and interpreted is rising.

Characterizing search, recognition, and decision in the detection of lung nodules on CT scans: elucidation with eye tracking

PURPOSE

To determine the effectiveness of radiologists' search, recognition, and acceptance of lung nodules on computed tomographic (CT) images by using eye tracking.

MATERIALS AND METHODS

This study was performed with a protocol approved by the institutional review board. All study subjects provided informed consent, and all private health information was protected in accordance with HIPAA. A remote eye tracker was used to record time-varying gaze paths while 13 radiologists interpreted 40 lung CT images with an average of 3.9 synthetic nodules (5-mm diameter) embedded randomly in the lung parenchyma. The radiologists' gaze volumes ( GV gaze volume s) were defined as the portion of the lung parenchyma within 50 pixels (approximately 3 cm) of all gaze points. The fraction of the total lung volume encompassed within the GV gaze volume s, the fraction of lung nodules encompassed within each GV gaze volume (search effectiveness), the fraction of lung nodules within the GV gaze volume detected by the reader (recognition-acceptance effectiveness), and overall sensitivity of lung nodule detection were measured.

RESULTS

Detected nodules were within 50 pixels of the nearest gaze point for 990 of 992 correct detections. On average, radiologists searched 26.7% of the lung parenchyma in 3 minutes and 16 seconds and encompassed between 86 and 143 of 157 nodules within their GV gaze volume s. Once encompassed within their GV gaze volume , the average sensitivity of nodule recognition and acceptance ranged from 47 of 100 nodules to 103 of 124 nodules (sensitivity, 0.47-0.82). Overall sensitivity ranged from 47 to 114 of 157 nodules (sensitivity, 0.30-0.73) and showed moderate correlation (r = 0.62, P = .02) with the fraction of lung volume searched.

CONCLUSION

Relationships between reader search, recognition and acceptance, and overall lung nodule detection rate can be studied with eye tracking. Radiologists appear to actively search less than half of the lung parenchyma, with substantial interreader variation in volume searched, fraction of nodules included within the search volume, sensitivity for nodules within the search volume, and overall detection rate.

Scanners and drillers: characterizing expert visual search through volumetric images

Modern imaging methods like computed tomography (CT) generate 3-D volumes of image data. How do radiologists search through such images? Are certain strategies more efficient? Although there is a large literature devoted to understanding search in 2-D, relatively little is known about search in volumetric space. In recent years, with the ever-increasing popularity of volumetric medical imaging, this question has taken on increased importance as we try to understand, and ultimately reduce, errors in diagnostic radiology. In the current study, we asked 24 radiologists to search chest CTs for lung nodules that could indicate lung cancer. To search, radiologists scrolled up and down through a "stack" of 2-D chest CT "slices." At each moment, we tracked eye movements in the 2-D image plane and coregistered eye position with the current slice. We used these data to create a 3-D representation of the eye movements through the image volume. Radiologists tended to follow one of two dominant search strategies: "drilling" and "scanning." Drillers restrict eye movements to a small region of the lung while quickly scrolling through depth. Scanners move more slowly through depth and search an entire level of the lung before moving on to the next level in depth. Driller performance was superior to the scanners on a variety of metrics, including lung nodule detection rate, percentage of the lung covered, and the percentage of search errors where a nodule was never fixated.

Informatics in radiology: what can you see in a single glance and how might this guide visual search in medical images?

Diagnostic accuracy for radiologists is above that expected by chance when they are exposed to a chest radiograph for only one-fifth of a second, a period too brief for more than a single voluntary eye movement. How do radiologists glean information from a first glance at an image? It is thought that this expert impression of the gestalt of an image is related to the everyday, immediate visual understanding of the gist of a scene. Several high-speed mechanisms guide our search of complex images. Guidance by basic features (such as color) requires no learning, whereas guidance by complex scene properties is learned. It is probable that both hardwired guidance by basic features and learned guidance by scene structure become part of radiologists' expertise. Search in scenes may be best explained by a two-pathway model: Object recognition is performed via a selective pathway in which candidate targets must be individually selected for recognition. A second, nonselective pathway extracts information from global or statistical information without selecting specific objects. An appreciation of the role of nonselective processing may be particularly useful for understanding what separates novice from expert radiologists and could help establish new methods of physician training based on medical image perception.

Investigation of viewing procedures for interpretation of breast tomosynthesis image volumes: a detection-task study with eye tracking

Objectives

To evaluate the efficiency of different methods of reading breast tomosynthesis (BT) image volumes.

Methods

All viewing procedures consisted of free scroll volume browsing and three were combined with initial cine loops at three different frame rates (9, 14 and 25 fps). The presentation modes consisted of vertically and horizontally orientated BT image volumes. Fifty-five normal BT image volumes in mediolateral oblique view were collected. In these, simulated lesions were inserted, creating four unique image sets, one for each viewing procedure. Four observers interpreted the cases in a free-response task. Time efficiency, visual attention and search were investigated using eye tracking.

Results

Horizontally orientated BT image volumes were read faster than vertically when using free scroll browsing only and when combined with fast cine loop. Cine loops at slow frame rates were ruled out as inefficient.

Conclusions

In general, horizontally oriented BT image volumes were read more efficiently. All viewing procedures except for slow frame rates were promising when assuming equivalent detection performance.

Key Points

• Breast tomosynthesis is increasingly used for breast cancer detection

• There is a benefit in reading breast tomosynthesis image volumes presented horizontally

• Align image content to visual field, especially for dynamic 3D images

• Reading at slow frame rates was considered inefficient

Characterization of radiologists' search strategies for lung nodule detection: slice-based versus volumetric displays

The goal of this study was to assess whether radiologists' search paths for lung nodule detection in chest computed tomography (CT) between different rendering and display schemes have reliable properties that can be exploited as an indicator of ergonomic efficiency for the purpose of comparing different display paradigms. Eight radiologists retrospectively viewed 30 lung cancer screening CT exams, containing a total of 91 nodules, in each of three display modes [i.e., slice-by-slice, orthogonal maximum intensity projection (MIP) and stereoscopic] for the purpose of detecting and classifying lung nodules. Radiologists' search patterns in the axial direction were recorded and analyzed along with the location, size, and shape for each detected feature, and the likelihood that the feature is an actual nodule. Nodule detection performance was analyzed by employing free-response receiver operating characteristic methods. Search paths were clearly different between slice-by-slice displays and volumetric displays but, aside from training and novelty effects, not between MIP and stereographic displays. Novelty and training effects were associated with the stereographic display mode, as evidenced by differences between the beginning and end of the study. The stereo display provided higher detection and classification performance with less interpretation time compared to other display modes tested in the study; however, the differences were not statistically significant. Our preliminary results indicate a potential role for the use of radiologists' search paths in evaluating the relative ergonomic efficiencies of different display paradigms, but systematic training and practice is necessary to eliminate training curve and novelty effects before search strategies can be meaningfully compared.

Holistic component of image perception in mammogram interpretation: gaze-tracking study

PURPOSE

To test the hypothesis that rapid and accurate performance of the proficient observer in mammogram interpretation involves a shift in the mechanism of image perception from a relatively slow search-to-find mode to a relatively fast holistic mode.

MATERIALS AND METHODS

This HIPAA-compliant study had institutional review board approval, and participant informed consent was obtained; patient informed consent was not required. The eye positions of three full-time mammographers, one attending radiologist, two mammography fellows, and three radiology residents were recorded during the interpretation of 20 normal and 20 subtly abnormal mammograms. The search time required to first locate a cancer, as well as the initial eye scan path, was determined and compared with diagnostic performance as measured with receiver operating characteristic (ROC) analysis.

RESULTS

The median time for all observers to fixate a cancer, regardless of the decision outcome, was 1.13 seconds, with a range of 0.68 second to 3.06 seconds. Even though most of the lesions were fixated, recognition of them as cancerous ranged from 85% (17 of 20) to 10% (two of 20), with corresponding areas under the ROC curve of 0.87-0.40. The ROC index of detectability, d(a), was linearly related to the time to first fixate a cancer with a correlation (r(2)) of 0.81.

CONCLUSION

The rapid initial fixation of a true abnormality is evidence for a global perceptual process capable of analyzing the visual input of the entire retinal image and pinpointing the spatial location of an abnormality. It appears to be more highly developed in the most proficient observers, replacing the less efficient initial search-to-find strategies.

Thin-section CT of the lungs: Eye-tracking analysis of the visual approach to reading tiled and stacked display formats

OBJECTIVE

To use eye-tracking analysis to identify the differences in approach to and efficiency of reading thin-section CT of the lungs presented tiled and stacked soft-copy displays.

MATERIALS AND METHODS

Four chest radiologists read 16 thin-section CT examinations displayed in either a tiled (four images at once) or stacked (full screen cine) format. Eye-movements were recorded and analysed in terms of movement type; saccade distance (classified by the calculated range of useful peripheral vision), number of fixations, duration and direction of gaze-comparison of the areas of the images viewed.

RESULTS

Cases presented in stacked format were read quicker than when presented in tiled format with a greater fixation frequency (5 fixations versus 4.5 fixations points per 100 data points; p<0.001) and a greater proportion of short saccades (97% versus 94%; p<0.005). The consistency with which the observers viewed equivalent areas of the scan images in different cases was greater when viewing in stacked format (mean kappa 0.45 versus 0.36; p<0.05) suggesting a more systematic approach to reading.

CONCLUSION

Eye-tracking data demonstrates why thin-section CT examinations of the lungs are read more efficiently when displayed in a stack as opposed to a tiled format.

Time-dependent observer errors in pulmonary nodule detection

The work was carried out to investigate differences in visual search characteristics between groups of observers with different levels of experience in the task of pulmonary nodule detection in chest radiology and we report here on these differences in respect of time related decisions. Volunteer observers were divided into three groups depending on their level of expertise. There were eight radiologists, eight radiographers and eight novices. Their task was to detect pulmonary nodules in a test bank of 120 digitized posteroanterior (PA) chest radiographs. Five of the eight radiographers were tested twice: once before and once after a 6-month training programme in interpretation of the adult chest radiograph. During each test session the observers' eye movements were tracked. Data on the observers' decisions through Alternate Free Response Operating Characteristic (AFROC) methodology were correlated to their eye-movement and fixation patterns. True negative decisions from all observers were associated with shorter fixation times than false negative decisions. No correct negative decisions were made after fixations exceeding 3 s.

Visual Skills in Airport-Security Screening

An experiment examined visual performance in a simulated luggage-screening task. Observers participated in five sessions of a task requiring them to search for knives hidden in x-ray images of cluttered bags. Sensitivity and response times improved reliably as a result of practice. Eye movement data revealed that sensitivity increases were produced entirely by changes in observers' ability to recognize target objects, and not by changes in the effectiveness of visual scanning. Moreover, recognition skills were in part stimulus-specific, such that performance was degraded by the introduction of unfamiliar target objects. Implications for screener training are discussed.

Visualising scanning patterns of pathologists in the grading of cervical intraepithelial neoplasia

AIM

To investigate how effectively eye tracking devices can visualise the scanning patterns of pathologists, for application in studies on diagnostic decision making.

METHODS

EyeCatcher, an eye tracking device, was used to visualise and compare the scanning patterns of five pathologists while they graded two projections of cervical intraepithelial neoplasia. Density cloud images were created from the scanning patterns. A questionnaire and interview provided information on the following steps in the diagnostic process.

RESULTS

EyeCatcher successfully registered the scanning patterns of the pathologists. A "scanning style" and a "selective style" of visual search were distinguished. The scanning patterns, in addition to the interpretation and combination of the information ultimately leading to a diagnosis, varied between the various observers, resulting in a broad range of final diagnoses.

CONCLUSIONS

Eye gaze tracking devices provide an excellent basis for further discussion on the interpretation and grading criteria of lesions. As such, they may play an important role in studies on diagnostic decision making in pathology and in the development of training and quality control programmes for pathologists.

Searching for nodules: what features attract attention and influence detection?

RATIONALE AND OBJECTIVES

The goal of the study was to determine whether there are certain physical features of pulmonary nodules that attract visual attention and contribute to increased recognition and detection by observers.

MATERIALS AND METHODS

A series of posteroanterior chest images with solitary pulmonary nodules were searched by six radiologists as their eye-position was recorded. The signal-to-noise ratio, size, conspicuity, location, and calcification status were measured for each nodule. Dwell parameters were correlated with nodule features and related to detection rates.

RESULTS

Only nodule size (F = 5.08, P = .0254) and conspicuity (F = 4.625, P = .0329) influenced total dwell time on nodules, with larger, more conspicuous nodules receiving less visual attention than smaller, less conspicuous nodules. All nodule features examined influenced overall detection performance (P < .05) even though most did not influence visual search and attention.

CONCLUSION

Individual nodule features do not attract attention as measured by "first hit" fixation data, but certain features do tend to hold attention once the nodule has been fixated. The combination of all features influences whether or not it is detected.

Segmentation of objects from backgrounds in visual search tasks

In most visual search experiments in the laboratory, objects are presented on an isolated, blank background. In most real world search tasks, however, the background is continuous and can be complex. In six experiments, we examine the ability of the visual system to separate search items from a background. The results support a view in which objects are separated from backgrounds in a single, preattentive step. This is followed by a limited-capacity search process that selects objects that might be targets for further identification. Identity information regarding the object's status (target or distractor) then accumulates through a limited capacity parallel process. The main effect of background complexity is to slow the accumulation of information in this later recognition stage. It may be that recognition is slowed because background noise causes the preattentive segmentation stage to deliver less effectively segmented objects to later stages. Only when backgrounds become nearly identical to the search objects does the background have the effect of slowing item-by-item selection.

Machine classification of dental images with visual search

RATIONALE AND OBJECTIVES

The authors performed this study to assess the performance of a computer-based classification system that uses gaze locations of observers to define the subspace for machine learning.

MATERIALS AND METHODS

Thirty-two dental radiographs were classified by an expert viewer into four categories of disease of the periapical region: no disease (normal tooth), mild disease (widened periodontal ligament space), moderate disease (destruction of the lamina dura), and severe disease (resorption of bone in the periapical area). There were eight images in each category. Six observers independently viewed the images while their eye gaze position was recorded. They then classified the images into one of the four categories. A sample of image space was used as input to a machine learning routine to develop a machine classifier. Sample space was determined with three techniques: visual gaze, random selection, and constrained random selection. K analyses were used to compare classification accuracies with the three sampling techniques.

RESULTS

With use of the expert classification as a standard of reference, observers classified images with 57% accuracy, and the machine classified images with 84% accuracy by using the same gaze-selected features and image space. Results of kappa analyses revealed mean values of 0.78 for gaze-selected sampling, 0.69 for random sampling, 0.68 for constrained random selection, and 0.44 for observers. The use of sample space selected with the visual gaze technique was superior to that selected with both random-selection techniques and by the observers.

CONCLUSION

Machine classification of dental images improves the accuracy of individual observers using gaze-selected image space.

Unobtrusively tracking eye gaze direction and pupil diameter of mammographers

RATIONALE AND OBJECTIVES

The visual process that radiologists use for diagnosis is incompletely understood. This study developed techniques to unobtrusively track direction and pupil diameter of radiologists reading a wide variety of films. We evaluated the eye gaze patterns of mammographic experts to gain knowledge that might improve the rate of early detection of breast cancer.

METHODS

A video camera with a near-infrared light filter is pointed at the mammographic expert who is reading mammograms. The video images are analyzed in real time on a personal computer to detect eye gaze direction and pupil diameter. Two separate trials were used: 1) to demonstrate the system's speed and ability to work with mammograms (a brief test with one mammographer was used) and 2) four mammographic experts evaluated 14 mammograms.

RESULTS

In the first trial, the system successfully tracked the eye gaze of a mammographer who quickly recognized the patient case, with the pupil diameter briefly increasing 40%, and then the gaze direction dwelling in an area of microcalcifications. In the second trial, 66% of the false-positive results for films with masses were associated with long eye gaze dwells, whereas 33% of the prolonged dwells for films with microcalcifications were associated with true-positive diagnoses.

CONCLUSIONS

This near-infrared light system successfully tracked the eye gaze direction and pupil diameter of mammographic experts evaluating films. The association of long eye gaze dwells with diagnostic accuracy varied with the type of object being viewed. In films with masses, false-positive diagnoses were associated with long dwells. In films with microcalcifications, true-positive diagnoses were associated with long dwells.