Using Cognitive Psychology Research to Inform Professional Visual Search Operations

An evaluation of image enhancements in three-dimensional computed tomography baggage screening

New computed tomography and image enhancement technologies are increasingly used in cabin baggage screening at airports. The current work sought to establish whether these technological advancements had pushed beyond human psychological capability by examining the impact of different image enhancements on threat detection. Sixty-one naïve adults participated in an online study and received targetless search training before being tested in a simulated baggage screening task - in addition to three-dimensional images this task incorporated two-dimensional slice views, material stripping and zoom functionality. The two-dimensional slice view improved threat detection sensitivity and lowered response criterion relative to a standard three-dimensional image, as screeners found dangerous items they had initially missed, likely due to improved ability to resolve superposition or identify edges. In contrast, material stripping and zoom did not impact sensitivity, criterion or screeners' confidence in their responses, suggesting that they contributed no additional informational value.

Effects of machine learning errors on human decision-making: manipulations of model accuracy, error types, and error importance

This study addressed the cognitive impacts of providing correct and incorrect machine learning (ML) outputs in support of an object detection task. The study consisted of five experiments that manipulated the accuracy and importance of mock ML outputs. In each of the experiments, participants were given the T and L task with T-shaped targets and L-shaped distractors. They were tasked with categorizing each image as target present or target absent. In Experiment 1, they performed this task without the aid of ML outputs. In Experiments 2-5, they were shown images with bounding boxes, representing the output of an ML model. The outputs could be correct (hits and correct rejections), or they could be erroneous (false alarms and misses). Experiment 2 manipulated the overall accuracy of these mock ML outputs. Experiment 3 manipulated the proportion of different types of errors. Experiments 4 and 5 manipulated the importance of specific types of stimuli or model errors, as well as the framing of the task in terms of human or model performance. These experiments showed that model misses were consistently harder for participants to detect than model false alarms. In general, as the model's performance increased, human performance increased as well, but in many cases the participants were more likely to overlook model errors when the model had high accuracy overall. Warning participants to be on the lookout for specific types of model errors had very little impact on their performance. Overall, our results emphasize the importance of considering human cognition when determining what level of model performance and types of model errors are acceptable for a given task.

Graded prioritisation of targets in search: reward diminishes the low prevalence effect

In many real-life contexts, observers are required to search for targets that are rarely present (e.g. tumours in X-rays; dangerous items in airport security screenings). Despite the rarity of these items, they are of enormous importance for the health and safety of the public, yet they are easily missed during visual search. This is referred to as the prevalence effect. In the current series of experiments, we investigate whether unequal reward can modulate the prevalence effect, in a multiple target search task. Having first established the impact of prevalence (Experiment 1) and reward (Experiment 2) on how efficiently participants can find one of several targets in the current paradigm, we then combined the two forms of priority to investigate their interaction. An unequal reward distribution (where lower prevalence items are more rewarded; Experiment 3) was found to diminish the effect of prevalence, compared to an equal reward distribution (Experiment 4) as indicated by faster response times and fewer misses. These findings suggest that when combined with an unequal reward distribution, the low prevalence effect can be diminished.

Delayed onsets are not necessary for generating distractor quitting thresholds effects in visual search

Salient distractors lower quitting thresholds in visual search. That is, when searching for the presence of a target among filler items, a large heterogeneously coloured distractor presented at a delayed onset produces quick target-absent judgements and increased target-present errors. The aim of the current study was to explore if the timing of the salient distractor modulates this Quitting Threshold Effect (QTE). In Experiment 1, participants completed a target detection search task in the presence or absence of a salient singleton distractor that either appeared simultaneously with other search items or appeared at a delayed onset (i.e., 100 ms or 250 ms after other array items appeared). In Experiment 2, a similar method was used, except that the salient singleton distractor appeared simultaneously, 100 ms before, or 100 ms after the other array items. Across both experiments, we observed robust distractor QTEs. Regardless of their onset, salient distractors decreased target-absent search speeds and increased target-present error rates. In all, the present findings suggest that delayed onsets are not required for lowered quitting thresholds in visual search.

Time on task and task load in visual inspection: A four-month field study with X-ray baggage screeners

Previous studies suggest that performance in visual inspection and typical vigilance tasks depend on time on task and task load. European regulation mandates that security officers (screeners) take a break or change tasks after 20 min of X-ray baggage screening. However, longer screening durations could reduce staffing challenges. We investigated the effects of time on task and task load on visual inspection performance in a four-month field study with screeners. At an international airport, 22 screeners inspected X-ray images of cabin baggage for up to 60 min, while a control group (N = 19) screened for 20 min. Hit rate remained stable for low and average task loads. However, when the task load was high, the screeners compensated by speeding up X-ray image inspection at the expense of the hit rate over time on task. Our results support the dynamic-allocation resource theory. Moreover, extending the permitted screening duration to 30 or 40 min should be considered.

Active visual search in naturalistic environments reflects individual differences in classic visual search performance

Visual search is a ubiquitous activity in real-world environments. Yet, traditionally, visual search is investigated in tightly controlled paradigms, where head-restricted participants locate a minimalistic target in a cluttered array that is presented on a computer screen. Do traditional visual search tasks predict performance in naturalistic settings, where participants actively explore complex, real-world scenes? Here, we leverage advances in virtual reality technology to test the degree to which classic and naturalistic search are limited by a common factor, set size, and the degree to which individual differences in classic search behavior predict naturalistic search behavior in a large sample of individuals (N = 75). In a naturalistic search task, participants looked for an object within their environment via a combination of head-turns and eye-movements using a head-mounted display. Then, in a classic search task, participants searched for a target within a simple array of colored letters using only eye-movements. In each task, we found that participants' search performance was impacted by increases in set size-the number of items in the visual display. Critically, we observed that participants' efficiency in classic search tasks-the degree to which set size slowed performance-indeed predicted efficiency in real-world scenes. These results demonstrate that classic, computer-based visual search tasks are excellent models of active, real-world search behavior.

The use of cognitive psychology-based human-computer interaction tax system in ceramic industry tax collection and management and economic development of Jingdezhen city

This work aims to solve the complex problems of non-linearity, instability, and multiple economic factors in the tax forecast of the ceramic industry to ensure the sustainable development of the ceramic industry. The key influential indicators of the tax forecast are obtained by analyzing the principal components affecting the tax index. In addition, a human-computer interaction (HCI) system is established based on cognitive psychology theory to improve the user-friendliness of tax analysis. At the same time, the tax data of the ceramic industry in Jingdezhen City, Jiangxi Province in different years are used for the empirical analysis of the tax prediction of different prediction models, including particle swarm optimization (PSO) algorithm, and fusion algorithm (FA), and support vector machine (SVM). This work comprehensively analyzes the influence of the optimized tax supervision mode on the economic development of the ceramic industry and provides ideas for the development of the ceramic industry in Jingdezhen. The research results demonstrate that the main indicator affecting tax revenue is the added value of the primary and secondary industries. The optimized SVM based on grid search method can provide a comprehensive data base for tax forecasting. The optimization of the computer system based on cognitive psychology improves the model prediction accuracy by 10%, and the absolute error decreases from 6.9 to 1.8%. The tax forecast results indicate that the tax imbalance in Jingdezhen is increasing. Therefore, the government needs to attach great importance to the development of the ceramic industry and strictly implement the tax policy. The tax supervision model can alleviate the problems of low fiscal contribution rate, tax evasion, and management loopholes. In addition, the SVM tax prediction model optimized by grid search method will lay a theoretical foundation for the research and application of taxation in the ceramic industry.

The influence of category representativeness on the low prevalence effect in visual search

Visual search is greatly affected by the appearance rate of given target types, such that low-prevalence items are harder to detect, which has consequences for real-world search tasks where target frequency cannot be balanced. However, targets that are highly representative of a categorically defined task set are also easier to find. We hypothesized that targets that are highly representative are less vulnerable to low-prevalence effects because an observer's attentional set prioritizes guidance toward them even when they are rare. We assessed this hypothesis by first determining the categorical structure of "prohibited carry-ons" via an exemplar-naming task, and used this structure to assess how category representativeness interacted with prevalence. Specifically, from the exemplar-naming task we selected a commonly named (knives) and rarely named (gas cans) target for a search task in which one of the targets was shown infrequently. As predicted, highly representative targets were found more easily than their less representative counterparts, but they also were less affected by prevalence manipulations. Experiment 1b replicated the results with targets matched for emotional valence (water bottles and fireworks). These findings demonstrate the powerful explanatory power of theories of attentional guidance that incorporate the dynamic influence of recent experience with the knowledge that comes from life experience to better predict behavioral outcomes associated with high-stakes search environments.

The low prevalence effect in fingerprint comparison amongst forensic science trainees and novices

The low prevalence effect is a phenomenon whereby target prevalence affects performance in visual search (e.g., baggage screening) and comparison (e.g., fingerprint examination) tasks, such that people more often fail to detect infrequent target stimuli. For example, when exposed to higher base-rates of 'matching' (i.e., from the same person) than 'non-matching' (i.e., from different people) fingerprint pairs, people more often misjudge 'non-matching' pairs as 'matches'-an error that can falsely implicate an innocent person for a crime they did not commit. In this paper, we investigated whether forensic science training may mitigate the low prevalence effect in fingerprint comparison. Forensic science trainees (n = 111) and untrained novices (n = 114) judged 100 fingerprint pairs as 'matches' or 'non-matches' where the matching pair occurrence was either high (90%) or equal (50%). Some participants were also asked to use a novel feature-comparison strategy as a potential attenuation technique for the low prevalence effect. Regardless of strategy, both trainees and novices were susceptible to the effect, such that they more often misjudged non-matching pairs as matches when non-matches were rare. These results support the robust nature of the low prevalence effect in visual comparison and have important applied implications for forensic decision-making in the criminal justice system.

The relationship between the subjective experience of real-world cognitive failures and objective target-detection performance in visual search

Visual search is a common occurrence in everyday life, such as searching for the location of keys, identifying a friend in a crowd, or scanning an upcoming intersection for hazards while driving. Visual search is also used in professional contexts, such as medical diagnostic imaging and airport baggage screening. These contexts are often characterised by low-prevalence or rare targets. Here we tested whether individual differences in the detection of targets in visual search could be predicted from variables derived from the rich informational source of participants' subjective experience of their cognitive and attentional function in everyday life. We tested this in both low-prevalence (Experiment 1) and high-prevalence (Experiment 2) visual search conditions. In both experiments, participants completed a visual search with arrays containing multiple photorealistic objects, and their task was to detect the presence of a gun. Following this, they completed the Cognitive Failures Questionnaire (CFQ) and the Attentional Control Scale (ACS). In Experiment 1, the target was present on 2% of trials, while in Experiment 2, it was present on 50%. In both experiments, participants' scores on the False Triggering component of the CFQ were negatively associated with accuracy on target-present trials, while participants' scores on the Forgetfulness component of the CFQ were positively associated with target-present accuracy. These results show that objective performance in visual search can be predicted from subjective experiences of cognitive function. They also highlight that the CFQ is not monolithic. Instead, the CFQ subfactors can have qualitatively different relationships with performance. Theoretical and practical implications are discussed.

The effect of categorical superiority in subsequent search misses

Subsequent search misses (SSM) refer to the decrease in accuracy of second target detection in dual-target visual search. One of the theoretical explanations of SSM errors is similarity bias - the tendency to search for similar targets and to miss the dissimilar ones. The current study focuses on both perceptual and categorical similarity and their individual roles in SSM. Five experiments investigated the role of perceptual and categorical similarity in subsequent search misses, wherein perceptual and categorical similarities were manipulated separately, and task relevance was controlled. The role of both perceptual and categorical similarity was revealed, however, the categorical similarity had greater impact on second target detection. The findings of this research suggest the revision of the traditional perceptual set hypothesis that mainly focuses on perceptual target similarity in multiple target visual search.

When experience does not promote expertise: security professionals fail to detect low prevalence fake IDs

Professional screeners frequently verify photograph IDs in such industries as professional security, bar tending, and sales of age-restricted materials. Moreover, security screening is a vital tool for law enforcement in the search for missing or wanted persons. Nevertheless, previous research demonstrates that novice participants fail to spot fake IDs when they are rare (i.e., the low prevalence effect; LPE). To address whether this phenomenon also occurs with professional screeners, we conducted three experiments. Experiment 1 compared security professional and non-professionals. Experiment 2 compared bar-security professionals, access-security professionals, and non-professionals. Finally, Experiment 3 added a newly created Professional Identity Training Questionnaire to determine whether and how aspects of professionals’ employment predict ID-matching accuracy. Across all three experiments, all participants were susceptible to the LPE regardless of professional status. Neither length/type of professional experience nor length/type of training experience affected ID verification performance. We discuss task performance and survey responses with aims to acknowledge and address this potential problem in real-world screening scenarios.

Visual search behavior and performance in luggage screening: effects of time pressure, automation aid, and target expectancy

Visual inspection of luggage using X-ray technology at airports is a time-sensitive task that is often supported by automated systems to increase performance and reduce workload. The present study evaluated how time pressure and automation support influence visual search behavior and performance in a simulated luggage screening task. Moreover, we also investigated how target expectancy (i.e., targets appearing in a target-often location or not) influenced performance and visual search behavior. We used a paradigm where participants used the mouse to uncover a portion of the screen which allowed us to track how much of the stimulus participants uncovered prior to their decision. Participants were randomly assigned to either a high (5-s time per trial) or a low (10-s time per trial) time-pressure condition. In half of the trials, participants were supported by an automated diagnostic aid (85% reliability) in deciding whether a threat item was present. Moreover, within each half, in target-present trials, targets appeared in a predictable location (i.e., 70% of targets appeared in the same quadrant of the image) to investigate effects of target expectancy. The results revealed better detection performance with low time pressure and faster response times with high time pressure. There was an overall negative effect of automation support because the automation was only moderately reliable. Participants also uncovered a smaller amount of the stimulus under high time pressure in target-absent trials. Target expectancy of target location improved accuracy, speed, and the amount of uncovered space needed for the search.Significance Statement Luggage screening is a safety-critical real-world visual search task which often has to be done under time pressure. The present research found that time pressure compromises performance and increases the risk to miss critical items even with automation support. Moreover, even highly reliable automated support may not improve performance if it does not exceed the manual capabilities of the human screener. Lastly, the present research also showed that heuristic search strategies (e.g., areas where targets appear more often) seem to guide attention also in luggage screening.

Eye tracking for work-related visual search: a cognitive task analysis

Cognitive Task Analysis (CTA) is an important methodology in ergonomics for studying workplaces and work patterns. Using eye tracking as a CTA methodology, this article explores visual search patterns in complex work environments and situations. It presents a simulated crime scene case study that applies eye tracking-based experiments in foraging and sense-making loops to elicit and represent knowledge on expert versus novice search patterns for complex work. The case probes the visual search task of preliminarily evaluating and documenting potential crime scene evidence. The experimental protocol relies on the ASL Mobile Eye and the analyses of experimental data include preliminary inspections of live-viewing data on eye-movements, precedence matrices detailing scan paths, and gaze charts that illustrate participants' attention based on fixation counts and durations. In line with the CTA methodology, the article uses concept maps to represent knowledge derived from different phases of the study. The article also discusses the research implications and methodologically reflects on the case study. Practitioner summary: This study offers valuable insights for work design. The use of eye tracking as a CTA methodology offers potentials for translating visual search tasks into defined visual search concepts for complex work environments and situations. The ability to model visual attention is valuable for work designs that improve complex work performance, reduce work stress, and promote work satisfaction.

Airport Security Screener Problem-Solving Knowledge and Implications

OBJECTIVE

This research investigates security screeners' knowledge and the effect that differences in knowledge have on the performance of problem-solving activities. We argue that the development of problem-solving knowledge enables security screeners to perform effective problem-solving activity, which assists search and decision-making processes.

BACKGROUND

Airport security screening research has investigated the many variables that affect security screeners' search and decision making during simulated threat-detection tasks. Although search and decision making are essential aspects of security screening, few studies have investigated the problem-solving knowledge and activities that support security screening task performance.

METHOD

Sixteen more-experienced and 24 less-experienced security screeners were observed as they performed x-ray screening in the field at an Australian international airport's departure security checkpoint. Participants wore eye-tracking glasses and delivered concurrent verbal protocol.

RESULTS

When interacting with other security screeners, more-experienced screeners demonstrated situational knowledge more than less-experienced screeners, whereas less-experienced screeners experienced more insufficient knowledge. Lag-sequential analysis using combined data from both screener groups showed that situational knowledge facilitated effective problem-solving activity to support search and decision making. Insufficient knowledge led screeners to seek assistance and defer decision making.

CONCLUSION

This study expands current understandings of airport security screening. It demonstrates that security screeners develop knowledge that is specific to problem solving. This knowledge assists effective problem-solving activity to support search and decision making, and to mitigate uncertainty during the x-ray screening task.

APPLICATION

Findings can inform future security screening processes, screener training, and technology support tools. Furthermore, findings are potentially transferable to other domains.

Effects of transcranial direct current stimulation over the posterior parietal cortex on novice X-ray screening performance

Existing theories of visual search are generally deduced from lab-based studies involving the identification of a target object among similar distractors. The role of the right parietal cortex in visual search is well-established. However, less is known about real-world visual search tasks, such as X-ray screening, which require targets to be disembedded from their background. Research has shown variations in the cognitive abilities required for these tasks and typical lab-based visual search tasks. Thus, the findings of traditional visual search studies do not always transfer into the applied domain. Although brain imaging studies have offered insights into visual search tasks involving disembedding, highlighting an association between the left parietal cortex and disembedding performance, no causal link has yet been established. To this end, we carried out a pilot study (n = 34, between-subjects) administering non-invasive brain stimulation over the posterior parietal cortex (PPC) prior to completing a security X-ray screening task. The findings suggested that anodal left PPC tDCS enhanced novice performance in X-ray screening over that of sham stimulation, in line with brain imaging findings. However, the efficacy of tDCS is under question, with a growing number of failed replications. With this in mind, this study aims to re-test our original hypothesis by examining the effects of left-side parietal stimulation on novice X-ray screener performance and comparing them to those of sham stimulation and of stimulation on a control site (right PPC). As such, this within-subjects study comprised three sessions (2 mA left PPC, 2 mA right PPC, low-intensity sham stimulation left PPC), to investigate effects of anodal tDCS on X-ray screening performance. The pre-registered analysis did not detect any significant differences between left PPC tDCS and sham tDCS or left PPC tDCS and right PPC tDCS on novice performance (d') in X-ray screening. Further exploratory analyses detected no effects of left PPC tDCS on any other indices of performance in the X-ray security screening task (c, RTs and accuracy), or a disembedding control task (RTs and accuracy). The use of alternative stimulation techniques, with replicable behavioural effects on the parietal lobe (or a multi-technique approach), and well-powered studies with a systematic variation of stimulation parameters, could help to choose between two possible interpretations: that neither left nor right PPC are causally related to either tasks or that tDCS was ineffective. Finally, low-intensity sham stimulation (.016 mA), previously shown to outperform other sham conditions in between-subjects designs, was found to be ineffective for blinding participants in a within-subjects design. Our findings raise concerns for the current lack of optimal control conditions and add to the growing literature highlighting the need for replication in the field.

Automation reliability, human-machine system performance, and operator compliance: A study with airport security screeners supported by automated explosives detection systems for cabin baggage screening

Using a simulated X-ray screening task, we tested 122 airport security screeners working with the support of explosives detection systems for cabin baggage screening (EDSCB) as low-level automation. EDSCB varied systematically on three automation reliability measures: accuracy, d', and positive predictive value (PPV). Results showed that when unaided performance was high, operator confidence was high, and automation provided only small benefits. When unaided performance was lower, operator confidence was lower, and automation with higher d' provided large benefits. Operator compliance depended on the PPV of automation: We found lower compliance for lower PPV. Automation with a high false alarm rate of 20% and a low PPV of .3 resulted in operators ignoring about one-half of the true automation alarms on difficult targets-a strong cry-wolf effect. Our results suggest that automation reliability described by d' and PPV is more valid than using accuracy alone. When the PPV is below .5, operators should receive clear instructions on how to respond to automation alarms.

Some cues are more equal than others: Cue plausibility for false alarms in baggage screening

This study investigated the effects of cue plausibility in a baggage screening task. 120 participants had to indicate whether a prohibited item was present in a series of grey-scaled X-ray images of baggage. They were assisted by a support system, which pointed at the location of a suspicious object. A 2 × 2 × 2 between-subjects design was used. Cue plausibility for false alarms (i.e. how the cued object was similar to a prohibited item) and support system reliability were manipulated at two levels (high/low). Furthermore, half of participants were provided with a rationale about automation failures (RAF) to reduce their negative impact on trust and performance. The results showed lower performance and more compliance with automation suggestions when cues were implausible than plausible. The RAF increased the response time and did not improve detection performance. Overall, this suggests that effective (computer-based) training is needed to reduce the negative effect of plausible cues.

Visual search does not always predict performance in tasks that require finding targets among distractors: The case of line-ending illusory contours

The standard visual search task is integral to the study of selective attention and in search tasks target present slopes are the primary index of attentional demand. However, there are times when similarities in slopes may obscure important differences between conditions. To demonstrate this point, we used the case of line-ending illusory contours, building on a study by Li, Cave, and Wolfe (2008) where orientation-based search for figures defined by line-ending illusory contours was compared to that for the corresponding real-contour controls. Consistent with Li et al. (2008), we found search to be efficient for both illusory contour figures and the corresponding real-contour controls, with no significant differences between them. However, major differences between illusory contours and the real-contour controls emerged in selective enumeration, a task where participants enumerated targets in a display of distractors, with the number of targets and distractors manipulated. When looking at the distractor slopes, the increase in RT to enumerate a single target as a function of the number of distractors (a direct analogue to target present trials, with identical displays), we found distractor costs for illusory contour figures to be over 100 ms/distractor higher than for the corresponding real-contour controls. Furthermore, the discrepancies in RT slope between 1-3 and 6-8 targets associated with subitizing were only seen in the real-contour controls. These results show that similarities in RT slopes in search may mask important differences between conditions that emerge in other tasks.

Expertise, Automation and Trust in X-Ray Screening of Cabin Baggage

X-ray screening of passenger baggage is a key component in aviation security. The current study investigated how experts and novices performed in an X-ray baggage screening task while being assisted by an adaptable diagnostic aid. Furthermore, it examined how both groups operated and trusted this automated system. 30 experts (certified screeners) and 31 novices (students) had to indicate whether a target item (either a knife or a gun) was present in a series of X-ray images of cabin baggage. Half of the participants could choose between three different support levels of the diagnostic aid (DA): (1) no support, (2) a cue indicating the presence of a potential target without locating it, or (3) a cue indicating the presence of a potential target by surrounding it with a red frame. As expected, experts achieved higher detection performance (d'), were more self-confident and felt more competent in achieving the task than novices. Furthermore, experts experienced less time pressure and fatigue. Although both groups used the DA in a comparable way (in terms of support level used and frequency of level switches), results showed a performance increase for novices working with the DA compared to novices without support. This benefit of DA was not observed for experts. Interestingly, despite no difference in perceived trust ratings, experts were more compliant (i.e., following DA recommendations when it indicated the presence of a target) and reliant (i.e., following DA recommendations when it indicated the absence of a target) than novices. Altogether, the results of the present study suggested that novices benefited more from a DA than experts. Furthermore, compliance and reliance on DA seemed to depend on expertise with the task. Since experts should be better at assessing the reliability of the DA than novices, they may have used the DA as 'back-up' to confirm their decisions based on expertise (confirmatory function), while novices may have used it as a guide to base their decisions on (support function). Finally, trust towards a DA was associated with the degree to which participants found the DA useful.