Detecting Bombs in X-Ray Images of Hold Baggage: 2D Versus 3D Imaging

Enhanced detonators detection in X-ray baggage inspection by image manipulation and deep convolutional neural networks

Detecting detonators is a challenging task because they can be easily mis-classified as being a harmless organic mass, especially in high baggage throughput scenarios. Of particular interest is the focus on automated security X-ray analysis for detonators detection. The complex security scenarios require increasingly advanced combinations of computer-assisted vision. We propose an extensive set of experiments to evaluate the ability of Convolutional Neural Network (CNN) models to detect detonators, when the quality of the input images has been altered through manipulation. We leverage recent advances in the field of wavelet transforms and established CNN architectures-as both of these can be used for object detection. Various methods of image manipulation are used and further, the performance of detection is evaluated. Both raw X-ray images and manipulated images with the Contrast Limited Adaptive Histogram Equalization (CLAHE), wavelet transform-based methods and the mixed CLAHE RGB-wavelet method were analyzed. The results showed that a significant number of operations, such as: edges enhancements, altered color information or different frequency components provided by wavelet transforms, can be used to differentiate between almost similar features. It was found that the wavelet-based CNN achieved the higher detection performance. Overall, this performance illustrates the potential for a combined use of the manipulation methods and deep CNNs for airport security applications.

Enhanced threat detection in three dimensions: An image-matched comparison of computed tomography and dual-view X-ray baggage screening

Computed Tomography (CT) is increasingly used in screening of cabin baggage in airports. The current study aimed to establish whether screening with CT confers a detection advantage over dual-view (DV) X-ray when resolution is controlled. We also evaluated whether a 'targetless' search strategy - in which screeners identify and reject safe items - improved detection relative to target-based methods. In an online study, 104 novice screeners were trained with either CT or DV, and either a targetless or a target-based search strategy. Screeners were then tested in a simulated cabin baggage screening task. CT screeners performed with greater sensitivity than DV screeners. Search strategy did not affect sensitivity, although the target-based strategy resulted in a more liberal criterion. We conclude that CT imaging confers a benefit to screening performance over DV when image resolution is controlled. This is likely due to the ability to rotate the image to resolve occlusions.

Visual search as an embodied process: The effects of perspective change and external reference on search performance

Traditional visual search tasks in the laboratories typically involve looking for targets in 2D displays with exemplar views of objects. In real life, visual search commonly entails 3D objects in 3D spaces with nonperpendicular viewing and relative motions between observers and search array items, both of which lead to transformations of objects’ projected images in lawful but unpredicted ways. Furthermore, observers often do not have to memorize a target before searching, but may refer to it while searching, for example, holding a picture of someone while looking for them from a crowd. Extending the traditional visual search task, in this study, we investigated the effects of image transformation as a result of perspective change yielded by discrete viewing angle change (Experiment 1) or continuous rotation of the search array (Experiment 2) and of having external references on visual search performance. Results showed that when searching from 3D objects with a non-zero viewing angle, performance was similar to searching from 2D exemplar views of objects; when searching for 3D targets from rotating arrays in virtual reality, performance was similar to searching from stationary arrays. In general, discrete or continuous perspective change did not affect the search outcomes in terms of accuracy, response time, and self-rated confidence, or the search process in terms of eye movement patterns. Therefore, visual search does not require the exact match of retinal images. Additionally, being able to see the target during the search improved search accuracy and observers’ confidence. It increased search time because, as revealed by the eye movements, observers actively checked back on the reference target. Thus, visual search is an embodied process that involves real-time information exchange between the observers and the environment.

How Realistic Is Threat Image Projection for X-ray Baggage Screening?

At airports, security officers (screeners) inspect X-ray images of passenger baggage in order to prevent threat items (bombs, guns, knives, etc.) from being brought onto an aircraft. Because threat items rarely occur, many airports use a threat-image-projection (TIP) system, which projects pre-recorded X-ray images of threat items onto some of the X-ray baggage images in order to improve the threat detection of screeners. TIP is regulatorily mandated in many countries and is also used to identify officers with insufficient threat-detection performance. However, TIP images sometimes look unrealistic because of artifacts and unrealistic scenarios, which could reduce the efficacy of TIP. Screeners rated a representative sample of TIP images regarding artifacts identified in a pre-study. We also evaluated whether specific image characteristics affect the occurrence rate of artifacts. 24% of the TIP images were rated to display artifacts and 26% to depict unrealistic scenarios, with 34% showing at least one of the two. With two-thirds of the TIP images having been perceived as realistic, we argue that TIP still serves its purpose, but artifacts and unrealistic scenarios should be reduced. Recommendations on how to improve the efficacy of TIP by considering image characteristics are provided.

Improved X-ray baggage screening sensitivity with 'targetless' search training

When searching for a known target, mental representations of target features, or templates, guide attention towards matching objects and facilitate recognition. When only distractor features are known, distractor templates allow irrelevant objects to be recognised and attention to be shifted away. This is particularly true in X-ray baggage search, a challenging real-world visual search task with implications for public safety, where targets may be unknown, difficult to predict and concealed by an adversary, but distractors are typically benign and easier to identify. In the present study, we draw on basic principles of distractor suppression and rejection to investigate a counterintuitive ‘targetless’ approach to training baggage search. In a simulated X-ray baggage search task, we observed significant benefits to target detection sensitivity (d′) for targetless relative to target-based training, but no effects of performance-contingent rewards or the inclusion of superordinate semantic categories during training. The benefits of targetless search training were most apparent for stimuli involving less spatial overlap (occlusion), which likely represents the difficulty and greater individual variation involved in searching more visually complex images. Together, these results demonstrate the effectiveness of a counterintuitive targetless approach to training target detection in X-ray baggage search, based on basic principles of distractor suppression and rejection, with potential for use as a real-world training tool.

Airport Security Screener Problem-Solving Knowledge and Implications

OBJECTIVE

This research investigates security screeners' knowledge and the effect that differences in knowledge have on the performance of problem-solving activities. We argue that the development of problem-solving knowledge enables security screeners to perform effective problem-solving activity, which assists search and decision-making processes.

BACKGROUND

Airport security screening research has investigated the many variables that affect security screeners' search and decision making during simulated threat-detection tasks. Although search and decision making are essential aspects of security screening, few studies have investigated the problem-solving knowledge and activities that support security screening task performance.

METHOD

Sixteen more-experienced and 24 less-experienced security screeners were observed as they performed x-ray screening in the field at an Australian international airport's departure security checkpoint. Participants wore eye-tracking glasses and delivered concurrent verbal protocol.

RESULTS

When interacting with other security screeners, more-experienced screeners demonstrated situational knowledge more than less-experienced screeners, whereas less-experienced screeners experienced more insufficient knowledge. Lag-sequential analysis using combined data from both screener groups showed that situational knowledge facilitated effective problem-solving activity to support search and decision making. Insufficient knowledge led screeners to seek assistance and defer decision making.

CONCLUSION

This study expands current understandings of airport security screening. It demonstrates that security screeners develop knowledge that is specific to problem solving. This knowledge assists effective problem-solving activity to support search and decision making, and to mitigate uncertainty during the x-ray screening task.

APPLICATION

Findings can inform future security screening processes, screener training, and technology support tools. Furthermore, findings are potentially transferable to other domains.

Automation reliability, human-machine system performance, and operator compliance: A study with airport security screeners supported by automated explosives detection systems for cabin baggage screening

Using a simulated X-ray screening task, we tested 122 airport security screeners working with the support of explosives detection systems for cabin baggage screening (EDSCB) as low-level automation. EDSCB varied systematically on three automation reliability measures: accuracy, d', and positive predictive value (PPV). Results showed that when unaided performance was high, operator confidence was high, and automation provided only small benefits. When unaided performance was lower, operator confidence was lower, and automation with higher d' provided large benefits. Operator compliance depended on the PPV of automation: We found lower compliance for lower PPV. Automation with a high false alarm rate of 20% and a low PPV of .3 resulted in operators ignoring about one-half of the true automation alarms on difficult targets-a strong cry-wolf effect. Our results suggest that automation reliability described by d' and PPV is more valid than using accuracy alone. When the PPV is below .5, operators should receive clear instructions on how to respond to automation alarms.

An approach for adaptive automatic threat recognition within 3D computed tomography images for baggage security screening

BACKGROUND

The screening of baggage using X-ray scanners is now routine in aviation security with automatic threat detection approaches, based on 3D X-ray computed tomography (CT) images, known as Automatic Threat Recognition (ATR) within the aviation security industry. These current strategies use pre-defined threat material signatures in contrast to adaptability towards new and emerging threat signatures. To address this issue, the concept of adaptive automatic threat recognition (AATR) was proposed in previous work.

OBJECTIVE

In this paper, we present a solution to AATR based on such X-ray CT baggage scan imagery. This aims to address the issues of rapidly evolving threat signatures within the screening requirements. Ideally, the detection algorithms deployed within the security scanners should be readily adaptable to different situations with varying requirements of threat characteristics (e.g., threat material, physical properties of objects).

METHODS

We tackle this issue using a novel adaptive machine learning methodology with our solution consisting of a multi-scale 3D CT image segmentation algorithm, a multi-class support vector machine (SVM) classifier for object material recognition and a strategy to enable the adaptability of our approach. Experiments are conducted on both open and sequestered 3D CT baggage image datasets specifically collected for the AATR study.

RESULTS

Our proposed approach performs well on both recognition and adaptation. Overall our approach can achieve the probability of detection around 90% with a probability of false alarm below 20%.

CONCLUSIONS

Our AATR shows the capabilities of adapting to varying types of materials, even the unknown materials which are not available in the training data, adapting to varying required probability of detection and adapting to varying scales of the threat object.

A reference architecture for plausible Threat Image Projection (TIP) within 3D X-ray computed tomography volumes

BACKGROUND

Threat Image Projection (TIP) is a technique used in X-ray security baggage screening systems that superimposes a threat object signature onto a benign X-ray baggage image in a plausible and realistic manner. It has been shown to be highly effective in evaluating the ongoing performance of human operators, improving their vigilance and performance on threat detection.

OBJECTIVE

With the increasing use of 3D Computed Tomography (CT) in aviation security for both hold and cabin baggage screening a significant challenge arises in extending TIP to 3D CT volumes due to the difficulty in 3D CT volume segmentation and the proper insertion location determination. In this paper, we present an approach for 3D TIP in CT volumes targeting realistic and plausible threat object insertion within 3D CT baggage images.

METHOD

The proposed approach consists of dual threat (source) and baggage (target) volume segmentation, particle swarm optimisation based insertion determination and metal artefact generation. In our experiments, real baggage data collected from airports are used to generate TIP volumes for evaluation. We also propose a TIP quality score metric to automatically estimate the quality of generated TIP volumes.

RESULT

In our experiments with real baggage CT volumes and varying threat items, 90.25% of the generated TIP volumes are graded as good by human evaluation, 7% of them are of medium quality with minor flaws and 2.75% of them are bad.

CONCLUSION

Qualitative evaluations on real 3D CT baggage imagery show that our approach is able to generate realistic and plausible TIP which are indiscernible from real CT volumes and the TIP quality scores are consistent with human evaluations.

Using Eye Movements to Understand how Security Screeners Search for Threats in X-Ray Baggage

There has been an increasing drive to understand failures in searches for weapons and explosives in X-ray baggage screening. Tracking eye movements during the search has produced new insights into the guidance of attention during the search, and the identification of targets once they are fixated. Here, we review the eye-movement literature that has emerged on this front over the last fifteen years, including a discussion of the problems that real-world searchers face when trying to detect targets that could do serious harm to people and infrastructure.