Finger Vein Identification Based on Large Kernel Convolution and Attention Mechanism

FV (finger vein) identification is a biometric identification technology that extracts the features of FV images for identity authentication. To address the limitations of CNN-based FV identification, particularly the challenge of small receptive fields and difficulty in capturing long-range dependencies, an FV identification method named Let-Net (large kernel and attention mechanism network) was introduced, which combines local and global information. Firstly, Let-Net employs large kernels to capture a broader spectrum of spatial contextual information, utilizing deep convolution in conjunction with residual connections to curtail the volume of model parameters. Subsequently, an integrated attention mechanism is applied to augment information flow within the channel and spatial dimensions, effectively modeling global information for the extraction of crucial FV features. The experimental results on nine public datasets show that Let-Net has excellent identification performance, and the EER and accuracy rate on the FV_USM dataset can reach 0.04% and 99.77%. The parameter number and FLOPs of Let-Net are only 0.89M and 0.25G, which means that the time cost of training and reasoning of the model is low, and it is easier to deploy and integrate into various applications.

Metabolite monitoring concept for the biometric identification of individuals from the skin surface

This study aims at proof of concept that constant monitoring of the concentrations of metabolites in three individuals' sweat over time can differentiate one from another at any given time, providing investigators and analysts with increased ability and means to individualize this bountiful biological sample. A technique was developed to collect and extract authentic sweat samples from three female volunteers for the analysis of lactate, urea, and L-alanine levels. These samples were collected 21 times over a 40-day period and quantified using a series of bioaffinity-based enzymatic assays with UV-vis spectrophotometric detection. Sweat samples were simultaneously dried, derivatized, and analyzed by a GC-MS technique for comparison. Both UV-vis and GC-MS analysis methods provided a statistically significant MANOVA result, demonstrating that the sum of the three metabolites could differentiate each individual at any given day of the time interval. Expanding upon previous studies, this experiment aims to establish a method of metabolite monitoring as opposed to single-point analyses for application to biometric identification from the skin surface.

Enhancement, integration, expansion: Activating representation of detailed features for occluded person re-identification

A proposed method, Enhancement, integration, and Expansion, aims to activate the representation of detailed features for occluded person re-identification. Region and context are two important and complementary features, and integrating them in an occluded environment can effectively improve the robustness of the model. Firstly, a self-enhancement module is designed. Based on the constructed multi-stream architecture, rich and meaningful feature interference is introduced in the feature extraction stage to enhance the model's ability to perceive noise. Next, a collaborative integration module similar to cascading cross-attention is proposed. By studying the intrinsic interaction patterns of regional and contextual features, it adaptively fuses features across streams and enhances the diverse and complete representation of internal information. The module is not only robust to complex occlusions, but also mitigates the feature interference problem due to similar appearances or scenes. Finally, a matching expansion module that enhances feature discriminability and completeness is proposed. Providing more stable and accurate features for recognition. Compared with state-of-the-art methods on two occluded and holistic datasets, the proposed method is proved to be advanced and the effectiveness of the module is proved by extensive ablation studies.

A Novel PPG-Based Biometric Authentication System Using a Hybrid CVT-ConvMixer Architecture with Dense and Self-Attention Layers

Biometric authentication is a widely used method for verifying individuals' identities using photoplethysmography (PPG) cardiac signals. The PPG signal is a non-invasive optical technique that measures the heart rate, which can vary from person to person. However, these signals can also be changed due to factors like stress, physical activity, illness, or medication. Ensuring the system can accurately identify and authenticate the user despite these variations is a significant challenge. To address these issues, the PPG signals were preprocessed and transformed into a 2-D image that visually represents the time-varying frequency content of multiple PPG signals from the same human using the scalogram technique. Afterward, the features fusion approach is developed by combining features from the hybrid convolution vision transformer (CVT) and convolutional mixer (ConvMixer), known as the CVT-ConvMixer classifier, and employing attention mechanisms for the classification of human identity. This hybrid model has the potential to provide more accurate and reliable authentication results in real-world scenarios. The sensitivity (SE), specificity (SP), F1-score, and area under the receiver operating curve (AUC) metrics are utilized to assess the model's performance in accurately distinguishing genuine individuals. The results of extensive experiments on the three PPG datasets were calculated, and the proposed method achieved ACCs of 95%, SEs of 97%, SPs of 95%, and an AUC of 0.96, which indicate the effectiveness of the CVT-ConvMixer system. These results suggest that the proposed method performs well in accurately classifying or identifying patterns within the PPG signals to perform continuous human authentication.

Audio-Visual Fusion Based on Interactive Attention for Person Verification

With the rapid development of multimedia technology, personnel verification systems have become increasingly important in the security field and identity verification. However, unimodal verification systems have performance bottlenecks in complex scenarios, thus triggering the need for multimodal feature fusion methods. The main problem with audio-visual multimodal feature fusion is how to effectively integrate information from different modalities to improve the accuracy and robustness of the system for individual identity. In this paper, we focus on how to improve multimodal person verification systems and how to combine audio and visual features. In this study, we use pretrained models to extract the embeddings from each modality and then perform fusion model experiments based on these embeddings. The baseline approach in this paper involves taking the fusion feature and passing it through a fully connected (FC) layer. Building upon this baseline, we propose three fusion models based on attentional mechanisms: attention, gated, and inter-attention. These fusion models are trained on the VoxCeleb1 development set and tested on the evaluation sets of the VoxCeleb1, NIST SRE19, and CNC-AV datasets. On the VoxCeleb1 dataset, the best system performance achieved in this study was an equal error rate (EER) of 0.23% and a detection cost function (minDCF) of 0.011. On the evaluation set of NIST SRE19, the EER was 2.60% and the minDCF was 0.283. On the evaluation set of the CNC-AV set, the EER was 11.30% and the minDCF was 0.443. These experimental results strongly demonstrate that the proposed fusion method can significantly improve the performance of multimodal character verification systems.

Populational Variations of Cheiloscopy Patterns: A cross-sectional observation pilot study

Purpose Lip prints are unique and have potential for use as a human identifier. The purpose of this study was to observe possible cheiloscopy differences of individuals with and without parafunctional oral habits such as smoking, vaping, playing a wind instrument or using an asthma inhaler.Methods This IRB approved blinded cross-sectional observation pilot study collected lip prints from sixty-six individuals, three of which were excluded. Participants cleansed their lips, then lipstick was applied to the vermillion zones of the upper and lower lips. Adhesive tape was applied to the lips and prints were transferred to white bond paper for viewing purposes. Each set of included lip prints was divided into quadrants and dichotomized into a group of those with an oral parafunctional habit or with no such habits. Each quadrant sample was then manually analyzed and classed according to the gold standard Suzuki and Tsuchihashi system.Results A total of 252 dichotomized lip print quadrants (with habits n=76, 30.2%, and without habits n=176, 69.8%) were analyzed. Type II patterns were the most common for examined quadrant samples; however, no statistically significant differences (Pearson's chi-squared test, p=0.366) were observed between pattern classifications of samples with and without parafunctional oral habits.Conclusion There is no statistically significant difference of lip print patterns between individuals with and without parafunctional oral habits. Further research on populational variations is needed for cheiloscopy to aid in human identifications.

Electrocardiogram-based biometrics for user identification - Using your heartbeat as a digital key

External biometrics such as thumbprint and facial recognition have become standard tools for securing our digital devices and protecting our data. These systems, however, are potentially prone to copying and cybercrime access. Researchers have therefore explored internal biometrics, such as the electrical patterns within an electrocardiogram (ECG). The heart's electrical signals carry sufficient distinctiveness to allow the ECG to be used as an internal biometric for user authentication and identification. Using the ECG in this way has many potential advantages and limitations. This article reviews the history of ECG biometrics and explores some of the technical and security considerations. It also explores current and future uses of the ECG as an internal biometric.

Comparison of Wrist and Forearm EMG for Multi-day Biometric Authentication

Recently, electromyography (EMG) has been established as a promising new biometric trait that provides a unique dual mode security: biometrics and knowledge. For authentication that is used daily and long-term by general consumers, the wrist is a suitable location, which could be easily integrated into the existing form of smartwatches and fitness trackers. However, current EMG-based biometrics still follow the historical path of powered prosthetics research, where EMG signals were usually recorded from forearm positions. Moreover, the robustness of EMG processing algorithms across multiple days is still an open problem that needs to be addressed before for long-term reliable use. This study intends to investigate the difference in authentication performance between wrist and forearm EMG signals, in a within-day and two cross-day analyses. Our open dataset (GRABMyo dataset) was used to examine this difference, which contains forearm and wrist EMG data collected from 43 participants over three different days with long separation (Days 1, 8, and 29). The results showed wrist EMG signals led to at least comparable with forearm EMG signals in within-day Equal-error rate (EER). In cross-day analysis, the EER of the wrist EMG signals was higher than that of forearm signals. In general, the low median EER (<0.1) of wrist EMG in cumulative cross-day analysis demonstrates the promise of using wrist EMG signals for authentication in long-term applications.

A Scalable ECG Identification System Based on Locality-Sensitive Hashing

Electrocardiograms (ECGs) have the inherent property of being intrinsic and dynamic and are shown to be unique among individuals, making them promising as a biometric trait. Although many ECG biometric recognition approaches have demonstrated accurate recognition results in small enrollment sets, they can suffer from performance degradation when many subjects are enrolled. This study proposes an ECG biometric identification system based on locality-sensitive hashing (LSH) that can accommodate a large number of registrants while maintaining satisfactory identification accuracy. By incorporating the concept of LSH, the identity of an unknown subject can be recognized without performing vector comparisons for all registered subjects. Moreover, a kernel density estimator-based method is used to exclude unregistered subjects. The ECGs of 285 subjects from the PTB dataset were used to evaluate the proposed scheme's performance. Experimental results demonstrated an IR and EER of 99% and 4%, respectively, when Nen/Nid = 15/3.

Photoplethysmogram Biometric Authentication Using a 1D Siamese Network

In the head-mounted display environment for experiencing metaverse or virtual reality, conventional input devices cannot be used, so a new type of nonintrusive and continuous biometric authentication technology is required. Since the wrist wearable device is equipped with a photoplethysmogram sensor, it is very suitable for use for nonintrusive and continuous biometric authentication purposes. In this study, we propose a one-dimensional Siamese network biometric identification model using a photoplethysmogram. To maintain the unique characteristics of each person and reduce noise in preprocessing, we adopted a multicycle averaging method without using a bandpass or low-pass filter. In addition, to verify the effectiveness of the multicycle averaging method, the number of cycles was changed and the results were compared. Genuine and impostor data were used to verify the biometric identification. We used the one-dimensional Siamese network to verify the similarity between the classes and found that the method with five overlapping cycles was the most effective. Tests were conducted on the overlapping data of five single-cycle signals and excellent identification results were observed, with an AUC score of 0.988 and an accuracy of 0.9723. Thus, the proposed biometric identification model is time-efficient and shows excellent security performance, even in devices with limited computational capabilities, such as wearable devices. Consequently, our proposed method has the following advantages compared with previous works. First, the effect of noise reduction and information preservation through multicycle averaging was experimentally verified by varying the number of photoplethysmogram cycles. Second, by analyzing authentication performance through genuine and impostor matching analysis based on a one-dimensional Siamese network, the accuracy that is not affected by the number of enrolled subjects was derived.

Acceptability and applicability of biometric iris scanning for the identification and follow up of highly mobile research participants living in fishing communities along the shores of Lake Victoria in Kenya, Tanzania, and Uganda

BACKGROUND

Recruitment and retention of participants in research studies conducted in fishing communities remain a challenge because of population mobility. Reliable and acceptable methods for identifying and tracking participants taking part in HIV prevention and treatment research are needed. The study aims to assess the acceptability, and technical feasibility of iris scans as a biometric identification method for research participants in fishing communities.

METHODS

This was a cross-sectional study conducted in eight fishing communities in Kenya, Tanzania, and Uganda, with follow-up after one month in a randomly selected subset of participants. All consenting participants had their iris scanned and then responded to the survey.

RESULTS

1,199 participants were recruited. The median age was 33 [Interquartile range (IQR) 24-42] years; 56% were women. The overall acceptability of iris scanning was 99%, and the success rate was 98%. Eighty one percent (n = 949) had a successful scan on first attempt, 116 (10%) on second and 113 (9%) after more than two attempts. A month later, 30% (n = 341) of participants were followed up. The acceptability of repeat iris scanning was 99% (n = 340). All participants who accepted repeat iris scanning had successful scans, with 307 (90%) scans succeeding on first attempt; 25 (7%) on second attempt, and 8 (2%) after several attempts. The main reason for refusing iris scanning was fear of possible side effects of the scan on the eyes or body.

CONCLUSION

The acceptability and applicability of biometric iris scan as a technique for unique identification of research participants is high in fishing communities. However, successful use of the iris scanning technology in research will require education regarding the safety of the procedure.

Lightweight Multimodal Domain Generic Person Reidentification Metric for Person-Following Robots

Recently, person-following robots have been increasingly used in many real-world applications, and they require robust and accurate person identification for tracking. Recent works proposed to use re-identification metrics for identification of the target person; however, these metrics suffer due to poor generalization, and due to impostors in nonlinear multi-modal world. This work learns a domain generic person re-identification to resolve real-world challenges and to identify the target person undergoing appearance changes when moving across different indoor and outdoor environments or domains. Our generic metric takes advantage of novel attention mechanism to learn deep cross-representations to address pose, viewpoint, and illumination variations, as well as jointly tackling impostors and style variations the target person randomly undergoes in various indoor and outdoor domains; thus, our generic metric attains higher recognition accuracy of target person identification in complex multi-modal open-set world, and attains 80.73% and 64.44% Rank-1 identification in multi-modal close-set PRID and VIPeR domains, respectively.

Face Off: Overcoming the Fifth Amendment Conflict Between Cybersecurity and Self-Incrimination

The Founders included the privilege against self-incrimination in the Constitution to protect individual privacy and ensure a fair judicial process. Courts have failed U.S. citizens by neglecting to protect them from compelled unlocking of biometrically encrypted devices. This inaction has created a loophole that contradicts the framework of the privilege against self-incrimination. To correct this mistake courts should reconsider the trend they have set for the Constitution and the Fifth Amendment and consider adopting a forward-thinking cybersecurity lens to conclude that biometric authentication is testimonial. Courts should consider that biometric encryption is akin to a compelled password entry for the purposes of the foregone conclusion doctrine. The foregone conclusion doctrine should be applied in limited circumstances with a specific and high burden of proof so that the "jealous protection of the privilege against self-incriminating testimony" can be preserved. Allowing law enforcement such easy access to smart devices narrows Fifth Amendment protections and the expansive foregone conclusion exception is contrary to both principles of cybersecurity and the spirit of the Fifth Amendment. Courts should move to remediate this at once. These liberties and values can only be guaranteed by courts that are willing to take on cases with issues revolving around biometric encryption, the Fifth Amendment, and the foregone conclusion doctrine.

Biometric-Based Key Generation and User Authentication Using Acoustic Characteristics of the Outer Ear and a Network of Correlation Neurons

Trustworthy AI applications such as biometric authentication must be implemented in a secure manner so that a malefactor is not able to take advantage of the knowledge and use it to make decisions. The goal of the present work is to increase the reliability of biometric-based key generation, which is used for remote authentication with the protection of biometric templates. Ear canal echograms were used as biometric images. Multilayer convolutional neural networks that belong to the autoencoder type were used to extract features from the echograms. A new class of neurons (correlation neurons) that analyzes correlations between features instead of feature values is proposed. A neuro-extractor model was developed to associate a feature vector with a cryptographic key or user password. An open data set of ear canal echograms to test the performance of the proposed model was used. The following indicators were achieved: EER = 0.0238 (FRR = 0.093, FAR < 0.001), with a key length of 8192 bits. The proposed model is superior to known analogues in terms of key length and probability of erroneous decisions. The ear canal parameters are hidden from direct observation and photography. This fact creates additional difficulties for the synthesis of adversarial examples.

A Smart Pen Based on Triboelectric Effects for Handwriting Pattern Tracking and Biometric Identification

The rapid development of artificial intelligence places high demands on human-machine interfaces. Various types of huma-machine interfaces have been implemented, including smart keyboards, electronic skins, and wearable motion sensors. Handwriting behavior has a high degree of interaction freedom, and handwriting characteristics offer high-security standards for human-machine systems. Herein, we propose a portable smart pen integrated with triboelectric displacement vector sensors to trace handwriting trajectories for human-machine interactions and biometric identification. A triboelectric pressure sensor array is evenly distributed along the pen case to sense displacement vectors, and an additional triboelectric sensor is placed on top of the pen to detect vertical force. By leveraging the resin pen refill as a tribopositive material and a nanowired polyethylene tribonegative layer attached to a Cu electrode, triboelectric signals are generated during the writing/moving process. The calculation and analysis of the sensor signals enable the recognition of handwritten patterns, including Latin letters and Arabic numerals. Moreover, the smart pen can be used to authenticate users based on their unique handwriting patterns, which can help take human-machine interfaces and cyber security to the next level. Furthermore, a custom smart pen operation mode that enables the control of a slide presentation demonstrates the smart pen's potential for various human-machine interface applications.

A Secure Online Fingerprint Authentication System for Industrial IoT Devices over 5G Networks

The development of 5G networks has rapidly increased the use of Industrial Internet of Things (IIoT) devices for control, monitoring, and processing purposes. Biometric-based user authentication can prevent unauthorized access to IIoT devices, thereby safeguarding data security during production. However, most biometric authentication systems in the IIoT have no template protection, thus risking raw biometric data stored as templates in central databases or IIoT devices. Moreover, traditional biometric authentication faces slow, limited database holding capacity and data transmission problems. To address these issues, in this paper we propose a secure online fingerprint authentication system for IIoT devices over 5G networks. The core of the proposed system is the design of a cancelable fingerprint template, which protects original minutia features and provides privacy and security guarantee for both entity users and the message content transmitted between IIoT devices and the cloud server via 5G networks.Compared with state-of-the-art methods, the proposed authentication system shows competitive performance on six public fingerprint databases, while saving computational costs and achieving fast online matching.

Dual-Biometric Human Identification Using Radar Deep Transfer Learning

Accurate human identification using radar has a variety of potential applications, such as surveillance, access control and security checkpoints. Nevertheless, radar-based human identification has been limited to a few motion-based biometrics that are solely reliant on micro-Doppler signatures. This paper proposes for the first time the use of combined radar-based heart sound and gait signals as biometrics for human identification. The proposed methodology starts by converting the extracted biometric signatures collected from 18 subjects to images, and then an image augmentation technique is applied and the deep transfer learning is used to classify each subject. A validation accuracy of 58.7% and 96% is reported for the heart sound and gait biometrics, respectively. Next, the identification results of the two biometrics are combined using the joint probability mass function (PMF) method to report a 98% identification accuracy. To the best of our knowledge, this is the highest reported in the literature to date. Lastly, the trained networks are tested in an actual scenario while being used in an office access control platform to identify different human subjects. We report an accuracy of 76.25%.

Individual Identification in Facial Appearance Biometrics Based on Macroscopical Comparison

Individual identification is one of the research hotspots in the practice of forensic science, and the judgment is usually built on the comparison of the unique biological characteristics of the individual, such as fingerprints, iris and DNA. With the dramatic increase in the number of cases related to video image investigations, there is an increasing need for the technology to identify individuals based on the macroscopic comparison of facial appearance biometrics. At present, with the introduction of computer three-dimensional (3D) modeling and 3D superimposition comparison technology, considerable progress has been made in individual identification methods based on macroscopic comparison of facial appearance biometrics. This paper reviews individual facial appearance biometric methods based on macroscopical comparison, comprehensively analyzes the advantages and limitations of different methods, and puts forward recommendations and prospects for subsequent research.

Biometric Identification Based on Keystroke Dynamics

The purpose of the paper is to study how changes in neural network architecture and its hyperparameters affect the results of biometric identification based on keystroke dynamics. The publicly available dataset of keystrokes was used, and the models with different parameters were trained using this data. Various neural network layers-convolutional, recurrent, and dense-in different configurations were employed together with pooling and dropout layers. The results were compared with the state-of-the-art model using the same dataset. The results varied, with the best-achieved accuracy equal to 82% for the identification (1 of 20) task.

PerAE: An Effective Personalized AutoEncoder for ECG-based Biometric in Augmented Reality System

With the development of the Augmented and Virtual Reality (AR/VR) technologies, massive biometric data are collected by different organizations. These data have great significance but also worsen the privacy risks. Electro-CardioGram (ECG)-based Identity Recognition (EIR) is a popular Biometric technology. An ECG record is an internal Biology feature of a person and has time continuity. Thus, compared with traditional Biometric methods like face recognition, EIR may be less vulnerable to attack. We propose an Autoencoder-based EIR system, called Personalized AutoEncoder (PerAE). PerAE maintains a small autoencoder model (called Attention-MemAE) for each registered user of a system. The Attention-MemAE enhances the autoencoder by using a memory module and two attention mechanisms. A users Attention-MemAE classifies the hearbeats of other users as anomalies. An Attention-MemAE can be updated when the distribution of the users ECG data is changed. By using personalized autoencoder, PerAE can improve the time efficiency and reduce the memory overhead. It improves the adaptability, scalability, and maintainability of EIR systems. Experiment results show that to train an Attention-MemAE with 90% identification accuracy for a user, we can just take five minutes to collect the users ECG data (around 500 heartbeat samples).

Distinguishability of keystroke dynamic template

When keystroke dynamics are used for authentication, users tend to get different levels of security due to differences in the quality of their templates. This paper addresses this issue by proposing a metric to quantify the quality of keystroke dynamic templates. That is, in behavioral biometric verification, the user’s templates are generally constructed using multiple enrolled samples to capture intra-user variation. This variation is then used to normalize the distance between a set of enrolled samples and a test sample. Then a normalized distance is compared against a predefined threshold value to derive a verification decision. As a result, the coverage area for accepted samples in the original space of vector representation is discrete. Therefore, users with the higher intra-user variation suffer higher false acceptance rates (FAR). This paper proposes a metric that can be used to reflect the verification performance of individual keystroke dynamic templates in terms of FAR. Specifically, the metric is derived from statistical information of user-specific feature variations, and it has a non-decreasing property when a new feature is added to a template. The experiments are performed based on two public keystroke dynamic datasets comprising of two main types of keystroke dynamics: constrained-text and free-text, namely the CMU keystroke dynamics dataset and the Web-Based Benchmark for keystroke dynamics dataset. Experimental results based on multiple classifiers demonstrate that the proposed metric can be a good indicator of the template’s false acceptance rate. Thus, it can be used to enhance the security of the user authentication system based on keystroke dynamics.

Reinforced Palmprint Reconstruction Attacks in Biometric Systems

Biometric signals can be acquired with different sensors and recognized in secure identity management systems. However, it is vulnerable to various attacks that compromise the security management in many applications, such as industrial IoT. In a real-world scenario, the target template stored in the database of a biometric system can possibly be leaked, and then used to reconstruct a fake image to fool the biometric system. As such, many reconstruction attacks have been proposed, yet unsatisfactory naturalness, poor visual quality or incompleteness remains as major limitations. Thus, two reinforced palmprint reconstruction attacks are proposed. Any palmprint image, which can be easily obtained, is used as the initial image, and the region of interest is iteratively modified with deep reinforcement strategies to reduce the matching distance. In the first attack, Modification Constraint within Neighborhood (MCwN) limits the modification extent and suppresses the reckless modification. In the second attack, Batch Member Selection (BMS) selects the significant pixels (SPs) to compose the batch, which are simultaneously modified to a slighter extent to reduce the matching number and the visual-quality degradation. The two reinforced attacks can satisfy all the requirements, which cannot be simultaneously satisfied by the existing attacks. The thorough experiments demonstrate that the two attacks have a highly successful attack rate for palmprint systems based on the most state-of-the-art coding-based methods.

On Learning Disentangled Representations for Gait Recognition

Gait, the walking pattern of individuals, is one of the important biometrics modalities. Most of the existing gait recognition methods take silhouettes or articulated body models as gait features. These methods suffer from degraded recognition performance when handling confounding variables, such as clothing, carrying and viewing angle. To remedy this issue, we propose a novel AutoEncoder framework, GaitNet, to explicitly disentangle appearance, canonical and pose features from RGB imagery. The LSTM integrates pose features over time as a dynamic gait feature while canonical features are averaged as a static gait feature. Both of them are utilized as classification features. In addition, we collect a Frontal-View Gait (FVG) dataset to focus on gait recognition from frontal-view walking, which is a challenging problem since it contains minimal gait cues compared to other views. FVG also includes other important variations, e.g., walking speed, carrying, and clothing. With extensive experiments on CASIA-B, USF, and FVG datasets, our method demonstrates superior performance to the SOTA quantitatively, the ability of feature disentanglement qualitatively, and promising computational efficiency. We further compare our GaitNet with state-of-the-art face recognition to demonstrate the advantages of gait biometrics identification under certain scenarios, e.g., long-distance/lower resolutions, cross viewing angles. Source code is available at http://cvlab.cse.msu.edu/project-gaitnet.html.

Increased Risks of Re-identification For Patients Posed by Deep Learning-Based ECG Identification Algorithms

ECGs analysis is an important tool in cardiac diagnosis. ECG data also have the potential to be used as a biometric source that allows precise person identification similar to the widely used fingerprint and iris recognition techniques. However, this phenomenon also raises serious privacy concerns. In this study, we provide a complete, multi-step ECG identification algorithm using a private database of ECG recordings. We train and validate our AI model on approximately 40k patients which makes this study by far the largest research project in this field. Moreover, our best model attained an exceptionally high accuracy of 94.56%. In addition to discussing the general implications of the deployment of such systems related to privacy, for the first time, we also assess the accuracy of ECG-based identification for distinct heart condition groups (and combinations of such conditions) and the corresponding privacy implications. For instance, we discovered that in contrast to initial expectation that identification accuracy for healthy normal sinus rhythm should be the highest, the identification accuracy is higher for patients with sinus tachycardia or patients who are diagnosed with both ST changes and supraventricular tachycardia. This puts these patients at a higher risk of privacy issues due to re-identification. On the other hand, we observed that patients with premature ventricular contractions have an identification accuracy as low as 78.54%. The identification rate for patients with a pacemaker is 80.2%.Clinical relevance-While ECG as a biometric can be a potentially useful technology, it also raises serious concerns regarding the privacy of cardiac patients. Especially, the ECG Identification algorithms empowered by deep learning can increase the risk of re-identification.

ECG-based Biometric Recognition without QRS Segmentation: A Deep Learning-Based Approach

Electrocardiogram (ECG)-based identification systems have been widely studied in the literature. Usually, an ECG trace needs to be segmented according to the detected R peaks to enable feature extraction from the ECGs of duration equal to nearly one cardiac cycle. Beat averaging should also be applied to reduce the influence of inter-beat variation on the extracted features and identification accuracy. Either detecting R peaks or collecting extra heartbeats for averaging will inevitably lead to a delay in the identification process. This paper proposes a deep learning-based ECG biometric identification scheme that allows identity recognition using a random ECG segment without needing R-peak detection and beat averaging. Moreover, the problem of being vulnerable to unregistered subjects in an identification system is also addressed. Experimental results demonstrated that an identification rate of 99.1% for an identification system having 235 enrollees with an equal error rate of 8.08% was achieved.

Motion Artifact Resilient SCG-based Biometric Authentication Using Machine Learning

On account of privacy preserving issue and health-care monitoring, physiological signal biometric authentication system has gained popularity in recent years. Seismocardiogram (SCG) is now easily accessible owing to the advance of wearable sensor technology. However, SCG biometric has not been widely explored due to the challenging motion artifact removal. In this paper, we design placing the sensors at different body parts under different activities to determine the best sensor location. In addition, we develop SCG noise removal algorithm and utilize machine learning approach to perform biometric authentication tasks. We validate the proposed methods on 20 healthy young adults. The dataset contains acceleration data of sitting, standing, walking, and sitting post-exercise activities with the sensor placed at the wrists, neck, heart and sternum. We demonstrate that vertical and dorsal-ventral SCG near the heart and the sternum produce reliable SCG biometric evidenced by achieving the state-of-the-art performance. Moreover, we present the efficacy of the devised noise removal procedure in the authentication during walking motion.Clinical relevance- A seismocardiography-based biometric authentication system can help serve privacy preserving and reveal cardiovascular functioning information in clinics.

A Brain Biometric-based Identification Approach Using Local Field Potentials

Traditional biometrics such as face, iris and fingerprint have been applied widely nowadays. Nevertheless, with more and more potential problems being exposed, such as privacy leak and fabricate attack, it is urgent to find new secure biometrics to meet the needs. Identification based on brain signals is a promising option due to its unique advantages of confidentiality, anti-spoofing, continuity and cancelability. Among various types of brain signals, local field potential (LFP) has long term stability, high signal to noise ratio and high spatial resolution, which is suitable for identification. In this paper, we propose a novel biometric which is extracted from LFP signals with a deep neural network. The proposed biometric can be generated in a task-related manner thus is cancelable. Experiments with ten rats demonstrate that, the proposed biometric achieves a high identification accuracy of 94.47%, and the performance is stable over several days.

PPG-based Biometric Identification: Discovering and Identifying a New User

The convenience of Photoplethysmography (PPG) signal acquisition from wearable devices makes it becomes a hot topic in biometric identification. A majority of studies focus on PPG biometric technology in a verification application rather than an identification application. Yet, in the identification application, it is an inevitable problem in discovering and identifying a new user. However, so far few works have investigated this problem. Existing approaches can only identify trained old users. Their identification model needs to be retrained when a new user joins, which reduces the identification accuracy. This work investigates the approach and performance of identifying both old users and new users on a deep neural network trained only by old users. We used a deep neural network as a feature extractor, and the distance of the feature vector to discover and identify a new user, which avoids retraining the identification model. On the BIDMC data set, we achieved an accuracy of more than 99% for old users, an accuracy of more than 90% for discovering a new user, and an average accuracy of about 90% for identifying a new user. Our proposed approach can accurately identify old users and has feasibility in discovering and identifying a new user without retraining in the identification application.

Attend to the Difference: Cross-Modality Person Re-Identification via Contrastive Correlation

The problem of cross-modality person re-identification has been receiving increasing attention recently, due to its practical significance. Motivated by the fact that human usually attend to the difference when they compare two similar objects, we propose a dual-path cross-modality feature learning framework which preserves intrinsic spatial structures and attends to the difference of input cross-modality image pairs. Our framework is composed by two main components: a Dual-path Spatial-structure-preserving Common Space Network (DSCSN) and a Contrastive Correlation Network (CCN). The former embeds cross-modality images into a common 3D tensor space without losing spatial structures, while the latter extracts contrastive features by dynamically comparing input image pairs. Note that the representations generated for the input RGB and Infrared images are mutually dependant to each other. We conduct extensive experiments on two public available RGB-IR ReID datasets, SYSU-MM01 and RegDB, and our proposed method outperforms state-of-the-art algorithms by a large margin with both full and simplified evaluation modes.

and Their Hybrid Progenies by Electrochemical Fingerprints

The use of electrochemical fingerprints for plant identification is an emerging application in biosensors. In this work, Taxodium ascendens, T. distichum, T. mucronatum, and 18 of their hybrid progenies were collected for this purpose. This is the first attempt to use electrochemical fingerprinting for the identification of plant hybrid progeny. Electrochemical fingerprinting in the leaves of Taxodium spp. was recorded under two conditions. The results showed that the electrochemical fingerprints of each species and progeny possessed very suitable reproducibility. These electrochemical fingerprints represent the electrochemical behavior of electrochemically active substances in leaf tissues under specific conditions. Since these species and progenies are very closely related to each other, it is challenging to identify them directly using a particular electrochemical fingerprinting. Therefore, electrochemical fingerprints measured under different conditions were used to perform pattern recognition. We can identify different species and progenies by locating the features in different pattern maps. We also performed a phylogenetic study with data from electrochemical fingerprinting. The results proved that the electrochemical classification results and the relationship between them are closely related.

Vascular Auscultation of Carotid Artery: Towards Biometric Identification and Verification of Individuals

BACKGROUND

Biometric sensing is a security method for protecting information and property. State-of-the-art biometric traits are behavioral and physiological in nature. However, they are vulnerable to tampering and forgery.

METHODS

The proposed approach uses blood flow sounds in the carotid artery as a source of biometric information. A handheld sensing device and an associated desktop application were built. Between 80 and 160 carotid recordings of 11 s in length were acquired from seven individuals each. Wavelet-based signal analysis was performed to assess the potential for biometric applications.

RESULTS

The acquired signals per individual proved to be consistent within one carotid sound recording and between multiple recordings spaced by several weeks. The averaged continuous wavelet transform spectra for all cardiac cycles of one recording showed specific spectral characteristics in the time-frequency domain, allowing for the discrimination of individuals, which could potentially serve as an individual fingerprint of the carotid sound. This is also supported by the quantitative analysis consisting of a small convolutional neural network, which was able to differentiate between different users with over 95% accuracy.

CONCLUSION

The proposed approach and processing pipeline appeared promising for the discrimination of individuals. The biometrical recognition could clinically be used to obtain and highlight differences from a previously established personalized audio profile and subsequently could provide information on the source of the deviation as well as on its effects on the individual's health. The limited number of individuals and recordings require a study in a larger population along with an investigation of the long-term spectral stability of carotid sounds to assess its potential as a biometric marker. Nevertheless, the approach opens the perspective for automatic feature extraction and classification.

Learning Modal-Invariant Angular Metric by Cyclic Projection Network for VIS-NIR Person Re-Identification

Person re-identification across visible and near-infrared cameras (VIS-NIR Re-ID) has widespread applications. The challenge of this task lies in heterogeneous image matching. Existing methods attempt to learn discriminative features via complex feature extraction strategies. Nevertheless, the distributions of visible and near-infrared features are disparate caused by modal gap, which significantly affects feature metric and makes the performance of the existing models poor. To address this problem, we propose a novel approach from the perspective of metric learning. We conduct metric learning on a well-designed angular space. Geometrically, features are mapped from the original space to the hypersphere manifold, which eliminates the variations of feature norm and concentrates on the angle between the feature and the target category. Specifically, we propose a cyclic projection network (CPN) that transforms features into an angle-related space while identity information is preserved. Furthermore, we proposed three kinds of loss functions, AICAL, LAL and DAL, in angular space for angular metric learning. Multiple experiments on two existing public datasets, SYSU-MM01 and RegDB, show that performance of our method greatly exceeds the SOTA performance.

Biometrics for Internet-of-Things Security: A Review

The large number of Internet-of-Things (IoT) devices that need interaction between smart devices and consumers makes security critical to an IoT environment. Biometrics offers an interesting window of opportunity to improve the usability and security of IoT and can play a significant role in securing a wide range of emerging IoT devices to address security challenges. The purpose of this review is to provide a comprehensive survey on the current biometrics research in IoT security, especially focusing on two important aspects, authentication and encryption. Regarding authentication, contemporary biometric-based authentication systems for IoT are discussed and classified based on different biometric traits and the number of biometric traits employed in the system. As for encryption, biometric-cryptographic systems, which integrate biometrics with cryptography and take advantage of both to provide enhanced security for IoT, are thoroughly reviewed and discussed. Moreover, challenges arising from applying biometrics to IoT and potential solutions are identified and analyzed. With an insight into the state-of-the-art research in biometrics for IoT security, this review paper helps advance the study in the field and assists researchers in gaining a good understanding of forward-looking issues and future research directions.

Biometric Identification Based on Eye Movement Dynamic Features

The paper presents studies on biometric identification methods based on the eye movement signal. New signal features were investigated for this purpose. They included its representation in the frequency domain and the largest Lyapunov exponent, which characterizes the dynamics of the eye movement signal seen as a nonlinear time series. These features, along with the velocities and accelerations used in the previously conducted works, were determined for 100-ms eye movement segments. 24 participants took part in the experiment, composed of two sessions. The users’ task was to observe a point appearing on the screen in 29 locations. The eye movement recordings for each point were used to create a feature vector in two variants: one vector for one point and one vector including signal for three consecutive locations. Two approaches for defining the training and test sets were applied. In the first one, 75% of randomly selected vectors were used as the training set, under a condition of equal proportions for each participant in both sets and the disjointness of the training and test sets. Among four classifiers: kNN (k = 5), decision tree, naïve Bayes, and random forest, good classification performance was obtained for decision tree and random forest. The efficiency of the last method reached 100%. The outcomes were much worse in the second scenario when the training and testing sets when defined based on recordings from different sessions; the possible reasons are discussed in the paper.

On the Effectiveness of Impedance-Based Fingerprint Presentation Attack Detection

Within the last few decades, the need for subject authentication has grown steadily, and biometric recognition technology has been established as a reliable alternative to passwords and tokens, offering automatic decisions. However, as unsupervised processes, biometric systems are vulnerable to presentation attacks targeting the capture devices, where presentation attack instruments (PAI) instead of bona fide characteristics are presented. Due to the capture devices being exposed to the public, any person could potentially execute such attacks. In this work, a fingerprint capture device based on thin film transistor (TFT) technology has been modified to additionally acquire the impedances of the presented fingers. Since the conductance of human skin differs from artificial PAIs, those impedance values were used to train a presentation attack detection (PAD) algorithm. Based on a dataset comprising 42 different PAI species, the results showed remarkable performance in detecting most attack presentations with an APCER = 2.89% in a user-friendly scenario specified by a BPCER = 0.2%. However, additional experiments utilising unknown attacks revealed a weakness towards particular PAI species.

Transcending Conventional Biometry Frontiers: Diffusive Dynamics PPG Biometry

This paper presents the first photoplethysmographic (PPG) signal dynamic-based biometric authentication system with a Siamese convolutional neural network (CNN). Our method extracts the PPG signal's biometric characteristics from its diffusive dynamics, characterized by geometric patterns in the (p,q)-planes specific to the 0-1 test. PPG signal diffusive dynamics are strongly dependent on the vascular bed's biostructure, unique to each individual. The dynamic characteristics of the PPG signal are more stable over time than its morphological features, particularly in the presence of psychosomatic conditions. Besides its robustness, our biometric method is anti-spoofing, given the complex nature of the blood network. Our proposal trains using a national research study database with 40 real-world PPG signals measured with commercial equipment. Biometric system results for input data, raw and preprocessed, are studied and compared with eight primary biometric methods related to PPG, achieving the best equal error rate (ERR) and processing times with a single attempt, among all of them.

High-security photoacoustic identity recognition by capturing hierarchical vascular structure of finger

Currently, most biometric methods mainly use single features, making them easily forged and cracked. In this study, a novel triple-layers biometric recognition method, based on photoacoustic microscopy, is proposed to improve the security of biometric identity recognition. Using the photoacoustic (PA) dermoscope, three-dimensional absorption-structure information of the fingers was obtained. Then, by combining U-Net, Gabor filtering, wavelet analysis and morphological transform, a lightweight algorithm called photoacoustic depth feature recognition algorithm (PADFR) was developed to automatically realize stratification (the fingerprint, blood vessel fingerprint and venous vascular), extracting feature points and identity recognition. The experimental results show that PADFR can automatically recognize the PA hierarchical features with an average accuracy equal to 92.99%. The proposed method is expected to be widely used in biometric identification system due to its high security.

Relation-Based Deep Attention Network with Hybrid Memory for One-Shot Person Re-Identification

One-shot person Re-identification, which owns one labeled sample among numerous unlabeled data for each identity, is proposed to tackle the problem of the shortage of labeled data. Considering the scenarios without sufficient labeled data, it is very challenging to keep abreast of the performance of the supervised task in which sufficient labeled samples are available. In this paper, we propose a relation-based attention network with hybrid memory, which can make full use of the global information to pay attention to the identity features for model training with the relation-based attention network. Importantly, our specially designed network architecture effectively reduces the interference of environmental noise. Moreover, we propose a hybrid memory to train the one-shot data and unlabeled data in a unified framework, which notably contributes to the performance of person Re-identification. In particular, our designed one-shot feature update mode effectively alleviates the problem of overfitting, which is caused by the lack of supervised information during the training process. Compared with state-of-the-art unsupervised and one-shot algorithms for person Re-identification, our method achieves considerable improvements of 6.7%, 4.6%, and 11.5% on Market-1501, DukeMTMC-reID, and MSMT17 datasets, respectively, and becomes the new state-of-the-art method for one-shot person Re-identification.

Triple-Type Feature Extraction for Palmprint Recognition

Palmprint recognition has received tremendous research interests due to its outstanding user-friendliness such as non-invasive and good hygiene properties. Most recent palmprint recognition studies such as deep-learning methods usually learn discriminative features from palmprint images, which usually require a large number of labeled samples to achieve a reasonable good recognition performance. However, palmprint images are usually limited because it is relative difficult to collect enough palmprint samples, making most existing deep-learning-based methods ineffective. In this paper, we propose a heuristic palmprint recognition method by extracting triple types of palmprint features without requiring any training samples. We first extract the most important inherent features of a palmprint, including the texture, gradient and direction features, and encode them into triple-type feature codes. Then, we use the block-wise histograms of the triple-type feature codes to form the triple feature descriptors for palmprint representation. Finally, we employ a weighted matching-score level fusion to calculate the similarity between two compared palmprint images of triple-type feature descriptors for palmprint recognition. Extensive experimental results on the three widely used palmprint databases clearly show the promising effectiveness of the proposed method.

On the Impact of the Data Acquisition Protocol on ECG Biometric Identification

Electrocardiographic (ECG) signals have been used for clinical purposes for a long time. Notwithstanding, they may also be used as the input for a biometric identification system. Several studies, as well as some prototypes, are already based on this principle. One of the methods already used for biometric identification relies on a measure of similarity based on the Kolmogorov Complexity, called the Normalized Relative Compression (NRC)—this approach evaluates the similarity between two ECG segments without the need to delineate the signal wave. This methodology is the basis of the present work. We have collected a dataset of ECG signals from twenty participants on two different sessions, making use of three different kits simultaneously—one of them using dry electrodes, placed on their fingers; the other two using wet sensors placed on their wrists and chests. The aim of this work was to study the influence of the ECG protocol collection, regarding the biometric identification system’s performance. Several variables in the data acquisition are not controllable, so some of them will be inspected to understand their influence in the system. Movement, data collection point, time interval between train and test datasets and ECG segment duration are examples of variables that may affect the system, and they are studied in this paper. Through this study, it was concluded that this biometric identification system needs at least 10 s of data to guarantee that the system learns the essential information. It was also observed that “off-the-person” data acquisition led to a better performance over time, when compared to “on-the-person” places.

Gait-Based Implicit Authentication Using Edge Computing and Deep Learning for Mobile Devices

Implicit authentication mechanisms are expected to prevent security and privacy threats for mobile devices using behavior modeling. However, recently, researchers have demonstrated that the performance of behavioral biometrics is insufficiently accurate. Furthermore, the unique characteristics of mobile devices, such as limited storage and energy, make it subject to constrained capacity of data collection and processing. In this paper, we propose an implicit authentication architecture based on edge computing, coined Edge computing-based mobile Device Implicit Authentication (EDIA), which exploits edge-based gait biometric identification using a deep learning model to authenticate users. The gait data captured by a device’s accelerometer and gyroscope sensors is utilized as the input of our optimized model, which consists of a CNN and a LSTM in tandem. Especially, we deal with extracting the features of gait signal in a two-dimensional domain through converting the original signal into an image, and then input it into our network. In addition, to reduce computation overhead of mobile devices, the model for implicit authentication is generated on the cloud server, and the user authentication process also takes place on the edge devices. We evaluate the performance of EDIA under different scenarios where the results show that i) we achieve a true positive rate of 97.77% and also a 2% false positive rate; and ii) EDIA still reaches high accuracy with limited dataset size.

3D facial phenotyping by biometric sibling matching used in contemporary genomic methodologies

The analysis of contemporary genomic data typically operates on one-dimensional phenotypic measurements (e.g. standing height). Here we report on a data-driven, family-informed strategy to facial phenotyping that searches for biologically relevant traits and reduces multivariate 3D facial shape variability into amendable univariate measurements, while preserving its structurally complex nature. We performed a biometric identification of siblings in a sample of 424 children, defining 1,048 sib-shared facial traits. Subsequent quantification and analyses in an independent European cohort (n = 8,246) demonstrated significant heritability for a subset of traits (0.17-0.53) and highlighted 218 genome-wide significant loci (38 also study-wide) associated with facial variation shared by siblings. These loci showed preferential enrichment for active chromatin marks in cranial neural crest cells and embryonic craniofacial tissues and several regions harbor putative craniofacial genes, thereby enhancing our knowledge on the genetic architecture of normal-range facial variation.

A decentralized hybrid computing consumer authentication framework for a reliable drone delivery as a service

The thriving adoption of drones for delivering parcels, packages, medicines, etc., is surging with time. The application of drones for delivery services results in faster delivery, fuel-saving, and less energy consumption. Giant companies like Google, Amazon, Facebook, etc., are actively working on developing, testing, and improving drone-based delivery systems. So far, a lot of work has been done for improving the design, speed, operating range, security of the delivery drones, etc. However, very limited work has been done to ensure a complete and reliable last-mile delivery from the merchant's store to the hands of the actual customer. To ensure a complete and reliable last-mile delivery, a drone must authenticate the consumer before dropping the package. Therefore, in this work, we propose a consumer authentication (Consumer-Auth) hybrid computing framework for drone delivery as a service to make sure that the parcel is perfectly delivered to the intended customer. The proposed Consumer-Auth framework enables a drone to reach the exact destination by using the GPS coordinates of the customer autonomously. After reaching the exact location, the drone waits for the customer to come to the specific pinned location then it starts a two-factor consumer authentication process, i.e., one-time password (OTP) verification and face Recognition. The experimental results manifest the effectiveness of the proposed Consumer-Auth framework to ensure a complete and reliable drone-based last-mile delivery.

Deep Residual Networks for User Authentication via Hand-Object Manipulations

With the ubiquity of wearable devices, various behavioural biometrics have been exploited for continuous user authentication during daily activities. However, biometric authentication using complex hand behaviours have not been sufficiently investigated. This paper presents an implicit and continuous user authentication model based on hand-object manipulation behaviour, using a finger-and hand-mounted inertial measurement unit (IMU)-based system and state-of-the-art deep learning models. We employed three convolutional neural network (CNN)-based deep residual networks (ResNets) with multiple depths (i.e., 50, 101, and 152 layers) and two recurrent neural network (RNN)-based long short-term memory (LSTMs): simple and bidirectional. To increase ecological validity, data collection of hand-object manipulation behaviours was based on three different age groups and simple and complex daily object manipulation scenarios. As a result, both the ResNets and LSTMs models acceptably identified users’ hand behaviour patterns, with the best average accuracy of 96.31% and F1-score of 88.08%. Specifically, in the simple hand behaviour authentication scenarios, more layers in residual networks tended to show better performance without showing conventional degradation problems (the ResNet-152 > ResNet-101 > ResNet-50). In a complex hand behaviour scenario, the ResNet models outperformed user authentication compared to the LSTMs. The 152-layered ResNet and bidirectional LSTM showed an average false rejection rate of 8.34% and 16.67% and an equal error rate of 1.62% and 9.95%, respectively.

ECG-based biometric under different psychological stress states

BACKGROUND AND OBJECTIVE

In recent years, people have been exploring methods for biometric identification through electrocardiogram (ECG) signals. Under the same psychological pressure state, biometric identification through ECG signals is a traditional verification method. However, ECG signals are affected by changes in psychological stress, and ECG-Based biometric under different psychological stress states are still challenging. In this paper, we propose a method combining manual and automatic features for ECG-based biometric under different psychological stress states. And propose a new indicator Stress Classification Coefficient (SCC) that assesses the effect of different psychological stress on heart rate variability (HRV) features.

METHODS

In our method, we obtain manual features to be a three-step process: first, HRV features obtained from the ECG signals. Second, based on HRV features, the mental state of the experimental subjects is assessed by using the Gaussian mixture model (GMM). Finally, use cluster centers to process the original HRV features to reduce the Stress Classification Coefficient (SCC). Also, the one-dimensional convolutional neural network is constructed to automatically extract the implied features of ECG signals. Finally, the manual feature and the automatic feature are combined, and the final recognition result is obtained through the support vector machine (SVM) model. The major attribute of the proposed method is that it can perform ECG biometric under different psychological stress states. The combination of manual and automatic features expands the application scenarios of ECG-based biometric.

RESULTS

Based on this method, we used the Montreal stress model with calculation experiment in the laboratory to induce stress on 23 healthy students (10 women and 13 men, aged 20-37), and obtain their ECG signals under different stress conditions. Through this method to recognize the above data, an average recognition rate of more than 95% can be achieved, the average F1 score is 0.97.

CONCLUSIONS

The proposed method in this article is a promising approach to deal with the effects of different psychological stresses on ECG-Based biometric. It provides the possibility of ECG-Based biometric under different psychological stress.

Self-Relative Evaluation Framework for EEG-Based Biometric Systems

In recent years, electroencephalogram (EEG) signals have been used as a biometric modality, and EEG-based biometric systems have received increasing attention. However, due to the sensitive nature of EEG signals, the extraction of identity information through processing techniques may lead to some loss in the extracted identity information. This may impact the distinctiveness between subjects in the system. In this context, we propose a new self-relative evaluation framework for EEG-based biometric systems. The proposed framework aims at selecting a more accurate identity information when the biometric system is open to the enrollment of novel subjects. The experiments were conducted on publicly available EEG datasets collected from 108 subjects in a resting state with closed eyes. The results show that the openness condition is useful for selecting more accurate identity information.

Batch Coherence-Driven Network for Part-Aware Person Re-Identification

Existing part-aware person re-identification methods typically employ two separate steps: namely, body part detection and part-level feature extraction. However, part detection introduces an additional computational cost and is inherently challenging for low-quality images. Accordingly, in this work, we propose a simple framework named Batch Coherence-Driven Network (BCD-Net) that bypasses body part detection during both the training and testing phases while still learning semantically aligned part features. Our key observation is that the statistics in a batch of images are stable, and therefore that batch-level constraints are robust. First, we introduce a batch coherence-guided channel attention (BCCA) module that highlights the relevant channels for each respective part from the output of a deep backbone model. We investigate channel-part correspondence using a batch of training images, then impose a novel batch-level supervision signal that helps BCCA to identify part-relevant channels. Second, the mean position of a body part is robust and consequently coherent between batches throughout the training process. Accordingly, we introduce a pair of regularization terms based on the semantic consistency between batches. The first term regularizes the high responses of BCD-Net for each part on one batch in order to constrain it within a predefined area, while the second encourages the aggregate of BCD-Net's responses for all parts covering the entire human body. The above constraints guide BCD-Net to learn diverse, complementary, and semantically aligned part-level features. Extensive experimental results demonstrate that BCD-Net consistently achieves state-of-the-art performance on four large-scale ReID benchmarks.

Deep Coattention-Based Comparator for Relative Representation Learning in Person Re-Identification

Person re-identification (re-ID) favors discriminative representations over unseen shots to recognize identities in disjoint camera views. Effective methods are developed via pair-wise similarity learning to detect a fixed set of region features, which can be mapped to compute the similarity value. However, relevant parts of each image are detected independently without referring to the correlation on the other image. Also, region-based methods spatially position local features for their aligned similarities. In this article, we introduce the deep coattention-based comparator (DCC) to fuse codependent representations of paired images so as to correlate the best relevant parts and produce their relative representations accordingly. The proposed approach mimics the human foveation to detect the distinct regions concurrently across images and alternatively attends to fuse them into the similarity learning. Our comparator is capable of learning representations relative to a test shot and well-suited to reidentifying pedestrians in surveillance. We perform extensive experiments to provide the insights and demonstrate the state of the arts achieved by our method in benchmark data sets: 1.2 and 2.5 points gain in mean average precision (mAP) on DukeMTMC-reID and Market-1501, respectively.

Stacking-Based Deep Neural Network: Deep Analytic Network for Pattern Classification

Stacking-based deep neural network (S-DNN) is aggregated with pluralities of basic learning modules, one after another, to synthesize a deep neural network (DNN) alternative for pattern classification. Contrary to the DNNs trained from end to end by backpropagation (BP), each S-DNN layer, that is, a self-learnable module, is to be trained decisively and independently without BP intervention. In this paper, a ridge regression-based S-DNN, dubbed deep analytic network (DAN), along with its kernelization (K-DAN), are devised for multilayer feature relearning from the pre-extracted baseline features and the structured features. Our theoretical formulation demonstrates that DAN/K-DAN relearn by perturbing the intra/interclass variations, apart from diminishing the prediction errors. We scrutinize the DAN/K-DAN performance for pattern classification on datasets of varying domains-faces, handwritten digits, generic objects, to name a few. Unlike the typical BP-optimized DNNs to be trained from gigantic datasets by GPU, we reveal that DAN/K-DAN are trainable using only CPU even for small-scale training sets. Our experimental results show that DAN/K-DAN outperform the present S-DNNs and also the BP-trained DNNs, including multiplayer perceptron, deep belief network, etc., without data augmentation applied.

TJU-DHD: A Diverse High-Resolution Dataset for Object Detection

Vehicles, pedestrians, and riders are the most important and interesting objects for the perception modules of self-driving vehicles and video surveillance. However, the state-of-the-art performance of detecting such important objects (esp. small objects) is far from satisfying the demand of practical systems. Large-scale, rich-diversity, and high-resolution datasets play an important role in developing better object detection methods to satisfy the demand. Existing public large-scale datasets such as MS COCO collected from websites do not focus on the specific scenarios. Moreover, the popular datasets (e.g., KITTI and Citypersons) collected from the specific scenarios are limited in the number of images and instances, the resolution, and the diversity. To attempt to solve the problem, we build a diverse high-resolution dataset (called TJU-DHD). The dataset contains 115354 high-resolution images (52% images have a resolution of 1624×1200 pixels and 48% images have a resolution of at least 2, 560×1.440 pixels) and 709 330 labeled objects in total with a large variance in scale and appearance. Meanwhile, the dataset has a rich diversity in season variance, illumination variance, and weather variance. In addition, a new diverse pedestrian dataset is further built. With the four different detectors (i.e., the one-stage RetinaNet, anchor-free FCOS, two-stage FPN, and Cascade R-CNN), experiments about object detection and pedestrian detection are conducted. We hope that the newly built dataset can help promote the research on object detection and pedestrian detection in these two scenes. The dataset is available at https://github.com/tjubiit/TJU-DHD.