Human movement detection and identification using pyroelectric infrared sensors

AI-Driven Sensing Technology: Review

Machine learning and deep learning technologies are rapidly advancing the capabilities of sensing technologies, bringing about significant improvements in accuracy, sensitivity, and adaptability. These advancements are making a notable impact across a broad spectrum of fields, including industrial automation, robotics, biomedical engineering, and civil infrastructure monitoring. The core of this transformative shift lies in the integration of artificial intelligence (AI) with sensor technology, focusing on the development of efficient algorithms that drive both device performance enhancements and novel applications in various biomedical and engineering fields. This review delves into the fusion of ML/DL algorithms with sensor technologies, shedding light on their profound impact on sensor design, calibration and compensation, object recognition, and behavior prediction. Through a series of exemplary applications, the review showcases the potential of AI algorithms to significantly upgrade sensor functionalities and widen their application range. Moreover, it addresses the challenges encountered in exploiting these technologies for sensing applications and offers insights into future trends and potential advancements.

Magnetron-Sputtered Lead Titanate Thin Films for Pyroelectric Applications: Part 2-Electrical Characteristics and Characterization Methods

Pyroelectric materials are naturally electrically polarized and exhibits a built-in spontaneous polarization in their unit cell structure even in the absence of any externally applied electric field. These materials are regarded as one of the ideal detector elements for infrared applications because they have a fast response time and uniform sensitivity at room temperature across all wavelengths. Crystals of the perovskite lead titanate (PbTiO3) family show pyroelectric characteristics and undergo structural phase transitions. They have a high Curie temperature (the temperature at which the material changes from the ferroelectric (polar) to the paraelectric (nonpolar) phase), high pyroelectric coefficient, high spontaneous polarization, low dielectric constant, and constitute important component materials not only useful for infrared detection, but also with vast applications in electronic, optic, and MEMS devices. However, the preparation of large perfect and pure single crystals PbTiO3 is challenging. Additionally, difficulties arise in the application of such bulk crystals in terms of connection to processing circuits, large size, and high voltages required for their operation. In this part of the review paper, we explain the electrical behavior and characterization techniques commonly utilized to unravel the pyroelectric properties of lead titanate and its derivatives. Further, it explains how the material preparation techniques affect the electrical characteristics of resulting thin films. It also provides an in-depth discussion of the measurement of pyroelectric coefficients using different techniques.

Association between a pyroelectric infrared sensor monitoring system and a 3-dimensional accelerometer to assess movement in preweaning dairy calves

The objective of this study was to correlate movement assessed by a pyroelectric infrared sensor system in preweaning dairy calves with lying and standing time assessed by a 3D accelerometer considering the temperature-humidity index (THI). A total of 35 dairy calves (1-7 d of age) were enrolled in the study and 20 calves were included in the final analyses. The lying and standing time of the calves was monitored with a 3D accelerometer (Hobo Pendant G Data Logger, Onset Computer Corporation, USA), which was used as the gold standard reference. The infrared sensor monitoring system (IMS; Calf Monitoring System, Futuro Farming GmbH, Germany) was fixed to the fence of the calf hutch within the calf's reach. Temperature-humidity was monitored with 2 validated THI sensors inside and on outside of each calf hutch. Additionally, one THI sensor was located near the calf hutches. The observation period lasted 14 consecutive days. The average standing time assessed by the 3D accelerometer was 13.4 ± 12.7 (mean ± standard deviation) min/h and the average lying time was 46.6 (±12.7) min/h. The median (25th percentile; 75th percentile) number of movements measured by the IMS was 360 (60; 919) movements per hour. Number of movements per hour measured by the IMS was compared with data obtained with a validated 3D accelerometer. The Pearson correlation coefficient between both standing and lying time and the number of movements was r = 0.85 and r = -0.85, respectively. The Pearson correlation coefficients were only slightly influenced by THI (low THI [<68]: r = 0.86; medium THI [68-72]: r = 0.85; high THI [>72]: r = 0.81). Our data show that the number of movements of dairy calves measured by IMS were highly correlated with the chosen gold standard reference method. High THI slightly affects the measurement accuracy of IMS.

Exploiting the PIR Sensor Analog Behavior as Thermoreceptor: Movement Direction Classification Based on Spiking Neurons

Pyroelectric infrared sensors (PIR) are widely used as infrared (IR) detectors due to their basic implementation, low cost, low power, and performance. Combined with a Fresnel lens, they can be used as a binary detector in applications of presence and motion control. Furthermore, due to their features, they can be used in autonomous intelligent devices or included in robotics applications or sensor networks. In this work, two neural processing architectures are presented: (1) an analog processing approach to achieve the behavior of a presynaptic neuron from a PIR sensor. An analog circuit similar to the leaky integrate and fire model is implemented to be able to generate spiking rates proportional to the IR stimuli received at a PIR sensor. (2) An embedded postsynaptic neuron where a spiking neural network matrix together with an algorithm based on digital processing techniques is introduced. This structure allows connecting a set of sensors to the post-synaptic circuit emulating an optic nerve. As a case study, the entire neural processing approach presented in this paper is applied to optical flow detection considering a four-PIR array as input. The results validate both the spiking approach for an analog sensor presented and the ability to retrieve the analog information sent as spike trains in a simulated optic nerve.

Edge-Based Transfer Learning for Classroom Occupancy Detection in a Smart Campus Context

Studies and systems that are aimed at the identification of the presence of people within an indoor environment and the monitoring of their activities and flows have been receiving more attention in recent years, specifically since the beginning of the COVID-19 pandemic. This paper proposes an approach for people counting that is based on the use of cameras and Raspberry Pi platforms, together with an edge-based transfer learning framework that is enriched with specific image processing strategies, with the aim of this approach being adopted in different indoor environments without the need for tailored training phases. The system was deployed on a university campus, which was chosen as the case study. The proposed system was able to work in classrooms with different characteristics. This paper reports a proposed architecture that could make the system scalable and privacy compliant and the evaluation tests that were conducted in different types of classrooms, which demonstrate the feasibility of this approach. Overall, the system was able to count the number of people in classrooms with a maximum mean absolute error of 1.23.

Empirical Study on Human Movement Classification Using Insole Footwear Sensor System and Machine Learning

This paper presents a plantar pressure sensor system (P2S2) integrated in the insoles of shoes to detect thirteen commonly used human movements including walking, stooping left and right, pulling a cart backward, squatting, descending, ascending stairs, running, and falling (front, back, right, left). Six force sensitive resistors (FSR) sensors were positioned on critical pressure points on the insoles to capture the electrical signature of pressure change in the various movements. A total of 34 adult participants were tested with the P2S2. The pressure data were collected and processed using a Principal Component Analysis (PCA) for input to the multiple machine learning (ML) algorithms, including k-NN, neural network and Support-Vector Machine (SVM) algorithms. The ML models were trained using four-fold cross-validation. Each fold kept subject data independent from other folds. The model proved effective with an accuracy of 86%, showing a promising result in predicting human movements using the P2S2 integrated in shoes.

Anthropometric Ratios for Lower-Body Detection Based on Deep Learning and Traditional Methods

Lower-body detection can be useful in many applications, such as the detection of falling and injuries during exercises. However, it can be challenging to detect the lower-body, especially under various lighting and occlusion conditions. This paper presents a novel lower-body detection framework using proposed anthropometric ratios and compares the performance of deep learning (convolutional neural networks and OpenPose) and traditional detection methods. According to the results, the proposed framework helps to successfully detect the accurate boundaries of the lower-body under various illumination and occlusion conditions for lower-limb monitoring. The proposed framework of anthropometric ratios combined with convolutional neural networks (A-CNNs) also achieves high accuracy (90.14%), while the combination of anthropometric ratios and traditional techniques (A-Traditional) for lower-body detection shows satisfactory performance with an averaged accuracy (74.81%). Although the accuracy of OpenPose (95.82%) is higher than the A-CNNs for lower-body detection, the A-CNNs provides lower complexity than the OpenPose, which is advantageous for lower-body detection and implementation on monitoring systems.

The Effect of Graphite Additives on Magnetization, Resistivity and Electrical Conductivity of Magnetorheological Plastomer

Common sensors in many applications are in the form of rigid devices that can react according to external stimuli. However, a magnetorheological plastomer (MRP) can offer a new type of sensing capability, as it is flexible in shape, soft, and responsive to an external magnetic field. In this study, graphite (Gr) particles are introduced into an MRP as an additive, to investigate the advantages of its electrical properties in MRPs, such as conductivity, which is absolutely required in a potential sensor. As a first step to achieve this, MRP samples containing carbonyl iron particles (CIPs) and various amounts of of Gr, from 0 to 10 wt.%, are prepared, and their magnetic-field-dependent electrical properties are experimentally evaluated. After the morphological aspect of Gr-MRP is characterized using environmental scanning electron microscopy (ESEM), the magnetic properties of MRP and Gr-MRP are evaluated via a vibrating sample magnetometer (VSM). The resistivities of the Gr-MRP samples are then tested under various applied magnetic flux densities, showing that the resistivity of Gr-MRP decreases with increasing of Gr content up to 10 wt.%. In addition, the electrical conductivity is tested using a test rig, showing that the conductivity increases as the amount of Gr additive increases, up to 10 wt.%. The conductivity of 10 wt.% Gr-MRP is found to be highest, at 178.06% higher than the Gr-MRP with 6 wt.%, for a magnetic flux density of 400 mT. It is observed that with the addition of Gr, the conductivity properties are improved with increases in the magnetic flux density, which could contribute to the potential usefulness of these materials as sensing detection devices.

Deep Learning-Based Industry 4.0 and Internet of Things towards Effective Energy Management for Smart Buildings

Worldwide, energy consumption and saving represent the main challenges for all sectors, most importantly in industrial and domestic sectors. The internet of things (IoT) is a new technology that establishes the core of Industry 4.0. The IoT enables the sharing of signals between devices and machines via the internet. Besides, the IoT system enables the utilization of artificial intelligence (AI) techniques to manage and control the signals between different machines based on intelligence decisions. The paper’s innovation is to introduce a deep learning and IoT based approach to control the operation of air conditioners in order to reduce energy consumption. To achieve such an ambitious target, we have proposed a deep learning-based people detection system utilizing the YOLOv3 algorithm to count the number of persons in a specific area. Accordingly, the operation of the air conditioners could be optimally managed in a smart building. Furthermore, the number of persons and the status of the air conditioners are published via the internet to the dashboard of the IoT platform. The proposed system enhances decision making about energy consumption. To affirm the efficacy and effectiveness of the proposed approach, intensive test scenarios are simulated in a specific smart building considering the existence of air conditioners. The simulation results emphasize that the proposed deep learning-based recognition algorithm can accurately detect the number of persons in the specified area, thanks to its ability to model highly non-linear relationships in data. The detection status can also be successfully published on the dashboard of the IoT platform. Another vital application of the proposed promising approach is in the remote management of diverse controllable devices.

On exploiting Data Visualization and IoT for Increasing Sustainability and Safety in a Smart Campus

In a world that is getting increasingly digital and interconnected, and where more and more physical objects are integrated into the information network (Internet of Things, IoT), Data Visualization can facilitate the understanding of huge volumes of data. In this paper, we present the design and implementation of a testbed where IoT and Data Visualization have been exploited to increase the sustainability and safety of the Cesena (Smart) Campus. In particular, we detail the overall system architecture and the interactive dashboard that facilitates the management of the campus premises and the timetabling. Exploiting our system, we show how we can improve the campus sustainability (in terms of energy saving) and safety (considering the COVID-19 restrictions and regulations).

Characterizing Variations in the Indoor Temperature and Humidity of Guest Rooms with an Occupancy-Based Climate Control Technology

This paper characterizes variations in the indoor temperature and humidity profiles of actual guest rooms equipped with Occupancy-Based Climate Control (OBCC) systems that were used to initiate a temperature setback to 15.6 °C in the winter and to 26.7 °C in the summer in the guest rooms. Empirical knowledge of these conditions can provide useful insights for an improved field demonstration and optimization of OBCC, as well as for a more realistic temperature and occupancy input for building simulations for hotel guest rooms. As a result, one year of one minute temperatures and humidity data was characterized against outdoor climate for three different occupancy modes, which was useful to identify the observed room-to-room variations in heat losses and resultant indoor temperatures during the heating season due to the different dynamic heat balance conditions of the guest rooms. This indicated potential discomfort in the rooms that appeared to have a stronger association between outdoor and indoor temperatures, which was also identified from the thermal comfort survey indicating thermostat-related discomfort sources. Interestingly enough, the guests who stayed in these rooms tended to set their thermostat at higher setpoint temperatures when they occupied the room, which appeared to compensate for the low balance-point temperatures of these rooms.

Motion sensors achieved from a conducting polymer-metal Schottky contact

Mechanical-to-electrical energy conversion devices show potential applications in the detection of movements. Previous studies on these sensor devices are mainly based on piezoelectricity or triboelectricity, which typically generates AC signals. In this study, a movement sensor that generated DC signals based on a conducting polymer-metal Schottky diode was prepared for the first time. Using the Al|poly(3,4-ethylenedioxythiophene)|Au device as a model, we showed that the sensor device could detect the touch and sliding movements. Both the pressure of the Al electrode touching the PEDOT surface and its sliding speed affected the voltage outputs. The device showed a high response speed of 1.7 s at 39.8 kPa. The modified device can even measure the sliding speed. The DC output allows the use of electrical energy for running other electronic devices. A conducting polymer-metal Schottky contact may be useful for the development of DC output movement sensors.

Mechanical-to-electrical energy conversion devices show potential applications in the detection of movements.

Multi-Residential Activity Labelling in Smart Homes with Wearable Tags Using BLE Technology

Smart home platforms show promising outcomes to provide a better quality of life for residents in their homes. One of the main challenges that exists with these platforms in multi-residential houses is activity labeling. As most of the activity sensors do not provide any information regarding the identity of the person who triggers them, it is difficult to label the sensor events in multi-residential smart homes. To deal with this challenge, individual localization in different areas can be a promising solution. The localization information can be used to automatically label the activity sensor data to individuals. Bluetooth low energy (BLE) is a promising technology for this application due to how easy it is to implement and its low energy footprint. In this approach, individuals wear a tag that broadcasts its unique identity (ID) in certain time intervals, while fixed scanners listen to the broadcasting packet to localize the tag and the individual. However, the localization accuracy of this method depends greatly on different settings of broadcasting signal strength, and the time interval of BLE tags. To achieve the best localization accuracy, this paper studies the impacts of different advertising time intervals and power levels, and proposes an efficient and applicable algorithm to select optimal value settings of BLE sensors. Moreover, it proposes an automatic activity labeling method, through integrating BLE localization information and ambient sensor data. The applicability and effectiveness of the proposed structure is also demonstrated in a real multi-resident smart home scenario.

Research on the Multiple Factors Influencing Human Identification Based on Pyroelectric Infrared Sensors

Analysis of the multiple factors affecting human identification ability based on pyroelectric infrared technology is a complex problem. First, we examine various sensed pyroelectric waveforms of the human body thermal infrared signal and reveal a mechanism for affecting human identification. Then, we find that the mechanism is decided by the distance, human target, pyroelectric infrared (PIR) sensor, the body type, human moving velocity, signal modulation mask, and Fresnel lens. The mapping relationship between the sensed waveform and multiple influencing factors is established, and a group of mathematical models are deduced which fuse the macro factors and micro factors. Finally, the experimental results show the macro-factors indirectly affect the recognition ability of human based on the pyroelectric technology. At the same time, the correctness and effectiveness of the mathematical models is also verified, which make it easier to obtain more pyroelectric infrared information about the human body for discriminating human targets.

An improved camera trap for amphibians, reptiles, small mammals, and large invertebrates

Camera traps are valuable sampling tools commonly used to inventory and monitor wildlife communities but are challenged to reliably sample small animals. We introduce a novel active camera trap system enabling the reliable and efficient use of wildlife cameras for sampling small animals, particularly reptiles, amphibians, small mammals and large invertebrates. It surpasses the detection ability of commonly used passive infrared (PIR) cameras for this application and eliminates problems such as high rates of false triggers and high variability in detection rates among cameras and study locations. Our system, which employs a HALT trigger, is capable of coupling to digital PIR cameras and is designed for detecting small animals traversing small tunnels, narrow trails, small clearings and along walls or drift fencing.

Graphene-based mid-infrared room-temperature pyroelectric bolometers with ultrahigh temperature coefficient of resistance

There is a growing number of applications demanding highly sensitive photodetectors in the mid-infrared. Thermal photodetectors, such as bolometers, have emerged as the technology of choice, because they do not need cooling. The performance of a bolometer is linked to its temperature coefficient of resistance (TCR, ∼2–4% K⁻¹ for state-of-the-art materials). Graphene is ideally suited for optoelectronic applications, with a variety of reported photodetectors ranging from visible to THz frequencies. For the mid-infrared, graphene-based detectors with TCRs ∼4–11% K⁻¹ have been demonstrated. Here we present an uncooled, mid-infrared photodetector, where the pyroelectric response of a LiNbO₃ crystal is transduced with high gain (up to 200) into resistivity modulation for graphene. This is achieved by fabricating a floating metallic structure that concentrates the pyroelectric charge on the top-gate capacitor of the graphene channel, leading to TCRs up to 900% K⁻¹, and the ability to resolve temperature variations down to 15 μK.

There is emerging interest in photodetectors in the mid-infrared driven by increasing need to monitor the environment for security and healthcare purposes. Sassi et al. show a thermal photodetector, based on the coupling between graphene and a pyroelectric crystal, which shows high temperature sensitivity.

A Tagless Indoor Localization System Based on Capacitive Sensing Technology

Accurate indoor person localization is essential for several services, such as assisted living. We introduce a tagless indoor person localization system based on capacitive sensing and localization algorithms that can determine the location with less than 0.2 m average error in a 3 m × 3 m room and has recall and precision better than 70%. We also discuss the effects of various noise types on the measurements and ways to reduce them using filters suitable for on-sensor implementation to lower communication energy consumption. We also compare the performance of several standard localization algorithms in terms of localization error, recall, precision, and accuracy of detection of the movement trajectory.

Abnormal Activity Detection Using Pyroelectric Infrared Sensors

Healthy aging is one of the most important social issues. In this paper, we propose a method for abnormal activity detection without any manual labeling of the training samples. By leveraging the Field of View (FOV) modulation, the spatio-temporal characteristic of human activity is encoded into low-dimension data stream generated by the ceiling-mounted Pyroelectric Infrared (PIR) sensors. The similarity between normal training samples are measured based on Kullback-Leibler (KL) divergence of each pair of them. The natural clustering of normal activities is discovered through a self-tuning spectral clustering algorithm with unsupervised model selection on the eigenvectors of a modified similarity matrix. Hidden Markov Models (HMMs) are employed to model each cluster of normal activities and form feature vectors. One-Class Support Vector Machines (OSVMs) are used to profile the normal activities and detect abnormal activities. To validate the efficacy of our method, we conducted experiments in real indoor environments. The encouraging results show that our method is able to detect abnormal activities given only the normal training samples, which aims to avoid the laborious and inconsistent data labeling process.

EMD-Based Symbolic Dynamic Analysis for the Recognition of Human and Nonhuman Pyroelectric Infrared Signals

In this paper, we propose an effective human and nonhuman pyroelectric infrared (PIR) signal recognition method to reduce PIR detector false alarms. First, using the mathematical model of the PIR detector, we analyze the physical characteristics of the human and nonhuman PIR signals; second, based on the analysis results, we propose an empirical mode decomposition (EMD)-based symbolic dynamic analysis method for the recognition of human and nonhuman PIR signals. In the proposed method, first, we extract the detailed features of a PIR signal into five symbol sequences using an EMD-based symbolization method, then, we generate five feature descriptors for each PIR signal through constructing five probabilistic finite state automata with the symbol sequences. Finally, we use a weighted voting classification strategy to classify the PIR signals with their feature descriptors. Comparative experiments show that the proposed method can effectively classify the human and nonhuman PIR signals and reduce PIR detector’s false alarms.