A CNN-LSTM neural network for recognition of puffing in smoking episodes using wearable sensors

Pharmacokinetics-Informed Neural Network for Predicting Opioid Administration Moments with Wearable Sensors

Long-term and high-dose prescription opioid use places individuals at risk for opioid misuse, opioid use disorder (OUD), and overdose. Existing methods for monitoring opioid use and detecting misuse rely on self-reports, which are prone to reporting bias, and toxicology testing, which may be infeasible in outpatient settings. Although wearable technologies for monitoring day-to-day health metrics have gained significant traction in recent years due to their ease of use, flexibility, and advancements in sensor technology, their application within the opioid use space remains underexplored. In the current work, we demonstrate that oral opioid administrations can be detected using physiological signals collected from a wrist sensor. More importantly, we show that models informed by opioid pharmacokinetics increase reliability in predicting the timing of opioid administrations. Forty-two individuals who were prescribed opioids as a part of their medical treatment in-hospital and after discharge were enrolled. Participants wore a wrist sensor throughout the study, while opioid administrations were tracked using electronic medical records and self-reports. We collected 1,983 hours of sensor data containing 187 opioid administrations from the inpatient setting and 927 hours of sensor data containing 40 opioid administrations from the outpatient setting. We demonstrate that a self-supervised pre-trained model, capable of learning the canonical time series of plasma concentration of the drug derived from opioid pharmacokinetics, can reliably detect opioid administration in both settings. Our work suggests the potential of pharmacokinetic-informed, data-driven models to objectively detect opioid use in daily life.

Smoking behavior detection algorithm based on YOLOv8-MNC

Introduction

The detection of smoking behavior is an emerging field faced with challenges in identifying small, frequently occluded objects like cigarette butts using existing deep learning technologies. Such challenges have led to unsatisfactory detection accuracy and poor model robustness.

Methods

To overcome these issues, this paper introduces a novel smoking detection algorithm, YOLOv8-MNC, which builds on the YOLOv8 network and includes a specialized layer for small target detection. The YOLOv8-MNC algorithm employs three key strategies: (1) It utilizes NWD Loss to mitigate the effects of minor deviations in object positions on IoU, thereby enhancing training accuracy; (2) It incorporates the Multi-head Self-Attention Mechanism (MHSA) to bolster the network's global feature learning capacity; and (3) It implements the lightweight general up-sampling operator CARAFE, in place of conventional nearest-neighbor interpolation up-sampling modules, minimizing feature information loss during the up-sampling process.

Results

Experimental results from a customized smoking behavior dataset demonstrate significant improvement in detection accuracy. The YOLOv8-MNC model achieved a detection accuracy of 85.887%, signifying a remarkable increase of 5.7% in the mean Average Precision (mAP@0.5) when compared to the previous algorithm.

Discussion

The YOLOv8-MNC algorithm represents a valuable step forward in resolving existing problems in smoking behavior detection. Its enhanced performance in both detection accuracy and robustness indicates potential applicability in related fields, thus illustrating a meaningful advancement in the sphere of smoking behavior detection. Future efforts will focus on refining this technique and exploring its application in broader contexts.

CigTrak: Smartwatch-Based Accurate Online Smoking Puff & Episode Detection with Gesture-Focused Windowing for CNN

Wearable-based motion sensing solutions are capable of automatically detecting and tracking individual smoking puffs and/or episodes to aid the users in their journey of smoking cessation. But they are either obtrusive to use, perform with a low accuracy, or have questionable ability of running fully on a low-power device like a smartwatch, all affecting their widespread adoption. We propose 'CigTrak', a novel pipeline for an accurate smoking puff and episode detection using 6-DoF motion sensor on a smartwatch. A multi-stage method for puff detection is devised, comprising a novel kinematic analysis of puffing motion enabling temporal localization of puff. A Convolutional Neural Network (CNN)-backed model uses this candidate puff as an input instance by re-sampling it to required input size for the final decision. Clusters of detected puffs are further used to detect episodes. Data from 13 subjects was used for evaluating puff detection, and 9 subjects for evaluating episode detection. CigTrak achieved a high subject-independent performance for puff detection (F1-score 0.94) and free-living episode detection (F1-score 0.89), surpassing state of the art performance. CigTrak was also implemented fully online on two different smartwatches for testing a real-time puff detection.Clinical Relevance- Cigarette smoking affects physical & mental well-being of a person, and is the leading cause of preventable diseases, adversely affecting cardiac and respiratory systems. With many adults wanting to quit smoking [1], a reliable way of auto-journaling of smoking activities can greatly aid in cessation efforts through self-help, and reduce burden on healthcare industry. CigTrak, with its high accuracy in detecting smoking puffs and episodes, and capability of running fully on a smartwatch, can be readily used for this purpose.

DeepPuff: Utilizing Deep Learning for Smoking Behavior Identification in Free-living Environment

A comprehensive assessment of cigarette smoking behavior and its effect on health requires a detailed examination of smoke exposure. We propose a CNN-LSTM-based deep learning architecture named DeepPuff to quantify Respiratory Smoke Exposure Metrics (RSEM). Smoke inhalations were detected from the breathing and hand gesture sensors of the Personal Automatic Cigarette Tracker v2 (PACT 2.0). The DeepPuff model for smoke inhalation detection was developed using data collected from 190 cigarette smoking events from 38 medium to heavy smokers and optimized for precision (avoidance of false positives). An independent dataset of 459 smoking events from 45 participants (90 smoking events in the lab and 369 smoking events in free-living conditions) was used for testing the model. The proposed model achieved a precision of 82.39% on the training and 93.80% on the testing dataset (95.88% in the lab and 93.78% in free-living). RSEM metrics were then computed from the breathing signal of each detected smoke inhalation. Results from the RSEM algorithm were compared with respiratory metrics obtained from video annotation. Smoke exposure metrics of puff duration, inhale-exhale duration, and inhalation duration were not statistically different from the ground truth generated through video annotation. The results suggest that DeepPuff may be used as a reliable means to measure respiratory smoke exposure metrics collected under free-living conditions.

Machine learning applications in tobacco research: a scoping review

OBJECTIVE

Identify and review the body of tobacco research literature that self-identified as using machine learning (ML) in the analysis.

DATA SOURCES

MEDLINE, EMABSE, PubMed, CINAHL Plus, APA PsycINFO and IEEE Xplore databases were searched up to September 2020. Studies were restricted to peer-reviewed, English-language journal articles, dissertations and conference papers comprising an empirical analysis where ML was identified to be the method used to examine human experience of tobacco. Studies of genomics and diagnostic imaging were excluded.

STUDY SELECTION

Two reviewers independently screened the titles and abstracts. The reference list of articles was also searched. In an iterative process, eligible studies were classified into domains based on their objectives and types of data used in the analysis.

DATA EXTRACTION

Using data charting forms, two reviewers independently extracted data from all studies. A narrative synthesis method was used to describe findings from each domain such as study design, objective, ML classes/algorithms, knowledge users and the presence of a data sharing statement. Trends of publication were visually depicted.

DATA SYNTHESIS

74 studies were grouped into four domains: ML-powered technology to assist smoking cessation (n=22); content analysis of tobacco on social media (n=32); smoker status classification from narrative clinical texts (n=6) and tobacco-related outcome prediction using administrative, survey or clinical trial data (n=14). Implications of these studies and future directions for ML researchers in tobacco control were discussed.

CONCLUSIONS

ML represents a powerful tool that could advance the research and policy decision-making of tobacco control. Further opportunities should be explored.

Analyzing Impact of Mouthpiece-based Puff Topography Devices on Smoking Behavior using Wearable Sensors

Detailed assessment of smoking topography (puffing and post-puffing metrics) can lead to a better understanding of factors that influence tobacco use. Research suggests that portable mouthpiece-based devices used for puff topography measurement may alter natural smoking behavior. This paper evaluated the impact of a portable puff topography device (CReSS Pocket) on puffing & post-puffing topography using a wearable system, the Personal Automatic Cigarette Tracker v2 (PACT 2.0) as a reference measurement. Data from 45 smokers who smoked one cigarette in the lab and an unrestricted number of cigarettes under free-living conditions over 4 consecutive days were used for analysis. PACT 2.0 was worn on all four days. A puff topography instrument (CReSS pocket) was used for cigarette smoking on two random days during the four days of study in the laboratory and free-living conditions. Smoke inhalations were automatically detected using PACT2.0 signals. Respiratory smoke exposure metrics (i.e., puff count, duration of cigarette, puff duration, inhale-exhale duration, inhale-exhale volume, volume over time, smoke hold duration, inter-puff interval) were computed for each puff/smoke inhalation. Analysis comparing respiratory smoke exposure metrics during CReSS days and days without CReSS revealed a significant difference in puff duration, inhale-exhale duration and volume, smoke hold duration, inter-puff interval, and volume over time. However, the number of cigarettes per day and number of puffs per cigarette were statistically the same irrespective of the use of the CReSS device. The results suggested that the use of mouthpiece-based puff topography devices may influence measures of smoking topography with corresponding changes in smoking behavior and smoke exposure.

RNN-based deep learning for physical activity recognition using smartwatch sensors: A case study of simple and complex activity recognition

Currently, identification of complex human activities is experiencing exponential growth through the use of deep learning algorithms. Conventional strategies for recognizing human activity generally rely on handcrafted characteristics from heuristic processes in time and frequency domains. The advancement of deep learning algorithms has addressed most of these issues by automatically extracting features from multimodal sensors to correctly classify human physical activity. This study proposed an attention-based bidirectional gated recurrent unit as Att-BiGRU to enhance recurrent neural networks. This deep learning model allowed flexible forwarding and reverse sequences to extract temporal-dependent characteristics for efficient complex activity recognition. The retrieved temporal characteristics were then used to exemplify essential information through an attention mechanism. A human activity recognition (HAR) methodology combined with our proposed model was evaluated using the publicly available datasets containing physical activity data collected by accelerometers and gyroscopes incorporated in a wristwatch. Simulation experiments showed that attention mechanisms significantly enhanced performance in recognizing complex human activity.

MODIS Land Surface Temperature Product Reconstruction Based on the SSA-BiLSTM Model

Land surface temperature (LST) is an important parameter indispensable for studying the substance and energy exchanges between the land surface and the atmosphere, climate changes, and other related aspects. However, due to cloud cover, there are many null values in MODIS (Moderate Resolution Imaging Spectroradiometer) LST data, which prevents such data from being widely used. Therefore, an LST reconstruction method is proposed by combining data decomposition with data prediction—SSA (Singular Spectrum Analysis) and BiLSTM (Bidirectional Long Short-Term Memory). This method consists of two major processes, namely, rough LST reconstruction based on the SSA model and refined LST reconstruction based on the BiLSTM model. The accuracy of the proposed method is verified through “removal–reconstruction–comparison” using remote sensing data and measured data. The verification results show that when the rate of original missing values in the LST time series for the study area is lower than 10%, the RMSE is smaller than 1.1 K, and the correlation coefficient is more significant than 0.98. Even when the rate of missing data is 40% and 50%, the proposed method remains accurate, the values of RMSE are 1.8331 K and 2.2929 K, and the importance of R2 are 0.9856 and 0.9800, respectively. The proposed method is compared with other existing LST reconstruction methods. The results of the comparative analysis indicate that the proposed method is superior to other methods in terms of reconstruction accuracy and stability. Moreover, the LST data reconstructed using the proposed method are highly consistent with the measured data, which further proves the accuracy of this method in LST reconstruction. The research findings provide a new technique and idea for accurate LST reconstruction.

Deep Learning in Human Activity Recognition with Wearable Sensors: A Review on Advances

Mobile and wearable devices have enabled numerous applications, including activity tracking, wellness monitoring, and human-computer interaction, that measure and improve our daily lives. Many of these applications are made possible by leveraging the rich collection of low-power sensors found in many mobile and wearable devices to perform human activity recognition (HAR). Recently, deep learning has greatly pushed the boundaries of HAR on mobile and wearable devices. This paper systematically categorizes and summarizes existing work that introduces deep learning methods for wearables-based HAR and provides a comprehensive analysis of the current advancements, developing trends, and major challenges. We also present cutting-edge frontiers and future directions for deep learning-based HAR.

Real-Time Littering Activity Monitoring Based on Image Classification Method

This paper describes the implementation of real time human activity recognition systems in public areas. The objective of the study is to develop an alarm system to identify people who do not care for their surrounding environment. In this research, the actions recognized are limited to littering activity using two methods, i.e., CNN and CNN-LSTM. The proposed system captures, classifies, and recognizes the activity by using two main components, a namely camera and mini-PC. The proposed system was implemented in two locations, i.e., Sekanak River and the mini garden near the Sekanak market. It was able to recognize the littering activity successfully. Based on the proposed model, the validation results from the prediction of the testing data in simulation show a loss value of 70% and an accuracy value of 56% for CNN of model 8 that used 500 epochs and a loss value of 10.61%, and an accuracy value of 97% for CNN-LSTM that used 100 epochs. For real experiment of CNN model 8, it is obtained 66.7% and 75% success for detecting littering activity at mini garden and Sekanak River respectively, while using CNN-LSTM in real experiment sequentially gives 94.4% and 100% success for mini garden and Sekanak river.

Effect of a Recliner Chair with Rocking Motions on Sleep Efficiency

In this study, we analyze the effect of a recliner chair with rocking motions on sleep quality of naps using automated sleep scoring and spindle detection models. The quality of sleep corresponding to the two rocking motions was measured quantitatively and qualitatively. For the quantitative evaluation, we conducted a sleep parameter analysis based on the results of the estimated sleep stages obtained on the brainwave and spindle estimation, and a sleep survey assessment from the participants was analyzed for the qualitative evaluation. The analysis showed that sleep in the recliner chair with rocking motions positively increased the duration of the spindles and deep sleep stage, resulting in improved sleep quality.

Dynamic models of stress-smoking responses based on high-frequency sensor data

Self-reports indicate that stress increases the risk for smoking; however, intensive data from sensors can provide a more nuanced understanding of stress in the moments leading up to and following smoking events. Identifying personalized dynamical models of stress-smoking responses can improve characterizations of smoking responses following stress, but techniques used to identify these models require intensive longitudinal data. This study leveraged advances in wearable sensing technology and digital markers of stress and smoking to identify person-specific models of stress and smoking system dynamics by considering stress immediately before, during, and after smoking events. Adult smokers (n = 45) wore the AutoSense chestband (respiration-inductive plethysmograph, electrocardiogram, accelerometer) with MotionSense (accelerometers, gyroscopes) on each wrist for three days prior to a quit attempt. The odds of minute-level smoking events were regressed on minute-level stress probabilities to identify person-specific dynamic models of smoking responses to stress. Simulated pulse responses to a continuous stress episode revealed a consistent pattern of increased odds of smoking either shortly after the beginning of the simulated stress episode or with a delay, for all participants. This pattern is followed by a dramatic reduction in the probability of smoking thereafter, for about half of the participants (49%). Sensor-detected stress probabilities indicate a vulnerability for smoking that may be used as a tailoring variable for just-in-time interventions to support quit attempts.

CNN-based bi-directional and directional long-short term memory network for determination of face mask

Context

The COVID-19 virus, exactly like in numerous other diseases, can be contaminated from person to person by inhalation. In order to prevent the spread of this virus, which led to a pandemic around the world, a series of rules have been set by governments that people must follow. The obligation to use face masks, especially in public spaces, is one of these rules.

Objective

The aim of this study is to determine whether people are wearing the face mask correctly by using deep learning methods.

Methods

A dataset consisting of 2000 images was created. In the dataset, images of a person from three different angles were collected in four classes, which are “masked”, “non-masked”, “masked but nose open”, and “masked but under the chin”. Using this data, new models are proposed by transferring the learning through AlexNet and VGG16, which are the Convolutional Neural network architectures. Classification layers of these models were removed and, Long-Short Term Memory and Bi-directional Long-Short Term Memory architectures were added instead.

Result and conclusions

Although there are four different classes to determine whether the face masks are used correctly, in the six models proposed, high success rates have been achieved. Among all models, the TrVGG16 + BiLSTM model has achieved the highest classification accuracy with 95.67%.

Significance

The study has proven that it can take advantage of the proposed models in conjunction with transfer learning to ensure the proper and effective use of the face mask, considering the benefit of society.

OpiTrack: A Wearable-based Clinical Opioid Use Tracker with Temporal Convolutional Attention Networks

Opioid use disorder is a medical condition with major social and economic consequences. While ubiquitous physiological sensing technologies have been widely adopted and extensively used to monitor day-to-day activities and deliver targeted interventions to improve human health, the use of these technologies to detect drug use in natural environments has been largely underexplored. The long-term goal of our work is to develop a mobile technology system that can identify high-risk opioid-related events (i.e., development of tolerance in the setting of prescription opioid use, return-to-use events in the setting of opioid use disorder) and deploy just-in-time interventions to mitigate the risk of overdose morbidity and mortality. In the current paper, we take an initial step by asking a crucial question: Can opioid use be detected using physiological signals obtained from a wrist-mounted sensor? Thirty-six individuals who were admitted to the hospital for an acute painful condition and received opioid analgesics as part of their clinical care were enrolled. Subjects wore a noninvasive wrist sensor during this time (1-14 days) that continuously measured physiological signals (heart rate, skin temperature, accelerometry, electrodermal activity, and interbeat interval). We collected a total of 2070 hours (≈ 86 days) of physiological data and observed a total of 339 opioid administrations. Our results are encouraging and show that using a Channel-Temporal Attention TCN (CTA-TCN) model, we can detect an opioid administration in a time-window with an F1-score of 0.80, a specificity of 0.77, sensitivity of 0.80, and an AUC of 0.77. We also predict the exact moment of administration in this time-window with a normalized mean absolute error of 8.6% and R ² coefficient of 0.85.

Real-Time Food Intake Monitoring Using Wearable Egocnetric Camera

With technological advancement, wearable egocentric camera systems have extensively been studied to develop food intake monitoring devices for the assessment of eating behavior. This paper provides a detailed description of the implementation of CNN based image classifier in the Cortex-M7 microcontroller. The proposed network classifies the captured images by the wearable egocentric camera as food and no food images in real-time. This real-time food image detection can potentially lead the monitoring devices to consume less power, less storage, and more user-friendly in terms of privacy by saving only images that are detected as food images. A derivative of pre-trained MobileNet is trained to detect food images from camera captured images. The proposed network needs 761.99KB of flash and 501.76KB of RAM to implement which is built for an optimal trade-off between accuracy, computational cost, and memory footprint considering implementation on a Cortex-M7 microcontroller. The image classifier achieved an average precision of 82%±3% and an average F-score of 74%±2% while testing on 15343 (2127 food images and 13216 no food images) images of five full days collected from five participants.