RisQ: Recognizing Smoking Gestures with Inertial Sensors on a Wristband

Smartband-based smoking detection and real-time brief mindfulness intervention: findings from a feasibility clinical trial

BACKGROUND

Smartbands can be used to detect cigarette smoking and deliver real time smoking interventions. Brief mindfulness interventions have been found to reduce smoking.

OBJECTIVE

This single arm feasibility trial used a smartband to detect smoking and deliver brief mindfulness exercises.

METHODS

Daily smokers who were motivated to reduce their smoking wore a smartband for 60 days. For 21 days, the smartband monitored, detected and notified the user of smoking in real time. After 21 days, a 'mindful smoking' exercise was triggered by detected smoking. After 28 days, a 'RAIN' (recognize, allow, investigate, nonidentify) exercise was delivered to predicted smoking. Participants received mindfulness exercises by text message and online mindfulness training. Feasibility measures included treatment fidelity, adherence and acceptability.

RESULTS

Participants (N=155) were 54% female, 76% white non-Hispanic, and treatment starters (n=115) were analyzed. Treatment fidelity cutoffs were met, including for detecting smoking and delivering mindfulness exercises. Adherence was mixed, including moderate smartband use and low completion of mindfulness exercises. Acceptability was mixed, including high helpfulness ratings and mixed user experiences data. Retention of treatment starters was high (81.9%).

CONCLUSIONS

Findings demonstrate the feasibility of using a smartband to track smoking and deliver quit smoking interventions contingent on smoking.

Personalized food consumption detection with deep learning and Inertial Measurement Unit sensor

For individuals diagnosed with diabetes mellitus, it is crucial to keep a record of the carbohydrates consumed during meals, as this should be done at least three times daily, amounting to an average of six meals. Unfortunately, many individuals tend to overlook this essential task. For those who use an artificial pancreas, carbohydrate intake proves to be a critical factor, as it can activate the insulin pump in the artificial pancreas to deliver insulin to the body. To address this need, we have developed personalized deep learning model that can accurately detect carbohydrate intake with a high degree of accuracy. Our study employed a publicly available dataset gathered by an Inertial Measurement Unit (IMU), which included accelerometer and gyroscope data. The data was sampled at a rate of 15 Hz, necessitating preprocessing. For our tailored to the patient model, we utilized a recurrent network comprising Long short-term memory (LSTM) layers. Our findings revealed a median F1 score of 0.99, indicating a high level of accuracy. Additionally, the confusion matrix displayed a difference of only 6 s, further validating the model's accuracy. Therefore, we can confidently assert that our model architecture exhibits a high degree of accuracy. Our model performed well above 90% on the dataset, with most results between 98%-99%. The recurrent networks improved the problem-solving capabilities significantly, though some outliers remained. The model's average prediction latency was 5.5 s, suggesting that later meal predictions result in extended meal progress predictions. The dataset's limitation of mostly single-day data points raises questions about multi-day performance, which could be explored by collecting multi-day data, including night periods. Future enhancements might involve transformer networks and shorter time windows to improve model responsiveness and accuracy. Therefore, we can confidently assert that our model exhibits a high degree of accuracy.

Sensors for Smoking Detection in Epidemiological Research: Scoping Review

BACKGROUND

The use of wearable sensors is being explored as a challenging way to accurately identify smoking behaviors by measuring physiological and environmental factors in real-life settings. Although they hold potential benefits for aiding smoking cessation, no single wearable device currently achieves high accuracy in detecting smoking events. Furthermore, it is crucial to emphasize that this area of study is dynamic and requires ongoing updates.

OBJECTIVE

This scoping review aims to map the scientific literature for identifying the main sensors developed or used for tobacco smoke detection, with a specific focus on wearable sensors, as well as describe their key features and categorize them by type.

METHODS

According to the PRISMA-ScR (Preferred Reporting Items for Systematic Reviews and Meta-Analyses extension for Scoping Reviews) protocol, an electronic search was conducted on the PubMed, MEDLINE, and Web of Science databases, using the following keywords: ("biosensors" OR "biosensor" OR "sensors" OR "sensor" OR "wearable") AND ("smoking" OR "smoke").

RESULTS

Among a total of 37 studies included in this scoping review published between 2012 and March 2024, 16 described sensors based on wearable bands, 15 described multisensory systems, and 6 described other strategies to detect tobacco smoke exposure. Included studies provided details about the design or application of wearable sensors based on an elastic band to detect different aspects of tobacco smoke exposure (eg, arm, wrist, and finger movements, and lighting events). Some studies proposed a system composed of different sensor modalities (eg, Personal Automatic Cigarette Tracker [PACT], PACT 2.0, and AutoSense).

CONCLUSIONS

Our scoping review has revealed both the obstacles and opportunities linked to wearable devices, offering valuable insights for future research initiatives. Tackling the recognized challenges and delving into potential avenues for enhancement could elevate wearable devices into even more effective tools for aiding smoking cessation. In this context, continuous research is essential to fine-tune and optimize these devices, guaranteeing their practicality and reliability in real-world applications.

CigTrak: Smartwatch-Based Accurate Online Smoking Puff & Episode Detection with Gesture-Focused Windowing for CNN

Wearable-based motion sensing solutions are capable of automatically detecting and tracking individual smoking puffs and/or episodes to aid the users in their journey of smoking cessation. But they are either obtrusive to use, perform with a low accuracy, or have questionable ability of running fully on a low-power device like a smartwatch, all affecting their widespread adoption. We propose 'CigTrak', a novel pipeline for an accurate smoking puff and episode detection using 6-DoF motion sensor on a smartwatch. A multi-stage method for puff detection is devised, comprising a novel kinematic analysis of puffing motion enabling temporal localization of puff. A Convolutional Neural Network (CNN)-backed model uses this candidate puff as an input instance by re-sampling it to required input size for the final decision. Clusters of detected puffs are further used to detect episodes. Data from 13 subjects was used for evaluating puff detection, and 9 subjects for evaluating episode detection. CigTrak achieved a high subject-independent performance for puff detection (F1-score 0.94) and free-living episode detection (F1-score 0.89), surpassing state of the art performance. CigTrak was also implemented fully online on two different smartwatches for testing a real-time puff detection.Clinical Relevance- Cigarette smoking affects physical & mental well-being of a person, and is the leading cause of preventable diseases, adversely affecting cardiac and respiratory systems. With many adults wanting to quit smoking [1], a reliable way of auto-journaling of smoking activities can greatly aid in cessation efforts through self-help, and reduce burden on healthcare industry. CigTrak, with its high accuracy in detecting smoking puffs and episodes, and capability of running fully on a smartwatch, can be readily used for this purpose.

DeepPuff: Utilizing Deep Learning for Smoking Behavior Identification in Free-living Environment

A comprehensive assessment of cigarette smoking behavior and its effect on health requires a detailed examination of smoke exposure. We propose a CNN-LSTM-based deep learning architecture named DeepPuff to quantify Respiratory Smoke Exposure Metrics (RSEM). Smoke inhalations were detected from the breathing and hand gesture sensors of the Personal Automatic Cigarette Tracker v2 (PACT 2.0). The DeepPuff model for smoke inhalation detection was developed using data collected from 190 cigarette smoking events from 38 medium to heavy smokers and optimized for precision (avoidance of false positives). An independent dataset of 459 smoking events from 45 participants (90 smoking events in the lab and 369 smoking events in free-living conditions) was used for testing the model. The proposed model achieved a precision of 82.39% on the training and 93.80% on the testing dataset (95.88% in the lab and 93.78% in free-living). RSEM metrics were then computed from the breathing signal of each detected smoke inhalation. Results from the RSEM algorithm were compared with respiratory metrics obtained from video annotation. Smoke exposure metrics of puff duration, inhale-exhale duration, and inhalation duration were not statistically different from the ground truth generated through video annotation. The results suggest that DeepPuff may be used as a reliable means to measure respiratory smoke exposure metrics collected under free-living conditions.

An LSTM-based Gesture-to-Speech Recognition System

Fast and flexible communication options are limited for speech-impaired people. Hand gestures coupled with fast, generated speech can enable a more natural social dynamic for those individuals - particularly individuals without the fine motor skills to type on a keyboard or tablet reliably. We created a mobile phone application prototype that generates audible responses associated with trained hand movements and collects and organizes the accelerometer data for rapid training to allow tailored models for individuals who may not be able to perform standard movements such as sign language. Six participants performed 11 distinct gestures to produce the dataset. A mobile application was developed that integrated a bidirectional LSTM network architecture which was trained from this data. After evaluation using nested subject-wise cross-validation, our integrated bidirectional LSTM model demonstrates an overall recall of 91.8% in recognition of these pre-selected 11 hand gestures, with recall at 95.8% when two commonly confused gestures were not assessed. This prototype is a step in creating a mobile phone system capable of capturing new gestures and developing tailored gesture recognition models for individuals in speech-impaired populations. Further refinement of this prototype can enable fast and efficient communication with the goal of further improving social interaction for individuals unable to speak.

Face touch monitoring using an instrumented wristband using dynamic time warping and k-nearest neighbours

One of the main factors in controlling infectious diseases such as COVID-19 is to prevent touching preoral and prenasal regions. Face touching is a habitual behaviour that occurs frequently. Studies showed that people touch their faces 23 times per hour on average. A contaminated hand could transmit the infection to the body by a facial touch. Since controlling this spontaneous habit is not easy, this study aimed to develop and validate a technology to detect and monitor face touch using dynamic time warping (DTW) and KNN (k-nearest neighbours) based on a wrist-mounted inertial measurement unit (IMU) in a controlled environment and natural environment trials. For this purpose, eleven volunteers were recruited and their hand motions were recorded in controlled and natural environment trials using a wrist-mounted IMU. Then the sensitivity, precision, and accuracy of our developed technology in detecting the face touch were evaluated. It was observed that the sensitivity, precision, and accuracy of the DTW-KNN classifier were 91%, 97%, and 85% in controlled environment trials and 79%, 92%, and 79% in natural environment trials (daily life). In conclusion, a wrist-mounted IMU, widely available in smartwatches, could detect the face touch with high sensitivity, precision, and accuracy and can be used as an ambulatory system to detect and monitor face touching as a high-risk habit in daily life.

SmokeMon: Unobtrusive Extraction of Smoking Topography Using Wearable Energy-Efficient Thermal

Smoking is the leading cause of preventable death worldwide. Cigarette smoke includes thousands of chemicals that are harmful and cause tobacco-related diseases. To date, the causality between human exposure to specific compounds and the harmful effects is unknown. A first step in closing the gap in knowledge has been measuring smoking topography, or how the smoker smokes the cigarette (puffs, puff volume, and duration). However, current gold-standard approaches to smoking topography involve expensive, bulky, and obtrusive sensor devices, creating unnatural smoking behavior and preventing their potential for real-time interventions in the wild. Although motion-based wearable sensors and their corresponding machine-learned models have shown promise in unobtrusively tracking smoking gestures, they are notorious for confounding smoking with other similar hand-to-mouth gestures such as eating and drinking. In this paper, we present SmokeMon, a chest-worn thermal-sensing wearable system that can capture spatial, temporal, and thermal information around the wearer and cigarette all day to unobtrusively and passively detect smoking events. We also developed a deep learning-based framework to extract puffs and smoking topography. We evaluate SmokeMon in both controlled and free-living experiments with a total of 19 participants, more than 110 hours of data, and 115 smoking sessions achieving an F1-score of 0.9 for puff detection in the laboratory and 0.8 in the wild. By providing SmokeMon as an open platform, we provide measurement of smoking topography in free-living settings to enable testing of smoking topography in the real world, with potential to facilitate timely smoking cessation interventions.

Monitoring Arm Movements Post-Stroke for Applications in Rehabilitation and Home Settings

Optimal recovery of arm function following stroke requires patients to perform a large number of functional arm movements in clinical therapy sessions, as well as at home. Technology to monitor adherence to this activity would be helpful to patients and clinicians. Current approaches to monitoring arm movements are limited because of challenges in distinguishing between functional and non-functional movements. Here, we present an Arm Rehabilitation Monitor (ARM), a device intended to make such measurements in an unobtrusive manner. The ARM device is based on a single Inertial Measurement Unit (IMU) worn on the wrist and uses machine learning techniques to interpret the resulting signals. We characterized the ability of the ARM to detect reaching actions in a functional assessment dataset (functional assessment tasks) and an Activities-of-Daily-Living (ADL) dataset (pizza-making and walking task) from 12 participants with stroke. The Convolutional Neural Network (CNN) and Random Forests (RF) classifiers had a Matthews Correlation Coefficient score of 0.59 and 0.58 when trained and tested on the functional dataset, 0.50 and 0.49 when trained and tested on the ADL dataset, and 0.37 and 0.36 when trained on the functional dataset and tested on the ADL dataset, respectively. The latter is the most relevant scenario for the intended application of training during a clinical visit for monitoring movements in the in-home setting. The classifiers showed good performance in estimating the time spent reaching and number of reaching gestures and showed low sensitivity to irrelevant arm movements produced during walking. We conclude that the ARM has sufficient accuracy and robustness to merit being used in preliminary studies to monitor arm activity in rehabilitation or home applications.

Applications and Techniques for Fast Machine Learning in Science

In this community review report, we discuss applications and techniques for fast machine learning (ML) in science-the concept of integrating powerful ML methods into the real-time experimental data processing loop to accelerate scientific discovery. The material for the report builds on two workshops held by the Fast ML for Science community and covers three main areas: applications for fast ML across a number of scientific domains; techniques for training and implementing performant and resource-efficient ML algorithms; and computing architectures, platforms, and technologies for deploying these algorithms. We also present overlapping challenges across the multiple scientific domains where common solutions can be found. This community report is intended to give plenty of examples and inspiration for scientific discovery through integrated and accelerated ML solutions. This is followed by a high-level overview and organization of technical advances, including an abundance of pointers to source material, which can enable these breakthroughs.

Smoking Cessation System for Preemptive Smoking Detection

Smoking cessation is a significant challenge for many people addicted to cigarettes and tobacco. Mobile health-related research into smoking cessation is primarily focused on mobile phone data collection either using self-reporting or sensor monitoring techniques. In the past 5 years with the increased popularity of smartwatch devices, research has been conducted to predict smoking movements associated with smoking behaviors based on accelerometer data analyzed from the internal sensors in a user's smartwatch. Previous smoking detection methods focused on classifying current user smoking behavior. For many users who are trying to quit smoking, this form of detection may be insufficient as the user has already relapsed. In this paper, we present a smoking cessation system utilizing a smartwatch and finger sensor that is capable of detecting pre-smoking activities to discourage users from future smoking behavior. Pre-smoking activities include grabbing a pack of cigarettes or lighting a cigarette and these activities are often immediately succeeded by smoking. Therefore, through accurate detection of pre-smoking activities, we can alert the user before they have relapsed. Our smoking cessation system combines data from a smartwatch for gross accelerometer and gyroscope information and a wearable finger sensor for detailed finger bend-angle information. We compare the results of a smartwatch-only system with a combined smartwatch and finger sensor system to illustrate the accuracy of each system. The combined smartwatch and finger sensor system performed at an 80.6% accuracy for the classification of pre-smoking activities compared to 47.0% accuracy of the smartwatch-only system.

CNN-Based Smoker Classification and Detection in Smart City Application

To better regulate smoking in no-smoking areas, we present a novel AI-based surveillance system for smart cities. In this paper, we intend to solve the issue of no-smoking area surveillance by introducing a framework for an AI-based smoker detection system for no-smoking areas in a smart city. Moreover, this research will provide a dataset for smoker detection problems in indoor and outdoor environments to help future research on this AI-based smoker detection system. The newly curated smoker detection image dataset consists of two classes, Smoking and NotSmoking. Further, to classify the Smoking and NotSmoking images, we have proposed a transfer learning-based solution using the pre-trained InceptionResNetV2 model. The performance of the proposed approach for predicting smokers and not-smokers was evaluated and compared with other CNN methods on different performance metrics. The proposed approach achieved an accuracy of 96.87% with 97.32% precision and 96.46% recall in predicting the Smoking and NotSmoking images on a challenging and diverse newly-created dataset. Although, we trained the proposed method on the image dataset, we believe the performance of the system will not be affected in real-time.

WristPrint: Characterizing User Re-identification Risks from Wrist-worn Accelerometry Data

Public release of wrist-worn motion sensor data is growing. They enable and accelerate research in developing new algorithms to passively track daily activities, resulting in improved health and wellness utilities of smartwatches and activity trackers. But, when combined with sensitive attribute inference attack and linkage attack via re-identification of the same user in multiple datasets, undisclosed sensitive attributes can be revealed to unintended organizations with potentially adverse consequences for unsuspecting data contributing users. To guide both users and data collecting researchers, we characterize the re-identification risks inherent in motion sensor data collected from wrist-worn devices in users' natural environment. For this purpose, we use an open-set formulation, train a deep learning architecture with a new loss function, and apply our model to a new data set consisting of 10 weeks of daily sensor wearing by 353 users. We find that re-identification risk increases with an increase in the activity intensity. On average, such risk is 96% for a user when sharing a full day of sensor data.

Using Smartwatches to Detect Face Touching

Frequent spontaneous facial self-touches, predominantly during outbreaks, have the theoretical potential to be a mechanism of contracting and transmitting diseases. Despite the recent advent of vaccines, behavioral approaches remain an integral part of reducing the spread of COVID-19 and other respiratory illnesses. The aim of this study was to utilize the functionality and the spread of smartwatches to develop a smartwatch application to identify motion signatures that are mapped accurately to face touching. Participants (n = 10, five women, aged 20–83) performed 10 physical activities classified into face touching (FT) and non-face touching (NFT) categories in a standardized laboratory setting. We developed a smartwatch application on Samsung Galaxy Watch to collect raw accelerometer data from participants. Data features were extracted from consecutive non-overlapping windows varying from 2 to 16 s. We examined the performance of state-of-the-art machine learning methods on face-touching movement recognition (FT vs. NFT) and individual activity recognition (IAR): logistic regression, support vector machine, decision trees, and random forest. While all machine learning models were accurate in recognizing FT categories, logistic regression achieved the best performance across all metrics (accuracy: 0.93 ± 0.08, recall: 0.89 ± 0.16, precision: 0.93 ± 0.08, F1-score: 0.90 ± 0.11, AUC: 0.95 ± 0.07) at the window size of 5 s. IAR models resulted in lower performance, where the random forest classifier achieved the best performance across all metrics (accuracy: 0.70 ± 0.14, recall: 0.70 ± 0.14, precision: 0.70 ± 0.16, F1-score: 0.67 ± 0.15) at the window size of 9 s. In conclusion, wearable devices, powered by machine learning, are effective in detecting facial touches. This is highly significant during respiratory infection outbreaks as it has the potential to limit face touching as a transmission vector.

OpiTrack: A Wearable-based Clinical Opioid Use Tracker with Temporal Convolutional Attention Networks

Opioid use disorder is a medical condition with major social and economic consequences. While ubiquitous physiological sensing technologies have been widely adopted and extensively used to monitor day-to-day activities and deliver targeted interventions to improve human health, the use of these technologies to detect drug use in natural environments has been largely underexplored. The long-term goal of our work is to develop a mobile technology system that can identify high-risk opioid-related events (i.e., development of tolerance in the setting of prescription opioid use, return-to-use events in the setting of opioid use disorder) and deploy just-in-time interventions to mitigate the risk of overdose morbidity and mortality. In the current paper, we take an initial step by asking a crucial question: Can opioid use be detected using physiological signals obtained from a wrist-mounted sensor? Thirty-six individuals who were admitted to the hospital for an acute painful condition and received opioid analgesics as part of their clinical care were enrolled. Subjects wore a noninvasive wrist sensor during this time (1-14 days) that continuously measured physiological signals (heart rate, skin temperature, accelerometry, electrodermal activity, and interbeat interval). We collected a total of 2070 hours (≈ 86 days) of physiological data and observed a total of 339 opioid administrations. Our results are encouraging and show that using a Channel-Temporal Attention TCN (CTA-TCN) model, we can detect an opioid administration in a time-window with an F1-score of 0.80, a specificity of 0.77, sensitivity of 0.80, and an AUC of 0.77. We also predict the exact moment of administration in this time-window with a normalized mean absolute error of 8.6% and R ² coefficient of 0.85.

Effect of a mobile just-in-time implementation intention intervention on momentary smoking lapses in smoking cessation attempts among Asian American young adults

Identifying vulnerable windows for a given problematic behavior and providing timely and appropriate support are critical for building an effective just-in-time (JIT) intervention for behavioral change. We developed and evaluated an implementation intention (II) based, JIT cessation intervention prototype to support Asian American young adult smokers to prevent lapses in their cessation attempts in real-time. We examined how a JIT II reminder may prevent lapses during self-identified high-risk smoking situation (HRSS) as a microtemporal process. We also tested whether the effect of JIT reminder changes over the course of study and differed between those who used their own versus project loan phones. Asian American young adult smokers (N = 57) who were interested in quitting or reducing smoking participated in a 4 week, mobile-based, cessation study (MyQuit USC, MQU). MQU is a JIT mobile app that deploys a user-specified II reminder at user-specified HRSS and assesses momentary lapse status. Generalized mixed linear models were conducted to assess the effect of the JIT intervention on lapse prevention. We found a significant interaction effect (p = .03) such that receiving JIT reminder reduced the likelihood of lapses for participants using their own phones but not for the loaners. The results also showed that when participants enacted the suggested II, they were less likely to lapse (p < .001). The JIT effect did not change over time in study (p = .21). This study provides evidence that receiving a reminder of a smoker's own plan just before a self-identified risky situation on a familiar device and successfully executing specified plans can be helpful in preventing lapses. Our results highlighted factors to consider when designing and refining a JIT intervention.

mTeeth: Identifying Brushing Teeth Surfaces Using Wrist-Worn Inertial Sensors

Ensuring that all the teeth surfaces are adequately covered during daily brushing can reduce the risk of several oral diseases. In this paper, we propose the mTeeth model to detect teeth surfaces being brushed with a manual toothbrush in the natural free-living environment using wrist-worn inertial sensors. To unambiguously label sensor data corresponding to different surfaces and capture all transitions that last only milliseconds, we present a lightweight method to detect the micro-event of brushing strokes that cleanly demarcates transitions among brushing surfaces. Using features extracted from brushing strokes, we propose a Bayesian Ensemble method that leverages the natural hierarchy among teeth surfaces and patterns of transition among them. For training and testing, we enrich a publicly-available wrist-worn inertial sensor dataset collected from the natural environment with time-synchronized precise labels of brushing surface timings and moments of transition. We annotate 10,230 instances of brushing on different surfaces from 114 episodes and evaluate the impact of wide between-person and within-person between-episode variability on machine learning model's performance for brushing surface detection.

Quantification of Smoking Characteristics Using Smartwatch Technology: Pilot Feasibility Study of New Technology

Background

While there have been many technological advances in studying the neurobiological and clinical basis of tobacco use disorder and nicotine addiction, there have been relatively minor advances in technologies for monitoring, characterizing, and intervening to prevent smoking in real time. Better understanding of real-time smoking behavior can be helpful in numerous applications without the burden and recall bias associated with self-report.

Objective

The goal of this study was to test the validity of using a smartwatch to advance the study of temporal patterns and characteristics of smoking in a controlled laboratory setting prior to its implementation in situ. Specifically, the aim was to compare smoking characteristics recorded by Automated Smoking PerceptIon and REcording (ASPIRE) on a smartwatch with the pocket Clinical Research Support System (CReSS) topography device, using video observation as the gold standard.

Methods

Adult smokers (N=27) engaged in a video-recorded laboratory smoking task using the pocket CReSS while also wearing a Polar M600 smartwatch. In-house software, ASPIRE, was used to record accelerometer data to identify the duration of puffs and interpuff intervals (IPIs). The recorded sessions from CReSS and ASPIRE were manually annotated to assess smoking topography. Agreement between CReSS-recorded and ASPIRE-recorded smoking behavior was compared.

Results

ASPIRE produced more consistent number of puffs and IPI durations relative to CReSS, when comparing both methods to visual puff count. In addition, CReSS recordings reported many implausible measurements in the order of milliseconds. After filtering implausible data recorded from CReSS, ASPIRE and CReSS produced consistent results for puff duration (R²=.79) and IPIs (R²=.73).

Conclusions

Agreement between ASPIRE and other indicators of smoking characteristics was high, suggesting that the use of ASPIRE is a viable method of passively characterizing smoking behavior. Moreover, ASPIRE was more accurate than CReSS for measuring puffs and IPIs. Results from this study provide the foundation for future utilization of ASPIRE to passively and accurately monitor and quantify smoking behavior in situ.

Automatic Functional Shoulder Task Identification and Sub-task Segmentation Using Wearable Inertial Measurement Units for Frozen Shoulder Assessment

Advanced sensor technologies have been applied to support frozen shoulder assessment. Sensor-based assessment tools provide objective, continuous and quantitative information for evaluation and diagnosis. However, the current tools for assessment of functional shoulder tasks mainly rely on manual operation. It may cause several technical issues to the reliability and usability of the assessment tool, including manual bias during the recording and additional efforts for data labeling. To tackle these issues, this pilot study aims to propose an automatic functional shoulder task identification and sub-task segmentation system using inertial measurement units to provide reliable shoulder task labeling and sub-task information for clinical professionals. The proposed method combines machine learning models and rule-based modification to identify shoulder tasks and segment sub-tasks accurately. A hierarchical design is applied to enhance the efficiency and performance of the proposed approach. Nine healthy subjects and nine frozen shoulder patients are invited to perform five common shoulder tasks in the lab-based and clinical environments, respectively. The experimental results show that the proposed method can achieve 87.11% F-score for shoulder task identification, and 83.23% F-score and 427 mean absolute time errors (milliseconds) for sub-task segmentation. The proposed approach demonstrates the feasibility of the proposed method to support reliable evaluation for clinical assessment.

RNN-Aided Human Velocity Estimation from a Single IMU

Pedestrian Dead Reckoning (PDR) uses inertial measurement units (IMUs) and combines velocity and orientation estimates to determine a position. The estimation of the velocity is still challenging, as the integration of noisy acceleration and angular speed signals over a long period of time causes large drifts. Classic approaches to estimate the velocity optimize for specific applications, sensor positions, and types of movement and require extensive parameter tuning. Our novel hybrid filter combines a convolutional neural network (CNN) and a bidirectional recurrent neural network (BLSTM) (that extract spatial features from the sensor signals and track their temporal relationships) with a linear Kalman filter (LKF) that improves the velocity estimates. Our experiments show the robustness against different movement states and changes in orientation, even in highly dynamic situations. We compare the new architecture with conventional, machine, and deep learning methods and show that from a single non-calibrated IMU, our novel architecture outperforms the state-of-the-art in terms of velocity (≤0.16 m/s) and traveled distance (≤3 m/km). It also generalizes well to different and varying movement speeds and provides accurate and precise velocity estimates.

A CNN-LSTM neural network for recognition of puffing in smoking episodes using wearable sensors

A detailed assessment of smoking behavior under free-living conditions is a key challenge for health behavior research. A number of methods using wearable sensors and puff topography devices have been developed for smoking and individual puff detection. In this paper, we propose a novel algorithm for automatic detection of puffs in smoking episodes by using a combination of Respiratory Inductance Plethysmography and Inertial Measurement Unit sensors. The detection of puffs was performed by using a deep network containing convolutional and recurrent neural networks. Convolutional neural networks (CNN) were utilized to automate feature learning from raw sensor streams. Long Short Term Memory (LSTM) network layers were utilized to obtain the temporal dynamics of sensor signals and classify sequence of time segmented sensor streams. An evaluation was performed by using a large, challenging dataset containing 467 smoking events from 40 participants under free-living conditions. The proposed approach achieved an F1-score of 78% in leave-one-subject-out cross-validation. The results suggest that CNN-LSTM based neural network architecture sufficiently detect puffing episodes in free-living condition. The proposed model be used as a detection tool for smoking cessation programs and scientific research.

A CNN-LSTM neural network for recognition of puffing in smoking episodes using wearable sensors

A detailed assessment of smoking behavior under free-living conditions is a key challenge for health behavior research. A number of methods using wearable sensors and puff topography devices have been developed for smoking and individual puff detection. In this paper, we propose a novel algorithm for automatic detection of puffs in smoking episodes by using a combination of Respiratory Inductance Plethysmography and Inertial Measurement Unit sensors. The detection of puffs was performed by using a deep network containing convolutional and recurrent neural networks. Convolutional neural networks (CNN) were utilized to automate feature learning from raw sensor streams. Long Short Term Memory (LSTM) network layers were utilized to obtain the temporal dynamics of sensor signals and classify sequence of time segmented sensor streams. An evaluation was performed by using a large, challenging dataset containing 467 smoking events from 40 participants under free-living conditions. The proposed approach achieved an F1-score of 78% in leave-one-subject-out cross-validation. The results suggest that CNN-LSTM based neural network architecture sufficiently detect puffing episodes in free-living condition. The proposed model be used as a detection tool for smoking cessation programs and scientific research.

A Report on Smoking Detection and Quitting Technologies

Mobile health technologies are being developed for personal lifestyle and medical healthcare support, of which a growing number are designed to assist smokers to quit. The potential impact of these technologies in the fight against smoking addiction and on improving quitting rates must be systematically evaluated. The aim of this report is to identify and appraise the most promising smoking detection and quitting technologies (e.g., smartphone apps, wearable devices) supporting smoking reduction or quitting programs. We searched PubMed and Scopus databases (2008-2019) for studies on mobile health technologies developed to assist smokers to quit using a combination of Medical Subject Headings topics and free text terms. A Google search was also performed to retrieve the most relevant smartphone apps for quitting smoking, considering the average user's rating and the ranking computed by the search engine algorithms. All included studies were evaluated using consolidated criteria for reporting qualitative research, such as applied methodologies and the performed evaluation protocol. Main outcome measures were usability and effectiveness of smoking detection and quitting technologies supporting smoking reduction or quitting programs. Our search identified 32 smoking detection and quitting technologies (12 smoking detection systems and 20 smoking quitting smartphone apps). Most of the existing apps for quitting smoking require the users to register every smoking event. Moreover, only a restricted group of them have been scientifically evaluated. The works supported by documented experimental evaluation show very high detection scores, however the experimental protocols usually lack in variability (e.g., only right-hand patients, not natural sequence of gestures) and have been conducted with limited numbers of patients as well as under constrained settings quite far from real-life use scenarios. Several recent scientific works show very promising results but, at the same time, present obstacles for the application on real-life daily scenarios.

SmokingOpp: Detecting the Smoking 'Opportunity' Context Using Mobile Sensors

Context plays a key role in impulsive adverse behaviors such as fights, suicide attempts, binge-drinking, and smoking lapse. Several contexts dissuade such behaviors, but some may trigger adverse impulsive behaviors. We define these latter contexts as 'opportunity' contexts, as their passive detection from sensors can be used to deliver context-sensitive interventions. In this paper, we define the general concept of 'opportunity' contexts and apply it to the case of smoking cessation. We operationalize the smoking 'opportunity' context, using self-reported smoking allowance and cigarette availability. We show its clinical utility by establishing its association with smoking occurrences using Granger causality. Next, we mine several informative features from GPS traces, including the novel location context of smoking spots, to develop the SmokingOpp model for automatically detecting the smoking 'opportunity' context. Finally, we train and evaluate the SmokingOpp model using 15 million GPS points and 3,432 self-reports from 90 newly abstinent smokers in a smoking cessation study.

StopWatch: The Preliminary Evaluation of a Smartwatch-Based System for Passive Detection of Cigarette Smoking

Introduction

Recent developments in smoking cessation support systems and interventions have highlighted the requirement for unobtrusive, passive ways to measure smoking behavior. A number of systems have been developed for this that either use bespoke sensing technology, or expensive combinations of wearables and smartphones. Here, we present StopWatch, a system for passive detection of cigarette smoking that runs on a low-cost smartwatch and does not require additional sensing or a connected smartphone.

Methods

Our system uses motion data from the accelerometer and gyroscope in an Android smartwatch to detect the signature hand movements of cigarette smoking. It uses machine learning techniques to transform raw motion data into motion features, and in turn into individual drags and instances of smoking. These processes run on the smartwatch, and do not require a smartphone.

Results

We conducted preliminary validations of the system in daily smokers (n = 13) in laboratory and free-living conditions running on an Android LG G-Watch. In free-living conditions, over a 24-h period, the system achieved precision of 86% and recall of 71%.

Conclusions

StopWatch is a system for passive measurement of cigarette smoking that runs entirely on a commercially available Android smartwatch. It requires no smartphone so the cost is low, and needs no bespoke sensing equipment so participant burden is also low. Performance is currently lower than other more expensive and complex systems, though adequate for some applications. Future developments will focus on enhancing performance, validation on a range of smartwatches, and detection of electronic cigarette use.

Implications

We present a low-cost, smartwatch-based system for passive detection of cigarette smoking. It uses data from the motion sensors in the watch to identify the signature hand movements of cigarette smoking. The system will provide the detailed measures of individual smoking behavior needed for context-triggered just-in-time smoking cessation support systems, and to enable just-in-time adaptive interventions. More broadly, the system will enable researchers to obtain detailed measures of individual smoking behavior in free-living conditions that are free from the recall errors and reporting biases associated with self-report of smoking.

Indoor acoustic localization: a survey

Applications of localization range from body tracking, gesture capturing, indoor plan construction to mobile health sensing. Technologies such as inertial sensors, radio frequency signals and cameras have been deeply excavated to locate targets. Among all the technologies, the acoustic signal gains enormous favor considering its comparatively high accuracy with common infrastructure and low time latency. Range-based localization falls into two categories: absolute range and relative range. Different mechanisms, such as Time of Flight, Doppler effect and phase shift, are widely studied to achieve the two genres of localization. The subcategories show distinguishing features but also face diverse challenges. In this survey, we present a comprehensive overview on various indoor localization systems derived from the various mechanisms. We also discuss the remaining issues and the future work.

Counting Bites With Bits: Expert Workshop Addressing Calorie and Macronutrient Intake Monitoring

BACKGROUND

Conventional diet assessment approaches such as the 24-hour self-reported recall are burdensome, suffer from recall bias, and are inaccurate in estimating energy intake. Wearable sensor technology, coupled with advanced algorithms, is increasingly showing promise in its ability to capture behaviors that provide useful information for estimating calorie and macronutrient intake.

OBJECTIVE

This paper aimed to summarize current technological approaches to monitoring energy intake on the basis of expert opinion from a workshop panel and to make recommendations to advance technology and algorithms to improve estimation of energy expenditure.

METHODS

A 1-day invitational workshop sponsored by the National Science Foundation was held at Northwestern University. A total of 30 participants, including population health researchers, engineers, and intervention developers, from 6 universities and the National Institutes of Health participated in a panel discussing the state of evidence with regard to monitoring calorie intake and eating behaviors.

RESULTS

Calorie monitoring using technological approaches can be characterized into 3 domains: (1) image-based sensing (eg, wearable and smartphone-based cameras combined with machine learning algorithms); (2) eating action unit (EAU) sensors (eg, to measure feeding gesture and chewing rate); and (3) biochemical measures (eg, serum and plasma metabolite concentrations). We discussed how each domain functions, provided examples of promising solutions, and highlighted potential challenges and opportunities in each domain. Image-based sensor research requires improved ground truth (context and known information about the foods), accurate food image segmentation and recognition algorithms, and reliable methods of estimating portion size. EAU-based domain research is limited by the understanding of when their systems (device and inference algorithm) succeed and fail, need for privacy-protecting methods of capturing ground truth, and uncertainty in food categorization. Although an exciting novel technology, the challenges of biochemical sensing range from a lack of adaptability to environmental effects (eg, temperature change) and mechanical impact, instability of wearable sensor performance over time, and single-use design.

CONCLUSIONS

Conventional approaches to calorie monitoring rely predominantly on self-reports. These approaches can gain contextual information from image-based and EAU-based domains that can map automatically captured food images to a food database and detect proxies that correlate with food volume and caloric intake. Although the continued development of advanced machine learning techniques will advance the accuracy of such wearables, biochemical sensing provides an electrochemical analysis of sweat using soft bioelectronics on human skin, enabling noninvasive measures of chemical compounds that provide insight into the digestive and endocrine systems. Future computing-based researchers should focus on reducing the burden of wearable sensors, aligning data across multiple devices, automating methods of data annotation, increasing rigor in studying system acceptability, increasing battery lifetime, and rigorously testing validity of the measure. Such research requires moving promising technological solutions from the controlled laboratory setting to the field.

Mobile alcohol biosensors and pharmacotherapy development research

Recent advances in biosensor technology herald a major shift in the way alcohol use will be tracked in humans. Wearable biosensors can passively and continuously monitor wearers' alcohol consumption in real time. An important application of these biosensors is to improve the way medication for alcohol use disorder (AUD) is tested in clinical research. Both laboratory-based screening paradigms and clinical trials have methodological problems that impact their efficiency and predictive validity. Medication screening using laboratory-based methods is a resource-intensive assessment of a single episode of behavior in a non-representative setting. Clinical trials rely on participant self-report to document medication-induced changes in drinking behavior. This review describes how mobile biosensors can be leveraged to improve AUD medication development research. We first review the current state of alcohol biosensor technology with a focus on strengths and limitations of the devices. We describe how multiple biosensors can be combined to create a far more detailed record of drinking compared to single biosensor platforms. We then discuss each phase of the medication development pipeline in turn (i.e., phases 1-4) and describe how mobile biosensors can be incorporated in standard medication testing paradigms to improve efficiency and predictive validity. We conclude with discussion of challenges associated with using currently available biosensors for medication testing and recommendations for researchers wishing to incorporate alcohol biosensors into their own research.

Wearable Sensors for Monitoring of Cigarette Smoking in Free-Living: A Systematic Review

Globally, cigarette smoking is widespread among all ages, and smokers struggle to quit. The design of effective cessation interventions requires an accurate and objective assessment of smoking frequency and smoke exposure metrics. Recently, wearable devices have emerged as a means of assessing cigarette use. However, wearable technologies have inherent limitations, and their sensor responses are often influenced by wearers' behavior, motion and environmental factors. This paper presents a systematic review of current and forthcoming wearable technologies, with a focus on sensing elements, body placement, detection accuracy, underlying algorithms and applications. Full-texts of 86 scientific articles were reviewed in accordance with the Preferred Reporting Items for Systematic Review and Meta-Analyses (PRISMA) guidelines to address three research questions oriented to cigarette smoking, in order to: (1) Investigate the behavioral and physiological manifestations of cigarette smoking targeted by wearable sensors for smoking detection; (2) explore sensor modalities employed for detecting these manifestations; (3) evaluate underlying signal processing and pattern recognition methodologies and key performance metrics. The review identified five specific smoking manifestations targeted by sensors. The results suggested that no system reached 100% accuracy in the detection or evaluation of smoking-related features. Also, the testing of these sensors was mostly limited to laboratory settings. For a realistic evaluation of accuracy metrics, wearable devices require thorough testing under free-living conditions.

Ambulatory Smoking Habits Investigation based on Physiology and Context (ASSIST) using wearable sensors and mobile phones: protocol for an observational study

INTRODUCTION

Smoking prevalence continues to be high over the world and smoking-induced diseases impose a heavy burden on the medical care system. As believed by many researchers, a promising way to promote healthcare and well-being at low cost for the large vulnerable smoking population is through eHealth solutions by providing self-help information about smoking cessation. But in the absence of first-hand knowledge about smoking habits in daily life settings, systems built on these methods often fail to deliver proactive and tailored interventions for different users and situations over time, thus resulting in low efficacy. To fill the gap, an observational study has been developed on the theme of objective and non-biased monitoring of smoking habits in a longitudinal and ambulatory mode. This paper presents the study protocol. The primary objective of the study is to reveal the contextual and physiological pattern of different smoking behaviours using wearable sensors and mobile phones. The secondary objectives are to (1) analyse cue factors and contextual situations of smoking events; (2) describe smoking types with regard to users' characteristics and (3) compare smoking types between and within subjects.

METHODS AND ANALYSES

This is an observational study aimed at reaching 100 participants. Inclusion criteria are adults aged between 18 and 65 years, current smoker and office worker. The primary outcome is a collection of a diverse and inclusive data set representing the daily smoking habits of the general smoking population from similar social context. Data analysation will revolve around our primary and secondary objectives. First, linear regression and linear mixed model will be used to estimate whether a factor or pattern have consistent (p value<0.05) correlation with smoking. Furthermore, multivariate multilevel analysis will be used to examine the influence of smokers' characteristics (sex, age, education, socioeconomic status, nicotine dependence, attitudes towards smoking, quit attempts, etc), contextual factors, and physical and emotional statuses on their smoking habits. Most recent machine learning techniques will also be explored to combine heterogeneous data for classification of smoking events and prediction of craving.

ETHICS AND DISSEMINATION

The study was designed together by an interdisciplinary group of researchers, including psychologist, psychiatrist, engineer and user involvement coordinator. The protocol was reviewed and approved by the ethical review board of UZ Leuven on 18 April 2016, with an approval number S60078. The study will allow us to characterise the types of smokers and triggering events. These findings will be disseminated through peer-reviewed articles.

To Mask or Not to Mask? Balancing Privacy with Visual Confirmation Utility in Activity-Oriented Wearable Cameras

Activity-oriented cameras are increasingly being used to provide visual confirmation of specific hand-related activities in real-world settings. However, recent studies have shown that bystander privacy concerns limit participant willingness to wear a camera. Researchers have investigated different image obfuscation methods as an approach to enhance bystander privacy; however, these methods may have varying effects on the visual confirmation utility of the image, which we define as the ability of a human viewer to interpret the activity of the wearer in the image. Visual confirmation utility is needed to annotate and validate hand-related activities for several behavioral-based applications, particularly in cases where a human in the loop method is needed to label (e.g., annotating gestures that cannot be automatically detected yet). We propose a new type of obfuscation, activity-oriented partial obfuscation, as a methodological contribution to researchers interested in obtaining visual confirmation of hand-related activities in the wild. We tested the effects of this approach by collecting ten diverse and realistic video scenarios that involved the wearer performing hand-related activities while bystanders performed activities that could be of concern if recorded. Then we conducted an online experiment with 367 participants to evaluate the effect of varying degrees of obfuscation on bystander privacy and visual confirmation utility. Our results show that activity-oriented partial obfuscation (1) maintains visual confirmation of the wearer's hand-related activity, especially when an object is present in the hand, and even when extreme filters are applied, while (2) significantly reducing bystander concerns and enhancing bystander privacy. Informed by our analysis, we further discuss the impact of the filter method used in activity-oriented partial obfuscation on bystander privacy and concerns.

A Method of Ultrasonic Finger Gesture Recognition Based on the Micro-Doppler Effect

With the popularity of small-screen smart mobile devices, gestures as a new type of human–computer interaction are highly demanded. Furthermore, finger gestures are more familiar to people in controlling devices. In this paper, a new method for recognizing finger gestures is proposed. Ultrasound was actively emitted to measure the micro-Doppler effect caused by finger motions and was obtained at high resolution. By micro-Doppler processing, micro-Doppler feature maps of finger gestures were generated. Since the feature map has a similar structure to the single channel color image, a recognition model based on a convolutional neural network was constructed for classification. The optimized recognition model achieved an average accuracy of 96.51% in the experiment.

Smoking detection based on regularity analysis of hand to mouth gestures

A number of studies have been introduced for the detection of smoking via a variety of features extracted from the wrist IMU data. However, none of the previous studies investigated gesture regularity as a way to detect smoking events. This study describes a novel method to detect smoking events by monitoring the regularity of hand gestures. Here, the regularity of hand gestures was estimated from a one axis accelerometer worn on the wrist of the dominant hand. To quantify the regularity score, this paper applied a novel approach of unbiased autocorrelation to process the temporal sequence of hand gestures. The comparison of regularity score of smoking events with other activities substantiated that hand-to-mouth gestures are highly regular during smoking events and have the potential to detect smoking from among a plethora of daily activities. This hypothesis was validated on a dataset of 140 cigarette smoking events generated by 35 regular smokers in a controlled setting. The regularity of gestures detected smoking events with an F1-score of 0.81. However, the accuracy dropped to 0.49 in the free-living study of same 35 smokers smoking 295 cigarettes. Nevertheless, regularity of gestures may be useful as a supportive tool for other detection methods. To validate that proposition, this paper further incorporated the regularity of gestures in an instrumented lighter based smoking detection algorithm and achieved an improvement in F1-score from 0.89 (lighter only) to 0.91 (lighter and regularity of hand gestures).

mORAL: An mHealth Model for Inferring Oral Hygiene Behaviors in-the-wild Using Wrist-worn Inertial Sensors

We address the open problem of reliably detecting oral health behaviors passively from wrist-worn inertial sensors. We present our model named mORAL (pronounced em oral) for detecting brushing and flossing behaviors, without the use of instrumented toothbrushes so that the model is applicable to brushing with still prevalent manual toothbrushes. We show that for detecting rare daily events such as toothbrushing, adopting a model that is based on identifying candidate windows based on events, rather than fixed-length timeblocks, leads to significantly higher performance. Trained and tested on 2,797 hours of sensor data collected over 192 days on 25 participants (using video annotations for ground truth labels), our brushing model achieves 100% median recall with a false positive rate of one event in every nine days of sensor wearing. The average error in estimating the start/end times of the detected event is 4.1% of the interval of the actual toothbrushing event.

Cigarette Smoking Detection with An Inertial Sensor and A Smart Lighter

In recent years, a number of wearable approaches have been introduced for objective monitoring of cigarette smoking based on monitoring of hand gestures, breathing or cigarette lighting events. However, non-reactive, objective and accurate measurement of everyday cigarette consumption in the wild remains a challenge. This study utilizes a wearable sensor system (Personal Automatic Cigarette Tracker 2.0, PACT2.0) and proposes a method that integrates information from an instrumented lighter and a 6-axis Inertial Measurement Unit (IMU) on the wrist for accurate detection of smoking events. The PACT2.0 was utilized in a study of 35 moderate to heavy smokers in both controlled (1.5–2 h) and unconstrained free-living conditions (~24 h). The collected dataset contained approximately 871 h of IMU data, 463 lighting events, and 443 cigarettes. The proposed method identified smoking events from the cigarette lighter data and estimated puff counts by detecting hand-to-mouth gestures (HMG) in the IMU data by a Support Vector Machine (SVM) classifier. The leave-one-subject-out (LOSO) cross-validation on the data from the controlled portion of the study achieved high accuracy and F1-score of smoking event detection and estimation of puff counts (97%/98% and 93%/86%, respectively). The results of validation in free-living demonstrate 84.9% agreement with self-reported cigarettes. These results suggest that an IMU and instrumented lighter may potentially be used in studies of smoking behavior under natural conditions.

Designing a wrist-worn sensor to improve medication adherence: accommodating diverse user behaviors and technology preferences

Objectives

High medication adherence is important for HIV suppression (antiretroviral therapy) and pre-exposure prophylaxis efficacy. We are developing sensor-based technologies to detect pill-taking gestures, trigger reminders, and generate adherence reports.

Materials and Methods

We collected interview, observation, and questionnaire data from individuals with and at-risk for HIV (N = 17). We assessed their medication-taking practices and physical actions, and feedback on our initial design.

Results

While participants displayed diverse medication taking practices and physical actions, most (67%) wanted to use the system to receive real-time and summative feedback, and most (69%) wanted to share data with their physicians. Participants preferred reminders via the wrist-worn device or mobile app, and summative feedback via mobile app or email.

Discussion

Adoption of these systems is promising if designs accommodate diverse behaviors and preferences.

Conclusion

Our findings may help improve the accuracy and adoption of the system by accounting for user behaviors, physical actions, and preferences.

Combining ecological momentary assessment with objective, ambulatory measures of behavior and physiology in substance-use research

Whereas substance-use researchers have long combined self-report with objective measures of behavior and physiology inside the laboratory, developments in mobile/wearable electronic technology are increasingly allowing for the collection of both subjective and objective information in participants' daily lives. For self-report, ecological momentary assessment (EMA), as implemented on contemporary smartphones or personal digital assistants, can provide researchers with near-real-time information on participants' behavior and mood in their natural environments. Data from portable/wearable electronic sensors measuring participants' internal and external environments can be combined with EMA (e.g., by timestamps recorded on questionnaires) to provide objective information useful in determining the momentary context of behavior and mood and/or validating participants' self-reports. Here, we review three objective ambulatory monitoring techniques that have been combined with EMA, with a focus on detecting drug use and/or measuring the behavioral or physiological correlates of mental events (i.e., emotions, cognitions): (1) collection and processing of biological samples in the field to measure drug use or participants' physiological activity (e.g., hypothalamic-pituitary-adrenal axis activity); (2) global positioning system (GPS) location information to link environmental characteristics (disorder/disadvantage, retail drug outlets) to drug use and affect; (3) ambulatory electronic physiological monitoring (e.g., electrocardiography) to detect drug use and mental events, as advances in machine learning algorithms make it possible to distinguish target changes from confounds (e.g., physical activity). Finally, we consider several other mobile/wearable technologies that hold promise to be combined with EMA, as well as potential challenges faced by researchers working with multiple mobile/wearable technologies simultaneously in the field.

The use of ambulatory assessment in smoking cessation

Ambulatory assessment of smoking behavior has greatly advanced our knowledge of the smoking cessation process. The current article first provides a brief overview of ecological momentary assessment for smoking cessation and highlights some of the primary advantages and scientific advancements made from this data collection method. Next, a discussion of how certain data collection tools (i.e., smoking topography and carbon monoxide detection) that have been traditionally used in lab-based settings are now being used to collect data in the real world. The second half of the paper focuses on the use of wearable wireless sensors to collect data during the smoking cessation process. Details regarding how these sensor-based technologies work, their application to newer tobacco products, and their potential to be used as intervention tools are discussed. Specific focus is placed on the opportunity to utilize novel intervention approaches, such as Just-In-Time Adaptive Interventions, to intervene upon smoking behavior. Finally, a discussion of some of the current challenges and limitations related to using sensor-based tools for smoking cessation are presented, along with suggestions for future research in this area.

Recognizing cigarette smoke inhalations using hidden Markov models

Previous studies with the Personal Automatic Cigarette Tracker (PACT) wearable system have found that smoking presents a distinct temporal breathing pattern, which might be well-suited for recognition by hidden Markov models (HMMs). In this work, we explored the feasibility of using HMMs to characterize the temporal information of smoking inhalations contained in the respiratory signals such as tidal volume, airflow, and the signal from the hand-to-mouth proximity sensor. Left-to-right HMMs were built to classify smoking and non-smoking inhalations using either only the respiratory signals, or both respiratory and hand proximity signals. Using a data set of 20 subjects, a leave-one-out cross-validation was performed on each HMM. In the recognition of smoke inhalations, the highest average recall, precision and F-score perceived by the HMMs was 42.39%, 88.19% and 56.38%, respectively, providing a 7.3% improvement in recall against a previously reported Support Vector Machines.

Concept Test of a Smoking Cessation Smart Case

Background: Wearable/portable devices that unobtrusively detect smoking and contextual data offer the potential to provide Just-In-Time Adaptive Intervention (JITAI) support for mobile cessation programs. Little has been reported on the development of these technologies. Introduction: To address this gap, we offer a case report of users' experiences with a prototype "smart" cigarette case that automatically tracks time and location of smoking. Small-scale user-experience studies are typical of iterative product design and are especially helpful when proposing novel ideas. The purpose of the study was to assess concept acceptability and potential for further development. Materials and Methods: We tested the prototype case with a small sample of potential users (n = 7). Participants used the hardware/software for 2 weeks and reconvened for a 90-min focus group to discuss experiences and provide feedback. Results: Participants liked the smart case in principle but found the prototype too bulky for easy portability. The potential for the case to convey positive messages about self also emerged as a finding. Participants indicated willingness to pay for improved technology (USD $15-$60 on a one-time basis). Discussion: The smart case is a viable concept, but design detail is critical to user acceptance. Future research should examine designs that maximize convenience and that explore the device's ability to cue intentions and other cognitions that would support cessation. Conclusions: This study is the first to our knowledge to report formative research on the smart case concept. This initial exploration provides insights that may be helpful to other developers of JITAI-support technology.

i 2 tag

Many radio frequency identification (RFID) applications, such as virtual shopping cart and tag-assisted gaming, involve sensing and recognizing tag mobility. However, existing RFID localization methods are mostly designed for static or slowly moving targets (less than 0.3m/sec). More importantly, we observe that prior methods suffer from serious performance degradation for detecting real-world moving tags in typical indoor environments with multipath interference. In this article, we present i 2 tag, an intelligent mobility-aware activity identification system for RFID tags in multipath-rich environments (e.g., indoors). i 2 tag employs a supervised learning framework based on our novel fine-grain mobility provile, which can quantify different levels of mobility. Unlike previous methods that mostly rely on phase measurement, i 2 tag takes into account various measurements, including RSSI variance, packet loss rate, and our novel relative phase--based fingerprint. Additionally, we design a multidimensional dynamic time warping--based algorithm to robustly detect mobility and the associated activities. We show that i 2 tag is readily deployable using off-the-shelf RFID devices. A prototype has been implemented using a ThingMagic reader and standard-compatible tags. Experimental results demonstrate its superiority in mobility detection and activity identification in various indoor environments.

Detecting Smoking Events Using Accelerometer Data Collected Via Smartwatch Technology: Validation Study

BACKGROUND

Smoking is the leading cause of preventable death in the world today. Ecological research on smoking in context currently relies on self-reported smoking behavior. Emerging smartwatch technology may more objectively measure smoking behavior by automatically detecting smoking sessions using robust machine learning models.

OBJECTIVE

This study aimed to examine the feasibility of detecting smoking behavior using smartwatches. The second aim of this study was to compare the success of observing smoking behavior with smartwatches to that of conventional self-reporting.

METHODS

A convenience sample of smokers was recruited for this study. Participants (N=10) recorded 12 hours of accelerometer data using a mobile phone and smartwatch. During these 12 hours, they engaged in various daily activities, including smoking, for which they logged the beginning and end of each smoking session. Raw data were classified as either smoking or nonsmoking using a machine learning model for pattern recognition. The accuracy of the model was evaluated by comparing the output with a detailed description of a modeled smoking session.

RESULTS

In total, 120 hours of data were collected from participants and analyzed. The accuracy of self-reported smoking was approximately 78% (96/123). Our model was successful in detecting 100 of 123 (81%) smoking sessions recorded by participants. After eliminating sessions from the participants that did not adhere to study protocols, the true positive detection rate of the smartwatch based-detection increased to more than 90%. During the 120 hours of combined observation time, only 22 false positive smoking sessions were detected resulting in a 2.8% false positive rate.

CONCLUSIONS

Smartwatch technology can provide an accurate, nonintrusive means of monitoring smoking behavior in natural contexts. The use of machine learning algorithms for passively detecting smoking sessions may enrich ecological momentary assessment protocols and cessation intervention studies that often rely on self-reported behaviors and may not allow for targeted data collection and communications around smoking events.

Development of a Multisensory Wearable System for Monitoring Cigarette Smoking Behavior in Free-Living Conditions

This paper presents the development and validation of a novel multi-sensory wearable system (Personal Automatic Cigarette Tracker v2 or PACT2.0) for monitoring of cigarette smoking in free-living conditions. The contributions of the PACT2.0 system are: (1) the implementation of a complete sensor suite for monitoring of all major behavioral manifestations of cigarette smoking (lighting events, hand-to-mouth gestures, and smoke inhalations); (2) a miniaturization of the sensor hardware to enable its applicability in naturalistic settings; and (3) an introduction of new sensor modalities that may provide additional insight into smoking behavior e.g., Global Positioning System (GPS), pedometer and Electrocardiogram(ECG) or provide an easy-to-use alternative (e.g., bio-impedance respiration sensor) to traditional sensors. PACT2.0 consists of three custom-built devices: an instrumented lighter, a hand module, and a chest module. The instrumented lighter is capable of recording the time and duration of all lighting events. The hand module integrates Inertial Measurement Unit (IMU) and a Radio Frequency (RF) transmitter to track the hand-to-mouth gestures. The module also operates as a pedometer. The chest module monitors the breathing (smoke inhalation) patterns (inductive and bio-impedance respiratory sensors), cardiac activity (ECG sensor), chest movement (three-axis accelerometer), hand-to-mouth proximity (RF receiver), and captures the geo-position of the subject (GPS receiver). The accuracy of PACT2.0 sensors was evaluated in bench tests and laboratory experiments. Use of PACT2.0 for data collection in the community was validated in a 24 h study on 40 smokers. Of 943 h of recorded data, 98.6% of the data was found usable for computer analysis. The recorded information included 549 lighting events, 522/504 consumed cigarettes (from lighter data/self-registered data, respectively), 20,158/22,207 hand-to-mouth gestures (from hand IMU/proximity sensor, respectively) and 114,217/112,175 breaths (from the respiratory inductive plethysmograph (RIP)/bio-impedance sensor, respectively). The proposed system scored 8.3 ± 0.31 out of 10 on a post-study acceptability survey. The results suggest that PACT2.0 presents a reliable platform for studying of smoking behavior at the community level.

Location-Enhanced Activity Recognition in Indoor Environments Using Off the Shelf Smart Watch Technology and BLE Beacons

Activity recognition in indoor spaces benefits context awareness and improves the efficiency of applications related to personalised health monitoring, building energy management, security and safety. The majority of activity recognition frameworks, however, employ a network of specialised building sensors or a network of body-worn sensors. As this approach suffers with respect to practicality, we propose the use of commercial off-the-shelf devices. In this work, we design and evaluate an activity recognition system composed of a smart watch, which is enhanced with location information coming from Bluetooth Low Energy (BLE) beacons. We evaluate the performance of this approach for a variety of activities performed in an indoor laboratory environment, using four supervised machine learning algorithms. Our experimental results indicate that our location-enhanced activity recognition system is able to reach a classification accuracy ranging from 92% to 100%, while without location information classification accuracy it can drop to as low as 50% in some cases, depending on the window size chosen for data segmentation.

Dynamic Computation Offloading for Low-Power Wearable Health Monitoring Systems

OBJECTIVE

The objective of this paper is to describe and evaluate an algorithm to reduce power usage and increase battery lifetime for wearable health-monitoring devices.

METHODS

We describe a novel dynamic computation offloading scheme for real-time wearable health monitoring devices that adjusts the partitioning of data processing between the wearable device and mobile application as a function of desired classification accuracy.

RESULTS

By making the correct offloading decision based on current system parameters, we show that we are able to reduce system power by as much as 20%.

CONCLUSION

We demonstrate that computation offloading can be applied to real-time monitoring systems, and yields significant power savings.

SIGNIFICANCE

Making correct offloading decisions for health monitoring devices can extend battery life and improve adherence.

mSieve: Differential Behavioral Privacy in Time Series of Mobile Sensor Data

Differential privacy concepts have been successfully used to protect anonymity of individuals in population-scale analysis. Sharing of mobile sensor data, especially physiological data, raise different privacy challenges, that of protecting private behaviors that can be revealed from time series of sensor data. Existing privacy mechanisms rely on noise addition and data perturbation. But the accuracy requirement on inferences drawn from physiological data, together with well-established limits within which these data values occur, render traditional privacy mechanisms inapplicable. In this work, we define a new behavioral privacy metric based on differential privacy and propose a novel data substitution mechanism to protect behavioral privacy. We evaluate the efficacy of our scheme using 660 hours of ECG, respiration, and activity data collected from 43 participants and demonstrate that it is possible to retain meaningful utility, in terms of inference accuracy (90%), while simultaneously preserving the privacy of sensitive behaviors.

Recognizing the Operating Hand and the Hand-Changing Process for User Interface Adjustment on Smartphones

As the size of smartphone touchscreens has become larger and larger in recent years, operability with a single hand is getting worse, especially for female users. We envision that user experience can be significantly improved if smartphones are able to recognize the current operating hand, detect the hand-changing process and then adjust the user interfaces subsequently. In this paper, we proposed, implemented and evaluated two novel systems. The first one leverages the user-generated touchscreen traces to recognize the current operating hand, and the second one utilizes the accelerometer and gyroscope data of all kinds of activities in the user's daily life to detect the hand-changing process. These two systems are based on two supervised classifiers constructed from a series of refined touchscreen trace, accelerometer and gyroscope features. As opposed to existing solutions that all require users to select the current operating hand or confirm the hand-changing process manually, our systems follow much more convenient and practical methods and allow users to change the operating hand frequently without any harm to the user experience. We conduct extensive experiments on Samsung Galaxy S4 smartphones, and the evaluation results demonstrate that our proposed systems can recognize the current operating hand and detect the hand-changing process with 94.1% and 93.9% precision and 94.1% and 93.7% True Positive Rates (TPR) respectively, when deciding with a single touchscreen trace or accelerometer-gyroscope data segment, and the False Positive Rates (FPR) are as low as 2.6% and 0.7% accordingly. These two systems can either work completely independently and achieve pretty high accuracies or work jointly to further improve the recognition accuracy.