Identification of cigarette smoke inhalations from wearable sensor data using a Support Vector Machine classifier

DeepPuff: Utilizing Deep Learning for Smoking Behavior Identification in Free-living Environment

A comprehensive assessment of cigarette smoking behavior and its effect on health requires a detailed examination of smoke exposure. We propose a CNN-LSTM-based deep learning architecture named DeepPuff to quantify Respiratory Smoke Exposure Metrics (RSEM). Smoke inhalations were detected from the breathing and hand gesture sensors of the Personal Automatic Cigarette Tracker v2 (PACT 2.0). The DeepPuff model for smoke inhalation detection was developed using data collected from 190 cigarette smoking events from 38 medium to heavy smokers and optimized for precision (avoidance of false positives). An independent dataset of 459 smoking events from 45 participants (90 smoking events in the lab and 369 smoking events in free-living conditions) was used for testing the model. The proposed model achieved a precision of 82.39% on the training and 93.80% on the testing dataset (95.88% in the lab and 93.78% in free-living). RSEM metrics were then computed from the breathing signal of each detected smoke inhalation. Results from the RSEM algorithm were compared with respiratory metrics obtained from video annotation. Smoke exposure metrics of puff duration, inhale-exhale duration, and inhalation duration were not statistically different from the ground truth generated through video annotation. The results suggest that DeepPuff may be used as a reliable means to measure respiratory smoke exposure metrics collected under free-living conditions.

A Report on Smoking Detection and Quitting Technologies

Mobile health technologies are being developed for personal lifestyle and medical healthcare support, of which a growing number are designed to assist smokers to quit. The potential impact of these technologies in the fight against smoking addiction and on improving quitting rates must be systematically evaluated. The aim of this report is to identify and appraise the most promising smoking detection and quitting technologies (e.g., smartphone apps, wearable devices) supporting smoking reduction or quitting programs. We searched PubMed and Scopus databases (2008-2019) for studies on mobile health technologies developed to assist smokers to quit using a combination of Medical Subject Headings topics and free text terms. A Google search was also performed to retrieve the most relevant smartphone apps for quitting smoking, considering the average user's rating and the ranking computed by the search engine algorithms. All included studies were evaluated using consolidated criteria for reporting qualitative research, such as applied methodologies and the performed evaluation protocol. Main outcome measures were usability and effectiveness of smoking detection and quitting technologies supporting smoking reduction or quitting programs. Our search identified 32 smoking detection and quitting technologies (12 smoking detection systems and 20 smoking quitting smartphone apps). Most of the existing apps for quitting smoking require the users to register every smoking event. Moreover, only a restricted group of them have been scientifically evaluated. The works supported by documented experimental evaluation show very high detection scores, however the experimental protocols usually lack in variability (e.g., only right-hand patients, not natural sequence of gestures) and have been conducted with limited numbers of patients as well as under constrained settings quite far from real-life use scenarios. Several recent scientific works show very promising results but, at the same time, present obstacles for the application on real-life daily scenarios.

StopWatch: The Preliminary Evaluation of a Smartwatch-Based System for Passive Detection of Cigarette Smoking

Introduction

Recent developments in smoking cessation support systems and interventions have highlighted the requirement for unobtrusive, passive ways to measure smoking behavior. A number of systems have been developed for this that either use bespoke sensing technology, or expensive combinations of wearables and smartphones. Here, we present StopWatch, a system for passive detection of cigarette smoking that runs on a low-cost smartwatch and does not require additional sensing or a connected smartphone.

Methods

Our system uses motion data from the accelerometer and gyroscope in an Android smartwatch to detect the signature hand movements of cigarette smoking. It uses machine learning techniques to transform raw motion data into motion features, and in turn into individual drags and instances of smoking. These processes run on the smartwatch, and do not require a smartphone.

Results

We conducted preliminary validations of the system in daily smokers (n = 13) in laboratory and free-living conditions running on an Android LG G-Watch. In free-living conditions, over a 24-h period, the system achieved precision of 86% and recall of 71%.

Conclusions

StopWatch is a system for passive measurement of cigarette smoking that runs entirely on a commercially available Android smartwatch. It requires no smartphone so the cost is low, and needs no bespoke sensing equipment so participant burden is also low. Performance is currently lower than other more expensive and complex systems, though adequate for some applications. Future developments will focus on enhancing performance, validation on a range of smartwatches, and detection of electronic cigarette use.

Implications

We present a low-cost, smartwatch-based system for passive detection of cigarette smoking. It uses data from the motion sensors in the watch to identify the signature hand movements of cigarette smoking. The system will provide the detailed measures of individual smoking behavior needed for context-triggered just-in-time smoking cessation support systems, and to enable just-in-time adaptive interventions. More broadly, the system will enable researchers to obtain detailed measures of individual smoking behavior in free-living conditions that are free from the recall errors and reporting biases associated with self-report of smoking.

Computation of Cigarette Smoke Exposure Metrics From Breathing

Traditional metrics of smoke exposure in cigarette smokers are derived either from self-report, biomarkers, or puff topography. Methods involving biomarkers measure concentrations of nicotine, nicotine metabolites, or carbon monoxide. Puff-topography methods employ portable instruments to measure puff count, puff volume, puff duration, and inter-puff interval. In this article, we propose smoke exposure metrics calculated from the breathing signal and describe a novel algorithm for the computation of these metrics. The Personal Automatic Cigarette Tracker v2 (PACT-2) sensors, puff topography devices (CReSS), and video observation were used in a study of 38 moderate to heavy smokers in a controlled environment. Parameters of smoke inhalation including the start and end of each puff, inhale and exhale cycle, and smoke holding were computed from the breathing signal. From these, the traditional metrics of puff duration, inhale-exhale cycle duration, smoke holding duration, inter-puff interval, and novel Respiratory Smoke Exposure Metrics (RSEMs) such as inhale-exhale cycle volume, and inhale-exhale volume over time were calculated. The proposed RSEM algorithm to extract smoke exposure metrics named generated interclass correlations (ICCs) of 0.85 and 0.87 and Pearson's correlations of 0.97 and 0.77 with video observation and CReSS, respectively, for puff duration. Similarly, for the inhale-exhale duration, an ICC of 0.84 and Pearson's correlation of 0.81 was obtained with video observation. The RSEMs provided measures previously unavailable in research that are proportional to the depth and duration of smoke inhalation. The results suggest that the breathing signal may be used to compute smoke exposure metrics.

Wearable Sensors for Monitoring of Cigarette Smoking in Free-Living: A Systematic Review

Globally, cigarette smoking is widespread among all ages, and smokers struggle to quit. The design of effective cessation interventions requires an accurate and objective assessment of smoking frequency and smoke exposure metrics. Recently, wearable devices have emerged as a means of assessing cigarette use. However, wearable technologies have inherent limitations, and their sensor responses are often influenced by wearers' behavior, motion and environmental factors. This paper presents a systematic review of current and forthcoming wearable technologies, with a focus on sensing elements, body placement, detection accuracy, underlying algorithms and applications. Full-texts of 86 scientific articles were reviewed in accordance with the Preferred Reporting Items for Systematic Review and Meta-Analyses (PRISMA) guidelines to address three research questions oriented to cigarette smoking, in order to: (1) Investigate the behavioral and physiological manifestations of cigarette smoking targeted by wearable sensors for smoking detection; (2) explore sensor modalities employed for detecting these manifestations; (3) evaluate underlying signal processing and pattern recognition methodologies and key performance metrics. The review identified five specific smoking manifestations targeted by sensors. The results suggested that no system reached 100% accuracy in the detection or evaluation of smoking-related features. Also, the testing of these sensors was mostly limited to laboratory settings. For a realistic evaluation of accuracy metrics, wearable devices require thorough testing under free-living conditions.

Objective Detection of Cigarette Smoking from Physiological Sensor Signals

Cigarette smoking has severe health impacts on those who smoke and the people around them. Several wearable sensing modalities have recently been investigated to collect objective data on daily smoking, including detection of smoking episodes from breathing patterns, hand to mouth behavior, and characteristic hand gestures or cigarette lighting events. In order to provide new insight into ongoing research on the objective collection of smoking-related events, this paper proposes a novel method to identify smoking events from the associated changes in heart rate parameters specific to smoking. The proposed method also accounts for the breathing rate and body motion of the person who is smoking to better distinguish these changes from intense physical activities. In this research, a human study was first performed on 20 daily cigarette smokers to record heart rate, breathing rate, and body acceleration collected from a wearable chest sensor consisting of an ECG and bioimpedance measurement sensor and a 3D inertial sensor. Each participant spent ~2 hours in a laboratory environment (mimicking daily activities that included smoking 4 cigarettes) and ~24 hours under unconstrained free-living conditions. A support vector machine-based classifier was developed to automatically detect smoking episodes from the captured sensor signals using fifteen features selected by a forward-feature selection method. In a leave one subject out cross-validation, the proposed approach detected smoking events (187 out of total 232) with the sensitivity and F-score accuracy of 0.87 and 0.79, respectively, in the laboratory setting (known activities) and 0.77 and 0.61, respectively, under free-living conditions. These results validate the proof-of-concept that, although further research is necessary for performance improvement, characteristic changes in heart rate parameters could be a useful indicator of cigarette smoking even under free-living conditions.

Cigarette Smoking Detection with An Inertial Sensor and A Smart Lighter

In recent years, a number of wearable approaches have been introduced for objective monitoring of cigarette smoking based on monitoring of hand gestures, breathing or cigarette lighting events. However, non-reactive, objective and accurate measurement of everyday cigarette consumption in the wild remains a challenge. This study utilizes a wearable sensor system (Personal Automatic Cigarette Tracker 2.0, PACT2.0) and proposes a method that integrates information from an instrumented lighter and a 6-axis Inertial Measurement Unit (IMU) on the wrist for accurate detection of smoking events. The PACT2.0 was utilized in a study of 35 moderate to heavy smokers in both controlled (1.5–2 h) and unconstrained free-living conditions (~24 h). The collected dataset contained approximately 871 h of IMU data, 463 lighting events, and 443 cigarettes. The proposed method identified smoking events from the cigarette lighter data and estimated puff counts by detecting hand-to-mouth gestures (HMG) in the IMU data by a Support Vector Machine (SVM) classifier. The leave-one-subject-out (LOSO) cross-validation on the data from the controlled portion of the study achieved high accuracy and F1-score of smoking event detection and estimation of puff counts (97%/98% and 93%/86%, respectively). The results of validation in free-living demonstrate 84.9% agreement with self-reported cigarettes. These results suggest that an IMU and instrumented lighter may potentially be used in studies of smoking behavior under natural conditions.

Combining ecological momentary assessment with objective, ambulatory measures of behavior and physiology in substance-use research

Whereas substance-use researchers have long combined self-report with objective measures of behavior and physiology inside the laboratory, developments in mobile/wearable electronic technology are increasingly allowing for the collection of both subjective and objective information in participants' daily lives. For self-report, ecological momentary assessment (EMA), as implemented on contemporary smartphones or personal digital assistants, can provide researchers with near-real-time information on participants' behavior and mood in their natural environments. Data from portable/wearable electronic sensors measuring participants' internal and external environments can be combined with EMA (e.g., by timestamps recorded on questionnaires) to provide objective information useful in determining the momentary context of behavior and mood and/or validating participants' self-reports. Here, we review three objective ambulatory monitoring techniques that have been combined with EMA, with a focus on detecting drug use and/or measuring the behavioral or physiological correlates of mental events (i.e., emotions, cognitions): (1) collection and processing of biological samples in the field to measure drug use or participants' physiological activity (e.g., hypothalamic-pituitary-adrenal axis activity); (2) global positioning system (GPS) location information to link environmental characteristics (disorder/disadvantage, retail drug outlets) to drug use and affect; (3) ambulatory electronic physiological monitoring (e.g., electrocardiography) to detect drug use and mental events, as advances in machine learning algorithms make it possible to distinguish target changes from confounds (e.g., physical activity). Finally, we consider several other mobile/wearable technologies that hold promise to be combined with EMA, as well as potential challenges faced by researchers working with multiple mobile/wearable technologies simultaneously in the field.

The use of ambulatory assessment in smoking cessation

Ambulatory assessment of smoking behavior has greatly advanced our knowledge of the smoking cessation process. The current article first provides a brief overview of ecological momentary assessment for smoking cessation and highlights some of the primary advantages and scientific advancements made from this data collection method. Next, a discussion of how certain data collection tools (i.e., smoking topography and carbon monoxide detection) that have been traditionally used in lab-based settings are now being used to collect data in the real world. The second half of the paper focuses on the use of wearable wireless sensors to collect data during the smoking cessation process. Details regarding how these sensor-based technologies work, their application to newer tobacco products, and their potential to be used as intervention tools are discussed. Specific focus is placed on the opportunity to utilize novel intervention approaches, such as Just-In-Time Adaptive Interventions, to intervene upon smoking behavior. Finally, a discussion of some of the current challenges and limitations related to using sensor-based tools for smoking cessation are presented, along with suggestions for future research in this area.

Recognizing cigarette smoke inhalations using hidden Markov models

Previous studies with the Personal Automatic Cigarette Tracker (PACT) wearable system have found that smoking presents a distinct temporal breathing pattern, which might be well-suited for recognition by hidden Markov models (HMMs). In this work, we explored the feasibility of using HMMs to characterize the temporal information of smoking inhalations contained in the respiratory signals such as tidal volume, airflow, and the signal from the hand-to-mouth proximity sensor. Left-to-right HMMs were built to classify smoking and non-smoking inhalations using either only the respiratory signals, or both respiratory and hand proximity signals. Using a data set of 20 subjects, a leave-one-out cross-validation was performed on each HMM. In the recognition of smoke inhalations, the highest average recall, precision and F-score perceived by the HMMs was 42.39%, 88.19% and 56.38%, respectively, providing a 7.3% improvement in recall against a previously reported Support Vector Machines.

Development of a Multisensory Wearable System for Monitoring Cigarette Smoking Behavior in Free-Living Conditions

This paper presents the development and validation of a novel multi-sensory wearable system (Personal Automatic Cigarette Tracker v2 or PACT2.0) for monitoring of cigarette smoking in free-living conditions. The contributions of the PACT2.0 system are: (1) the implementation of a complete sensor suite for monitoring of all major behavioral manifestations of cigarette smoking (lighting events, hand-to-mouth gestures, and smoke inhalations); (2) a miniaturization of the sensor hardware to enable its applicability in naturalistic settings; and (3) an introduction of new sensor modalities that may provide additional insight into smoking behavior e.g., Global Positioning System (GPS), pedometer and Electrocardiogram(ECG) or provide an easy-to-use alternative (e.g., bio-impedance respiration sensor) to traditional sensors. PACT2.0 consists of three custom-built devices: an instrumented lighter, a hand module, and a chest module. The instrumented lighter is capable of recording the time and duration of all lighting events. The hand module integrates Inertial Measurement Unit (IMU) and a Radio Frequency (RF) transmitter to track the hand-to-mouth gestures. The module also operates as a pedometer. The chest module monitors the breathing (smoke inhalation) patterns (inductive and bio-impedance respiratory sensors), cardiac activity (ECG sensor), chest movement (three-axis accelerometer), hand-to-mouth proximity (RF receiver), and captures the geo-position of the subject (GPS receiver). The accuracy of PACT2.0 sensors was evaluated in bench tests and laboratory experiments. Use of PACT2.0 for data collection in the community was validated in a 24 h study on 40 smokers. Of 943 h of recorded data, 98.6% of the data was found usable for computer analysis. The recorded information included 549 lighting events, 522/504 consumed cigarettes (from lighter data/self-registered data, respectively), 20,158/22,207 hand-to-mouth gestures (from hand IMU/proximity sensor, respectively) and 114,217/112,175 breaths (from the respiratory inductive plethysmograph (RIP)/bio-impedance sensor, respectively). The proposed system scored 8.3 ± 0.31 out of 10 on a post-study acceptability survey. The results suggest that PACT2.0 presents a reliable platform for studying of smoking behavior at the community level.

RisQ: Recognizing Smoking Gestures with Inertial Sensors on a Wristband

Smoking-induced diseases are known to be the leading cause of death in the United States. In this work, we design RisQ, a mobile solution that leverages a wristband containing a 9-axis inertial measurement unit to capture changes in the orientation of a person's arm, and a machine learning pipeline that processes this data to accurately detect smoking gestures and sessions in real-time. Our key innovations are fourfold: a) an arm trajectory-based method that extracts candidate hand-to-mouth gestures, b) a set of trajectory-based features to distinguish smoking gestures from confounding gestures including eating and drinking, c) a probabilistic model that analyzes sequences of hand-to-mouth gestures and infers which gestures are part of individual smoking sessions, and d) a method that leverages multiple IMUs placed on a person's body together with 3D animation of a person's arm to reduce burden of self-reports for labeled data collection. Our experiments show that our gesture recognition algorithm can detect smoking gestures with high accuracy (95.7%), precision (91%) and recall (81%). We also report a user study that demonstrates that we can accurately detect the number of smoking sessions with very few false positives over the period of a day, and that we can reliably extract the beginning and end of smoking session periods.

Laboratory Validation of Inertial Body Sensors to Detect Cigarette Smoking Arm Movements

Cigarette smoking remains the leading cause of preventable death in the United States. Traditional in-clinic cessation interventions may fail to intervene and interrupt the rapid progression to relapse that typically occurs following a quit attempt. The ability to detect actual smoking behavior in real-time is a measurement challenge for health behavior research and intervention. The successful detection of real-time smoking through mobile health (mHealth) methodology has substantial implications for developing highly efficacious treatment interventions. The current study was aimed at further developing and testing the ability of inertial sensors to detect cigarette smoking arm movements among smokers. The current study involved four smokers who smoked six cigarettes each in a laboratory-based assessment. Participants were outfitted with four inertial body movement sensors on the arms, which were used to detect smoking events at two levels: the puff level and the cigarette level. Two different algorithms (Support Vector Machines (SVM) and Edge-Detection based learning) were trained to detect the features of arm movement sequences transmitted by the sensors that corresponded with each level. The results showed that performance of the SVM algorithm at the cigarette level exceeded detection at the individual puff level, with low rates of false positive puff detection. The current study is the second in a line of programmatic research demonstrating the proof-of-concept for sensor-based tracking of smoking, based on movements of the arm and wrist. This study demonstrates efficacy in a real-world clinical inpatient setting and is the first to provide a detection rate against direct observation, enabling calculation of true and false positive rates. The study results indicate that the approach performs very well with some participants, whereas some challenges remain with participants who generate more frequent non-smoking movements near the face. Future work may allow for tracking smoking in real-world environments, which would facilitate developing more effective, just-in-time smoking cessation interventions.

Detection of cigarette smoke inhalations from respiratory signals using reduced feature set

A combination of wearable Respiratory Inductive Plethysmograph and a hand-to-mouth Proximity Sensor (PS) can be used to monitor smoking habits and smoke exposure in cigarette smokers. In our previous work, detection of smoke inhalations was achieved by using a Support Vector Machine (SVM) classifier applied to raw sensor signals with 1503-element feature vectors. This study uses empirically-defined 27 features computed from the sensor signals to reduce the length of vectors. Further reduction in the length of the feature vectors was achieved by a forward feature selection algorithm, identifying from 2 to 16 features most critical for smoke inhalations detection. For individual detection models, the 1503-element feature vectors, 27-element feature vectors and reduced feature vectors resulted in F-scores of 90.1%, 68.7% and 94% respectively. For the group models, F-scores were 81.3%, 65% and 67% respectively. These results demonstrate feasibility of detecting smoke inhalations with a computed feature set, but suggest high individuality of breathing patterns associated with smoking.

I am your smartphone, and I know you are about to smoke: the application of mobile sensing and computing approaches to smoking research and treatment

Much is known about the immediate and predictive antecedents of smoking lapse, which include situations (e.g., presence of other smokers), activities (e.g., alcohol consumption), and contexts (e.g., outside). This commentary suggests smartphone-based systems could be used to infer these predictive antecedents in real time and provide the smoker with just-in-time intervention. The smartphone of today is equipped with an array of sensors, including GPS, cameras, light sensors, barometers, accelerometers, and so forth, that provide information regarding physical location, human movement, ambient sounds, and visual imagery. We propose that libraries of algorithms to infer these antecedents can be developed and then incorporated into diverse mobile research and personalized treatment applications. While a number of challenges to the development and implementation of such applications are recognized, our field benefits from a database of known antecedents to a problem behavior, and further research and development in this exciting area are warranted.