Reduction of energy intake using just-in-time feedback from a wearable sensor system

An End-to-End Energy-Efficient Approach for Intake Detection With Low Inference Time Using Wrist-Worn Sensor

Automated detection of intake gestures with wearable sensors has been a critical area of research for advancing our understanding and ability to intervene in people's eating behavior. Numerous algorithms have been developed and evaluated in terms of accuracy. However, ensuring the system is not only accurate in making predictions but also efficient in doing so is critical for real-world deployment. Despite the growing research on accurate detection of intake gestures using wearables, many of these algorithms are often energy inefficient, impeding on-device deployment for continuous and real-time monitoring of diet. This article presents a template-based optimized multicenter classifier that enables accurate intake gesture detection while maintaining low-inference time and energy consumption using a wrist-worn accelerometer and gyroscope. We designed an Intake Gesture Counter smartphone application (CountING) and validated the practicality of our algorithm against seven state-of-the-art approaches on three public datasets (In-lab FIC, Clemson, and OREBA). Compared with other methods, we achieved optimal accuracy (81.60% F1 score) and very low inference time (15.97 msec per 2.20-sec data sample) on the Clemson dataset, and among the top performing algorithms, we achieve comparable accuracy (83.0% F1 score compared with 85.6% in the top performing algorithm) but superior inference time (13.8x faster, 33.14 msec per 2.20-sec data sample) on the In-lab FIC dataset and comparable accuracy (83.40% F1 score compared with 88.10% in the top-performing algorithm) but superior inference time (33.9x faster, 16.71 msec inference time per 2.20-sec data sample) on the OREBA dataset. On average, our approach achieved a 25-hour battery lifetime (44% to 52% improvement over state-of-the-art approaches) when tested on a commercial smartwatch for continuous real-time detection. Our approach demonstrates an effective and efficient method, enabling real-time intake gesture detection using wrist-worn devices in longitudinal studies.

Implementing just-in-time inventory management to address contextual operational issues: a case study of a commercial livestock farm in southern Nigeria

Purpose

This research focuses on the implementation of Just-in-Time (JIT) inventory management, drawing on a case study of a commercial livestock farm located in a swampy area of southern Nigeria.

Design/methodology/approach

The research adopts a qualitative approach. Interviews and workshops were used for data collection.

Findings

Findings from the study reveal that the commitment on the internal organisational members and skilful collaboration with supply chain partners are required for effective use of JIT, especially in an odd contextual situation such as the case in this study. This also justifies the embraced of additional cost of securing JIT inventory management practices such as the situation in the case study organisation that could not allow conventional inventory management.

Originality/value

It is suggested for further research to consider the topic from a mixed method approach as well as extend the focus on the possibility of legal regulations and government support to exceptional operational practices among organisations, especially those in the context of the food production sector, where this research was based.

Detection of Food Intake Sensor's Wear Compliance in Free-Living

Objective detection of periods of wear and non-wear is critical for human studies that rely on information from wearable sensors, such as food intake sensors. In this paper, we present a novel method of compliance detection on the example of the Automatic Ingestion Monitor v2 (AIM-2) sensor, containing a tri-axial accelerometer, a still camera, and a chewing sensor. The method was developed and validated using data from a study of 30 participants aged 18-39, each wearing the AIM-2 for two days (a day in pseudo-free-living and a day in free-living). Four types of wear compliance were analyzed: 'normal-wear', 'non-compliant-wear', 'non-wear-carried', and 'non-wear-stationary'. The ground truth of those four types of compliance was obtained by reviewing the images of the egocentric camera. The features for compliance detection were the standard deviation of acceleration, average pitch, and roll angles, and mean square error of two consecutive images. These were used to train three random forest classifiers 1) accelerometer-based, 2) image-based, and 3) combined accelerometer and image-based. Satisfactory wear compliance measurement accuracy was obtained using the combined classifier (89.24%) on leave one subject out cross-validation. The average duration of compliant wear in the study was 9h with a standard deviation of 2h or 70.96% of total on-time. This method can be used to calculate the wear and non-wear time of AIM-2, and potentially be extended to other devices. The study also included assessments of sensor burden and privacy concerns. The survey results suggest recommendations that may be used to increase wear compliance.

Noninvasive Continuous Monitoring of Vital Signs With Wearables: Fit for Medical Use?

BACKGROUND

Wearables (= wearable computer) enable continuous and noninvasive monitoring of a range of vital signs. Mobile and cost-effective devices, combined with powerful data analysis tools, open new dimensions in assessing body functions ("digital biomarkers").

METHODS

To answer the question whether wearables are ready for use in the medical context, a PubMed literature search and analysis for their clinical-scientific use using publications from the years 2008 to 2018 was performed.

RESULTS

A total of 79 out of 314 search hits were publications on clinical trials with wearables, of which 16 were randomized controlled trials. Motion sensors were most frequently used to measure defined movements, movement disorders, or general physical activity. Approximately 20% of the studies used sensors to detect cardiovascular parameters. As for the sensor location, the wrist was chosen in most studies (22.8%).

CONCLUSION

Wearables can be used in a precisely defined medical context, when taking into account complex influencing factors.

NeckSense: A Multi-Sensor Necklace for Detecting Eating Activities in Free-Living Conditions

We present the design, implementation, and evaluation of a multi-sensor, low-power necklace, NeckSense, for automatically and unobtrusively capturing fine-grained information about an individual's eating activity and eating episodes, across an entire waking day in a naturalistic setting. NeckSense fuses and classifies the proximity of the necklace from the chin, the ambient light, the Lean Forward Angle, and the energy signals to determine chewing sequences, a building block of the eating activity. It then clusters the identified chewing sequences to determine eating episodes. We tested NeckSense on 11 participants with and 9 participants without obesity, across two studies, where we collected more than 470 hours of data in a naturalistic setting. Our results demonstrate that NeckSense enables reliable eating detection for individuals with diverse body mass index (BMI) profiles, across an entire waking day, even in free-living environments. Overall, our system achieves an F1-score of 81.6% in detecting eating episodes in an exploratory study. Moreover, our system can achieve an F1-score of 77.1% for episodes even in an all-day-long free-living setting. With more than 15.8 hours of battery life, NeckSense will allow researchers and dietitians to better understand natural chewing and eating behaviors. In the future, researchers and dietitians can use NeckSense to provide appropriate real-time interventions when an eating episode is detected or when problematic eating is identified.

A Systematic Review of Technology-Driven Methodologies for Estimation of Energy Intake

Accurate measurement of energy intake (EI) is important for estimation of energy balance, and, correspondingly, body weight dynamics. Traditional measurements of EI rely on self-report, which may be inaccurate and underestimate EI. The imperfections in traditional methodologies such as 24-hour dietary recall, dietary record, and food frequency questionnaire stipulate development of technology-driven methods that rely on wearable sensors and imaging devices to achieve an objective and accurate assessment of EI. The aim of this research was to systematically review and examine peer-reviewed papers that cover the estimation of EI in humans, with the focus on emerging technology-driven methodologies. Five major electronic databases were searched for articles published from January 2005 to August 2017: Pubmed, Science Direct, IEEE Xplore, ACM library, and Google Scholar. Twenty-six eligible studies were retrieved that met the inclusion criteria. The review identified that while the current methods of estimating EI show promise, accurate estimation of EI in free-living individuals presents many challenges and opportunities. The most accurate result identified for EI (kcal) estimation had an average accuracy of 94%. However, collectively, the results were obtained from a limited number of food items (i.e., 19), small sample sizes (i.e., 45 meal images), and primarily controlled conditions. Therefore, new methods that accurately estimate EI over long time periods in free-living conditions are needed.

Validation of Sensor-Based Food Intake Detection by Multicamera Video Observation in an Unconstrained Environment

Video observations have been widely used for providing ground truth for wearable systems for monitoring food intake in controlled laboratory conditions; however, video observation requires participants be confined to a defined space. The purpose of this analysis was to test an alternative approach for establishing activity types and food intake bouts in a relatively unconstrained environment. The accuracy of a wearable system for assessing food intake was compared with that from video observation, and inter-rater reliability of annotation was also evaluated. Forty participants were enrolled. Multiple participants were simultaneously monitored in a 4-bedroom apartment using six cameras for three days each. Participants could leave the apartment overnight and for short periods of time during the day, during which time monitoring did not take place. A wearable system (Automatic Ingestion Monitor, AIM) was used to detect and monitor participants’ food intake at a resolution of 30 s using a neural network classifier. Two different food intake detection models were tested, one trained on the data from an earlier study and the other on current study data using leave-one-out cross validation. Three trained human raters annotated the videos for major activities of daily living including eating, drinking, resting, walking, and talking. They further annotated individual bites and chewing bouts for each food intake bout. Results for inter-rater reliability showed that, for activity annotation, the raters achieved an average (±standard deviation (STD)) kappa value of 0.74 (±0.02) and for food intake annotation the average kappa (Light’s kappa) of 0.82 (±0.04). Validity results showed that AIM food intake detection matched human video-annotated food intake with a kappa of 0.77 (±0.10) and 0.78 (±0.12) for activity annotation and for food intake bout annotation, respectively. Results of one-way ANOVA suggest that there are no statistically significant differences among the average eating duration estimated from raters’ annotations and AIM predictions (p-value = 0.19). These results suggest that the AIM provides accuracy comparable to video observation and may be used to reliably detect food intake in multi-day observational studies.

Statistical models for meal-level estimation of mass and energy intake using features derived from video observation and a chewing sensor

Accurate and objective assessment of energy intake remains an ongoing problem. We used features derived from annotated video observation and a chewing sensor to predict mass and energy intake during a meal without participant self-report. 30 participants each consumed 4 different meals in a laboratory setting and wore a chewing sensor while being videotaped. Subject-independent models were derived from bite, chew, and swallow features obtained from either video observation or information extracted from the chewing sensor. With multiple regression analysis, a forward selection procedure was used to choose the best model. The best estimates of meal mass and energy intake had (mean ± standard deviation) absolute percentage errors of 25.2% ± 18.9% and 30.1% ± 33.8%, respectively, and mean ± standard deviation estimation errors of −17.7 ± 226.9 g and −6.1 ± 273.8 kcal using features derived from both video observations and sensor data. Both video annotation and sensor-derived features may be utilized to objectively quantify energy intake.

Into the Wild: The Challenges of Physiological Stress Detection in Laboratory and Ambulatory Settings

Stress and mental health have become major concerns worldwide. Research has already extensively investigated physiological signals as quantitative and continuous markers of stress. In recent years, the focus of the field has shifted from the laboratory to the ambulatory environment. We provide an overview of physiological stress detection in laboratory settings with a focus on identifying physiological sensing priorities, including electrocardiogram, skin conductance, and electromyogram, and the most suitable machine learning techniques, of which the choice depends on the context of the application. Additionally, an overview is given of new challenges ahead to move toward the ambulant environment, including the influence of physical activity, lower signal quality due to motion artifacts, the lack of a stress reference, and the subject-dependent nature of the physiological stress response. Finally, several recommendations for future research are listed, focusing on large-scale, longitudinal trials across different population groups and just-in-time interventions to move toward disease prevention and interception.

Advances and Controversies in Diet and Physical Activity Measurement in Youth

Technological advancements in the past decades have improved dietary intake and physical activity measurements. This report reviews current developments in dietary intake and physical activity assessment in youth. Dietary intake assessment has relied predominantly on self-report or image-based methods to measure key aspects of dietary intake (e.g., food types, portion size, eating occasion), which are prone to notable methodologic (e.g., recall bias) and logistic (e.g., participant and researcher burden) challenges. Although there have been improvements in automatic eating detection, artificial intelligence, and sensor-based technologies, participant input is often needed to verify food categories and portions. Current physical activity assessment methods, including self-report, direct observation, and wearable devices, provide researchers with reliable estimations for energy expenditure and bodily movement. Recent developments in algorithms that incorporate signals from multiple sensors and technology-augmented self-reporting methods have shown preliminary efficacy in measuring specific types of activity patterns and relevant contextual information. However, challenges in detecting resistance (e.g., in resistance training, weight lifting), prolonged physical activity monitoring, and algorithm (non)equivalence remain to be addressed. In summary, although dietary intake assessment methods have yet to achieve the same validity and reliability as physical activity measurement, recent developments in wearable technologies in both arenas have the potential to improve current assessment methods.

THEME INFORMATION

This article is part of a theme issue entitled Innovative Tools for Assessing Diet and Physical Activity for Health Promotion, which is sponsored by the North American branch of the International Life Sciences Institute.

The SPLENDID Eating Detection Sensor: Development and Feasibility Study

Background

The available methods for monitoring food intake—which for a great part rely on self-report—often provide biased and incomplete data. Currently, no good technological solutions are available. Hence, the SPLENDID eating detection sensor (an ear-worn device with an air microphone and a photoplethysmogram [PPG] sensor) was developed to enable complete and objective measurements of eating events. The technical performance of this device has been described before. To date, literature is lacking a description of how such a device is perceived and experienced by potential users.

Objective

The objective of our study was to explore how potential users perceive and experience the SPLENDID eating detection sensor.

Methods

Potential users evaluated the eating detection sensor at different stages of its development: (1) At the start, 12 health professionals (eg, dieticians, personal trainers) were interviewed and a focus group was held with 5 potential end users to find out their thoughts on the concept of the eating detection sensor. (2) Then, preliminary prototypes of the eating detection sensor were tested in a laboratory setting where 23 young adults reported their experiences. (3) Next, the first wearable version of the eating detection sensor was tested in a semicontrolled study where 22 young, overweight adults used the sensor on 2 separate days (from lunch till dinner) and reported their experiences. (4) The final version of the sensor was tested in a 4-week feasibility study by 20 young, overweight adults who reported their experiences.

Results

Throughout all the development stages, most individuals were enthusiastic about the eating detection sensor. However, it was stressed multiple times that it was critical that the device be discreet and comfortable to wear for a longer period. In the final study, the eating detection sensor received an average grade of 3.7 for wearer comfort on a scale of 1 to 10. Moreover, experienced discomfort was the main reason for wearing the eating detection sensor <2 hours a day. The participants reported having used the eating detection sensor on 19/28 instructed days on average.

Conclusions

The SPLENDID eating detection sensor, which uses an air microphone and a PPG sensor, is a promising new device that can facilitate the collection of reliable food intake data, as shown by its technical potential. Potential users are enthusiastic, but to be successful wearer comfort and discreetness of the device need to be improved.

Physical Activity, Sedentary Behavior, and Diet-Related eHealth and mHealth Research: Bibliometric Analysis

Background

Electronic health (eHealth) and mobile health (mHealth) approaches to address low physical activity levels, sedentary behavior, and unhealthy diets have received significant research attention. However, attempts to systematically map the entirety of the research field are lacking. This gap can be filled with a bibliometric study, where publication-specific data such as citations, journals, authors, and keywords are used to provide a systematic overview of a specific field. Such analyses will help researchers better position their work.

Objective

The objective of this review was to use bibliometric data to provide an overview of the eHealth and mHealth research field related to physical activity, sedentary behavior, and diet.

Methods

The Web of Science (WoS) Core Collection was searched to retrieve all existing and highly cited (as defined by WoS) physical activity, sedentary behavior, and diet related eHealth and mHealth research papers published in English between January 1, 2000 and December 31, 2016. Retrieved titles were screened for eligibility, using the abstract and full-text where needed. We described publication trends over time, which included journals, authors, and countries of eligible papers, as well as their keywords and subject categories. Citations of eligible papers were compared with those expected based on published data. Additionally, we described highly-cited papers of the field (ie, top ranked 1%).

Results

The search identified 4805 hits, of which 1712 (including 42 highly-cited papers) were included in the analyses. Publication output increased on an average of 26% per year since 2000, with 49.00% (839/1712) of papers being published between 2014 and 2016. Overall and throughout the years, eHealth and mHealth papers related to physical activity, sedentary behavior, and diet received more citations than expected compared with papers in the same WoS subject categories. The Journal of Medical Internet Research published most papers in the field (9.58%, 164/1712). Most papers originated from high-income countries (96.90%, 1659/1717), in particular the United States (48.83%, 836/1712). Most papers were trials and studied physical activity. Beginning in 2013, research on Generation 2 technologies (eg, smartphones, wearables) sharply increased, while research on Generation 1 (eg, text messages) technologies increased at a reduced pace. Reviews accounted for 20 of the 42 highly-cited papers (n=19 systematic reviews). Social media, smartphone apps, and wearable activity trackers used to encourage physical activity, less sedentary behavior, and/or healthy eating were the focus of 14 highly-cited papers.

Conclusions

This study highlighted the rapid growth of the eHealth and mHealth physical activity, sedentary behavior, and diet research field, emphasized the sizeable contribution of research from high-income countries, and pointed to the increased research interest in Generation 2 technologies. It is expected that the field will grow and diversify further and that reviews and research on most recent technologies will continue to strongly impact the field.

Meal Microstructure Characterization from Sensor-Based Food Intake Detection

To avoid the pitfalls of self-reported dietary intake, wearable sensors can be used. Many food ingestion sensors offer the ability to automatically detect food intake using time resolutions that range from 23 ms to 8 min. There is no defined standard time resolution to accurately measure ingestive behavior or a meal microstructure. This paper aims to estimate the time resolution needed to accurately represent the microstructure of meals such as duration of eating episode, the duration of actual ingestion, and number of eating events. Twelve participants wore the automatic ingestion monitor (AIM) and kept a standard diet diary to report their food intake in free-living conditions for 24 h. As a reference, participants were also asked to mark food intake with a push button sampled every 0.1 s. The duration of eating episodes, duration of ingestion, and number of eating events were computed from the food diary, AIM, and the push button resampled at different time resolutions (0.1–30s). ANOVA and multiple comparison tests showed that the duration of eating episodes estimated from the diary differed significantly from that estimated by the AIM and the push button (p-value <0.001). There were no significant differences in the number of eating events for push button resolutions of 0.1, 1, and 5 s, but there were significant differences in resolutions of 10–30s (p-value <0.05). The results suggest that the desired time resolution of sensor-based food intake detection should be ≤5 s to accurately detect meal microstructure. Furthermore, the AIM provides more accurate measurement of the eating episode duration than the diet diary.