A new method for measuring meal intake in humans via automated wrist motion tracking

Eating Speed Measurement Using Wrist-Worn IMU Sensors Towards Free-Living Environments

Eating speed is an important indicator that has been widely investigated in nutritional studies. The relationship between eating speed and several intake-related problems such as obesity, diabetes, and oral health has received increased attention from researchers. However, existing studies mainly use self-reported questionnaires to obtain participants' eating speed, where they choose options from slow, medium, and fast. Such a non-quantitative method is highly subjective and coarse at the individual level. This study integrates two classical tasks in automated food intake monitoring domain: bite detection and eating episode detection, to advance eating speed measurement in near-free-living environments automatically and objectively. Specifically, a temporal convolutional network combined with a multi-head attention module (TCN-MHA) is developed to detect bites (including eating and drinking gestures) from IMU data. The predicted bite sequences are then clustered into eating episodes. Eating speed is calculated by using the time taken to finish the eating episode to divide the number of bites. To validate the proposed approach on eating speed measurement, a 7-fold cross validation is applied to the self-collected fine-annotated full-day-I (FD-I) dataset, and a holdout experiment is conducted on the full-day-II (FD-II) dataset. The two datasets are collected from 61 participants with a total duration of 513 h, which are publicly available. Experimental results show that the proposed approach achieves a mean absolute percentage error (MAPE) of 0.110 and 0.146 in the FD-I and FD-II datasets, respectively, showcasing the feasibility of automated eating speed measurement in near-free-living environments.

Exploring the Impact of the NULL Class on In-the-Wild Human Activity Recognition

Monitoring activities of daily living (ADLs) plays an important role in measuring and responding to a person's ability to manage their basic physical needs. Effective recognition systems for monitoring ADLs must successfully recognize naturalistic activities that also realistically occur at infrequent intervals. However, existing systems primarily focus on either recognizing more separable, controlled activity types or are trained on balanced datasets where activities occur more frequently. In our work, we investigate the challenges associated with applying machine learning to an imbalanced dataset collected from a fully in-the-wild environment. This analysis shows that the combination of preprocessing techniques to increase recall and postprocessing techniques to increase precision can result in more desirable models for tasks such as ADL monitoring. In a user-independent evaluation using in-the-wild data, these techniques resulted in a model that achieved an event-based F1-score of over 0.9 for brushing teeth, combing hair, walking, and washing hands. This work tackles fundamental challenges in machine learning that will need to be addressed in order for these systems to be deployed and reliably work in the real world.

sEMG-based automatic characterization of swallowed materials

Monitoring of ingestive activities is critically important for managing the health and wellness of individuals with various health conditions, including the elderly, diabetics, and individuals seeking better weight control. Monitoring swallowing events can be an ideal surrogate for developing streamlined methods for effective monitoring and quantification of eating or drinking events. Swallowing is an essential process for maintaining life. This seemingly simple process is the result of coordinated actions of several muscles and nerves in a complex fashion. In this study, we introduce automated methods for the detection and quantification of various eating and drinking activities. Wireless surface electromyography (sEMG) was used to detect chewing and swallowing from sEMG signals obtained from the sternocleidomastoid muscle, in addition to signals obtained from a wrist-mounted IMU sensor. A total of 4675 swallows were collected from 55 participants in the study. Multiple methods were employed to estimate bolus volumes in the case of fluid intake, including regression and classification models. Among the tested models, neural networks-based regression achieved an R2 of 0.88 and a root mean squared error of 0.2 (minimum bolus volume was 10 ml). Convolutional neural networks-based classification (when considering each bolus volume as a separate class) achieved an accuracy of over 99% using random cross-validation and around 66% using cross-subject validation. Multiple classification methods were also used for solid bolus type detection, including SVM and decision trees (DT), which achieved an accuracy above 99% with random validation and above 94% in cross-subject validation. Finally, regression models with both random and cross-subject validation were used for estimating the solid bolus volume with an R2 value that approached 1 and root mean squared error values as low as 0.00037 (minimum solid bolus weight was 3 gm). These reported results lay the foundation for a cost-effective and non-invasive method for monitoring swallowing activities which can be extremely beneficial in managing various chronic health conditions, such as diabetes and obesity.

Objective Determination of Eating Occasion Timing: Combining Self-Report, Wrist Motion, and Continuous Glucose Monitoring to Detect Eating Occasions in Adults With Prediabetes and Obesity

BACKGROUND

Accurately identifying eating patterns, specifically the timing, frequency, and distribution of eating occasions (EOs), is important for assessing eating behaviors, especially for preventing and managing obesity and type 2 diabetes (T2D). However, existing methods to study EOs rely on self-report, which may be prone to misreporting and bias and has a high user burden. Therefore, objective methods are needed.

METHODS

We aim to compare EO timing using objective and subjective methods. Participants self-reported EO with a smartphone app (self-report [SR]), wore the ActiGraph GT9X on their dominant wrist, and wore a continuous glucose monitor (CGM, Abbott Libre Pro) for 10 days. EOs were detected from wrist motion (WM) using a motion-based classifier and from CGM using a simulation-based system. We described EO timing and explored how timing identified with WM and CGM compares with SR.

RESULTS

Participants (n = 39) were 59 ± 11 years old, mostly female (62%) and White (51%) with a body mass index (BMI) of 34.2 ± 4.7 kg/m2. All had prediabetes or moderately controlled T2D. The median time-of-day first EO (and interquartile range) for SR, WM, and CGM were 08:24 (07:00-09:59), 9:42 (07:46-12:26), and 06:55 (04:23-10:03), respectively. The median last EO for SR, WM, and CGM were 20:20 (16:50-21:42), 20:12 (18:30-21:41), and 21:43 (20:35-22:16), respectively. The overlap between SR and CGM was 55% to 80% of EO detected with tolerance periods of ±30, 60, and 120 minutes. The overlap between SR and WM was 52% to 65% EO detected with tolerance periods of ±30, 60, and 120 minutes.

CONCLUSION

The continuous glucose monitor and WM detected overlapping but not identical meals and may provide complementary information to self-reported EO.

An explanation for the accuracy of sensor-based measures of energy intake: Amount of food consumed matters more than dietary composition

Understanding and intervening on eating behavior often necessitates measurement of energy intake (EI); however, commonly utilized and widely accepted methods vary in accuracy and place significant burden on users (e.g., food diaries), or are costly to implement (e.g., doubly labeled water). Thus, researchers have sought to leverage inexpensive and low-burden technologies such as wearable sensors for EI estimation. Paradoxically, one such methodology that estimates EI via smartwatch-based bite counting has demonstrated high accuracy in laboratory and free-living studies, despite only measuring the amount, not the composition, of food consumed. This secondary analysis sought to further explore this phenomenon by evaluating the degree to which EI can be explained by a sensor-based estimate of the amount consumed versus the energy density (ED) of the food consumed. Data were collected from 82 adults in free-living conditions (51.2% female, 31.7% racial and/or ethnic minority; Mage = 33.5, SD = 14.7) who wore a bite counter device on their wrist and used smartphone app to implement the Remote Food Photography Method (RFPM) to assess EI and ED for two weeks. Bite-based estimates of EI were generated via a previously validated algorithm. At a per-meal level, linear mixed effect models indicated that bite-based EI estimates accounted for 23.4% of the variance in RFPM-measured EI, while ED and presence of a beverage accounted for only 0.2% and 0.1% of the variance, respectively. For full days of intake, bite-based EI estimates and ED accounted for 41.5% and 0.2% of the variance, respectively. These results help to explain the viability of sensor-based EI estimation even in the absence of information about dietary composition.

Detecting Eating Episodes From Wrist Motion Using Daily Pattern Analysis

This paper presents new methods to detect eating from wrist motion. Our main novelty is that we analyze a full day of wrist motion data as a single sample so that the detection of eating occurrences can benefit from diurnal context. We develop a two-stage framework to facilitate a feasible full-day analysis. The first-stage model calculates local probabilities of eating P(Ew) within windows of data, and the second-stage model calculates enhanced probabilities of eating P(Ed) by treating all P(Ew) within a single day as one sample. The framework also incorporates an augmentation technique, which involves the iterative retraining of the first-stage model. This allows us to generate a sufficient number of day-length samples from datasets of limited size. We test our methods on the publicly available Clemson All-Day (CAD) dataset and FreeFIC dataset, and find that the inclusion of day-length analysis substantially improves accuracy in detecting eating episodes. We also benchmark our results against several state-of-the-art methods. Our approach achieved an eating episode true positive rate (TPR) of 89% with 1.4 false positives per true positive (FP/TP), and a time weighted accuracy of 84%, which are the highest accuracies reported on the CAD dataset. Our results show that the daily pattern classifier substantially improves meal detections and in particular reduces transient false detections that tend to occur when relying on shorter windows to look for individual ingestion or consumption events.

The Fully Understanding Eating and Lifestyle Behaviors (FUEL) trial: Protocol for a cohort study harnessing digital health tools to phenotype dietary non-adherence behaviors during lifestyle intervention

Objective

Lifestyle intervention can produce clinically significant weight loss and reduced disease risk/severity for many individuals with overweight/obesity. Dietary lapses, instances of non-adherence to the recommended dietary goal(s) in lifestyle intervention, are associated with less weight loss and higher energy intake. There are distinct "types" of dietary lapse (e.g., eating an off-plan food, eating a larger portion), and behavioral, psychosocial, and contextual mechanisms may differ across dietary lapse types. Some lapse types also appear to impact weight more than others. Elucidating clear lapse types thus has potential for understanding and improving adherence to lifestyle intervention.

Methods

This 18-month observational cohort study will use real-time digital assessment tools within a multi-level factor analysis framework to uncover "lapse phenotypes" and understand their impact on clinical outcomes. Adults with overweight/obesity (n = 150) will participate in a 12-month online lifestyle intervention and 6-month weight loss maintenance period. Participants will complete 14-day lapse phenotyping assessment periods at baseline, 3, 6, 12, and 18 months in which smartphone surveys, wearable devices, and geolocation will assess dietary lapses and relevant phenotyping characteristics. Energy intake (via 24-h dietary recall) and weight will be collected at each assessment period.

Results

This trial is ongoing; data collection began on 31 October 2022 and is scheduled to complete by February 2027.

Conclusion

Results will inform novel precision tools to improve dietary adherence in lifestyle intervention, and support updated theoretical models of adherence behavior. Additionally, these phenotyping methods can likely be leveraged to better understand non-adherence to other health behavior interventions.

Trial Registration

This study was prospectively registered https://clinicaltrials.gov/study/NCT05562427.

Technology to Automatically Record Eating Behavior in Real Life: A Systematic Review

To monitor adherence to diets and to design and evaluate nutritional interventions, it is essential to obtain objective knowledge about eating behavior. In most research, measures of eating behavior are based on self-reporting, such as 24-h recalls, food records (food diaries) and food frequency questionnaires. Self-reporting is prone to inaccuracies due to inaccurate and subjective recall and other biases. Recording behavior using nonobtrusive technology in daily life would overcome this. Here, we provide an up-to-date systematic overview encompassing all (close-to) publicly or commercially available technologies to automatically record eating behavior in real-life settings. A total of 1328 studies were screened and, after applying defined inclusion and exclusion criteria, 122 studies were included for in-depth evaluation. Technologies in these studies were categorized by what type of eating behavior they measure and which type of sensor technology they use. In general, we found that relatively simple sensors are often used. Depending on the purpose, these are mainly motion sensors, microphones, weight sensors and photo cameras. While several of these technologies are commercially available, there is still a lack of publicly available algorithms that are needed to process and interpret the resulting data. We argue that future work should focus on developing robust algorithms and validating these technologies in real-life settings. Combining technologies (e.g., prompting individuals for self-reports at sensed, opportune moments) is a promising route toward ecologically valid studies of eating behavior.

An End-to-End Energy-Efficient Approach for Intake Detection With Low Inference Time Using Wrist-Worn Sensor

Automated detection of intake gestures with wearable sensors has been a critical area of research for advancing our understanding and ability to intervene in people's eating behavior. Numerous algorithms have been developed and evaluated in terms of accuracy. However, ensuring the system is not only accurate in making predictions but also efficient in doing so is critical for real-world deployment. Despite the growing research on accurate detection of intake gestures using wearables, many of these algorithms are often energy inefficient, impeding on-device deployment for continuous and real-time monitoring of diet. This article presents a template-based optimized multicenter classifier that enables accurate intake gesture detection while maintaining low-inference time and energy consumption using a wrist-worn accelerometer and gyroscope. We designed an Intake Gesture Counter smartphone application (CountING) and validated the practicality of our algorithm against seven state-of-the-art approaches on three public datasets (In-lab FIC, Clemson, and OREBA). Compared with other methods, we achieved optimal accuracy (81.60% F1 score) and very low inference time (15.97 msec per 2.20-sec data sample) on the Clemson dataset, and among the top performing algorithms, we achieve comparable accuracy (83.0% F1 score compared with 85.6% in the top performing algorithm) but superior inference time (13.8x faster, 33.14 msec per 2.20-sec data sample) on the In-lab FIC dataset and comparable accuracy (83.40% F1 score compared with 88.10% in the top-performing algorithm) but superior inference time (33.9x faster, 16.71 msec inference time per 2.20-sec data sample) on the OREBA dataset. On average, our approach achieved a 25-hour battery lifetime (44% to 52% improvement over state-of-the-art approaches) when tested on a commercial smartwatch for continuous real-time detection. Our approach demonstrates an effective and efficient method, enabling real-time intake gesture detection using wrist-worn devices in longitudinal studies.

Clinical assessment for diet prescription

Accurate nutritional assessment based on dietary intake, physical activity, genetic makeup, and metabolites is required to prevent from developing and/or to treat people suffering from malnutrition as well as other nutrition related health issues. Nutritional screening ought to be considered as an essential part of clinical assessment for every patient on admission to healthcare setups, as well as on change in clinical conditions. Therefore, a detailed nutritional assessment must be performed every time nutritional imbalances are observed or suspected. In this review we have explored different techniques used for nutritional and physical activity assessment. Dietary Intake (DI) assessment is a multidimensional and complex process. Traditionally, dietary intake is assessed through self-report techniques, but due to limitations like biases, random errors, misestimations, and nutrient databases-linked errors, questions arise about the adequacy of self-reporting dietary intake procedures. Despite the limitations in assessing dietary intake (DI) and physical activity (PA), new methods and improved technologies such as biomarkers analysis, blood tests, genetic assessments, metabolomic analysis, DEXA (Dual-energy X-ray absorptiometry), MRI (Magnetic resonance imaging), and CT (computed tomography) scanning procedures have made much progress in the improvement of these measures. Genes also plays a crucial role in dietary intake and physical activity. Similarly, metabolites are also involved in different nutritional pathways. This is why integrating knowledge about the genetic and metabolic markers along with the latest technologies for dietary intake (DI) and physical activity (PA) assessment holds the key for accurately assessing one's nutritional status and prevent malnutrition and its related complications.

Combining passive eating monitoring and ecological momentary assessment to characterize dietary lapses from a lifestyle modification intervention

Dietary lapses (i.e., specific instances of nonadherence to recommended dietary goals) contribute to suboptimal weight loss outcomes during lifestyle modification programs. Passive eating monitoring could enhance lapse measurement via objective assessment of eating characteristics that could be markers for lapse (e.g., more bites consumed). The purpose of this study was to evaluate if passively-inferred eating characteristics (i.e., bites, eating duration, and eating rate), measured via wrist-worn device, could distinguish dietary lapses from non-lapse eating. Adults (n = 25) with overweight/obesity received a 24-week lifestyle modification intervention. Participants completed ecological momentary assessment (EMA; repeated smartphone surveys) biweekly to self-report on dietary lapses and non-lapse eating episodes. Participants wore a wrist device that captured continuous wrist motion. Previously-validated algorithms inferred eating episodes from wrist data, and calculated bite count, duration, and rate (seconds per bite). Mixed effects logistic regressions revealed no simple effects of bite count, duration, or eating rate on the likelihood of dietary lapse. Moderation analyses revealed that eating episodes in the evening were more likely to be lapses if they involved fewer bites (B = -0.16, p < .05), were shorter (B = -0.54, p < .05), or had a slower rate (B = 1.27, p < .001). Statistically significant interactions between eating characteristics (Bs = -0.30 to -0.08, ps < .001) revealed two distinct patterns. Eating episodes that were 1. smaller, slower, and shorter than average, or 2. larger, quicker, and longer than average were associated with increased probability of lapse. This study is the first to use objective eating monitoring to characterize dietary lapses throughout a lifestyle modification intervention. Results demonstrate the potential of sensors to identify non-adherence using only patterns of passively-sensed eating characteristics, thereby minimizing the need for self-report in future studies. CLINICAL TRIALS REGISTRY NUMBER: NCT03739151.

Smart Piezoelectric-Based Wearable System for Calorie Intake Estimation Using Machine Learning

Eating an appropriate food volume, maintaining the required calorie count, and making good nutritional choices are key factors for reducing the risk of obesity, which has many consequences such as Osteoarthritis (OA) that affects the patient’s knee. In this paper, we present a wearable sensor in the form of a necklace embedded with a piezoelectric sensor, that detects skin movement from the lower trachea while eating. In contrast to the previous state-of-the-art piezoelectric sensor-based system that used spectral features, our system fully exploits temporal amplitude-varying signals for optimal features, and thus classifies foods more accurately. Through evaluation of the frame length and the position of swallowing in the frame, we found the best performance was with a frame length of 30 samples (1.5 s), with swallowing located towards the end of the frame. This demonstrates that the chewing sequence carries important information for classification. Additionally, we present a new approach in which the weight of solid food can be estimated from the swallow count, and the calorie count of food can be calculated from their estimated weight. Our system based on a smartphone app helps users live healthily by providing them with real-time feedback about their ingested food types, volume, and calorie count.

Antagonistically Functionalized Diatom Biosilica for Bio-Triboelectric Generators

Although biomaterial-based triboelectric nanogenerators (Bio-TENGs) for use in wearable electronics and implantable sensors have been developed, power generation is not suitable for satisfying the basic requirements for practical applications. Here, to greatly enhance output performances of Bio-TENG devices, an antagonistic approach of diatom frustules (DFs) with amine and fluorine chemical functionalizations is reported. The DFs are treated with piranha solution to increase the density of hydroxyl groups and tribo-positive and tribo-negative composite films are designed with antagonistically functionalized DFs. The tribo-positive composites having electron donating functionality consist of aminated DFs and cellulose nanocrystals (CNCs), while the tribo-negative composite is composed of fluorinated DFs and polydimethylsiloxane (PDMS). An antagonistically and chemically functionalized TENG (ACF TENG) with an efficient contact area of 9.6 cm² under a force of 8 N and a frequency of 5 Hz exhibits an output voltage of 248 V, a short-circuit current of 16.4 µA, and a power density of 2.01 W m^-2 , which is 16.6 times higher than a reference (CNC:PDMS) TENG. This study shows a simple antagonistic approach for chemical functionalization as an efficient method to manipulate the tribo-polarity of bio-additives for enhancing power generation of Bio-TENGs.

Applications and Techniques for Fast Machine Learning in Science

In this community review report, we discuss applications and techniques for fast machine learning (ML) in science-the concept of integrating powerful ML methods into the real-time experimental data processing loop to accelerate scientific discovery. The material for the report builds on two workshops held by the Fast ML for Science community and covers three main areas: applications for fast ML across a number of scientific domains; techniques for training and implementing performant and resource-efficient ML algorithms; and computing architectures, platforms, and technologies for deploying these algorithms. We also present overlapping challenges across the multiple scientific domains where common solutions can be found. This community report is intended to give plenty of examples and inspiration for scientific discovery through integrated and accelerated ML solutions. This is followed by a high-level overview and organization of technical advances, including an abundance of pointers to source material, which can enable these breakthroughs.

Predicting the macronutrient composition of mixed meals from dietary biomarkers in blood

Diet monitoring is an essential intervention component for a number of diseases, from type 2 diabetes to cardiovascular diseases. However, current methods for diet monitoring are burdensome and often inaccurate. In prior work, we showed that continuous glucose monitors (CGMs) may be used to predict the macronutrients in a meal (e.g., carbohydrates, protein, and fat) by analyzing the shape of the post-prandial glucose response. The objective of this study was to examine a number of additional dietary biomarkers in blood by their ability to improve the prediction of meal macronutrients, compared to using CGMs alone. As our experimental method, we conducted a nutritional study where (n=10) participants consumed nine different mixed meals with varied but known macronutrient amounts, and we analyzed the concentration of 33 dietary biomarkers (including amino acids and their combinations, insulin, triglycerides, and 3 independent measures of glucose) at various times post-prandially. As our computational method, we built machine learning models to predict the macronutrient amounts from (1) individual biomarkers and (2) their combinations. The major result from this work is that the additional blood biomarkers provide complementary information, and more importantly, achieve higher prediction performance for the three macronutrients in terms of normalized root mean squared error (carbohydrates: 22.9%; protein: 23.4%; fat: 32.3%) than CGMs alone (carbohydrates: 28.2%, p = 0.08; protein: 42.9%, p<0.001; fat: 41.4%, p<0.05}). Our main conclusion is that augmenting CGMs to measure these additional dietary biomarkers improves macronutrient prediction performance, and may ultimately lead to the development of automated methods to monitor monitor nutritional intake. This work is significant to biomedical research as it provides a potential solution to the long-standing problem of diet monitoring, facilitating new interventions for a number of diseases.

Optimizing a Just-in-Time Adaptive Intervention to Improve Dietary Adherence in Behavioral Obesity Treatment: Protocol for a Microrandomized Trial

BACKGROUND

Behavioral obesity treatment (BOT) is a gold standard approach to weight loss and reduces the risk of cardiovascular disease. However, frequent lapses from the recommended diet stymie weight loss and prevent individuals from actualizing the health benefits of BOT. There is a need for innovative treatment solutions to improve adherence to the prescribed diet in BOT.

OBJECTIVE

The aim of this study is to optimize a smartphone-based just-in-time adaptive intervention (JITAI) that uses daily surveys to assess triggers for dietary lapses and deliver interventions when the risk of lapse is high. A microrandomized trial design will evaluate the efficacy of any interventions (ie, theory-driven or a generic alert to risk) on the proximal outcome of lapses during BOT, compare the effects of theory-driven interventions with generic risk alerts on the proximal outcome of lapse, and examine contextual moderators of interventions.

METHODS

Adults with overweight or obesity and cardiovascular disease risk (n=159) will participate in a 6-month web-based BOT while using the JITAI to prevent dietary lapses. Each time the JITAI detects elevated lapse risk, the participant will be randomized to no intervention, a generic risk alert, or 1 of 4 theory-driven interventions (ie, enhanced education, building self-efficacy, fostering motivation, and improving self-regulation). The primary outcome will be the occurrence of lapse in the 2.5 hours following randomization. Contextual moderators of intervention efficacy will also be explored (eg, location and time of day). The data will inform an optimized JITAI that selects the theory-driven approach most likely to prevent lapses in a given moment.

RESULTS

The recruitment for the microrandomized trial began on April 19, 2021, and is ongoing.

CONCLUSIONS

This study will optimize a JITAI for dietary lapses so that it empirically tailors the provision of evidence-based intervention to the individual and context. The finalized JITAI will be evaluated for efficacy in a future randomized controlled trial of distal health outcomes (eg, weight loss).

TRIAL REGISTRATION

ClinicalTrials.gov NCT04784585; http://clinicaltrials.gov/ct2/show/NCT04784585.

INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID)

DERR1-10.2196/33568.

Association of number of bites and eating speed with energy intake: Wearable technology results under free-living conditions

Personalized weight management strategies are gaining interest. However, knowledge is limited regarding eating habits and association with energy intake, and current technologies limit assessment in free-living situations. We assessed associations between eating behavior and time of day with energy intake using a wearable camera under free-living conditions and explored if obesity modifies the associations. Sixteen participants (50% with obesity) recorded free-living eating behaviors using a wearable fish-eye camera for 14 days. Videos were viewed by trained annotators who confirmed number of bites, eating speed, and time of day for each eating episode. Energy intake was determined by a trained dietitian performing 24-h diet recalls. Greater number of bites, reduced eating speed, and increased BMI significantly predicted higher energy intake among all participants (P < 0.05, each). There were no significant interactions between obesity and number of bites, eating speed, or time of day (p > 0.05). Greater number of bites and reduced eating speed were significantly associated with higher energy intake in participants without obesity. Results show that under free-living conditions, more bites and slower eating speed predicted higher energy intake when examining consumption of foods with beverages. Obesity did not modify these associations. Findings highlight how eating behaviors can impact energy balance and can inform weight management interventions using wearable technology.

Single-Stage Intake Gesture Detection Using CTC Loss and Extended Prefix Beam Search

Accurate detection of individual intake gestures is a key step towards automatic dietary monitoring. Both inertial sensor data of wrist movements and video data depicting the upper body have been used for this purpose. The most advanced approaches to date use a two-stage approach, in which (i) frame-level intake probabilities are learned from the sensor data using a deep neural network, and then (ii) sparse intake events are detected by finding the maxima of the frame-level probabilities. In this study, we propose a single-stage approach which directly decodes the probabilities learned from sensor data into sparse intake detections. This is achieved by weakly supervised training using Connectionist Temporal Classification (CTC) loss, and decoding using a novel extended prefix beam search decoding algorithm. Benefits of this approach include (i) end-to-end training for detections, (ii) simplified timing requirements for intake gesture labels, and (iii) improved detection performance compared to existing approaches. Across two separate datasets, we achieve relative F₁ score improvements between 1.9% and 6.2% over the two-stage approach for intake detection and eating/drinking detection tasks, for both video and inertial sensors.

The potential of artificial intelligence in enhancing adult weight loss: a scoping review

OBJECTIVE

To present an overview of how artificial intelligence (AI) could be used to regulate eating and dietary behaviours, exercise behaviours and weight loss.

DESIGN

A scoping review of global literature published from inception to 15 December 2020 was conducted according to Arksey and O'Malley's five-step framework. Eight databases (CINAHL, Cochrane-Central, Embase, IEEE Xplore, PsycINFO, PubMed, Scopus and Web of Science) were searched. Included studies were independently screened for eligibility by two reviewers with good interrater reliability (k = 0·96).

RESULTS

Sixty-six out of 5573 potential studies were included, representing more than 2031 participants. Three tenets of self-regulation were identified - self-monitoring (n 66, 100 %), optimisation of goal setting (n 10, 15·2 %) and self-control (n 10, 15·2 %). Articles were also categorised into three AI applications, namely machine perception (n 50), predictive analytics only (n 6) and real-time analytics with personalised micro-interventions (n 10). Machine perception focused on recognising food items, eating behaviours, physical activities and estimating energy balance. Predictive analytics focused on predicting weight loss, intervention adherence, dietary lapses and emotional eating. Studies on the last theme focused on evaluating AI-assisted weight management interventions that instantaneously collected behavioural data, optimised prediction models for behavioural lapse events and enhance behavioural self-control through adaptive and personalised nudges/prompts. Only six studies reported average weight losses (2·4-4·7 %) of which two were statistically significant.

CONCLUSION

The use of AI for weight loss is still undeveloped. Based on the current study findings, we proposed a framework on the applicability of AI for weight loss but cautioned its contingency upon engagement and contextualisation.

The Behavioral Intervention with Technology for E-Weight Loss Study (BITES): Incorporating Energy Balance Models and the Bite Counter into an Online Behavioral Weight Loss Program

This study evaluated feasibility and acceptability of adding energy balance modeling displayed on weight graphs combined with a wrist-worn bite counting sensor against a traditional online behavioral weight loss program. Adults with a BMI of 27–45 kg/m2 (83.3% women) were randomized to receive a 12-week online behavioral weight loss program with 12 weeks of continued contact (n = 9; base program), the base program plus a graph of their actual and predicted weight change based on individualized physiological parameters (n = 7), or the base program, graph, and a Bite Counter device for monitoring and limiting eating (n = 8). Participants attended weekly clinic weigh-ins plus baseline, midway (12 weeks), and study culmination (24 weeks) assessments of feasibility, acceptability, weight, and behavioral outcomes. In terms of feasibility, participants completed online lessons (M = 7.04 of 12 possible lessons, SD = 4.02) and attended weigh-ins (M = 16.81 visits, SD = 7.24). Six-month retention appears highest among nomogram participants, and weigh-in attendance and lesson completion appear highest in Bite Counter participants. Acceptability was sufficient across groups. Bite Counter use (days with ≥ 2 eating episodes) was moderate (47.8%) and comparable to other studies. Participants lost 4.6% ± 4.5 of their initial body weight at 12 weeks and 4.5% ± 5.8 at 24 weeks. All conditions increased their total physical activity minutes and use of weight control strategies (behavioral outcomes). Although all groups lost weight and the study procedures were feasible, acceptability can be improved with advances in the technology. Participants were satisfied with the online program and nomograms, and future research on engagement, adherence, and integration with other owned devices is needed. ClinicalTrials.gov Identifier: NCT02857595

Counting Bites and Recognizing Consumed Food from Videos for Passive Dietary Monitoring

Assessing dietary intake in epidemiological studies are predominantly based on self-reports, which are subjective, inefficient, and also prone to error. Technological approaches are therefore emerging to provide objective dietary assessments. Using only egocentric dietary intake videos, this work aims to provide accurate estimation on individual dietary intake through recognizing consumed food items and counting the number of bites taken. This is different from previous studies that rely on inertial sensing to count bites, and also previous studies that only recognize visible food items but not consumed ones. As a subject may not consume all food items visible in a meal, recognizing those consumed food items is more valuable. A new dataset that has 1,022 dietary intake video clips was constructed to validate our concept of bite counting and consumed food item recognition from egocentric videos. 12 subjects participated and 52 meals were captured. A total of 66 unique food items, including food ingredients and drinks, were labelled in the dataset along with a total of 2,039 labelled bites. Deep neural networks were used to perform bite counting and food item recognition in an end-to-end manner. Experiments have shown that counting bites directly from video clips can reach 74.15% top-1 accuracy (classifying between 0-4 bites in 20-second clips), and a MSE value of 0.312 (when using regression). Our experiments on video-based food recognition also show that recognizing consumed food items is indeed harder than recognizing visible ones, with a drop of 25% in F1 score.

Artificial Neural Network-Based Automatic Detection of Food Intake for Neuromodulation in Treating Obesity and Diabetes

PURPOSE

Neuromodulation, such as vagal nerve stimulation and intestinal electrical stimulation, has been introduced for the treatment of obesity and diabetes. Ideally, neuromodulation should be applied automatically after food intake. The purpose of this study was to develop a method of automatic food intake detection through dynamic analysis of heart rate variability (HRV).

MATERIALS AND METHODS

Two experiments were conducted: (1) a small sample series with a standard test meal and (2) a large sample series with varying meal size. Electrocardiograms (ECGs) were collected in the fasting and postprandial states. Each ECG was processed to compute the HRV. For each HRV segment, time- and frequency-domain features were derived and used as inputs to train and test an artificial neural network (ANN). The ANN was trained and tested with different cross-validation methods.

RESULTS

The highest classification accuracy reached with leave-one-subject-out-leave-one-sample-out cross-validation was 0.93 in experiment 1 and 0.88 in experiment 2. Retraining the ANN on recordings of a subject drastically increased the achieved accuracy for that subject to values of 0.995 and 0.95 in experiments 1 and 2, respectively.

CONCLUSIONS

Automatic food intake detection by ANNs, using features from the HRV, is feasible and may have a great potential for neuromodulation-based treatments of meal-related disorders.

"Automatic Ingestion Monitor Version 2" - A Novel Wearable Device for Automatic Food Intake Detection and Passive Capture of Food Images

Use of food image capture and/or wearable sensors for dietary assessment has grown in popularity. "Active" methods rely on the user to take an image of each eating episode. "Passive" methods use wearable cameras that continuously capture images. Most of "passively" captured images are not related to food consumption and may present privacy concerns. In this paper, we propose a novel wearable sensor (Automatic Ingestion Monitor, AIM-2) designed to capture images only during automatically detected eating episodes. The capture method was validated on a dataset collected from 30 volunteers in the community wearing the AIM-2 for 24h in pseudo-free-living and 24h in a free-living environment. The AIM-2 was able to detect food intake over 10-second epochs with a (mean and standard deviation) F1-score of 81.8 ± 10.1%. The accuracy of eating episode detection was 82.7%. Out of a total of 180,570 images captured, 8,929 (4.9%) images belonged to detected eating episodes. Privacy concerns were assessed by a questionnaire on a scale 1-7. Continuous capture had concern value of 5.0 ± 1.6 (concerned) while image capture only during food intake had concern value of 1.9 ±1.7 (not concerned). Results suggest that AIM-2 can provide accurate detection of food intake, reduce the number of images for analysis and alleviate the privacy concerns of the users.

A Data Driven End-to-End Approach for In-the-Wild Monitoring of Eating Behavior Using Smartwatches

The increased worldwide prevalence of obesity has sparked the interest of the scientific community towards tools that objectively and automatically monitor eating behavior. Despite the study of obesity being in the spotlight, such tools can also be used to study eating disorders (e.g. anorexia nervosa) or provide a personalized monitoring platform for patients or athletes. This paper presents a complete framework towards the automated i) modeling of in-meal eating behavior and ii) temporal localization of meals, from raw inertial data collected in-the-wild using commercially available smartwatches. Initially, we present an end-to-end Neural Network which detects food intake events (i.e. bites). The proposed network uses both convolutional and recurrent layers that are trained simultaneously. Subsequently, we show how the distribution of the detected bites throughout the day can be used to estimate the start and end points of meals, using signal processing algorithms. We perform extensive evaluation on each framework part individually. Leave-one-subject-out (LOSO) evaluation shows that our bite detection approach outperforms four state-of-the-art algorithms towards the detection of bites during the course of a meal (0.923 F1 score). Furthermore, LOSO and held-out set experiments regarding the estimation of meal start/end points reveal that the proposed approach outperforms a relevant approach found in the literature (Jaccard Index of 0.820 and 0.821 for the LOSO and held-out experiments, respectively). Experiments are performed using our publicly available FIC and the newly introduced FreeFIC datasets.

Combining ecological momentary assessment, wrist-based eating detection, and dietary assessment to characterize dietary lapse: A multi-method study protocol

OBJECTIVES

Behavioral obesity treatment (BOT) produces clinically significant weight loss and health benefits for many individuals with overweight/obesity. Yet, many individuals in BOT do not achieve clinically significant weight loss and/or experience weight regain. Lapses (i.e., eating that deviates from the BOT prescribed diet) could explain poor outcomes, but the behavior is understudied because it can be difficult to assess. We propose to study lapses using a multi-method approach, which allows us to identify objectively-measured characteristics of lapse behavior (e.g., eating rate, duration), examine the association between lapse and weight change, and estimate nutrition composition of lapse.

METHOD

We are recruiting participants (n = 40) with overweight/obesity to enroll in a 24-week BOT. Participants complete biweekly 7-day ecological momentary assessment (EMA) to self-report on eating behavior, including dietary lapses. Participants continuously wear the wrist-worn ActiGraph Link to characterize eating behavior. Participants complete 24-hour dietary recalls via structured interview at 6-week intervals to measure the composition of all food and beverages consumed.

RESULTS

While data collection for this trial is still ongoing, we present data from three pilot participants who completed EMA and wore the ActiGraph to illustrate the feasibility, benefits, and challenges of this work.

CONCLUSION

This protocol will be the first multi-method study of dietary lapses in BOT. Upon completion, this will be one of the largest published studies of passive eating detection and EMA-reported lapse. The integration of EMA and passive sensing to characterize eating provides contextually rich data that will ultimately inform a nuanced understanding of lapse behavior and enable novel interventions.Trial registration: Registered clinical trial NCT03739151; URL: https://clinicaltrials.gov/ct2/show/NCT03739151.

The Age of Artificial Intelligence: Use of Digital Technology in Clinical Nutrition

PURPOSE OF REVIEW

Computing advances over the decades have catalyzed the pervasive integration of digital technology in the medical industry, now followed by similar applications for clinical nutrition. This review discusses the implementation of such technologies for nutrition, ranging from the use of mobile apps and wearable technologies to the development of decision support tools for parenteral nutrition and use of telehealth for remote assessment of nutrition.

RECENT FINDINGS

Mobile applications and wearable technologies have provided opportunities for real-time collection of granular nutrition-related data. Machine learning has allowed for more complex analyses of the increasing volume of data collected. The combination of these tools has also translated into practical clinical applications, such as decision support tools, risk prediction, and diet optimization.

SUMMARY

The state of digital technology for clinical nutrition is still young, although there is much promise for growth and disruption in the future.

Real-Time Food Intake Monitoring Using Wearable Egocnetric Camera

With technological advancement, wearable egocentric camera systems have extensively been studied to develop food intake monitoring devices for the assessment of eating behavior. This paper provides a detailed description of the implementation of CNN based image classifier in the Cortex-M7 microcontroller. The proposed network classifies the captured images by the wearable egocentric camera as food and no food images in real-time. This real-time food image detection can potentially lead the monitoring devices to consume less power, less storage, and more user-friendly in terms of privacy by saving only images that are detected as food images. A derivative of pre-trained MobileNet is trained to detect food images from camera captured images. The proposed network needs 761.99KB of flash and 501.76KB of RAM to implement which is built for an optimal trade-off between accuracy, computational cost, and memory footprint considering implementation on a Cortex-M7 microcontroller. The image classifier achieved an average precision of 82%±3% and an average F-score of 74%±2% while testing on 15343 (2127 food images and 13216 no food images) images of five full days collected from five participants.

Future Directions for Integrative Objective Assessment of Eating Using Wearable Sensing Technology

Established methods for nutritional assessment suffer from a number of important limitations. Diaries are burdensome to complete, food frequency questionnaires only capture average food intake, and both suffer from difficulties in self estimation of portion size and biases resulting from misreporting. Online and app versions of these methods have been developed, but issues with misreporting and portion size estimation remain. New methods utilizing passive data capture are required that address reporting bias, extend timescales for data collection, and transform what is possible for measuring habitual intakes. Digital and sensing technologies are enabling the development of innovative and transformative new methods in this area that will provide a better understanding of eating behavior and associations with health. In this article we describe how wrist-worn wearables, on-body cameras, and body-mounted biosensors can be used to capture data about when, what, and how much people eat and drink. We illustrate how these new techniques can be integrated to provide complete solutions for the passive, objective assessment of a wide range of traditional dietary factors, as well as novel measures of eating architecture, within person variation in intakes, and food/nutrient combinations within meals. We also discuss some of the challenges these new approaches will bring.

Automatic Count of Bites and Chews From Videos of Eating Episodes

Methods for measuring of eating behavior (known as meal microstructure) often rely on manual annotation of bites, chews, and swallows on meal videos or wearable sensor signals. The manual annotation may be time consuming and erroneous, while wearable sensors may not capture every aspect of eating (e.g. chews only). The aim of this study is to develop a method to detect and count bites and chews automatically from meal videos. The method was developed on a dataset of 28 volunteers consuming unrestricted meals in the laboratory under video observation. First, the faces in the video (regions of interest, ROI) were detected using Faster R-CNN. Second, a pre-trained AlexNet was trained on the detected faces to classify images as a bite/no bite image. Third, the affine optical flow was applied in consecutively detected faces to find the rotational movement of the pixels in the ROIs. The number of chews in a meal video was counted by converting the 2-D images to a 1-D optical flow parameter and finding peaks. The developed bite and chew count algorithm was applied to 84 meal videos collected from 28 volunteers. A mean accuracy (±STD) of 85.4% (±6.3%) with respect to manual annotation was obtained for the number of bites and 88.9% (±7.4%) for the number of chews. The proposed method for an automatic bite and chew counting shows promising results that can be used as an alternative solution to manual annotation.

NeckSense: A Multi-Sensor Necklace for Detecting Eating Activities in Free-Living Conditions

We present the design, implementation, and evaluation of a multi-sensor, low-power necklace, NeckSense, for automatically and unobtrusively capturing fine-grained information about an individual's eating activity and eating episodes, across an entire waking day in a naturalistic setting. NeckSense fuses and classifies the proximity of the necklace from the chin, the ambient light, the Lean Forward Angle, and the energy signals to determine chewing sequences, a building block of the eating activity. It then clusters the identified chewing sequences to determine eating episodes. We tested NeckSense on 11 participants with and 9 participants without obesity, across two studies, where we collected more than 470 hours of data in a naturalistic setting. Our results demonstrate that NeckSense enables reliable eating detection for individuals with diverse body mass index (BMI) profiles, across an entire waking day, even in free-living environments. Overall, our system achieves an F1-score of 81.6% in detecting eating episodes in an exploratory study. Moreover, our system can achieve an F1-score of 77.1% for episodes even in an all-day-long free-living setting. With more than 15.8 hours of battery life, NeckSense will allow researchers and dietitians to better understand natural chewing and eating behaviors. In the future, researchers and dietitians can use NeckSense to provide appropriate real-time interventions when an eating episode is detected or when problematic eating is identified.

Emotional eating in healthy individuals and patients with an eating disorder: evidence from psychometric, experimental and naturalistic studies

Emotional eating has traditionally been defined as (over)eating in response to negative emotions. Such overeating can impact general health because of excess energy intake and mental health, due to the risks of developing binge eating. Yet, there is still significant controversy on the validity of the emotional eating concept and several theories compete in explaining its mechanisms. The present paper examines the emotional eating construct by reviewing and integrating recent evidence from psychometric, experimental and naturalistic research. Several psychometric questionnaires are available and some suggest that emotions differ fundamentally in how they affect eating (i.e. overeating, undereating). However, the general validity of such questionnaires in predicting actual food intake in experimental studies is questioned and other eating styles such as restrained eating seem to be better predictors of increased food intake under negative emotions. Also, naturalistic studies, involving the repeated assessment of momentary emotions and eating behaviour in daily life, are split between studies supporting and studies contradicting emotional eating in healthy individuals. Individuals with clinical forms of overeating (i.e. binge eating) consistently show positive relationships between negative emotions and eating in daily life. We will conclude with a summary of the controversies around the emotional eating construct and provide recommendations for future research and treatment development.

Using Intelligent Personal Annotations to Improve Human Activity Recognition for Movements in Natural Environments

Personal tracking algorithms for health monitoring are critical for understanding an individual's life-style and personal choices in natural environments (NE). In order to train such tracking algorithms in NE, however, annotated data is needed, particularly when tracking a variety of activities of daily living. These algorithms are often trained in laboratory settings, with expectations that they will perform equally well in NE, which is often not the case; they must be trained on annotated data collected in NE and wearable computers provide opportunities to collect such data, though the process is burdensome. Therefore, we propose an intelligent scoring algorithm that limits the number of user annotation requests through the confidence of predictions generated by the tracking algorithm and automatically annotating data with high confidence. We enhance our scoring algorithm by providing improvements in our tracking algorithm by obtaining context data from nearable sensors. Each specific context of a user bounds the set of activities that can likely occur, which in turn improves the tracking algorithm and confidence. Finally, we propose a hierarchical annotation approach, where repeated use allows us to ask for detailed annotations that differentiate fine-grained differences in ways individuals perform activities. We validate our approach in a diet monitoring case study. We vary the number of annotations requested per day to evaluate model accuracy; we improve accuracy in NE by 8% when restricting requests to 20 per day and improve F1-score of activities by 11% with hierarchical annotations, while discussing implementation, accuracy, and power consumption in real-time use.

Indoor acoustic localization: a survey

Applications of localization range from body tracking, gesture capturing, indoor plan construction to mobile health sensing. Technologies such as inertial sensors, radio frequency signals and cameras have been deeply excavated to locate targets. Among all the technologies, the acoustic signal gains enormous favor considering its comparatively high accuracy with common infrastructure and low time latency. Range-based localization falls into two categories: absolute range and relative range. Different mechanisms, such as Time of Flight, Doppler effect and phase shift, are widely studied to achieve the two genres of localization. The subcategories show distinguishing features but also face diverse challenges. In this survey, we present a comprehensive overview on various indoor localization systems derived from the various mechanisms. We also discuss the remaining issues and the future work.

Counting Bites With Bits: Expert Workshop Addressing Calorie and Macronutrient Intake Monitoring

BACKGROUND

Conventional diet assessment approaches such as the 24-hour self-reported recall are burdensome, suffer from recall bias, and are inaccurate in estimating energy intake. Wearable sensor technology, coupled with advanced algorithms, is increasingly showing promise in its ability to capture behaviors that provide useful information for estimating calorie and macronutrient intake.

OBJECTIVE

This paper aimed to summarize current technological approaches to monitoring energy intake on the basis of expert opinion from a workshop panel and to make recommendations to advance technology and algorithms to improve estimation of energy expenditure.

METHODS

A 1-day invitational workshop sponsored by the National Science Foundation was held at Northwestern University. A total of 30 participants, including population health researchers, engineers, and intervention developers, from 6 universities and the National Institutes of Health participated in a panel discussing the state of evidence with regard to monitoring calorie intake and eating behaviors.

RESULTS

Calorie monitoring using technological approaches can be characterized into 3 domains: (1) image-based sensing (eg, wearable and smartphone-based cameras combined with machine learning algorithms); (2) eating action unit (EAU) sensors (eg, to measure feeding gesture and chewing rate); and (3) biochemical measures (eg, serum and plasma metabolite concentrations). We discussed how each domain functions, provided examples of promising solutions, and highlighted potential challenges and opportunities in each domain. Image-based sensor research requires improved ground truth (context and known information about the foods), accurate food image segmentation and recognition algorithms, and reliable methods of estimating portion size. EAU-based domain research is limited by the understanding of when their systems (device and inference algorithm) succeed and fail, need for privacy-protecting methods of capturing ground truth, and uncertainty in food categorization. Although an exciting novel technology, the challenges of biochemical sensing range from a lack of adaptability to environmental effects (eg, temperature change) and mechanical impact, instability of wearable sensor performance over time, and single-use design.

CONCLUSIONS

Conventional approaches to calorie monitoring rely predominantly on self-reports. These approaches can gain contextual information from image-based and EAU-based domains that can map automatically captured food images to a food database and detect proxies that correlate with food volume and caloric intake. Although the continued development of advanced machine learning techniques will advance the accuracy of such wearables, biochemical sensing provides an electrochemical analysis of sweat using soft bioelectronics on human skin, enabling noninvasive measures of chemical compounds that provide insight into the digestive and endocrine systems. Future computing-based researchers should focus on reducing the burden of wearable sensors, aligning data across multiple devices, automating methods of data annotation, increasing rigor in studying system acceptability, increasing battery lifetime, and rigorously testing validity of the measure. Such research requires moving promising technological solutions from the controlled laboratory setting to the field.

The importance of field experiments in testing of sensors for dietary assessment and eating behavior monitoring

The field of sensor-based dietary assessment and behavioral monitoring is rapidly expanding. New devices and methods for detection for food intake and characterization of ingestive behavior, energy intake and nutrition have been introduced. Quite often the testing of new devices is limited to restricted meals in laboratory setting, which has the advantage of being controlled, but may not be representative of real life conditions. To illustrate the importance of field testing, we performed a statistical comparison of meal microstructure metrics acquired in laboratory versus a field-like study. In the laboratory study, individual participants ate a self-selected meal in isolation. In the field-like study, participants consumed selfselected meals in a social setting. In both studies, the participants were monitored by both video observation and wearable food intake sensors. Statistically significant differences were observed in the duration of the meals, duration of ingestion, number of bouts of ingestion, duration of pauses between ingestive bouts, number of bites and other metrics. These results suggest that field testing presents a far different picture of ingestion process and therefore is needed for any realistic assessment of the monitoring devices.

Effects of a novel bites, steps and eating rate-focused weight loss randomised controlled trial intervention on body weight and eating behaviours

BACKGROUND

Eating rate (ER), comprising the amount of food consumed per unit of time, is associated with obesity and energy intake (EI).

METHODS

The present study tested whether adding a self-monitoring wearable device to a multifaceted 8-week weight loss intervention increased weight loss. In addition, the device's effect on secondary change outcomes in EI, ER and estimated energy expenditure was explored. Tertiary outcomes included examining eating behaviours measured by the Weight-Related Eating Questionnaire (WREQ). Seventy-two adults who were overweight or obese [mean (SD) age, 37.7 (15.3) years; body mass index, 31.3 (3.2) kg m^-2 ] were randomised into two groups: intervention workbook plus device (WD) or intervention workbook only (WO). Three 24-h dietary recalls were obtained before weeks 0 and 8. Participants were weighed, consumed a test meal and completed 7-day Physical Activity Recall and WREQ at weeks 0 and 8.

RESULTS

There was no significant difference between WD and WO groups with respect to weight change [-0.46 (1.11) vs. 0.26 (0.82) kg, respectively], ER, EI, energy expenditure or WREQ scores, although there were significant changes over time, and within-group changes on all of these variables. At week 8, participants were dichotomised into weight loss or weight stable/gainers groups. A significant time by group change was seen in susceptibility to external cues scores, with significant time effects for susceptibility and restraint.

CONCLUSIONS

An intervention focused on reducing ER, energy density and increasing steps was effective for weight loss, although the wearable device provided no additional benefit. Participants with higher susceptibility to external eating may be more responsive to this intervention.

Moving beyond self-report data collection in the natural environment: A review of the past and future directions for ambulatory assessment in eating disorders

OBJECTIVE

In recent years, ecological momentary assessment (EMA) has been used to repeatedly assess eating disorder (ED) symptoms in naturalistic settings, which has allowed for increased understanding of temporal processes that potentiate ED behaviors. However, there remain notable limitations of self-report EMA, and with the rapid proliferation of technology there are ever-increasing possibilities to improve ambulatory assessment methods to further the understanding and treatment of EDs. Therefore, the purpose of this review was to (a) systematically review the studies in EDs that have utilized ambulatory assessment methods other than self-report, and (b) provide directions for future research and clinical applications.

METHOD

A systematic literature search of electronic databases was conducted, and data regarding study characteristics and methodological quality were extracted.

RESULTS

The search identified 17 studies that used ambulatory assessment methods to gather objective data, and focused primarily on autonomic functioning, physical activity, and cognitive processes in ED and control groups.

DISCUSSION

Together the literature demonstrates the promise of using a range of ecologically valid ambulatory assessment approaches in EDs, though there remains limited research that has utilized methods other than self-report (e.g., wearable sensors), particularly in recent years. Going forward, there are several technology-enhanced momentary assessment methods that have potential to improve the understanding and treatment of EDs.

Food Craving, Seeking, and Consumption Behaviors: Conceptual Phases and Assessment Methods Used in Animal and Human Studies

What drives us to eat? It is one of the most fundamental questions in the obesity research field which have been investigated for centuries. Numerous novel in vivo technologies in the neuroscience field allows us to reevaluate the multiple components and phases of food-related behaviors. Focused on the cognitive, executive, behavioral and temporal aspects, food-related behaviors can be distinguished into appetitive phase (food craving→food seeking) and consummatory phase (food consumption). Food craving phase is an internal state or stage in which the animal has the motivation to eat the food but there is no actual food specific behaviors or actions. Food seeking phase entails repeated behaviors with a food searching purpose until the animal discovers the food (or food-related cue) and the approach behavior stage after the discovery of food. Food consumption phase is the step that the animal grabs, chews and intake the food. This review will specifically focus on characteristics and evaluation methods for each phase of food-related behavior in rodent, non-human primates and human.

Nutritional Risk Screening and Assessment

Malnutrition is an independent risk factor that negatively influences patients’ clinical outcomes, quality of life, body function, and autonomy. Early identification of patients at risk of malnutrition or who are malnourished is crucial in order to start a timely and adequate nutritional support. Nutritional risk screening, a simple and rapid first-line tool to detect patients at risk of malnutrition, should be performed systematically in patients at hospital admission. Patients with nutritional risk should subsequently undergo a more detailed nutritional assessment to identify and quantify specific nutritional problems. Such an assessment includes subjective and objective parameters such as medical history, current and past dietary intake (including energy and protein balance), physical examination and anthropometric measurements, functional and mental assessment, quality of life, medications, and laboratory values. Nutritional care plans should be developed in a multidisciplinary approach, and implemented to maintain and improve patients’ nutritional condition. Standardized nutritional management including systematic risk screening and assessment may also contribute to reduced healthcare costs. Adequate and timely implementation of nutritional support has been linked with favorable outcomes such as a decrease in length of hospital stay, reduced mortality, and reductions in the rate of severe complications, as well as improvements in quality of life and functional status. The aim of this review article is to provide a comprehensive overview of nutritional screening and assessment methods that can contribute to an effective and well-structured nutritional management (process cascade) of hospitalized patients.

Defining Adherence to Mobile Dietary Self-Monitoring and Assessing Tracking Over Time: Tracking at Least Two Eating Occasions per Day Is Best Marker of Adherence within Two Different Mobile Health Randomized Weight Loss Interventions

BACKGROUND

Mobile dietary self-monitoring methods allow for objective assessment of adherence to self-monitoring; however, the best way to define self-monitoring adherence is not known.

OBJECTIVE

The objective was to identify the best criteria for defining adherence to dietary self-monitoring with mobile devices when predicting weight loss.

DESIGN

This was a secondary data analysis from two 6-month randomized trials: Dietary Intervention to Enhance Tracking with Mobile Devices (n=42 calorie tracking app or n=39 wearable Bite Counter device) and Self-Monitoring Assessment in Real Time (n=20 kcal tracking app or n=23 photo meal app).

PARTICIPANTS/SETTING

Adults (n=124; mean body mass index=34.7±5.6) participated in one of two remotely delivered weight-loss interventions at a southeastern university between 2015 and 2017.

INTERVENTION

All participants received the same behavioral weight loss information via twice-weekly podcasts. Participants were randomly assigned to a specific diet tracking method.

MAIN OUTCOME MEASURES

Seven methods of tracking adherence to self-monitoring (eg, number of days tracked, and number of eating occasions tracked) were examined, as was weight loss at 6 months.

STATISTICAL ANALYSES PERFORMED

Linear regression models estimated the strength of association (R²) between each method of tracking adherence and weight loss, adjusting for age and sex.

RESULTS

Among all study completers combined (N=91), adherence defined as the overall number of days participants tracked at least two eating occasions explained the most variance in weight loss at 6 months (R²=0.27; P<0.001). Self-monitoring declined over time; all examined adherence methods had fewer than half the sample still tracking after Week 10.

CONCLUSIONS

Using the total number of days at least two eating occasions are tracked using a mobile self-monitoring method may be the best way to assess self-monitoring adherence during weight loss interventions. This study shows that self-monitoring rates decline quickly and elucidates potential times for early interventions to stop the reductions in self-monitoring.

Food Composition Impacts the Accuracy of Wearable Devices When Estimating Energy Intake from Energy-Dense Food

The present study aimed to assess the feasibility and reliability of an a3utomatic food intake measurement device in estimating energy intake from energy-dense foods. Eighteen volunteers aged 20-36 years were recruited from the University of Padova. The device used in the present study was the Bite Counter (Bite Technologies, Pendleton, USA). The rationale of the device is that the wrist movements occurring in the act of bringing food to the mouth present unique patterns that are recognized and recorded by the Bite Counter. Subjects were asked to wear the Bite Counter on the wrist of the dominant hand, to turn the device on before the first bite and to turn it off once he or she finished his or her meal. The accuracy of caloric intake was significantly different among the methods used. In addition, the device's accuracy in estimating energy intake varied according to the type and amount of macronutrients present, and the difference was independent of the number of bites recorded. Further research is needed to overcome the current limitations of wearable devices in estimating caloric intake, which is not independent of the food being eaten.

Assessing Eating Behaviour Using Upper Limb Mounted Motion Sensors: A Systematic Review

Wearable motion tracking sensors are now widely used to monitor physical activity, and have recently gained more attention in dietary monitoring research. The aim of this review is to synthesise research to date that utilises upper limb motion tracking sensors, either individually or in combination with other technologies (e.g., cameras, microphones), to objectively assess eating behaviour. Eleven electronic databases were searched in January 2019, and 653 distinct records were obtained. Including 10 studies found in backward and forward searches, a total of 69 studies met the inclusion criteria, with 28 published since 2017. Fifty studies were conducted exclusively in laboratory settings, 13 exclusively in free-living settings, and three in both settings. The most commonly used motion sensor was an accelerometer (64) worn on the wrist (60) or lower arm (5), while in most studies (45), accelerometers were used in combination with gyroscopes. Twenty-six studies used commercial-grade smartwatches or fitness bands, 11 used professional grade devices, and 32 used standalone sensor chipsets. The most used machine learning approaches were Support Vector Machine (SVM, n = 21), Random Forest (n = 19), Decision Tree (n = 16), Hidden Markov Model (HMM, n = 10) algorithms, and from 2017 Deep Learning (n = 5). While comparisons of the detection models are not valid due to the use of different datasets, the models that consider the sequential context of data across time, such as HMM and Deep Learning, show promising results for eating activity detection. We discuss opportunities for future research and emerging applications in the context of dietary assessment and monitoring.

Examining changes in respiratory exchange ratio within an 8-week weight loss intervention

BACKGROUND

Maintaining weight loss is difficult, partly as a result of accompanying reductions in fat oxidation. The present study examined fat oxidation [reflected by respiratory exchange ratio (RER)] within an 8-week, self-led weight loss intervention. Changes in RER, body fat (BF%) and estimated energy expenditure (EE) were examined.

METHODS

Twenty-two adults [13 females, nine males; mean (SD) age 34.6 (16.5) years; body mass index 32.0 (4.3) kg m^-2 ] received a self-directed workbook; twelve were also randomised to receive a self-monitoring wrist-worn device. At weeks 0 and 8, RER (indirect calorimetry), BF% (BodPod) and estimated EE [7-day physical activity recall (PAR-EE) were collected. Participants were pooled and paired t-tests were used to examine changes over time. Correlations explored associations among variables. Participants were then dichotomised into weight loss group (WL) or weight stable/gainers group (WSG) and eating behaviours [Intuitive Eating Scale (IES-2)] were examined by 2 × 2 repeated measures multivariate analysis of covariance.

RESULTS

There were no significant changes in RER, body fat percentage and PAR-EE. A significant negative association was found between week 8 PAR-EE and week 8 RER, as well as between BF% change and RER change. There was a significant time by WL versus WSG group effect of IES-2 scores, with the WL group self-reporting significantly increased scores in Eating for Physical Reasons rather than Emotional Reasons (EPR) subscale.

CONCLUSIONS

Increased physical activity after an 8-week weight loss intervention was associated with a higher fasting fat oxidation. Participants who increased EPR scores were more successful in weight loss than those without a change in this subscale.

Statistical models for meal-level estimation of mass and energy intake using features derived from video observation and a chewing sensor

Accurate and objective assessment of energy intake remains an ongoing problem. We used features derived from annotated video observation and a chewing sensor to predict mass and energy intake during a meal without participant self-report. 30 participants each consumed 4 different meals in a laboratory setting and wore a chewing sensor while being videotaped. Subject-independent models were derived from bite, chew, and swallow features obtained from either video observation or information extracted from the chewing sensor. With multiple regression analysis, a forward selection procedure was used to choose the best model. The best estimates of meal mass and energy intake had (mean ± standard deviation) absolute percentage errors of 25.2% ± 18.9% and 30.1% ± 33.8%, respectively, and mean ± standard deviation estimation errors of −17.7 ± 226.9 g and −6.1 ± 273.8 kcal using features derived from both video observations and sensor data. Both video annotation and sensor-derived features may be utilized to objectively quantify energy intake.

Modeling Wrist Micromovements to Measure In-Meal Eating Behavior From Inertial Sensor Data

Overweight and obesity are both associated with in-meal eating parameters such as eating speed. Recently, the plethora of available wearable devices in the market ignited the interest of both the scientific community and the industry toward unobtrusive solutions for eating behavior monitoring. In this paper, we present an algorithm for automatically detecting the in-meal food intake cycles using the inertial signals (acceleration and orientation velocity) from an off-the-shelf smartwatch. We use five specific wrist micromovements to model the series of actions leading to and following an intake event (i.e., bite). Food intake detection is performed in two steps. In the first step, we process windows of raw sensor streams and estimate their micromovement probability distributions by means of a convolutional neural network. In the second step, we use a long short-term memory network to capture the temporal evolution and classify sequences of windows as food intake cycles. Evaluation is performed using a challenging dataset of 21 meals from 12 subjects. In our experiments, we compare the performance of our algorithm against three state-of-the-art approaches, where our approach achieves the highest F1 detection score (0.913 in the leave-one-subject-out experiment). The dataset used in the experiments is available at https://mug.ee.auth.gr/intake-cycle-detection/.

Mobile Ecological Momentary Diet Assessment Methods for Behavioral Research: Systematic Review

BACKGROUND

New methods for assessing diet in research are being developed to address the limitations of traditional dietary assessment methods. Mobile device-assisted ecological momentary diet assessment (mEMDA) is a new dietary assessment method that has not yet been optimized and has the potential to minimize recall biases and participant burden while maximizing ecological validity. There have been limited efforts to characterize the use of mEMDA in behavioral research settings.

OBJECTIVE

The aims of this study were to summarize mEMDA protocols used in research to date, to characterize key aspects of these assessment approaches, and to discuss the advantages and disadvantages of mEMDA compared with the traditional dietary assessment methods as well as implications for future mEMDA research.

METHODS

Studies that used mobile devices and described mEMDA protocols to assess dietary intake were included. Data were extracted according to Preferred Reporting of Systematic Reviews and Meta-Analyses and Cochrane guidelines and then synthesized narratively.

RESULTS

The review included 20 studies with unique mEMDA protocols. Of these, 50% (10/20) used participant-initiated reports of intake at eating events (event-contingent mEMDA), and 50% (10/20) used researcher-initiated prompts requesting that participants report recent dietary intake (signal-contingent mEMDA). A majority of the study protocols (60%, 12/20) enabled participants to use mobile phones to report dietary data. Event-contingent mEMDA protocols most commonly assessed diet in real time, used dietary records for data collection (60%, 6/10), and provided estimates of energy and nutrient intake (60%, 6/10). All signal-contingent mEMDA protocols used a near real-time recall approach with unannounced (ie, random) abbreviated diet surveys. Most signal-contingent protocols (70%, 7/10) assessed the frequency with which (targeted) foods or food groups were consumed. Relatively few (30%, 6/20) studies compared mEMDA with the traditional dietary assessment methods.

CONCLUSIONS

This review demonstrates that mEMDA has the potential to reduce participant burden and recall bias, thus advancing the field beyond current dietary assessment methods while maximizing ecological validity.

Personalised meal eating behaviour analysis via semi-supervised learning

Automated monitoring and analysis of eating behaviour patterns, i.e., "how one eats", has recently received much attention by the research community, owing to the association of eating patterns with health-related problems and especially obesity and its comorbidities. In this work, we introduce an improved method for meal micro-structure analysis. Stepping on a previous methodology of ours that combines feature extraction, SVM micro-movement classification and LSTM sequence modelling, we propose a method to adapt a pretrained IMU-based food intake cycle detection model to a new subject, with the purpose of improving model performance for that subject. We split model training into two stages. First, the model is trained using standard supervised learning techniques. Then, an adaptation step is performed, where the model is fine-tuned on unlabeled samples of the target subject via semisupervised learning. Evaluation is performed on a publicly available dataset that was originally created and used in [1] and has been extended here to demonstrate the effect of the semisupervised approach, where the proposed method improves over the baseline method.