Automatic Analysis of Food Intake and Meal Microstructure Based on Continuous Weight Measurements

Toward Systems Models for Obesity Prevention: A Big Role for Big Data

The relation among the various causal factors of obesity is not well understood, and there remains a lack of viable data to advance integrated, systems models of its etiology. The collection of big data has begun to allow the exploration of causal associations between behavior, built environment, and obesity-relevant health outcomes. Here, the traditional epidemiologic and emerging big data approaches used in obesity research are compared, describing the research questions, needs, and outcomes of 3 broad research domains: eating behavior, social food environments, and the built environment. Taking tangible steps at the intersection of these domains, the recent European Union project "BigO: Big data against childhood obesity" used a mobile health tool to link objective measurements of health, physical activity, and the built environment. BigO provided learning on the limitations of big data, such as privacy concerns, study sampling, and the balancing of epidemiologic domain expertise with the required technical expertise. Adopting big data approaches will facilitate the exploitation of data concerning obesity-relevant behaviors of a greater variety, which are also processed at speed, facilitated by mobile-based data collection and monitoring systems, citizen science, and artificial intelligence. These approaches will allow the field to expand from causal inference to more complex, systems-level predictive models, stimulating ambitious and effective policy interventions.

Perspective: Opportunities and Challenges of Technology Tools in Dietary and Activity Assessment: Bridging Stakeholder Viewpoints

The science and tools of measuring energy intake and output in humans have rapidly advanced in the last decade. Engineered devices such as wearables and sensors, software applications, and Web-based tools are now ubiquitous in both research and consumer environments. The assessment of energy expenditure in particular has progressed from reliance on self-report instruments to advanced technologies requiring collaboration across multiple disciplines, from optics to accelerometry. In contrast, assessing energy intake still heavily relies on self-report mechanisms. Although these tools have improved, moving from paper-based to online reporting, considerable room for refinement remains in existing tools, and great opportunities exist for novel, transformational tools, including those using spectroscopy and chemo-sensing. This report reviews the state of the science, and the opportunities and challenges in existing and emerging technologies, from the perspectives of 3 key stakeholders: researchers, users, and developers. Each stakeholder approaches these tools with unique requirements: researchers are concerned with validity, accuracy, data detail and abundance, and ethical use; users with ease of use and privacy; and developers with high adherence and utilization, intellectual property, licensing rights, and monetization. Cross-cutting concerns include frequent updating and integration of the food and nutrient databases on which assessments rely, improving accessibility and reducing disparities in use, and maintaining reliable technical assistance. These contextual challenges are discussed in terms of opportunities and further steps in the direction of personalized health.

A Data Driven End-to-End Approach for In-the-Wild Monitoring of Eating Behavior Using Smartwatches

The increased worldwide prevalence of obesity has sparked the interest of the scientific community towards tools that objectively and automatically monitor eating behavior. Despite the study of obesity being in the spotlight, such tools can also be used to study eating disorders (e.g. anorexia nervosa) or provide a personalized monitoring platform for patients or athletes. This paper presents a complete framework towards the automated i) modeling of in-meal eating behavior and ii) temporal localization of meals, from raw inertial data collected in-the-wild using commercially available smartwatches. Initially, we present an end-to-end Neural Network which detects food intake events (i.e. bites). The proposed network uses both convolutional and recurrent layers that are trained simultaneously. Subsequently, we show how the distribution of the detected bites throughout the day can be used to estimate the start and end points of meals, using signal processing algorithms. We perform extensive evaluation on each framework part individually. Leave-one-subject-out (LOSO) evaluation shows that our bite detection approach outperforms four state-of-the-art algorithms towards the detection of bites during the course of a meal (0.923 F1 score). Furthermore, LOSO and held-out set experiments regarding the estimation of meal start/end points reveal that the proposed approach outperforms a relevant approach found in the literature (Jaccard Index of 0.820 and 0.821 for the LOSO and held-out experiments, respectively). Experiments are performed using our publicly available FIC and the newly introduced FreeFIC datasets.

Challenges in the interpretation and therapeutic manipulation of human ingestive microstructure

This minireview focuses on the interpretative value of ingestive microstructure by summarizing observations from both rodent and human studies. Preliminary data on the therapeutic manipulation of distinct microstructural components of eating are also outlined. In rodents, the interpretative framework of ingestive microstructure mainly concentrates on deprivation state, palatability, satiation, and the role of learning from previous experiences. In humans, however, the control of eating is further influenced by genetic, psychosocial, cultural, and environmental factors, which add complexity and challenges to the interpretation of the microstructure of meal intake. Nevertheless, the presented findings stress the importance of microstructural analyses of ingestion, as a method to investigate specific behavioral variables that underlie the regulation of appetite control.

Validation of a Deep Learning System for the Full Automation of Bite and Meal Duration Analysis of Experimental Meal Videos

Eating behavior can have an important effect on, and be correlated with, obesity and eating disorders. Eating behavior is usually estimated through self-reporting measures, despite their limitations in reliability, based on ease of collection and analysis. A better and widely used alternative is the objective analysis of eating during meals based on human annotations of in-meal behavioral events (e.g., bites). However, this methodology is time-consuming and often affected by human error, limiting its scalability and cost-effectiveness for large-scale research. To remedy the latter, a novel "Rapid Automatic Bite Detection" (RABiD) algorithm that extracts and processes skeletal features from videos was trained in a video meal dataset (59 individuals; 85 meals; three different foods) to automatically measure meal duration and bites. In these settings, RABiD achieved near perfect agreement between algorithmic and human annotations (Cohen's kappa κ = 0.894; F1-score: 0.948). Moreover, RABiD was used to analyze an independent eating behavior experiment (18 female participants; 45 meals; three different foods) and results showed excellent correlation between algorithmic and human annotations. The analyses revealed that, despite the changes in food (hash vs. meatballs), the total meal duration remained the same, while the number of bites were significantly reduced. Finally, a descriptive meal-progress analysis revealed that different types of food affect bite frequency, although overall bite patterns remain similar (the outcomes were the same for RABiD and manual). Subjects took bites more frequently at the beginning and the end of meals but were slower in-between. On a methodological level, RABiD offers a valid, fully automatic alternative to human meal-video annotations for the experimental analysis of human eating behavior, at a fraction of the cost and the required time, without any loss of information and data fidelity.

Predicting Real-Life Eating Behaviours Using Single School Lunches in Adolescents

Large portion sizes and a high eating rate are associated with high energy intake and obesity. Most individuals maintain their food intake weight (g) and eating rate (g/min) rank in relation to their peers, despite food and environmental manipulations. Single meal measures may enable identification of “large portion eaters” and “fast eaters,” finding individuals at risk of developing obesity. The aim of this study was to predict real-life food intake weight and eating rate based on one school lunch. Twenty-four high-school students with a mean (±SD) age of 16.8 yr (±0.7) and body mass index of 21.9 (±4.1) were recruited, using no exclusion criteria. Food intake weight and eating rate was first self-rated (“Less,” “Average” or “More than peers”), then objectively recorded during one school lunch (absolute weight of consumed food in grams). Afterwards, subjects recorded as many main meals (breakfasts, lunches and dinners) as possible in real-life for a period of at least two weeks, using a Bluetooth connected weight scale and a smartphone application. On average participants recorded 18.9 (7.3) meals during the study. Real-life food intake weight was 327.4 g (±110.6), which was significantly lower (p = 0.027) than the single school lunch, at 367.4 g (±167.2). When the intra-class correlation of food weight intake between the objectively recorded real-life and school lunch meals was compared, the correlation was excellent (R = 0.91). Real-life eating rate was 33.5 g/min (±14.8), which was significantly higher (p = 0.010) than the single school lunch, at 27.7 g/min (±13.3). The intra-class correlation of the recorded eating rate between real-life and school lunch meals was very large (R = 0.74). The participants’ recorded food intake weights and eating rates were divided into terciles and compared between school lunches and real-life, with moderate or higher agreement (κ = 0.75 and κ = 0.54, respectively). In contrast, almost no agreement was observed between self-rated and real-life recorded rankings of food intake weight and eating rate (κ = 0.09 and κ = 0.08, respectively). The current study provides evidence that both food intake weight and eating rates per meal vary considerably in real-life per individual. However, based on these behaviours, most students can be correctly classified in regard to their peers based on single school lunches. In contrast, self-reported food intake weight and eating rate are poor predictors of real-life measures. Finally, based on the recorded individual variability of real-life food intake weight and eating rate, it is not advised to rank individuals based on single recordings collected in real-life settings.

Potential moderators of the portion size effect

AIM

The robust effect of portion size on intake has led to growing interest in why individuals consume more food when served larger portions. A number of explanations have been proposed, and this review aims to provide insight into potential underlying factors by summarizing recent studies testing moderators of the portion size effect.

SUMMARY OF FINDINGS

Provision of portion size information, such as through labeling or training in portion control, failed to attenuate food intake in response to increasing meal size. This indicates that a lack of knowledge about appropriate portions may not be sufficient to explain the portion size effect. In contrast, there is evidence for a role of decision making in the response to large portions, with value being one consideration of importance. The portion size effect may be more closely related to the inherent value of food than monetary value, since provision of the opportunity to take away uneaten food after a meal, which can reduce food waste, attenuated the portion size effect but variations in pricing did not. A number of studies also support an influence of orosensory processing on the portion size effect; large portions have been shown to relate to increased bite size and faster eating rate. Reduced oral processing time when consuming large portions could contribute to the effect by delaying sensory-specific satiety. Findings from a recent study supported this by demonstrating that sensory-specific satiety did not differ between larger and smaller portions despite substantial differences in intake.

CONCLUSIONS

A number of moderators of the portion size effect have been identified, including factors related to the environment, the food, and the individual. It is likely that multiple variables contribute to the response to large portions. Future research should aim to determine the relative contribution of explanatory variables across different contexts and individuals.

Modeling Wrist Micromovements to Measure In-Meal Eating Behavior From Inertial Sensor Data

Overweight and obesity are both associated with in-meal eating parameters such as eating speed. Recently, the plethora of available wearable devices in the market ignited the interest of both the scientific community and the industry toward unobtrusive solutions for eating behavior monitoring. In this paper, we present an algorithm for automatically detecting the in-meal food intake cycles using the inertial signals (acceleration and orientation velocity) from an off-the-shelf smartwatch. We use five specific wrist micromovements to model the series of actions leading to and following an intake event (i.e., bite). Food intake detection is performed in two steps. In the first step, we process windows of raw sensor streams and estimate their micromovement probability distributions by means of a convolutional neural network. In the second step, we use a long short-term memory network to capture the temporal evolution and classify sequences of windows as food intake cycles. Evaluation is performed using a challenging dataset of 21 meals from 12 subjects. In our experiments, we compare the performance of our algorithm against three state-of-the-art approaches, where our approach achieves the highest F1 detection score (0.913 in the leave-one-subject-out experiment). The dataset used in the experiments is available at https://mug.ee.auth.gr/intake-cycle-detection/.