Wearable Food Intake Monitoring Technologies: A Comprehensive Review

Mealtime prediction using wearable insulin pump data to support diabetes management

Many patients with diabetes struggle with post-meal high blood glucose due to missed or untimely meal-related insulin doses. To address this challenge, our research aims to: (1) study mealtime patterns in patients with type 1 diabetes using wearable insulin pump data, and (2) develop personalized models for predicting future mealtimes to support timely insulin dose administration. Using two independent datasets with over 45,000 meal logs from 82 patients with diabetes, we find that the majority of people ( ∼ 60%) have irregular and inconsistent mealtime patterns that change notably through the course of each day and across months in their own historical data. We also show the feasibility of predicting future mealtimes with personalized LSTM-based models that achieve an average F1 score of > 95% with less than 0.25 false positives per day. Our research lays the groundwork for developing a meal prediction system that can nudge patients with diabetes to administer bolus insulin doses before meal consumption to reduce the occurrence of post-meal high blood glucose.

Applications of Artificial Intelligence, Machine Learning, and Deep Learning in Nutrition: A Systematic Review

In industry 4.0, where the automation and digitalization of entities and processes are fundamental, artificial intelligence (AI) is increasingly becoming a pivotal tool offering innovative solutions in various domains. In this context, nutrition, a critical aspect of public health, is no exception to the fields influenced by the integration of AI technology. This study aims to comprehensively investigate the current landscape of AI in nutrition, providing a deep understanding of the potential of AI, machine learning (ML), and deep learning (DL) in nutrition sciences and highlighting eventual challenges and futuristic directions. A hybrid approach from the systematic literature review (SLR) guidelines and the preferred reporting items for systematic reviews and meta-analyses (PRISMA) guidelines was adopted to systematically analyze the scientific literature from a search of major databases on artificial intelligence in nutrition sciences. A rigorous study selection was conducted using the most appropriate eligibility criteria, followed by a methodological quality assessment ensuring the robustness of the included studies. This review identifies several AI applications in nutrition, spanning smart and personalized nutrition, dietary assessment, food recognition and tracking, predictive modeling for disease prevention, and disease diagnosis and monitoring. The selected studies demonstrated the versatility of machine learning and deep learning techniques in handling complex relationships within nutritional datasets. This study provides a comprehensive overview of the current state of AI applications in nutrition sciences and identifies challenges and opportunities. With the rapid advancement in AI, its integration into nutrition holds significant promise to enhance individual nutritional outcomes and optimize dietary recommendations. Researchers, policymakers, and healthcare professionals can utilize this research to design future projects and support evidence-based decision-making in AI for nutrition and dietary guidance.

Detecting Eating Episodes From Wrist Motion Using Daily Pattern Analysis

This paper presents new methods to detect eating from wrist motion. Our main novelty is that we analyze a full day of wrist motion data as a single sample so that the detection of eating occurrences can benefit from diurnal context. We develop a two-stage framework to facilitate a feasible full-day analysis. The first-stage model calculates local probabilities of eating P(Ew) within windows of data, and the second-stage model calculates enhanced probabilities of eating P(Ed) by treating all P(Ew) within a single day as one sample. The framework also incorporates an augmentation technique, which involves the iterative retraining of the first-stage model. This allows us to generate a sufficient number of day-length samples from datasets of limited size. We test our methods on the publicly available Clemson All-Day (CAD) dataset and FreeFIC dataset, and find that the inclusion of day-length analysis substantially improves accuracy in detecting eating episodes. We also benchmark our results against several state-of-the-art methods. Our approach achieved an eating episode true positive rate (TPR) of 89% with 1.4 false positives per true positive (FP/TP), and a time weighted accuracy of 84%, which are the highest accuracies reported on the CAD dataset. Our results show that the daily pattern classifier substantially improves meal detections and in particular reduces transient false detections that tend to occur when relying on shorter windows to look for individual ingestion or consumption events.

Continuous glucose monitoring for automatic real-time assessment of eating events and nutrition: a scoping review

Background

Accurate dietary assessment remains a challenge, particularly in free-living settings. Continuous glucose monitoring (CGM) shows promise in optimizing the assessment and monitoring of ingestive activity (IA, i.e., consumption of calorie-containing foods/beverages), and it might enable administering dietary Just-In-Time Adaptive Interventions (JITAIs).

Objective

In a scoping review, we aimed to answer the following questions: (1) Which CGM approaches to automatically detect IA in (near-)real-time have been investigated? (2) How accurate are these approaches? (3) Can they be used in the context of JITAIs?

Methods

We systematically searched four databases until October 2023 and included publications in English or German that used CGM-based approaches for human (all ages) IA detection. Eligible publications included a ground-truth method as a comparator. We synthesized the evidence qualitatively and critically appraised publication quality.

Results

Of 1,561 potentially relevant publications identified, 19 publications (17 studies, total N = 311; for 2 studies, 2 publications each were relevant) were included. Most publications included individuals with diabetes, often using meal announcements and/or insulin boluses accompanying meals. Inpatient and free-living settings were used. CGM-only approaches and CGM combined with additional inputs were deployed. A broad range of algorithms was tested. Performance varied among the reviewed methods, ranging from unsatisfactory to excellent (e.g., 21% vs. 100% sensitivity). Detection times ranged from 9.0 to 45.0 min.

Conclusion

Several CGM-based approaches are promising for automatically detecting IA. However, response times need to be faster to enable JITAIs aimed at impacting acute IA. Methodological issues and overall heterogeneity among articles prevent recommending one single approach; specific cases will dictate the most suitable approach.

Artificial intelligence & clinical nutrition: What the future might have in store

Artificial Intelligence (AI) is a rapidly emerging technology in healthcare that has the potential to revolutionise clinical nutrition. AI can assist in analysing complex data, interpreting medical images, and providing personalised nutrition interventions for patients. Clinical nutrition is a critical aspect of patient care, and AI can help clinicians make more informed decisions regarding patients' nutritional requirements, disease prevention, and management. AI algorithms can analyse large datasets to identify novel associations between diet and disease outcomes, enabling clinicians to make evidence-based nutritional recommendations. AI-powered devices and applications can also assist in tracking dietary intake, providing feedback, and motivating patients towards healthier food choices. However, the adoption of AI in clinical nutrition raises several ethical and regulatory concerns, such as data privacy and bias. Further research is needed to assess the clinical effectiveness and safety of AI-powered nutrition interventions. In conclusion, AI has the potential to transform clinical nutrition, but its integration into clinical practice should be carefully monitored to ensure patient safety and benefit. This article discusses the current and future applications of AI in clinical nutrition and highlights its potential benefits.

Technology to Automatically Record Eating Behavior in Real Life: A Systematic Review

To monitor adherence to diets and to design and evaluate nutritional interventions, it is essential to obtain objective knowledge about eating behavior. In most research, measures of eating behavior are based on self-reporting, such as 24-h recalls, food records (food diaries) and food frequency questionnaires. Self-reporting is prone to inaccuracies due to inaccurate and subjective recall and other biases. Recording behavior using nonobtrusive technology in daily life would overcome this. Here, we provide an up-to-date systematic overview encompassing all (close-to) publicly or commercially available technologies to automatically record eating behavior in real-life settings. A total of 1328 studies were screened and, after applying defined inclusion and exclusion criteria, 122 studies were included for in-depth evaluation. Technologies in these studies were categorized by what type of eating behavior they measure and which type of sensor technology they use. In general, we found that relatively simple sensors are often used. Depending on the purpose, these are mainly motion sensors, microphones, weight sensors and photo cameras. While several of these technologies are commercially available, there is still a lack of publicly available algorithms that are needed to process and interpret the resulting data. We argue that future work should focus on developing robust algorithms and validating these technologies in real-life settings. Combining technologies (e.g., prompting individuals for self-reports at sensed, opportune moments) is a promising route toward ecologically valid studies of eating behavior.

Vision-Based Methods for Food and Fluid Intake Monitoring: A Literature Review

Food and fluid intake monitoring are essential for reducing the risk of dehydration, malnutrition, and obesity. The existing research has been preponderantly focused on dietary monitoring, while fluid intake monitoring, on the other hand, is often neglected. Food and fluid intake monitoring can be based on wearable sensors, environmental sensors, smart containers, and the collaborative use of multiple sensors. Vision-based intake monitoring methods have been widely exploited with the development of visual devices and computer vision algorithms. Vision-based methods provide non-intrusive solutions for monitoring. They have shown promising performance in food/beverage recognition and segmentation, human intake action detection and classification, and food volume/fluid amount estimation. However, occlusion, privacy, computational efficiency, and practicality pose significant challenges. This paper reviews the existing work (253 articles) on vision-based intake (food and fluid) monitoring methods to assess the size and scope of the available literature and identify the current challenges and research gaps. This paper uses tables and graphs to depict the patterns of device selection, viewing angle, tasks, algorithms, experimental settings, and performance of the existing monitoring systems.

Toward a robust swallowing detection for an implantable active artificial larynx: a survey

Total laryngectomy consists in the removal of the larynx and is intended as a curative treatment for laryngeal cancer, but it leaves the patient with no possibility to breathe, talk, and swallow normally anymore. A tracheostomy is created to restore breathing through the throat, but the aero-digestive tracts are permanently separated and the air no longer passes through the nasal tracts, which allowed filtration, warming, humidification, olfaction, and acceleration of the air for better tissue oxygenation. As for phonation restoration, various techniques allow the patient to talk again. The main one consists of a tracheo-esophageal valve prosthesis that makes the air passes from the esophagus to the pharynx, and makes the air vibrate to allow speech through articulation. Finally, swallowing is possible through the original tract as it is now isolated from the trachea. Yet, many methods exist to detect and assess a swallowing, but none is intended as a definitive restoration technique of the natural airway, which would permanently close the tracheostomy and avoid its adverse effects. In addition, these methods are non-invasive and lack detection accuracy. The feasibility of an effective early detection of swallowing would allow to further develop an implantable active artificial larynx and therefore restore the aero-digestive tracts. A previous attempt has been made on an artificial larynx implanted in 2012, but no active detection was included and the system was completely mechanic. This led to residues in the airway because of the imperfect sealing of the mechanism. An active swallowing detection coupled with indwelling measurements would thus likely add a significant reliability on such a system as it would allow to actively close an artificial larynx. So, after a brief explanation of the swallowing mechanism, this survey intends to first provide a detailed consideration of the anatomical region involved in swallowing, with a detection perspective. Second, the swallowing mechanism following total laryngectomy surgery is detailed. Third, the current non-invasive swallowing detection technique and their limitations are discussed. Finally, the previous points are explored with regard to the inherent requirements for the feasibility of an effective swallowing detection for an artificial larynx. Graphical Abstract.

Mobile Computer Vision-Based Applications for Food Recognition and Volume and Calorific Estimation: A Systematic Review

The growing awareness of the influence of "what we eat" on lifestyle and health has led to an increase in the use of embedded food analysis and recognition systems. These solutions aim to effectively monitor daily food consumption, and therefore provide dietary recommendations to enable and support lifestyle changes. Mobile applications, due to their high accessibility, are ideal for real-life food recognition, volume estimation and calorific estimation. In this study, we conducted a systematic review based on articles that proposed mobile computer vision-based solutions for food recognition, volume estimation and calorific estimation. In addition, we assessed the extent to which these applications provide explanations to aid the users to understand the related classification and/or predictions. Our results show that 90.9% of applications do not distinguish between food and non-food. Similarly, only one study that proposed a mobile computer vision-based application for dietary intake attempted to provide explanations of features that contribute towards classification. Mobile computer vision-based applications are attracting a lot of interest in healthcare. They have the potential to assist in the management of chronic illnesses such as diabetes, ensuring that patients eat healthily and reducing complications associated with unhealthy food. However, to improve trust, mobile computer vision-based applications in healthcare should provide explanations of how they derive their classifications or volume and calorific estimations.

Toward Systems Models for Obesity Prevention: A Big Role for Big Data

The relation among the various causal factors of obesity is not well understood, and there remains a lack of viable data to advance integrated, systems models of its etiology. The collection of big data has begun to allow the exploration of causal associations between behavior, built environment, and obesity-relevant health outcomes. Here, the traditional epidemiologic and emerging big data approaches used in obesity research are compared, describing the research questions, needs, and outcomes of 3 broad research domains: eating behavior, social food environments, and the built environment. Taking tangible steps at the intersection of these domains, the recent European Union project "BigO: Big data against childhood obesity" used a mobile health tool to link objective measurements of health, physical activity, and the built environment. BigO provided learning on the limitations of big data, such as privacy concerns, study sampling, and the balancing of epidemiologic domain expertise with the required technical expertise. Adopting big data approaches will facilitate the exploitation of data concerning obesity-relevant behaviors of a greater variety, which are also processed at speed, facilitated by mobile-based data collection and monitoring systems, citizen science, and artificial intelligence. These approaches will allow the field to expand from causal inference to more complex, systems-level predictive models, stimulating ambitious and effective policy interventions.

National Food Intake Assessment: Technologies to Advance Traditional Methods

National dietary surveillance produces dietary intake data used for various purposes including development and evaluation of national policies in food and nutrition. Since 2000, What We Eat in America, the dietary component of the National Health and Nutrition Examination Survey, has collected dietary data and reported on the dietary intake of the US population. Continual innovations are required to improve methods of data collection, quality, and relevance. This review article evaluates the strengths and limitations of current and newer methods in national dietary data collection, underscoring the use of technology and emerging technology applications. We offer four objectives for national dietary surveillance that serve as guiding principles in the evaluation. Moving forward, national dietary surveillance must take advantage of new technologies for their potential in enhanced efficiency and objectivity in data operations while continuing to collect accurate dietary information that is standardized, validated, and publicly transparent.

Valuing the Diversity of Research Methods to Advance Nutrition Science

The ASN Board of Directors appointed the Nutrition Research Task Force to develop a report on scientific methods used in nutrition science to advance discovery, interpretation, and application of knowledge in the field. The genesis of this report was growing concern about the tone of discourse among nutrition professionals and the implications of acrimony on the productive study and translation of nutrition science. Too often, honest differences of opinion are cast as conflicts instead of areas of needed collaboration. Recognition of the value (and limitations) of contributions from well-executed nutrition science derived from the various approaches used in the discipline, as well as appreciation of how their layering will yield the strongest evidence base, will provide a basis for greater productivity and impact. Greater collaborative efforts within the field of nutrition science will require an understanding that each method or approach has a place and function that should be valued and used together to create the nutrition evidence base. Precision nutrition was identified as an important emerging nutrition topic by the preponderance of task force members, and this theme was adopted for the report because it lent itself to integration of many approaches in nutrition science. Although the primary audience for this report is nutrition researchers and other nutrition professionals, a secondary aim is to develop a document useful for the various audiences that translate nutrition research, including journalists, clinicians, and policymakers. The intent is to promote accurate, transparent, verifiable evidence-based communication about nutrition science. This will facilitate reasoned interpretation and application of emerging findings and, thereby, improve understanding and trust in nutrition science and appropriate characterization, development, and adoption of recommendations.

iHearken: Chewing sound signal analysis based food intake recognition system using Bi-LSTM softmax network

BACKGROUND AND OBJECTIVE

Food ingestion is an integral part of health and wellness. Continues monitoring of different food types and observing the amount being consumed prevents gastrointestinal diseases and weight-related issues. Food intake recognition (FIR) systems, thus have significant impact on everyday life. The purpose of this study is to develop an automatic approach for the FIR using a contemporary wearable hardware and machine learning technique. This will assist clinicians and concern person to manage health issues associated with food intake.

METHODS

In this work, we present a novel hardware iHearken, a headphone-like wearable sensor-based system to monitor eating activities and recognize food intake type in the free-living condition. State-of-the-art hardware is designed for data acquisition where 16 subjects are recruited and 20 different food items are used for data collection. Further, chewing sound signals are analyzed for FIR using bottleneck features. The proposed model is divided into 4 distinct phases: data acquisition, event detection using a pre-trained model, bottleneck feature extraction, and classification based on bidirectional long short-term memory (Bi-LSTM) softmax model. The Bi-LSTM network with softmax function is applied to calculate the identification score for apiece chewing signal which further classifies the chewing signal data into liquid / solid food classes.

RESULTS

The results of proposed model performance is evaluated in (%) for accuracy, precision, recall and F-score as 97.422, 96.808, 98.0, and 97.512, respectively, and root mean square error (RMSE), and mean absolute percentage error (MAPE) as 0.160 1.030 respectively for numbers of correct food type recognized. Further, we also evaluated our model's performance for food classification into solid and liquid and achieved an accuracy (96.66%), precision (96.40%), recall (95.230%), F-score (95.79%), RMSE (0.182), and MAPE (2.22). We also demonstrated that the food recognition accuracy of different models with the proposed model differed statistically.

CONCLUSION

An informatics complexity study of the proposed model was subsequently explored to review the effectiveness of the proposed wearable device and the methodology. The medical importance of this investigation is the reliable monitoring of the clinical development of the food intake classification methods via food chew event detection in the ambulatory environment has been justified.

Enhancing Nutrition Care Through Real-Time, Sensor-Based Capture of Eating Occasions: A Scoping Review

As food intake patterns become less structured, different methods of dietary assessment may be required to capture frequently omitted snacks, smaller meals, and the time of day when they are consumed. Incorporating sensors that passively and objectively detect eating behavior may assist in capturing these eating occasions into dietary assessment methods. The aim of this study was to identify and collate sensor-based technologies that are feasible for dietitians to use to assist with performing dietary assessments in real-world practice settings. A scoping review was conducted using the PRISMA extension for scoping reviews (PRISMA-ScR) framework. Studies were included if they were published between January 2016 and December 2021 and evaluated the performance of sensor-based devices for identifying and recording the time of food intake. Devices from included studies were further evaluated against a set of feasibility criteria to determine whether they could potentially be used to assist dietitians in conducting dietary assessments. The feasibility criteria were, in brief, consisting of an accuracy ≥80%; tested in settings where subjects were free to choose their own foods and activities; social acceptability and comfort; a long battery life; and a relatively rapid detection of an eating episode. Fifty-four studies describing 53 unique devices and 4 device combinations worn on the wrist (n = 18), head (n = 16), neck (n = 9), and other locations (n = 14) were included. Whilst none of the devices strictly met all feasibility criteria currently, continuous refinement and testing of device software and hardware are likely given the rapidly changing nature of this emerging field. The main reasons devices failed to meet the feasibility criteria were: an insufficient or lack of reporting on battery life (91%), the use of a limited number of foods and behaviors to evaluate device performance (63%), and the device being socially unacceptable or uncomfortable to wear for long durations (46%). Until sensor-based dietary assessment tools have been designed into more inconspicuous prototypes and are able to detect most food and beverage consumption throughout the day, their use will not be feasible for dietitians in practice settings.

Prospects and Pitfalls of Machine Learning in Nutritional Epidemiology

Nutritional epidemiology employs observational data to discover associations between diet and disease risk. However, existing analytic methods of dietary data are often sub-optimal, with limited incorporation and analysis of the correlations between the studied variables and nonlinear behaviours in the data. Machine learning (ML) is an area of artificial intelligence that has the potential to improve modelling of nonlinear associations and confounding which are found in nutritional data. These opportunities notwithstanding, the applications of ML in nutritional epidemiology must be approached cautiously to safeguard the scientific quality of the results and provide accurate interpretations. Given the complex scenario around ML, judicious application of such tools is necessary to offer nutritional epidemiology a novel analytical resource for dietary measurement and assessment and a tool to model the complexity of dietary intake and its relation to health. This work describes the applications of ML in nutritional epidemiology and provides guidelines to avoid common pitfalls encountered in applying predictive statistical models to nutritional data. Furthermore, it helps unfamiliar readers better assess the significance of their results and provides new possible future directions in the field of ML in nutritional epidemiology.

Opportunities and Challenges of Integrating Food Practice into Clinical Decision-Making

BACKGROUND

Food practice plays an important role in health. Food practice data collected in daily living settings can inform clinical decisions. However, integrating such data into clinical decision-making is burdensome for both clinicians and patients, resulting in poor adherence and limited utilization. Automation offers benefits in this regard, minimizing this burden resulting in a better fit with a patient's daily living routines, and creating opportunities for better integration into clinical workflow. Although the literature on patient-generated health data (PGHD) can serve as a starting point for the automation of food practice data, more diverse characteristics of food practice data provide additional challenges.

OBJECTIVES

We describe a series of steps for integrating food practices into clinical decision-making. These steps include the following: (1) sensing food practice; (2) capturing food practice data; (3) representing food practice; (4) reflecting the information to the patient; (5) incorporating data into the EHR; (6) presenting contextualized food practice information to clinicians; and (7) integrating food practice into clinical decision-making.

METHODS

We elaborate on automation opportunities and challenges in each step, providing a summary visualization of the flow of food practice-related data from daily living settings to clinical settings.

RESULTS

We propose four implications of automating food practice hereinafter. First, there are multiple ways of automating workflow related to food practice. Second, steps may occur in daily living and others in clinical settings. Food practice data and the necessary contextual information should be integrated into clinical decision-making to enable action. Third, as accuracy becomes important for food practice data, macrolevel data may have advantages over microlevel data in some situations. Fourth, relevant systems should be designed to eliminate disparities in leveraging food practice data.

CONCLUSION

Our work confirms previously developed recommendations in the context of PGHD work and provides additional specificity on how these recommendations apply to food practice.

Recent Advances of Point-of-Care Devices Integrated with Molecularly Imprinted Polymers-Based Biosensors: From Biomolecule Sensing Design to Intraoral Fluid Testing

Recent developments of point-of-care testing (POCT) and in vitro diagnostic medical devices have provided analytical capabilities and reliable diagnostic results for rapid access at or near the patient's location. Nevertheless, the challenges of reliable diagnosis still remain an important factor in actual clinical trials before on-site medical treatment and making clinical decisions. New classes of POCT devices depict precise diagnostic technologies that can detect biomarkers in biofluids such as sweat, tears, saliva or urine. The introduction of a novel molecularly imprinted polymer (MIP) system as an artificial bioreceptor for the POCT devices could be one of the emerging candidates to improve the analytical performance along with physicochemical stability when used in harsh environments. Here, we review the potential availability of MIP-based biorecognition systems as custom artificial receptors with high selectivity and chemical affinity for specific molecules. Further developments to the progress of advanced MIP technology for biomolecule recognition are introduced. Finally, to improve the POCT-based diagnostic system, we summarized the perspectives for high expandability to MIP-based periodontal diagnosis and the future directions of MIP-based biosensors as a wearable format.

Validity and Feasibility of the Monitoring and Modeling Family Eating Dynamics System to Automatically Detect In-field Family Eating Behavior: Observational Study

Background

The field of dietary assessment has a long history, marked by both controversies and advances. Emerging technologies may be a potential solution to address the limitations of self-report dietary assessment methods. The Monitoring and Modeling Family Eating Dynamics (M2FED) study uses wrist-worn smartwatches to automatically detect real-time eating activity in the field. The ecological momentary assessment (EMA) methodology was also used to confirm whether eating occurred (ie, ground truth) and to measure other contextual information, including positive and negative affect, hunger, satiety, mindful eating, and social context.

Objective

This study aims to report on participant compliance (feasibility) to the 2 distinct EMA protocols of the M2FED study (hourly time-triggered and eating event–triggered assessments) and on the performance (validity) of the smartwatch algorithm in automatically detecting eating events in a family-based study.

Methods

In all, 20 families (58 participants) participated in the 2-week, observational, M2FED study. All participants wore a smartwatch on their dominant hand and responded to time-triggered and eating event–triggered mobile questionnaires via EMA while at home. Compliance to EMA was calculated overall, for hourly time-triggered mobile questionnaires, and for eating event–triggered mobile questionnaires. The predictors of compliance were determined using a logistic regression model. The number of true and false positive eating events was calculated, as well as the precision of the smartwatch algorithm. The Mann-Whitney U test, Kruskal-Wallis test, and Spearman rank correlation were used to determine whether there were differences in the detection of eating events by participant age, gender, family role, and height.

Results

The overall compliance rate across the 20 deployments was 89.26% (3723/4171) for all EMAs, 89.7% (3328/3710) for time-triggered EMAs, and 85.7% (395/461) for eating event–triggered EMAs. Time of day (afternoon odds ratio [OR] 0.60, 95% CI 0.42-0.85; evening OR 0.53, 95% CI 0.38-0.74) and whether other family members had also answered an EMA (OR 2.07, 95% CI 1.66-2.58) were significant predictors of compliance to time-triggered EMAs. Weekend status (OR 2.40, 95% CI 1.25-4.91) and deployment day (OR 0.92, 95% CI 0.86-0.97) were significant predictors of compliance to eating event–triggered EMAs. Participants confirmed that 76.5% (302/395) of the detected events were true eating events (ie, true positives), and the precision was 0.77. The proportion of correctly detected eating events did not significantly differ by participant age, gender, family role, or height (P>.05).

Conclusions

This study demonstrates that EMA is a feasible tool to collect ground-truth eating activity and thus evaluate the performance of wearable sensors in the field. The combination of a wrist-worn smartwatch to automatically detect eating and a mobile device to capture ground-truth eating activity offers key advantages for the user and makes mobile health technologies more accessible to nonengineering behavioral researchers.

A Novel Approach to Dining Bowl Reconstruction for Image-Based Food Volume Estimation

Knowing the amounts of energy and nutrients in an individual's diet is important for maintaining health and preventing chronic diseases. As electronic and AI technologies advance rapidly, dietary assessment can now be performed using food images obtained from a smartphone or a wearable device. One of the challenges in this approach is to computationally measure the volume of food in a bowl from an image. This problem has not been studied systematically despite the bowl being the most utilized food container in many parts of the world, especially in Asia and Africa. In this paper, we present a new method to measure the size and shape of a bowl by adhering a paper ruler centrally across the bottom and sides of the bowl and then taking an image. When observed from the image, the distortions in the width of the paper ruler and the spacings between ruler markers completely encode the size and shape of the bowl. A computational algorithm is developed to reconstruct the three-dimensional bowl interior using the observed distortions. Our experiments using nine bowls, colored liquids, and amorphous foods demonstrate high accuracy of our method for food volume estimation involving round bowls as containers. A total of 228 images of amorphous foods were also used in a comparative experiment between our algorithm and an independent human estimator. The results showed that our algorithm overperformed the human estimator who utilized different types of reference information and two estimation methods, including direct volume estimation and indirect estimation through the fullness of the bowl.

Top-Down Detection of Eating Episodes by Analyzing Large Windows of Wrist Motion Using a Convolutional Neural Network

In this work, we describe a new method to detect periods of eating by tracking wrist motion during everyday life. Eating uses hand-to-mouth gestures for ingestion, each of which lasts a few seconds. Previous works have detected these gestures individually and then aggregated them to identify meals. The novelty of our approach is that we analyze a much longer window (0.5–15 min) using a convolutional neural network. Longer windows can contain other gestures related to eating, such as cutting or manipulating food, preparing foods for consumption, and resting between ingestion events. The context of these other gestures can improve the detection of periods of eating. We test our methods on the public Clemson all-day dataset, which consists of 354 recordings containing 1063 eating episodes. We found that accuracy at detecting eating increased by 15% in ≥4 min windows compared to ≤15 s windows. Using a 6 min window, we detected 89% of eating episodes, with 1.7 false positives for every true positive (FP/TP). These are the best results achieved to date on this dataset.

Perspective: Opportunities and Challenges of Technology Tools in Dietary and Activity Assessment: Bridging Stakeholder Viewpoints

The science and tools of measuring energy intake and output in humans have rapidly advanced in the last decade. Engineered devices such as wearables and sensors, software applications, and Web-based tools are now ubiquitous in both research and consumer environments. The assessment of energy expenditure in particular has progressed from reliance on self-report instruments to advanced technologies requiring collaboration across multiple disciplines, from optics to accelerometry. In contrast, assessing energy intake still heavily relies on self-report mechanisms. Although these tools have improved, moving from paper-based to online reporting, considerable room for refinement remains in existing tools, and great opportunities exist for novel, transformational tools, including those using spectroscopy and chemo-sensing. This report reviews the state of the science, and the opportunities and challenges in existing and emerging technologies, from the perspectives of 3 key stakeholders: researchers, users, and developers. Each stakeholder approaches these tools with unique requirements: researchers are concerned with validity, accuracy, data detail and abundance, and ethical use; users with ease of use and privacy; and developers with high adherence and utilization, intellectual property, licensing rights, and monetization. Cross-cutting concerns include frequent updating and integration of the food and nutrient databases on which assessments rely, improving accessibility and reducing disparities in use, and maintaining reliable technical assistance. These contextual challenges are discussed in terms of opportunities and further steps in the direction of personalized health.

Food Detection and Segmentation from Egocentric Camera Images

Tracking an individual's food intake provides useful insight into their eating habits. Technological advancements in wearable sensors such as the automatic capture of food images from wearable cameras have made the tracking of food intake efficient and feasible. For accurate food intake monitoring, an automated food detection technique is needed to recognize foods from unstaged real-world images. This work presents a novel food detection and segmentation pipeline to detect the presence of food in images acquired from an egocentric wearable camera, and subsequently segment the food image. An ensemble of YOLOv5 detection networks is trained to detect and localize food items among other objects present in captured images. The model achieves an overall 80.6% mean average precision on four objects-Food, Beverage, Screen, and Person. Post object detection, the predicted food objects which are sufficiently sharp were considered for segmentation. The Normalized-Graph-Cut algorithm was used to segment the different parts of the food resulting in an average IoU of 82%.Clinical relevance- The automatic monitoring of food intake using wearable devices can play a pivotal role in the treatment and prevention of eating disorders, obesity, malnutrition and other related issues. It can aid in understanding the pattern of nutritional intake and make personalized adjustments to lead a healthy life.

Determination of Chewing Count from Video Recordings Using Discrete Wavelet Decomposition and Low Pass Filtration

Several studies have shown the importance of proper chewing and the effect of chewing speed on the human health in terms of caloric intake and even cognitive functions. This study aims at designing algorithms for determining the chew count from video recordings of subjects consuming food items. A novel algorithm based on image and signal processing techniques has been developed to continuously capture the area of interest from the video clips, determine facial landmarks, generate the chewing signal, and process the signal with two methods: low pass filter, and discrete wavelet decomposition. Peak detection was used to determine the chew count from the output of the processed chewing signal. The system was tested using recordings from 100 subjects at three different chewing speeds (i.e., slow, normal, and fast) without any constraints on gender, skin color, facial hair, or ambience. The low pass filter algorithm achieved the best mean absolute percentage error of 6.48%, 7.76%, and 8.38% for the slow, normal, and fast chewing speeds, respectively. The performance was also evaluated using the Bland-Altman plot, which showed that most of the points lie within the lines of agreement. However, the algorithm needs improvement for faster chewing, but it surpasses the performance of the relevant literature. This research provides a reliable and accurate method for determining the chew count. The proposed methods facilitate the study of the chewing behavior in natural settings without any cumbersome hardware that may affect the results. This work can facilitate research into chewing behavior while using smart devices.

Effects of diet and fitness apps on eating disorder behaviours: qualitative study

Background

Diet and fitness apps are often promoted in university and college settings and touted as a means to improve health with little attention given to unanticipated negative effects, especially among those at risk for or with eating disorders.

Aims

Few researchers have studied how these apps affect women with eating disorders in university and college settings. This research investigates the unintended negative consequences of engaging with these tools.

Method

Data collection sessions comprised three components conducted with 24 participants: survey (demographic and eating disorder symptoms), think-aloud exercise and semi-structured interview. Thematic analysis was used to analyse data.

Results

Participants reported that diet and fitness apps trigger and exacerbate symptoms by focusing heavily on quantification, promoting overuse and providing certain types of feedback. Eight themes of negative consequences emerged: fixation on numbers, rigid diet, obsession, app dependency, high sense of achievement, extreme negative emotions, motivation from ‘negative’ messages, and excess competition. Although these themes were common when users’ focus was to lose weight or eat less, they were also prevalent when users wanted to focus explicitly on eating disorder recovery.

Conclusions

Unintended negative consequences are linked to the quantified self movement, conception of appropriate usage, and visual cues and feedback. This paper critically examines diet and fitness app design and discusses implications for designers, educators and clinicians. Ultimately, this research emphasises the need for a fundamental shift in how diet and fitness apps promote health, with mental health at the forefront.

Single-Stage Intake Gesture Detection Using CTC Loss and Extended Prefix Beam Search

Accurate detection of individual intake gestures is a key step towards automatic dietary monitoring. Both inertial sensor data of wrist movements and video data depicting the upper body have been used for this purpose. The most advanced approaches to date use a two-stage approach, in which (i) frame-level intake probabilities are learned from the sensor data using a deep neural network, and then (ii) sparse intake events are detected by finding the maxima of the frame-level probabilities. In this study, we propose a single-stage approach which directly decodes the probabilities learned from sensor data into sparse intake detections. This is achieved by weakly supervised training using Connectionist Temporal Classification (CTC) loss, and decoding using a novel extended prefix beam search decoding algorithm. Benefits of this approach include (i) end-to-end training for detections, (ii) simplified timing requirements for intake gesture labels, and (iii) improved detection performance compared to existing approaches. Across two separate datasets, we achieve relative F₁ score improvements between 1.9% and 6.2% over the two-stage approach for intake detection and eating/drinking detection tasks, for both video and inertial sensors.

Short and Long-Term Innovations on Dietary Behavior Assessment and Coaching: Present Efforts and Vision of the Pride and Prejudice Consortium

Overweight, obesity and cardiometabolic diseases are major global health concerns. Lifestyle factors, including diet, have been acknowledged to play a key role in the solution of these health risks. However, as shown by numerous studies, and in clinical practice, it is extremely challenging to quantify dietary behaviors as well as influencing them via dietary interventions. As shown by the limited success of 'one-size-fits-all' nutritional campaigns catered to an entire population or subpopulation, the need for more personalized coaching approaches is evident. New technology-based innovations provide opportunities to further improve the accuracy of dietary assessment and develop approaches to coach individuals towards healthier dietary behaviors. Pride & Prejudice (P&P) is a unique multi-disciplinary consortium consisting of researchers in life, nutrition, ICT, design, behavioral and social sciences from all four Dutch Universities of Technology. P&P focuses on the development and integration of innovative technological techniques such as artificial intelligence (AI), machine learning, conversational agents, behavior change theory and personalized coaching to improve current practices and establish lasting dietary behavior change.

Fluid Intake Monitoring Systems for the Elderly: A Review of the Literature

Fluid intake monitoring is an essential component in preventing dehydration and overhydration, especially for the senior population. Numerous critical health problems are associated with poor or excessive drinking such as swelling of the brain and heart failure. Real-time systems for monitoring fluid intake will not only measure the exact amount consumed by the users, but could also motivate people to maintain a healthy lifestyle by providing feedback to encourage them to hydrate regularly throughout the day. This paper reviews the most recent solutions to automatic fluid intake monitoring both commercially and in the literature. The available technologies are divided into four categories: wearables, surfaces with embedded sensors, vision- and environmental-based solutions, and smart containers. A detailed performance evaluation was carried out considering detection accuracy, usability and availability. It was observed that the most promising results came from studies that used data fusion from multiple technologies, compared to using an individual technology. The areas that need further research and the challenges for each category are discussed in detail.

Wearable sensors for continuous oral cavity and dietary monitoring toward personalized healthcare and digital medicine

Oral monitoring plays an essential role in preventing and diagnosing systemic diseases through saliva in the mouth. Dietary monitoring is also crucial to reduce the likelihood of chronic diseases such as hypertension and diabetes by analyzing food types, amounts and diet patterns. Therefore, the oral cavity and dietary monitoring are vital for accurate personalized healthcare management that can improve healthcare. To perform continuous oral cavity and dietary monitoring in real-time, a wearable sensing system capable of continuous analysis is necessary. In this review, we classify chemical and physical biosensing methods and summarize recent progress in wearable sensor development for oral cavity and dietary monitoring for personalized healthcare and digital medicine. We also discuss future perspectives and the potential of wearable sensors to provide robust data for food-intake monitoring and the saliva analysis of super-aged/aging societies, non-face-to-face social life, and global pandemic disease issues. We believe that this review will result in a paradigm shift toward personalized healthcare and digital medicine using wearable sensors through the analysis of massively parallel healthcare data.

Investigating sex differences in the accuracy of dietary assessment methods to measure energy intake in adults: a systematic review and meta-analysis

BACKGROUND

To inform the interpretation of dietary data in the context of sex differences in diet-disease relations, it is important to understand whether there are any sex differences in accuracy of dietary reporting.

OBJECTIVE

To quantify sex differences in self-reported total energy intake (TEI) compared with a reference measure of total energy expenditure (TEE).

METHODS

Six electronic databases were systematically searched for published original research articles between 1980 and April 2020. Studies were included if they were conducted in adult populations with measures for both females and males of self-reported TEI and TEE from doubly labeled water (DLW). Studies were screened and quality assessed independently by 2 authors. Random-effects meta-analyses were conducted to pool the mean differences between TEI and TEE for, and between, females and males, by method of dietary assessment.

RESULTS

From 1313 identified studies, 31 met the inclusion criteria. The studies collectively included information on 4518 individuals (54% females). Dietary assessment methods included 24-h recalls (n = 12, 2 with supplemental photos of food items consumed), estimated food records (EFRs; n = 11), FFQs (n = 10), weighed food records (WFRs, n = 5), and diet histories (n = 2). Meta-analyses identified underestimation of TEI by females and males, ranging from -1318 kJ/d (95% CI: -1967, -669) for FFQ to -2650 kJ/d (95% CI: -3492, -1807) for 24-h recalls for females, and from -1764 kJ/d (95% CI: -2285, -1242) for FFQ to -3438 kJ/d (95% CI: -5382, -1494) for WFR for males. There was no difference in the level of underestimation by sex, except when using EFR, for which males underestimated energy intake more than females (by 590 kJ/d, 95% CI: 35, 1,146).

CONCLUSION

Substantial underestimation of TEI across a range of dietary assessment methods was identified, similar by sex. These underestimations should be considered when assessing TEI and interpreting diet-disease relations.

Artificial Neural Network-Based Automatic Detection of Food Intake for Neuromodulation in Treating Obesity and Diabetes

PURPOSE

Neuromodulation, such as vagal nerve stimulation and intestinal electrical stimulation, has been introduced for the treatment of obesity and diabetes. Ideally, neuromodulation should be applied automatically after food intake. The purpose of this study was to develop a method of automatic food intake detection through dynamic analysis of heart rate variability (HRV).

MATERIALS AND METHODS

Two experiments were conducted: (1) a small sample series with a standard test meal and (2) a large sample series with varying meal size. Electrocardiograms (ECGs) were collected in the fasting and postprandial states. Each ECG was processed to compute the HRV. For each HRV segment, time- and frequency-domain features were derived and used as inputs to train and test an artificial neural network (ANN). The ANN was trained and tested with different cross-validation methods.

RESULTS

The highest classification accuracy reached with leave-one-subject-out-leave-one-sample-out cross-validation was 0.93 in experiment 1 and 0.88 in experiment 2. Retraining the ANN on recordings of a subject drastically increased the achieved accuracy for that subject to values of 0.995 and 0.95 in experiments 1 and 2, respectively.

CONCLUSIONS

Automatic food intake detection by ANNs, using features from the HRV, is feasible and may have a great potential for neuromodulation-based treatments of meal-related disorders.

Food/Non-Food Classification of Real-Life Egocentric Images in Low- and Middle-Income Countries Based on Image Tagging Features

Malnutrition, including both undernutrition and obesity, is a significant problem in low- and middle-income countries (LMICs). In order to study malnutrition and develop effective intervention strategies, it is crucial to evaluate nutritional status in LMICs at the individual, household, and community levels. In a multinational research project supported by the Bill & Melinda Gates Foundation, we have been using a wearable technology to conduct objective dietary assessment in sub-Saharan Africa. Our assessment includes multiple diet-related activities in urban and rural families, including food sources (e.g., shopping, harvesting, and gathering), preservation/storage, preparation, cooking, and consumption (e.g., portion size and nutrition analysis). Our wearable device ("eButton" worn on the chest) acquires real-life images automatically during wake hours at preset time intervals. The recorded images, in amounts of tens of thousands per day, are post-processed to obtain the information of interest. Although we expect future Artificial Intelligence (AI) technology to extract the information automatically, at present we utilize AI to separate the acquired images into two binary classes: images with (Class 1) and without (Class 0) edible items. As a result, researchers need only to study Class-1 images, reducing their workload significantly. In this paper, we present a composite machine learning method to perform this classification, meeting the specific challenges of high complexity and diversity in the real-world LMIC data. Our method consists of a deep neural network (DNN) and a shallow learning network (SLN) connected by a novel probabilistic network interface layer. After presenting the details of our method, an image dataset acquired from Ghana is utilized to train and evaluate the machine learning system. Our comparative experiment indicates that the new composite method performs better than the conventional deep learning method assessed by integrated measures of sensitivity, specificity, and burden index, as indicated by the Receiver Operating Characteristic (ROC) curve.

A Data Driven End-to-End Approach for In-the-Wild Monitoring of Eating Behavior Using Smartwatches

The increased worldwide prevalence of obesity has sparked the interest of the scientific community towards tools that objectively and automatically monitor eating behavior. Despite the study of obesity being in the spotlight, such tools can also be used to study eating disorders (e.g. anorexia nervosa) or provide a personalized monitoring platform for patients or athletes. This paper presents a complete framework towards the automated i) modeling of in-meal eating behavior and ii) temporal localization of meals, from raw inertial data collected in-the-wild using commercially available smartwatches. Initially, we present an end-to-end Neural Network which detects food intake events (i.e. bites). The proposed network uses both convolutional and recurrent layers that are trained simultaneously. Subsequently, we show how the distribution of the detected bites throughout the day can be used to estimate the start and end points of meals, using signal processing algorithms. We perform extensive evaluation on each framework part individually. Leave-one-subject-out (LOSO) evaluation shows that our bite detection approach outperforms four state-of-the-art algorithms towards the detection of bites during the course of a meal (0.923 F1 score). Furthermore, LOSO and held-out set experiments regarding the estimation of meal start/end points reveal that the proposed approach outperforms a relevant approach found in the literature (Jaccard Index of 0.820 and 0.821 for the LOSO and held-out experiments, respectively). Experiments are performed using our publicly available FIC and the newly introduced FreeFIC datasets.

Current Advances in Wearable Devices and Their Sensors in Patients With Depression

In this study, a literature survey was conducted of research into the development and use of wearable devices and sensors in patients with depression. We collected 18 studies that had investigated wearable devices for assessment, monitoring, or prediction of depression. In this report, we examine the sensors of the various types of wearable devices (e.g., actigraphy units, wristbands, fitness trackers, and smartwatches) and parameters measured through sensors in people with depression. In addition, we discuss future trends, referring to research in other areas employing wearable devices, and suggest the challenges of using wearable devices in the field of depression. Real-time objective monitoring of symptoms and novel approaches for diagnosis and treatment using wearable devices will lead to changes in management of patients with depression. During the process, it is necessary to overcome several issues, including limited types of collected data, reliability, user adherence, and privacy concerns.

The Age of Artificial Intelligence: Use of Digital Technology in Clinical Nutrition

PURPOSE OF REVIEW

Computing advances over the decades have catalyzed the pervasive integration of digital technology in the medical industry, now followed by similar applications for clinical nutrition. This review discusses the implementation of such technologies for nutrition, ranging from the use of mobile apps and wearable technologies to the development of decision support tools for parenteral nutrition and use of telehealth for remote assessment of nutrition.

RECENT FINDINGS

Mobile applications and wearable technologies have provided opportunities for real-time collection of granular nutrition-related data. Machine learning has allowed for more complex analyses of the increasing volume of data collected. The combination of these tools has also translated into practical clinical applications, such as decision support tools, risk prediction, and diet optimization.

SUMMARY

The state of digital technology for clinical nutrition is still young, although there is much promise for growth and disruption in the future.

Real-Time Food Intake Monitoring Using Wearable Egocnetric Camera

With technological advancement, wearable egocentric camera systems have extensively been studied to develop food intake monitoring devices for the assessment of eating behavior. This paper provides a detailed description of the implementation of CNN based image classifier in the Cortex-M7 microcontroller. The proposed network classifies the captured images by the wearable egocentric camera as food and no food images in real-time. This real-time food image detection can potentially lead the monitoring devices to consume less power, less storage, and more user-friendly in terms of privacy by saving only images that are detected as food images. A derivative of pre-trained MobileNet is trained to detect food images from camera captured images. The proposed network needs 761.99KB of flash and 501.76KB of RAM to implement which is built for an optimal trade-off between accuracy, computational cost, and memory footprint considering implementation on a Cortex-M7 microcontroller. The image classifier achieved an average precision of 82%±3% and an average F-score of 74%±2% while testing on 15343 (2127 food images and 13216 no food images) images of five full days collected from five participants.

Future Directions for Integrative Objective Assessment of Eating Using Wearable Sensing Technology

Established methods for nutritional assessment suffer from a number of important limitations. Diaries are burdensome to complete, food frequency questionnaires only capture average food intake, and both suffer from difficulties in self estimation of portion size and biases resulting from misreporting. Online and app versions of these methods have been developed, but issues with misreporting and portion size estimation remain. New methods utilizing passive data capture are required that address reporting bias, extend timescales for data collection, and transform what is possible for measuring habitual intakes. Digital and sensing technologies are enabling the development of innovative and transformative new methods in this area that will provide a better understanding of eating behavior and associations with health. In this article we describe how wrist-worn wearables, on-body cameras, and body-mounted biosensors can be used to capture data about when, what, and how much people eat and drink. We illustrate how these new techniques can be integrated to provide complete solutions for the passive, objective assessment of a wide range of traditional dietary factors, as well as novel measures of eating architecture, within person variation in intakes, and food/nutrient combinations within meals. We also discuss some of the challenges these new approaches will bring.

Epidermal Enzymatic Biosensors for Sweat Vitamin C: Toward Personalized Nutrition

Recent advances in wearable sensor technologies offer new opportunities for improving dietary adherence. However, despite their tremendous promise, the potential of wearable chemical sensors for guiding personalized nutrition solutions has not been reported. Herein, we present an epidermal biosensor aimed at following the dynamics of sweat vitamin C after the intake of vitamin C pills and fruit juices. Such skin-worn noninvasive electrochemical detection of sweat vitamin C has been realized by immobilizing the enzyme ascorbate oxidase (AAOx) on flexible printable tattoo electrodes and monitoring changes in the vitamin C level through changes in the reduction current of the oxygen cosubstrate. The flexible vitamin C tattoo patch was fabricated on a polyurethane substrate and combined with a localized iontophoretic sweat stimulation system along with amperometric cathodic detection of the oxygen depletion during the enzymatic reaction. The enzyme biosensor offers a highly selective response compared to the common direct (nonenzymatic) voltammetric measurements, with no effect on electroactive interfering species such as uric acid or acetaminophen. Temporal vitamin C profiles in sweat are demonstrated using different subjects taking varying amounts of commercial vitamin C pills or vitamin C-rich beverages. The dynamic rise and fall of such vitamin C sweat levels is thus demonstrated with no interference from other sweat constituents. Differences in such dynamics among the individual subjects indicate the potential of the epidermal biosensor for personalized nutrition solutions. The flexible tattoo patch displayed mechanical resiliency to multiple stretching and bending deformations. In addition, the AAOx biosensor is shown to be useful as a disposable strip for the rapid in vitro detection of vitamin C in untreated raw saliva and tears following pill or juice intake. These results demonstrate the potential of wearable chemical sensors for noninvasive nutrition status assessments and tracking of nutrient uptake toward detecting and correcting nutritional deficiencies, assessing adherence to vitamin intake, and supporting dietary behavior change.

Sensory Interactive Table (SIT)-Development of a Measurement Instrument to Support Healthy Eating in a Social Dining Setting

This paper presents the Sensory Interactive Table (SIT): an instrumented, interactive dining table. Through the use of load cells and LEDs that are embedded in the table surface, SIT allows us to study: (1) the eating behaviors of people in a social setting, (2) the social interactions around the eating behaviors of people in a social setting, and (3) the continuous cycle of feedback through LEDs on people’s eating behavior and their response to this feedback in real time, to ultimately create an effective dietary support system. This paper presents the hard- and software specifications of the system, and it shows the potential of the system to capture mass-related dimensions in real time and with high accuracy and spatial resolution.

Image-Based Food Classification and Volume Estimation for Dietary Assessment: A Review

A daily dietary assessment method named 24-hour dietary recall has commonly been used in nutritional epidemiology studies to capture detailed information of the food eaten by the participants to help understand their dietary behaviour. However, in this self-reporting technique, the food types and the portion size reported highly depends on users' subjective judgement which may lead to a biased and inaccurate dietary analysis result. As a result, a variety of visual-based dietary assessment approaches have been proposed recently. While these methods show promises in tackling issues in nutritional epidemiology studies, several challenges and forthcoming opportunities, as detailed in this study, still exist. This study provides an overview of computing algorithms, mathematical models and methodologies used in the field of image-based dietary assessment. It also provides a comprehensive comparison of the state of the art approaches in food recognition and volume/weight estimation in terms of their processing speed, model accuracy, efficiency and constraints. It will be followed by a discussion on deep learning method and its efficacy in dietary assessment. After a comprehensive exploration, we found that integrated dietary assessment systems combining with different approaches could be the potential solution to tackling the challenges in accurate dietary intake assessment.

Automatic, wearable-based, in-field eating detection approaches for public health research: a scoping review

Dietary intake, eating behaviors, and context are important in chronic disease development, yet our ability to accurately assess these in research settings can be limited by biased traditional self-reporting tools. Objective measurement tools, specifically, wearable sensors, present the opportunity to minimize the major limitations of self-reported eating measures by generating supplementary sensor data that can improve the validity of self-report data in naturalistic settings. This scoping review summarizes the current use of wearable devices/sensors that automatically detect eating-related activity in naturalistic research settings. Five databases were searched in December 2019, and 618 records were retrieved from the literature search. This scoping review included N = 40 studies (from 33 articles) that reported on one or more wearable sensors used to automatically detect eating activity in the field. The majority of studies (N = 26, 65%) used multi-sensor systems (incorporating > 1 wearable sensors), and accelerometers were the most commonly utilized sensor (N = 25, 62.5%). All studies (N = 40, 100.0%) used either self-report or objective ground-truth methods to validate the inferred eating activity detected by the sensor(s). The most frequently reported evaluation metrics were Accuracy (N = 12) and F1-score (N = 10). This scoping review highlights the current state of wearable sensors' ability to improve upon traditional eating assessment methods by passively detecting eating activity in naturalistic settings, over long periods of time, and with minimal user interaction. A key challenge in this field, wide variation in eating outcome measures and evaluation metrics, demonstrates the need for the development of a standardized form of comparability among sensors/multi-sensor systems and multidisciplinary collaboration.

Counting Bites With Bits: Expert Workshop Addressing Calorie and Macronutrient Intake Monitoring

BACKGROUND

Conventional diet assessment approaches such as the 24-hour self-reported recall are burdensome, suffer from recall bias, and are inaccurate in estimating energy intake. Wearable sensor technology, coupled with advanced algorithms, is increasingly showing promise in its ability to capture behaviors that provide useful information for estimating calorie and macronutrient intake.

OBJECTIVE

This paper aimed to summarize current technological approaches to monitoring energy intake on the basis of expert opinion from a workshop panel and to make recommendations to advance technology and algorithms to improve estimation of energy expenditure.

METHODS

A 1-day invitational workshop sponsored by the National Science Foundation was held at Northwestern University. A total of 30 participants, including population health researchers, engineers, and intervention developers, from 6 universities and the National Institutes of Health participated in a panel discussing the state of evidence with regard to monitoring calorie intake and eating behaviors.

RESULTS

Calorie monitoring using technological approaches can be characterized into 3 domains: (1) image-based sensing (eg, wearable and smartphone-based cameras combined with machine learning algorithms); (2) eating action unit (EAU) sensors (eg, to measure feeding gesture and chewing rate); and (3) biochemical measures (eg, serum and plasma metabolite concentrations). We discussed how each domain functions, provided examples of promising solutions, and highlighted potential challenges and opportunities in each domain. Image-based sensor research requires improved ground truth (context and known information about the foods), accurate food image segmentation and recognition algorithms, and reliable methods of estimating portion size. EAU-based domain research is limited by the understanding of when their systems (device and inference algorithm) succeed and fail, need for privacy-protecting methods of capturing ground truth, and uncertainty in food categorization. Although an exciting novel technology, the challenges of biochemical sensing range from a lack of adaptability to environmental effects (eg, temperature change) and mechanical impact, instability of wearable sensor performance over time, and single-use design.

CONCLUSIONS

Conventional approaches to calorie monitoring rely predominantly on self-reports. These approaches can gain contextual information from image-based and EAU-based domains that can map automatically captured food images to a food database and detect proxies that correlate with food volume and caloric intake. Although the continued development of advanced machine learning techniques will advance the accuracy of such wearables, biochemical sensing provides an electrochemical analysis of sweat using soft bioelectronics on human skin, enabling noninvasive measures of chemical compounds that provide insight into the digestive and endocrine systems. Future computing-based researchers should focus on reducing the burden of wearable sensors, aligning data across multiple devices, automating methods of data annotation, increasing rigor in studying system acceptability, increasing battery lifetime, and rigorously testing validity of the measure. Such research requires moving promising technological solutions from the controlled laboratory setting to the field.

Learning Deep Representations for Video-Based Intake Gesture Detection

Automatic detection of individual intake gestures during eating occasions has the potential to improve dietary monitoring and support dietary recommendations. Existing studies typically make use of on-body solutions such as inertial and audio sensors, while video is used as ground truth. Intake gesture detection directly based on video has rarely been attempted. In this study, we address this gap and show that deep learning architectures can successfully be applied to the problem of video-based detection of intake gestures. For this purpose, we collect and label video data of eating occasions using 360-degree video of 102 participants. Applying state-of-the-art approaches from video action recognition, our results show that (1) the best model achieves an F₁ score of 0.858, (2) appearance features contribute more than motion features, and (3) temporal context in form of multiple video frames is essential for top model performance.

Systematic Literature Review of Food-Intake Monitoring in an Aging Population

The dietary habits of people directly impact their health conditions. Especially in elder populations (in 2017, 6.7% of the world's population was over 65 years of age), these habits could lead to important-nutrient losses that could seriously affect their cognitive and functional state. Recently, a great research effort has been devoted to using different technologies and proposing different techniques for monitoring food-intake. Nevertheless, these techniques are usually generic but make use of the most innovative technologies and methodologies to obtain the best possible monitoring results. However, a large percentage of elderly people live in depopulated rural areas (in Spain, 28.1% of the elderly population lives in this type of area) with a fragile cultural and socioeconomic context. The use of these techniques in these environments is crucial to improving this group's quality of life (and even reducing their healthcare expenses). At the same time, it is especially challenging since they have very specific and strict requirements regarding the use and application of technology. In this Systematic Literature Review (SLR), we analyze the most important proposed technologies and techniques in order to identify whether they can be applied in this context and if they can be used to improve the quality of life of this fragile collective. In this SLR, we have analyzed 326 papers. From those, 29 proposals have been completely analyzed, taking into account the characteristics and requirements of this population.

Issues in Measuring and Interpreting Diet and Its Contribution to Obesity

PURPOSE OF REVIEW

This review summarises the issues related to the measurement and interpretation of dietary intake in individuals with overweight and obesity, as well as identifies future research priorities.

RECENT FINDINGS

Some aspects of the assessment of dietary intake have improved through the application of technology-based methods and the use of dietary biomarkers. In populations with overweight and obesity, misreporting bias related to social desirability is a prominent issue. Future efforts should focus on combining technology-based dietary methods with the use of dietary biomarkers to help reduce and account for the impact of these biases. Future research will be important in terms of strengthening methods used in the assessment and interpretation of dietary intake data in the context of overweight and obesity.

Food Composition Impacts the Accuracy of Wearable Devices When Estimating Energy Intake from Energy-Dense Food

The present study aimed to assess the feasibility and reliability of an a3utomatic food intake measurement device in estimating energy intake from energy-dense foods. Eighteen volunteers aged 20-36 years were recruited from the University of Padova. The device used in the present study was the Bite Counter (Bite Technologies, Pendleton, USA). The rationale of the device is that the wrist movements occurring in the act of bringing food to the mouth present unique patterns that are recognized and recorded by the Bite Counter. Subjects were asked to wear the Bite Counter on the wrist of the dominant hand, to turn the device on before the first bite and to turn it off once he or she finished his or her meal. The accuracy of caloric intake was significantly different among the methods used. In addition, the device's accuracy in estimating energy intake varied according to the type and amount of macronutrients present, and the difference was independent of the number of bites recorded. Further research is needed to overcome the current limitations of wearable devices in estimating caloric intake, which is not independent of the food being eaten.

Assessing Eating Behaviour Using Upper Limb Mounted Motion Sensors: A Systematic Review

Wearable motion tracking sensors are now widely used to monitor physical activity, and have recently gained more attention in dietary monitoring research. The aim of this review is to synthesise research to date that utilises upper limb motion tracking sensors, either individually or in combination with other technologies (e.g., cameras, microphones), to objectively assess eating behaviour. Eleven electronic databases were searched in January 2019, and 653 distinct records were obtained. Including 10 studies found in backward and forward searches, a total of 69 studies met the inclusion criteria, with 28 published since 2017. Fifty studies were conducted exclusively in laboratory settings, 13 exclusively in free-living settings, and three in both settings. The most commonly used motion sensor was an accelerometer (64) worn on the wrist (60) or lower arm (5), while in most studies (45), accelerometers were used in combination with gyroscopes. Twenty-six studies used commercial-grade smartwatches or fitness bands, 11 used professional grade devices, and 32 used standalone sensor chipsets. The most used machine learning approaches were Support Vector Machine (SVM, n = 21), Random Forest (n = 19), Decision Tree (n = 16), Hidden Markov Model (HMM, n = 10) algorithms, and from 2017 Deep Learning (n = 5). While comparisons of the detection models are not valid due to the use of different datasets, the models that consider the sequential context of data across time, such as HMM and Deep Learning, show promising results for eating activity detection. We discuss opportunities for future research and emerging applications in the context of dietary assessment and monitoring.