Automatic identification of the number of food items in a meal using clustering techniques based on the monitoring of swallowing and chewing

Statistical models for meal-level estimation of mass and energy intake using features derived from video observation and a chewing sensor

Accurate and objective assessment of energy intake remains an ongoing problem. We used features derived from annotated video observation and a chewing sensor to predict mass and energy intake during a meal without participant self-report. 30 participants each consumed 4 different meals in a laboratory setting and wore a chewing sensor while being videotaped. Subject-independent models were derived from bite, chew, and swallow features obtained from either video observation or information extracted from the chewing sensor. With multiple regression analysis, a forward selection procedure was used to choose the best model. The best estimates of meal mass and energy intake had (mean ± standard deviation) absolute percentage errors of 25.2% ± 18.9% and 30.1% ± 33.8%, respectively, and mean ± standard deviation estimation errors of −17.7 ± 226.9 g and −6.1 ± 273.8 kcal using features derived from both video observations and sensor data. Both video annotation and sensor-derived features may be utilized to objectively quantify energy intake.

Unobtrusive and Wearable Systems for Automatic Dietary Monitoring

The threat of obesity, diabetes, anorexia, and bulimia in our society today has motivated extensive research on dietary monitoring. Standard self-report methods such as 24-h recall and food frequency questionnaires are expensive, burdensome, and unreliable to handle the growing health crisis. Long-term activity monitoring in daily living is a promising approach to provide individuals with quantitative feedback that can encourage healthier habits. Although several studies have attempted automating dietary monitoring using wearable, handheld, smart-object, and environmental systems, it remains an open research problem. This paper aims to provide a comprehensive review of wearable and hand-held approaches from 2004 to 2016. Emphasis is placed on sensor types used, signal analysis and machine learning methods, as well as a benchmark of state-of-the art work in this field. Key issues, challenges, and gaps are highlighted to motivate future work toward development of effective, reliable, and robust dietary monitoring systems.

A Novel Chewing Detection System Based on PPG, Audio, and Accelerometry

In the context of dietary management, accurate monitoring of eating habits is receiving increased attention. Wearable sensors, combined with the connectivity and processing of modern smartphones, can be used to robustly extract objective and real-time measurements of human behavior. In particular, for the task of chewing detection, several approaches based on an in-ear microphone can be found in the literature, while other types of sensors have also been reported, such as strain sensors. In this paper, performed in the context of the SPLENDID project, we propose to combine an in-ear microphone with a photoplethysmography (PPG) sensor placed in the ear concha, in a new high accuracy and low sampling rate prototype chewing detection system. We propose a pipeline that initially processes each sensor signal separately, and then fuses both to perform the final detection. Features are extracted from each modality, and support vector machine (SVM) classifiers are used separately to perform snacking detection. Finally, we combine the SVM scores from both signals in a late-fusion scheme, which leads to increased eating detection accuracy. We evaluate the proposed eating monitoring system on a challenging, semifree living dataset of 14 subjects, which includes more than 60 h of audio and PPG signal recordings. Results show that fusing the audio and PPG signals significantly improves the effectiveness of eating event detection, achieving accuracy up to 0.938 and class-weighted accuracy up to 0.892.

An overview of the state of the art of automated capture of dietary intake information

Significant benefits arise from being able to capture dietary or nutritional intake information automatically or semi-automatically. These include the ability for individuals to know and understand their nutritional intake and hence improve their diet and health. To date, only highly manual processes such as 24-hour recall, food diaries, and food journals have been utilized which have been overly cumbersome for widespread adoption. Emerging informatics, computer vision, mobile computing, and sensor-based approaches are likely to play a role in further automating the capture of dietary intake information and these are becoming increasingly utilizable through such advents as the rapid and ubiquitous uptake of smartphones with built-in digital cameras and other sensors. In this paper, we review the state of the art of technologies for automatic capture of dietary intake information and identify significant outstanding research problems and promising directions.

Automatic food intake detection based on swallowing sounds

This paper presents a novel fully automatic food intake detection methodology, an important step toward objective monitoring of ingestive behavior. The aim of such monitoring is to improve our understanding of eating behaviors associated with obesity and eating disorders. The proposed methodology consists of two stages. First, acoustic detection of swallowing instances based on mel-scale Fourier spectrum features and classification using support vector machines is performed. Principal component analysis and a smoothing algorithm are used to improve swallowing detection accuracy. Second, the frequency of swallowing is used as a predictor for detection of food intake episodes. The proposed methodology was tested on data collected from 12 subjects with various degrees of adiposity. Average accuracies of >80% and >75% were obtained for intra-subject and inter-subject models correspondingly with a temporal resolution of 30s. Results obtained on 44.1 hours of data with a total of 7305 swallows show that detection accuracies are comparable for obese and lean subjects. They also suggest feasibility of food intake detection based on swallowing sounds and potential of the proposed methodology for automatic monitoring of ingestive behavior. Based on a wearable non-invasive acoustic sensor the proposed methodology may potentially be used in free-living conditions.