1
|
Li X, Yin A, Choi HY, Chan V, Allman-Farinelli M, Chen J. Evaluating the Quality and Comparative Validity of Manual Food Logging and Artificial Intelligence-Enabled Food Image Recognition in Apps for Nutrition Care. Nutrients 2024; 16:2573. [PMID: 39125452 PMCID: PMC11314244 DOI: 10.3390/nu16152573] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/08/2024] [Revised: 07/25/2024] [Accepted: 08/01/2024] [Indexed: 08/12/2024] Open
Abstract
For artificial intelligence (AI) to support nutrition care, high quality and accuracy of its features within smartphone applications (apps) are essential. This study evaluated popular apps' features, quality, behaviour change potential, and comparative validity of dietary assessment via manual logging and AI. The top 200 free and paid nutrition-related apps from Australia's Apple App and Google Play stores were screened (n = 800). Apps were assessed using MARS (quality) and ABACUS (behaviour change potential). Nutritional outputs from manual food logging and AI-enabled food-image recognition apps were compared with food records for Western, Asian, and Recommended diets. Among 18 apps, Noom scored highest on MARS (mean = 4.44) and ABACUS (21/21). From 16 manual food-logging apps, energy was overestimated for Western (mean: 1040 kJ) but underestimated for Asian (mean: -1520 kJ) diets. MyFitnessPal and Fastic had the highest accuracy (97% and 92%, respectively) out of seven AI-enabled food image recognition apps. Apps with more AI integration demonstrated better functionality, but automatic energy estimations from AI-enabled food image recognition were inaccurate. To enhance the integration of apps into nutrition care, collaborating with dietitians is essential for improving their credibility and comparative validity by expanding food databases. Moreover, training AI models are needed to improve AI-enabled food recognition, especially for mixed dishes and culturally diverse foods.
Collapse
Affiliation(s)
- Xinyi Li
- Discipline of Nutrition and Dietetics, Susan Wakil School of Nursing and Midwifery, Faculty of Medicine and Health, The University of Sydney, Camperdown, NSW 2006, Australia
- Charles Perkins Centre, The University of Sydney, Sydney, NSW 2006, Australia
| | - Annabelle Yin
- Discipline of Nutrition and Dietetics, Susan Wakil School of Nursing and Midwifery, Faculty of Medicine and Health, The University of Sydney, Camperdown, NSW 2006, Australia
- Charles Perkins Centre, The University of Sydney, Sydney, NSW 2006, Australia
| | - Ha Young Choi
- Discipline of Nutrition and Dietetics, Susan Wakil School of Nursing and Midwifery, Faculty of Medicine and Health, The University of Sydney, Camperdown, NSW 2006, Australia
- Charles Perkins Centre, The University of Sydney, Sydney, NSW 2006, Australia
| | - Virginia Chan
- Discipline of Nutrition and Dietetics, Susan Wakil School of Nursing and Midwifery, Faculty of Medicine and Health, The University of Sydney, Camperdown, NSW 2006, Australia
- Charles Perkins Centre, The University of Sydney, Sydney, NSW 2006, Australia
| | - Margaret Allman-Farinelli
- Discipline of Nutrition and Dietetics, Susan Wakil School of Nursing and Midwifery, Faculty of Medicine and Health, The University of Sydney, Camperdown, NSW 2006, Australia
- Charles Perkins Centre, The University of Sydney, Sydney, NSW 2006, Australia
| | - Juliana Chen
- Discipline of Nutrition and Dietetics, Susan Wakil School of Nursing and Midwifery, Faculty of Medicine and Health, The University of Sydney, Camperdown, NSW 2006, Australia
- Charles Perkins Centre, The University of Sydney, Sydney, NSW 2006, Australia
| |
Collapse
|
2
|
Yin YH, Liu JYW, Välimäki M. "How difficult it is to change dietary behaviour" experience of older people with sarcopenic obesity: a qualitative study. BMC Geriatr 2024; 24:568. [PMID: 38951785 PMCID: PMC11218247 DOI: 10.1186/s12877-024-05157-0] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/04/2024] [Accepted: 06/17/2024] [Indexed: 07/03/2024] Open
Abstract
BACKGROUND Dietary intervention is an important method to manage sarcopenic obesity, but the implementation in real world is difficult to achieve an ideal condition. This study aimed to the experiences of older people with sarcopenic obesity during the implementation of dietary behavioural change (DBC) intervention. METHODS This study is a semi-structured individual interview embedded within a pilot randomized controlled trial on community-dwelling older people with sarcopenic obesity. Purposive sampling was applied to invite 21 participants who had received a 15-week DBC intervention. The interviews were audio-recorded and transcribed verbatim. Content analysis was performed to analyze the data. RESULTS The themes for facilitators included: (a) Attach importance to self's health; (b) Family's support; (c) Concern self's body shape; (d) Instructor's support; (e) Regular food diary taken. The themes for barriers included: (a) Difficulties of taking food diary; (b) Difficulties of calculating the food amount; (c) Yield to offspring's appetite; (d) Misjudging self's or family's appetite. CONCLUSION Support from family members and instructor, caring about self's health and body image facilitated the intervention implementation. The complication of food amount estimation and diary record, personal sacrifice for next generations, and previous living experience were barriers for implementing the intervention. Overall, the older people with sarcopenic obesity can accept the design of DBC intervention program and have great willing to join.
Collapse
Affiliation(s)
- Yue-Heng Yin
- School of Nursing, Nanjing Medical University, 101 Long Mian Avenue, Nanjing, 211166, China.
- School of Nursing, The Hong Kong Polytechnic University, Hong Kong SAR, China.
| | - Justina Yat Wa Liu
- School of Nursing, The Hong Kong Polytechnic University, Hong Kong SAR, China
| | - Maritta Välimäki
- Faculty of Medicine, Department of Public Health, University of Helsinki, Helsinki, Finland
- , Turku, Finland
| |
Collapse
|
3
|
Hassan EA, Khalifa Y, Morsy AA. sEMG-based automatic characterization of swallowed materials. Biomed Eng Online 2024; 23:48. [PMID: 38760808 PMCID: PMC11100060 DOI: 10.1186/s12938-024-01241-z] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/21/2023] [Accepted: 04/30/2024] [Indexed: 05/19/2024] Open
Abstract
Monitoring of ingestive activities is critically important for managing the health and wellness of individuals with various health conditions, including the elderly, diabetics, and individuals seeking better weight control. Monitoring swallowing events can be an ideal surrogate for developing streamlined methods for effective monitoring and quantification of eating or drinking events. Swallowing is an essential process for maintaining life. This seemingly simple process is the result of coordinated actions of several muscles and nerves in a complex fashion. In this study, we introduce automated methods for the detection and quantification of various eating and drinking activities. Wireless surface electromyography (sEMG) was used to detect chewing and swallowing from sEMG signals obtained from the sternocleidomastoid muscle, in addition to signals obtained from a wrist-mounted IMU sensor. A total of 4675 swallows were collected from 55 participants in the study. Multiple methods were employed to estimate bolus volumes in the case of fluid intake, including regression and classification models. Among the tested models, neural networks-based regression achieved an R2 of 0.88 and a root mean squared error of 0.2 (minimum bolus volume was 10 ml). Convolutional neural networks-based classification (when considering each bolus volume as a separate class) achieved an accuracy of over 99% using random cross-validation and around 66% using cross-subject validation. Multiple classification methods were also used for solid bolus type detection, including SVM and decision trees (DT), which achieved an accuracy above 99% with random validation and above 94% in cross-subject validation. Finally, regression models with both random and cross-subject validation were used for estimating the solid bolus volume with an R2 value that approached 1 and root mean squared error values as low as 0.00037 (minimum solid bolus weight was 3 gm). These reported results lay the foundation for a cost-effective and non-invasive method for monitoring swallowing activities which can be extremely beneficial in managing various chronic health conditions, such as diabetes and obesity.
Collapse
Affiliation(s)
- Eman A Hassan
- Biomedical Engineering Dept., Cairo University, Giza, Egypt.
| | - Yassin Khalifa
- Biomedical Engineering Dept., Cairo University, Giza, Egypt
- Center for Research Computing, University of Pittsburgh, Pittsburgh, PA, USA
- Information Technology Analytics, University of Pittsburgh, Pittsburgh, PA, USA
| | - Ahmed A Morsy
- Biomedical Engineering Dept., Cairo University, Giza, Egypt
| |
Collapse
|
4
|
Hossain MB, LaMunion SR, Crouter SE, Melanson EL, Sazonov E. A CNN Model for Physical Activity Recognition and Energy Expenditure Estimation from an Eyeglass-Mounted Wearable Sensor. SENSORS (BASEL, SWITZERLAND) 2024; 24:3046. [PMID: 38793899 PMCID: PMC11125058 DOI: 10.3390/s24103046] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 04/04/2024] [Revised: 05/04/2024] [Accepted: 05/10/2024] [Indexed: 05/26/2024]
Abstract
Metabolic syndrome poses a significant health challenge worldwide, prompting the need for comprehensive strategies integrating physical activity monitoring and energy expenditure. Wearable sensor devices have been used both for energy intake and energy expenditure (EE) estimation. Traditionally, sensors are attached to the hip or wrist. The primary aim of this research is to investigate the use of an eyeglass-mounted wearable energy intake sensor (Automatic Ingestion Monitor v2, AIM-2) for simultaneous recognition of physical activity (PAR) and estimation of steady-state EE as compared to a traditional hip-worn device. Study data were collected from six participants performing six structured activities, with the reference EE measured using indirect calorimetry (COSMED K5) and reported as metabolic equivalents of tasks (METs). Next, a novel deep convolutional neural network-based multitasking model (Multitasking-CNN) was developed for PAR and EE estimation. The Multitasking-CNN was trained with a two-step progressive training approach for higher accuracy, where in the first step the model for PAR was trained, and in the second step the model was fine-tuned for EE estimation. Finally, the performance of Multitasking-CNN on AIM-2 attached to eyeglasses was compared to the ActiGraph GT9X (AG) attached to the right hip. On the AIM-2 data, Multitasking-CNN achieved a maximum of 95% testing accuracy of PAR, a minimum of 0.59 METs mean square error (MSE), and 11% mean absolute percentage error (MAPE) in EE estimation. Conversely, on AG data, the Multitasking-CNN model achieved a maximum of 82% testing accuracy in PAR, a minimum of 0.73 METs MSE, and 13% MAPE in EE estimation. These results suggest the feasibility of using an eyeglass-mounted sensor for both PAR and EE estimation.
Collapse
Affiliation(s)
- Md Billal Hossain
- Department of Electrical and Computer Engineering, The University of Alabama, Tuscaloosa, AL 35487, USA;
| | - Samuel R. LaMunion
- Department of Kinesiology, Recreation and Sport Studies, The University of Tennessee, Knoxville, TN 37996, USA; (S.R.L.); (S.E.C.)
| | - Scott E. Crouter
- Department of Kinesiology, Recreation and Sport Studies, The University of Tennessee, Knoxville, TN 37996, USA; (S.R.L.); (S.E.C.)
| | - Edward L. Melanson
- USA Division of Endocrinology, Metabolism, and Diabetes, University of Colorado Anschutz Medical Campus, Aurora, CO 80045, USA;
| | - Edward Sazonov
- Department of Electrical and Computer Engineering, The University of Alabama, Tuscaloosa, AL 35487, USA;
| |
Collapse
|
5
|
Thomas DM, Knight R, Gilbert JA, Cornelis MC, Gantz MG, Burdekin K, Cummiskey K, Sumner SCJ, Pathmasiri W, Sazonov E, Gabriel KP, Dooley EE, Green MA, Pfluger A, Kleinberg S. Transforming Big Data into AI-ready data for nutrition and obesity research. Obesity (Silver Spring) 2024; 32:857-870. [PMID: 38426232 PMCID: PMC11180473 DOI: 10.1002/oby.23989] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 09/16/2023] [Revised: 12/15/2023] [Accepted: 12/26/2023] [Indexed: 03/02/2024]
Abstract
OBJECTIVE Big Data are increasingly used in obesity and nutrition research to gain new insights and derive personalized guidance; however, this data in raw form are often not usable. Substantial preprocessing, which requires machine learning (ML), human judgment, and specialized software, is required to transform Big Data into artificial intelligence (AI)- and ML-ready data. These preprocessing steps are the most complex part of the entire modeling pipeline. Understanding the complexity of these steps by the end user is critical for reducing misunderstanding, faulty interpretation, and erroneous downstream conclusions. METHODS We reviewed three popular obesity/nutrition Big Data sources: microbiome, metabolomics, and accelerometry. The preprocessing pipelines, specialized software, challenges, and how decisions impact final AI- and ML-ready products were detailed. RESULTS Opportunities for advances to improve quality control, speed of preprocessing, and intelligent end user consumption were presented. CONCLUSIONS Big Data have the exciting potential for identifying new modifiable factors that impact obesity research. However, to ensure accurate interpretation of conclusions arising from Big Data, the choices involved in preparing AI- and ML-ready data need to be transparent to investigators and clinicians relying on the conclusions.
Collapse
Affiliation(s)
- Diana M. Thomas
- Department of Mathematical Sciences, United States Military Academy, West Point, NY 10996
| | - Rob Knight
- Bioinformatics and Systems Biology Program, University of San Diego, La Jolla, CA 92037
| | - Jack A. Gilbert
- Department of Pediatrics and Scripps Institution of Oceanography, University of California San Diego, La Jolla, CA 92037
| | - Marilyn C. Cornelis
- Department of Preventive Medicine, Northwestern University Feinberg School of Medicine, Chicago, IL 60611
| | - Marie G. Gantz
- RTI International, Biostatics and Epidemiology Division, Research Triangle Park, NC 27709
| | - Kate Burdekin
- RTI International, Biostatics and Epidemiology Division, Research Triangle Park, NC 27709
| | - Kevin Cummiskey
- Department of Mathematical Sciences, United States Military Academy, West Point, NY 10996
| | - Susan C. J. Sumner
- Department of Nutrition, Nutrition Research Institute, UNC Chapel Hill, 500 Laureate Way, Kannapolis, NC 28081
| | - Wimal Pathmasiri
- Department of Nutrition, Nutrition Research Institute, UNC Chapel Hill, 500 Laureate Way, Kannapolis, NC 28081
| | - Edward Sazonov
- Electrical and Computer Engineering Department, The University of Alabama, Tuscaloosa, AL 35487
| | - Kelley Pettee Gabriel
- Department of Epidemiology, The University of Alabama at Birmingham, Birmingham, AL 35294
| | - Erin E. Dooley
- Department of Epidemiology, The University of Alabama at Birmingham, Birmingham, AL 35294
| | - Mark A. Green
- Department of Geography & Planning, University of Liverpool, Liverpool, L69 3BX, UK
| | - Andrew Pfluger
- Department of Geography and Environmental Engineering, United States Military Academy, West Point, NY 10996
| | - Samantha Kleinberg
- Computer Science Department, Stevens Institute of Technology, Hoboken, NJ 07030
| |
Collapse
|
6
|
Moore J, McMeekin P, Parkes T, Walker R, Morris R, Stuart S, Hetherington V, Godfrey A. Contextualizing remote fall risk: Video data capture and implementing ethical AI. NPJ Digit Med 2024; 7:61. [PMID: 38448611 PMCID: PMC10917734 DOI: 10.1038/s41746-024-01050-7] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/03/2023] [Accepted: 02/16/2024] [Indexed: 03/08/2024] Open
Abstract
Wearable inertial measurement units (IMUs) are being used to quantify gait characteristics that are associated with increased fall risk, but the current limitation is the lack of contextual information that would clarify IMU data. Use of wearable video-based cameras would provide a comprehensive understanding of an individual's habitual fall risk, adding context to clarify abnormal IMU data. Generally, there is taboo when suggesting the use of wearable cameras to capture real-world video, clinical and patient apprehension due to ethical and privacy concerns. This perspective proposes that routine use of wearable cameras could be realized within digital medicine through AI-based computer vision models to obfuscate/blur/shade sensitive information while preserving helpful contextual information for a comprehensive patient assessment. Specifically, no person sees the raw video data to understand context, rather AI interprets the raw video data first to blur sensitive objects and uphold privacy. That may be more routinely achieved than one imagines as contemporary resources exist. Here, to showcase/display the potential an exemplar model is suggested via off-the-shelf methods to detect and blur sensitive objects (e.g., people) with an accuracy of 88%. Here, the benefit of the proposed approach includes a more comprehensive understanding of an individual's free-living fall risk (from free-living IMU-based gait) without compromising privacy. More generally, the video and AI approach could be used beyond fall risk to better inform habitual experiences and challenges across a range of clinical cohorts. Medicine is becoming more receptive to wearables as a helpful toolbox, camera-based devices should be plausible instruments.
Collapse
Affiliation(s)
- Jason Moore
- Department of Computer and Information Sciences, Northumbria University, Newcastle upon Tyne, UK
| | - Peter McMeekin
- Nursing, Midwifery and Health, Northumbria University, Newcastle upon Tyne, UK
| | - Thomas Parkes
- Department of Computer and Information Sciences, Northumbria University, Newcastle upon Tyne, UK
| | - Richard Walker
- Northumbria Healthcare NHS Foundation Trust, North Tyneside, Newcastle upon Tyne, UK
| | - Rosie Morris
- Northumbria Healthcare NHS Foundation Trust, North Tyneside, Newcastle upon Tyne, UK
- Department of Sport, Exercise and Rehabilitation, Northumbria University, Newcastle upon Tyne, UK
| | - Samuel Stuart
- Northumbria Healthcare NHS Foundation Trust, North Tyneside, Newcastle upon Tyne, UK
- Department of Sport, Exercise and Rehabilitation, Northumbria University, Newcastle upon Tyne, UK
- Department of Neurology, Oregon Health & Science University, Portland, OR, USA
| | - Victoria Hetherington
- Cumbria, Northumberland, Tyne and Wear NHS Foundation Trust, Wolfson Research Centre, Campus for Ageing and Vitality, Newcastle upon Tyne, UK
| | - Alan Godfrey
- Department of Computer and Information Sciences, Northumbria University, Newcastle upon Tyne, UK.
| |
Collapse
|
7
|
Tang Z, Patyk A, Jolly J, Goldstein SP, Thomas JG, Hoover A. Detecting Eating Episodes From Wrist Motion Using Daily Pattern Analysis. IEEE J Biomed Health Inform 2024; 28:1054-1065. [PMID: 38079368 PMCID: PMC10904729 DOI: 10.1109/jbhi.2023.3341077] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/12/2024]
Abstract
This paper presents new methods to detect eating from wrist motion. Our main novelty is that we analyze a full day of wrist motion data as a single sample so that the detection of eating occurrences can benefit from diurnal context. We develop a two-stage framework to facilitate a feasible full-day analysis. The first-stage model calculates local probabilities of eating P(Ew) within windows of data, and the second-stage model calculates enhanced probabilities of eating P(Ed) by treating all P(Ew) within a single day as one sample. The framework also incorporates an augmentation technique, which involves the iterative retraining of the first-stage model. This allows us to generate a sufficient number of day-length samples from datasets of limited size. We test our methods on the publicly available Clemson All-Day (CAD) dataset and FreeFIC dataset, and find that the inclusion of day-length analysis substantially improves accuracy in detecting eating episodes. We also benchmark our results against several state-of-the-art methods. Our approach achieved an eating episode true positive rate (TPR) of 89% with 1.4 false positives per true positive (FP/TP), and a time weighted accuracy of 84%, which are the highest accuracies reported on the CAD dataset. Our results show that the daily pattern classifier substantially improves meal detections and in particular reduces transient false detections that tend to occur when relying on shorter windows to look for individual ingestion or consumption events.
Collapse
|
8
|
Wang C, T SK, De Raedt W, Camps G, Hallez H, Vanrumste B. Eat-Radar: Continuous Fine-Grained Intake Gesture Detection Using FMCW Radar and 3D Temporal Convolutional Network With Attention. IEEE J Biomed Health Inform 2024; 28:1000-1011. [PMID: 38051610 DOI: 10.1109/jbhi.2023.3339703] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/07/2023]
Abstract
Unhealthy dietary habits are considered as the primary cause of various chronic diseases, including obesity and diabetes. The automatic food intake monitoring system has the potential to improve the quality of life (QoL) of people with diet-related diseases through dietary assessment. In this work, we propose a novel contactless radar-based approach for food intake monitoring. Specifically, a Frequency Modulated Continuous Wave (FMCW) radar sensor is employed to recognize fine-grained eating and drinking gestures. The fine-grained eating/drinking gesture contains a series of movements from raising the hand to the mouth until putting away the hand from the mouth. A 3D temporal convolutional network with self-attention (3D-TCN-Att) is developed to detect and segment eating and drinking gestures in meal sessions by processing the Range-Doppler Cube (RD Cube). Unlike previous radar-based research, this work collects data in continuous meal sessions (more realistic scenarios). We create a public dataset comprising 70 meal sessions (4,132 eating gestures and 893 drinking gestures) from 70 participants with a total duration of 1,155 minutes. Four eating styles (fork & knife, chopsticks, spoon, hand) are included in this dataset. To validate the performance of the proposed approach, seven-fold cross-validation method is applied. The 3D-TCN-Att model achieves a segmental F1-score of 0.896 and 0.868 for eating and drinking gestures, respectively. The results of the proposed approach indicate the feasibility of using radar for fine-grained eating and drinking gesture detection and segmentation in meal sessions.
Collapse
|
9
|
Ghosh T, Han Y, Raju V, Hossain D, McCrory MA, Higgins J, Boushey C, Delp EJ, Sazonov E. Integrated image and sensor-based food intake detection in free-living. Sci Rep 2024; 14:1665. [PMID: 38238423 PMCID: PMC10796396 DOI: 10.1038/s41598-024-51687-3] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/19/2021] [Accepted: 01/08/2024] [Indexed: 01/22/2024] Open
Abstract
The first step in any dietary monitoring system is the automatic detection of eating episodes. To detect eating episodes, either sensor data or images can be used, and either method can result in false-positive detection. This study aims to reduce the number of false positives in the detection of eating episodes by a wearable sensor, Automatic Ingestion Monitor v2 (AIM-2). Thirty participants wore the AIM-2 for two days each (pseudo-free-living and free-living). The eating episodes were detected by three methods: (1) recognition of solid foods and beverages in images captured by AIM-2; (2) recognition of chewing from the AIM-2 accelerometer sensor; and (3) hierarchical classification to combine confidence scores from image and accelerometer classifiers. The integration of image- and sensor-based methods achieved 94.59% sensitivity, 70.47% precision, and 80.77% F1-score in the free-living environment, which is significantly better than either of the original methods (8% higher sensitivity). The proposed method successfully reduces the number of false positives in the detection of eating episodes.
Collapse
Affiliation(s)
- Tonmoy Ghosh
- Electrical and Computer Engineering Department, University of Alabama, Tuscaloosa, AL, 35401, USA.
| | - Yue Han
- Electrical and Computer Engineering Department, Purdue University, West Lafayette, IN, 47907, USA
| | - Viprav Raju
- Electrical and Computer Engineering Department, University of Alabama, Tuscaloosa, AL, 35401, USA
| | - Delwar Hossain
- Electrical and Computer Engineering Department, University of Alabama, Tuscaloosa, AL, 35401, USA
| | - Megan A McCrory
- Department of Health Sciences, Boston University, Boston, MA, 02215, USA
| | - Janine Higgins
- Department of Pediatrics-Endocrinology, University of Colorado, Denver, CO, 80045, USA
| | - Carol Boushey
- Epidemiology Program, University of Hawaii Cancer Center, Honolulu, HI, 96813, USA
| | - Edward J Delp
- Electrical and Computer Engineering Department, Purdue University, West Lafayette, IN, 47907, USA
| | - Edward Sazonov
- Electrical and Computer Engineering Department, University of Alabama, Tuscaloosa, AL, 35401, USA
| |
Collapse
|
10
|
Jia W, Li B, Zheng Y, Mao ZH, Sun M. Estimating Amount of Food in a Circular Dining Bowl from a Single Image. MADIMA '23 : PROCEEDINGS OF THE 8TH INTERNATIONAL WORKSHOP ON MULTIMEDIA ASSISTED DIETARY MANAGEMENT 2023; 2023:1-9. [PMID: 38288389 PMCID: PMC10823382 DOI: 10.1145/3607828.3617789] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 01/31/2024]
Abstract
Unhealthy diet is a top risk factor causing obesity and numerous chronic diseases. To help the public adopt healthy diet, nutrition scientists need user-friendly tools to conduct Dietary Assessment (DA). In recent years, new DA tools have been developed using a smartphone or a wearable device which acquires images during a meal. These images are then processed to estimate calories and nutrients of the consumed food. Although considerable progress has been made, 2D food images lack scale reference and 3D volumetric information. In addition, food must be sufficiently observable from the image. This basic condition can be met when the food is stand-alone (no food container is used) or it is contained in a shallow plate. However, the condition cannot be met easily when a bowl is used. The food is often occluded by the bowl edge, and the shape of the bowl may not be fully determined from the image. However, bowls are the most utilized food containers by billions of people in many parts of the world, especially in Asia and Africa. In this work, we propose to premeasure plates and bowls using a marked adhesive strip before a dietary study starts. This simple procedure eliminates the use of a scale reference throughout the DA study. In addition, we use mathematical models and image processing to reconstruct the bowl in 3D. Our key idea is to estimate how full the bowl is rather than how much food is (in either volume or weight) in the bowl. This idea reduces the effect of occlusion. The experimental data have shown satisfactory results of our methods which enable accurate DA studies using both plates and bowls with reduced burden on research participants.
Collapse
Affiliation(s)
- Wenyan Jia
- Department of Electrical and Computer Engineering, University of Pittsburgh, Pittsburgh, PA, USA
| | - Boyang Li
- Department of Electrical and Computer Engineering, University of Pittsburgh, Pittsburgh, PA, USA
| | - Yaguang Zheng
- Rory Meyers College of Nursing, New York University, New York, NY, USA
| | - Zhi-Hong Mao
- Departments of Electrical and Computer Engineering, and Bioengineering, University of Pittsburgh, Pittsburgh, PA, USA
| | - Mingui Sun
- Departments of Neurosurgery Electrical and Computer Engineering and Bioengineering, University of Pittsburgh, Pittsburgh, PA, USA
| |
Collapse
|
11
|
Jobarteh ML, McCrory MA, Lo B, Triantafyllidis KK, Qiu J, Griffin JP, Sazonov E, Sun M, Jia W, Baranowski T, Anderson AK, Maitland K, Frost G. Evaluation of Acceptability, Functionality, and Validity of a Passive Image-Based Dietary Intake Assessment Method in Adults and Children of Ghanaian and Kenyan Origin Living in London, UK. Nutrients 2023; 15:4075. [PMID: 37764857 PMCID: PMC10537234 DOI: 10.3390/nu15184075] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/31/2023] [Revised: 09/18/2023] [Accepted: 09/19/2023] [Indexed: 09/29/2023] Open
Abstract
BACKGROUND Accurate estimation of dietary intake is challenging. However, whilst some progress has been made in high-income countries, low- and middle-income countries (LMICs) remain behind, contributing to critical nutritional data gaps. This study aimed to validate an objective, passive image-based dietary intake assessment method against weighed food records in London, UK, for onward deployment to LMICs. METHODS Wearable camera devices were used to capture food intake on eating occasions in 18 adults and 17 children of Ghanaian and Kenyan origin living in London. Participants were provided pre-weighed meals of Ghanaian and Kenyan cuisine and camera devices to automatically capture images of the eating occasions. Food images were assessed for portion size, energy, nutrient intake, and the relative validity of the method compared to the weighed food records. RESULTS The Pearson and Intraclass correlation coefficients of estimates of intakes of food, energy, and 19 nutrients ranged from 0.60 to 0.95 and 0.67 to 0.90, respectively. Bland-Altman analysis showed good agreement between the image-based method and the weighed food record. Under-estimation of dietary intake by the image-based method ranged from 4 to 23%. CONCLUSIONS Passive food image capture and analysis provides an objective assessment of dietary intake comparable to weighed food records.
Collapse
Affiliation(s)
- Modou L. Jobarteh
- Department of Population Health, London School of Hygiene and Tropical Medicine, London WC1E 7HT, UK
| | - Megan A. McCrory
- Department of Health Sciences, Boston University, Boston, MA 02215, USA;
| | - Benny Lo
- Hamlyn Centre, Department of Surgery and Cancer, Imperial College London, London SW7 2AZ, UK; (B.L.); (J.Q.)
| | - Konstantinos K. Triantafyllidis
- Section for Nutrition Research, Department of Metabolism, Digestion and Reproduction, Imperial College London, London SW7 2BX, UK; (K.K.T.); (J.P.G.); (G.F.)
| | - Jianing Qiu
- Hamlyn Centre, Department of Surgery and Cancer, Imperial College London, London SW7 2AZ, UK; (B.L.); (J.Q.)
| | - Jennifer P. Griffin
- Section for Nutrition Research, Department of Metabolism, Digestion and Reproduction, Imperial College London, London SW7 2BX, UK; (K.K.T.); (J.P.G.); (G.F.)
| | - Edward Sazonov
- Department of Electrical and Computer Engineering, University of Alabama, Tuscaloosa, AL 35487, USA;
| | - Mingui Sun
- Department of Neurological Surgery, University of Pittsburgh, Pittsburgh, PA 15261, USA; (M.S.); (W.J.)
| | - Wenyan Jia
- Department of Neurological Surgery, University of Pittsburgh, Pittsburgh, PA 15261, USA; (M.S.); (W.J.)
| | - Tom Baranowski
- USDA/ARS Children’s Nutrition Research Center, Department of Pediatrics, Baylor College of Medicine, Houston, TX 77030, USA;
| | - Alex K. Anderson
- Department of Nutritional Sciences, University of Georgia, Athens, GA 30602, USA;
| | | | - Gary Frost
- Section for Nutrition Research, Department of Metabolism, Digestion and Reproduction, Imperial College London, London SW7 2BX, UK; (K.K.T.); (J.P.G.); (G.F.)
| |
Collapse
|
12
|
Hiraguchi H, Perone P, Toet A, Camps G, Brouwer AM. Technology to Automatically Record Eating Behavior in Real Life: A Systematic Review. SENSORS (BASEL, SWITZERLAND) 2023; 23:7757. [PMID: 37765812 PMCID: PMC10534458 DOI: 10.3390/s23187757] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Received: 07/14/2023] [Revised: 09/04/2023] [Accepted: 09/05/2023] [Indexed: 09/29/2023]
Abstract
To monitor adherence to diets and to design and evaluate nutritional interventions, it is essential to obtain objective knowledge about eating behavior. In most research, measures of eating behavior are based on self-reporting, such as 24-h recalls, food records (food diaries) and food frequency questionnaires. Self-reporting is prone to inaccuracies due to inaccurate and subjective recall and other biases. Recording behavior using nonobtrusive technology in daily life would overcome this. Here, we provide an up-to-date systematic overview encompassing all (close-to) publicly or commercially available technologies to automatically record eating behavior in real-life settings. A total of 1328 studies were screened and, after applying defined inclusion and exclusion criteria, 122 studies were included for in-depth evaluation. Technologies in these studies were categorized by what type of eating behavior they measure and which type of sensor technology they use. In general, we found that relatively simple sensors are often used. Depending on the purpose, these are mainly motion sensors, microphones, weight sensors and photo cameras. While several of these technologies are commercially available, there is still a lack of publicly available algorithms that are needed to process and interpret the resulting data. We argue that future work should focus on developing robust algorithms and validating these technologies in real-life settings. Combining technologies (e.g., prompting individuals for self-reports at sensed, opportune moments) is a promising route toward ecologically valid studies of eating behavior.
Collapse
Affiliation(s)
- Haruka Hiraguchi
- TNO Human Factors, Netherlands Organization for Applied Scientific Research, Kampweg 55, 3769 DE Soesterberg, The Netherlands (A.-M.B.)
- Kikkoman Europe R&D Laboratory B.V., Nieuwe Kanaal 7G, 6709 PA Wageningen, The Netherlands
| | - Paola Perone
- TNO Human Factors, Netherlands Organization for Applied Scientific Research, Kampweg 55, 3769 DE Soesterberg, The Netherlands (A.-M.B.)
| | - Alexander Toet
- TNO Human Factors, Netherlands Organization for Applied Scientific Research, Kampweg 55, 3769 DE Soesterberg, The Netherlands (A.-M.B.)
| | - Guido Camps
- Division of Human Nutrition and Health, Wageningen University & Research, Stippeneng 4, 6708 WE Wageningen, The Netherlands
- OnePlanet Research Center, Plus Ultra II, Bronland 10, 6708 WE Wageningen, The Netherlands
| | - Anne-Marie Brouwer
- TNO Human Factors, Netherlands Organization for Applied Scientific Research, Kampweg 55, 3769 DE Soesterberg, The Netherlands (A.-M.B.)
- Department of Artificial Intelligence, Radboud University, Thomas van Aquinostraat 4, 6525 GD Nijmegen, The Netherlands
| |
Collapse
|
13
|
Zimmer M, Obbagy J, Scanlon KS, Gibbs K, Lerman JL, Hamner HC, Pannucci T, Sharfman A, Reedy J, Herrick KA. Count Every Bite to Make "Every Bite Count": Measurement Gaps and Future Directions for Assessing Diet From Birth to 24 Months. J Acad Nutr Diet 2023; 123:1269-1279.e1. [PMID: 37196980 PMCID: PMC10809843 DOI: 10.1016/j.jand.2023.05.011] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/28/2022] [Revised: 04/01/2023] [Accepted: 05/09/2023] [Indexed: 05/19/2023]
Affiliation(s)
- Meghan Zimmer
- Harvard University, Cambridge, Massachusetts; U.S. Department of Health and Human Services, National Cancer Institute, Bethesda, MD
| | - Julie Obbagy
- Food and Nutrition Service, USDA, Alexandria, Virginia
| | - Kelley S Scanlon
- Supplemental Nutrition and Safety Research and Analysis Division, Office of Policy Support, USDA Food and Nutrition Service, Alexandria, Virginia
| | - Kimberlea Gibbs
- Eunice Kennedy Shriver National Institute of Child Health and Human Development, Bethesda, Maryland
| | - Jennifer L Lerman
- U.S. Department of Health and Human Services, National Cancer Institute, Bethesda, MD
| | | | | | | | - Jill Reedy
- U.S. Department of Health and Human Services, National Cancer Institute, Bethesda, MD
| | - Kirsten A Herrick
- U.S. Department of Health and Human Services, National Cancer Institute, Bethesda, MD.
| |
Collapse
|
14
|
Ghosh T, McCrory MA, Marden T, Higgins J, Anderson AK, Domfe CA, Jia W, Lo B, Frost G, Steiner-Asiedu M, Baranowski T, Sun M, Sazonov E. I2N: image to nutrients, a sensor guided semi-automated tool for annotation of images for nutrition analysis of eating episodes. Front Nutr 2023; 10:1191962. [PMID: 37575335 PMCID: PMC10415029 DOI: 10.3389/fnut.2023.1191962] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/22/2023] [Accepted: 07/05/2023] [Indexed: 08/15/2023] Open
Abstract
Introduction Dietary assessment is important for understanding nutritional status. Traditional methods of monitoring food intake through self-report such as diet diaries, 24-hour dietary recall, and food frequency questionnaires may be subject to errors and can be time-consuming for the user. Methods This paper presents a semi-automatic dietary assessment tool we developed - a desktop application called Image to Nutrients (I2N) - to process sensor-detected eating events and images captured during these eating events by a wearable sensor. I2N has the capacity to offer multiple food and nutrient databases (e.g., USDA-SR, FNDDS, USDA Global Branded Food Products Database) for annotating eating episodes and food items. I2N estimates energy intake, nutritional content, and the amount consumed. The components of I2N are three-fold: 1) sensor-guided image review, 2) annotation of food images for nutritional analysis, and 3) access to multiple food databases. Two studies were used to evaluate the feasibility and usefulness of I2N: 1) a US-based study with 30 participants and a total of 60 days of data and 2) a Ghana-based study with 41 participants and a total of 41 days of data). Results In both studies, a total of 314 eating episodes were annotated using at least three food databases. Using I2N's sensor-guided image review, the number of images that needed to be reviewed was reduced by 93% and 85% for the two studies, respectively, compared to reviewing all the images. Discussion I2N is a unique tool that allows for simultaneous viewing of food images, sensor-guided image review, and access to multiple databases in one tool, making nutritional analysis of food images efficient. The tool is flexible, allowing for nutritional analysis of images if sensor signals aren't available.
Collapse
Affiliation(s)
- Tonmoy Ghosh
- Department of Electrical and Computer Engineering, University of Alabama, Tuscaloosa, AL, United States
| | - Megan A. McCrory
- Department of Health Sciences, Boston University, Boston, MA, United States
| | - Tyson Marden
- Colorado Clinical and Translational Sciences Institute, University of Colorado, Denver, CO, United States
| | - Janine Higgins
- Department of Medicine, University of Colorado Anschutz Medical Campus, Aurora, CO, United States
| | - Alex Kojo Anderson
- Department of Nutritional Sciences, University of Georgia, Athens, GA, United States
| | | | - Wenyan Jia
- Department of Electrical and Computer Engineering, University of Pittsburgh, Pittsburgh, PA, United States
| | - Benny Lo
- Department of Surgery and Cancer, Imperial College, London, United Kingdom
| | - Gary Frost
- Department of Metabolism, Digestion and Reproduction, Imperial College, London, United Kingdom
| | | | - Tom Baranowski
- Children’s Nutrition Research Center, Department of Pediatrics, Baylor College of Medicine, Houston, TX, United States
| | - Mingui Sun
- Department of Neurological Surgery, University of Pittsburgh, Pittsburgh, PA, United States
| | - Edward Sazonov
- Department of Electrical and Computer Engineering, University of Alabama, Tuscaloosa, AL, United States
| |
Collapse
|
15
|
Chen X, Kamavuako EN. Vision-Based Methods for Food and Fluid Intake Monitoring: A Literature Review. SENSORS (BASEL, SWITZERLAND) 2023; 23:6137. [PMID: 37447988 PMCID: PMC10346353 DOI: 10.3390/s23136137] [Citation(s) in RCA: 2] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 04/10/2023] [Revised: 05/28/2023] [Accepted: 06/28/2023] [Indexed: 07/15/2023]
Abstract
Food and fluid intake monitoring are essential for reducing the risk of dehydration, malnutrition, and obesity. The existing research has been preponderantly focused on dietary monitoring, while fluid intake monitoring, on the other hand, is often neglected. Food and fluid intake monitoring can be based on wearable sensors, environmental sensors, smart containers, and the collaborative use of multiple sensors. Vision-based intake monitoring methods have been widely exploited with the development of visual devices and computer vision algorithms. Vision-based methods provide non-intrusive solutions for monitoring. They have shown promising performance in food/beverage recognition and segmentation, human intake action detection and classification, and food volume/fluid amount estimation. However, occlusion, privacy, computational efficiency, and practicality pose significant challenges. This paper reviews the existing work (253 articles) on vision-based intake (food and fluid) monitoring methods to assess the size and scope of the available literature and identify the current challenges and research gaps. This paper uses tables and graphs to depict the patterns of device selection, viewing angle, tasks, algorithms, experimental settings, and performance of the existing monitoring systems.
Collapse
Affiliation(s)
- Xin Chen
- Department of Engineering, King’s College London, London WC2R 2LS, UK;
| | - Ernest N. Kamavuako
- Department of Engineering, King’s College London, London WC2R 2LS, UK;
- Faculté de Médecine, Université de Kindu, Site de Lwama II, Kindu, Maniema, Democratic Republic of the Congo
| |
Collapse
|
16
|
Breit M, Padia J, Marden T, Forjan D, Zhaoxing P, Zhou W, Ghosh T, Thomas G, McCrory MA, Sazonov E, Higgins J. The spectrum of eating environments encountered in free living adults documented using a passive capture food intake wearable device. Front Nutr 2023; 10:1119542. [PMID: 37252243 PMCID: PMC10212618 DOI: 10.3389/fnut.2023.1119542] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/08/2022] [Accepted: 03/29/2023] [Indexed: 05/31/2023] Open
Abstract
Introduction The aim of this feasibility and proof-of-concept study was to examine the use of a novel wearable device for automatic food intake detection to capture the full range of free-living eating environments of adults with overweight and obesity. In this paper, we document eating environments of individuals that have not been thoroughly described previously in nutrition software as current practices rely on participant self-report and methods with limited eating environment options. Methods Data from 25 participants and 116 total days (7 men, 18 women, Mage = 44 ± 12 years, BMI 34.3 ± 5.2 kg/mm2), who wore the passive capture device for at least 7 consecutive days (≥12h waking hours/d) were analyzed. Data were analyzed at the participant level and stratified amongst meal type into breakfast, lunch, dinner, and snack categories. Out of 116 days, 68.1% included breakfast, 71.5% included lunch, 82.8% included dinner, and 86.2% included at least one snack. Results The most prevalent eating environment among all eating occasions was at home and with one or more screens in use (breakfast: 48.1%, lunch: 42.2%, dinner: 50%, and snacks: 55%), eating alone (breakfast: 75.9%, lunch: 89.2%, dinner: 74.3%, snacks: 74.3%), in the dining room (breakfast: 36.7%, lunch: 30.1%, dinner: 45.8%) or living room (snacks: 28.0%), and in multiple locations (breakfast: 44.3%, lunch: 28.8%, dinner: 44.8%, snacks: 41.3%). Discussion Results suggest a passive capture device can provide accurate detection of food intake in multiple eating environments. To our knowledge, this is the first study to classify eating occasions in multiple eating environments and may be a useful tool for future behavioral research studies to accurately codify eating environments.
Collapse
Affiliation(s)
- Matthew Breit
- Department of Medicine, Division of Endocrinology, Metabolism, and Diabetes, University of Colorado Anschutz Medical Campus, Aurora, CO, United States
| | - Jonathan Padia
- Colorado Clinical and Translational Sciences Institute, University of Colorado Anschutz Medical Campus, Aurora, CO, United States
| | - Tyson Marden
- Colorado Clinical and Translational Sciences Institute, University of Colorado Anschutz Medical Campus, Aurora, CO, United States
| | - Dan Forjan
- Colorado Clinical and Translational Sciences Institute, University of Colorado Anschutz Medical Campus, Aurora, CO, United States
| | - Pan Zhaoxing
- Research Institute Biostatistics Core, Department of Pediatrics, University of Colorado Anschutz Medical Campus, Aurora, CO, United States
| | - Wenru Zhou
- Research Institute Biostatistics Core, Department of Pediatrics, University of Colorado Anschutz Medical Campus, Aurora, CO, United States
| | - Tonmoy Ghosh
- Department of Electrical and Computer Engineering, The University of Alabama, Tuscaloosa, AL, United States
| | - Graham Thomas
- Department of Psychiatry and Human Behavior, Weight Control and Diabetes Research Center, Alpert Medical School of Brown University and The Miriam Hospital, Providence, RI, United States
| | - Megan A. McCrory
- Department of Health Sciences, Boston University, Boston, MA, United States
| | - Edward Sazonov
- Department of Electrical and Computer Engineering, The University of Alabama, Tuscaloosa, AL, United States
| | - Janine Higgins
- Department of Medicine, Division of Endocrinology, Metabolism, and Diabetes, University of Colorado Anschutz Medical Campus, Aurora, CO, United States
| |
Collapse
|
17
|
Elbassuoni S, Ghattas H, El Ati J, Zoughby Y, Semaan A, Akl C, Trabelsi T, Talhouk R, Ben Gharbia H, Shmayssani Z, Mourad A. Capturing children food exposure using wearable cameras and deep learning. PLOS DIGITAL HEALTH 2023; 2:e0000211. [PMID: 36972212 PMCID: PMC10042366 DOI: 10.1371/journal.pdig.0000211] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 10/21/2022] [Accepted: 02/09/2023] [Indexed: 03/29/2023]
Abstract
Children's dietary habits are influenced by complex factors within their home, school and neighborhood environments. Identifying such influencers and assessing their effects is traditionally based on self-reported data which can be prone to recall bias. We developed a culturally acceptable machine-learning-based data-collection system to objectively capture school-children's exposure to food (including food items, food advertisements, and food outlets) in two urban Arab centers: Greater Beirut, in Lebanon, and Greater Tunis, in Tunisia. Our machine-learning-based system consists of 1) a wearable camera that captures continuous footage of children's environment during a typical school day, 2) a machine learning model that automatically identifies images related to food from the collected data and discards any other footage, 3) a second machine learning model that classifies food-related images into images that contain actual food items, images that contain food advertisements, and images that contain food outlets, and 4) a third machine learning model that classifies images that contain food items into two classes, corresponding to whether the food items are being consumed by the child wearing the camera or whether they are consumed by others. This manuscript reports on a user-centered design study to assess the acceptability of using wearable cameras to capture food exposure among school children in Greater Beirut and Greater Tunis. We then describe how we trained our first machine learning model to detect food exposure images using data collected from the Web and utilizing the latest trends in deep learning for computer vision. Next, we describe how we trained our other machine learning models to classify food-related images into their respective categories using a combination of public data and data acquired via crowdsourcing. Finally, we describe how the different components of our system were packed together and deployed in a real-world case study and we report on its performance.
Collapse
Affiliation(s)
- Shady Elbassuoni
- Computer Science Department, American University of Beirut, Beirut, Lebanon
| | - Hala Ghattas
- Center for Research on Population and Health, American University of Beirut, Beirut, Lebanon
- Department of Health Promotion, Education, and Behavior, University of South Carolina, Columbia, South Carolina, United States of America
| | - Jalila El Ati
- SURVEN Research Laboratory, National Institute of Nutrition and Food Technology, Tunis, Tunisia
| | - Yorgo Zoughby
- Computer Science Department, American University of Beirut, Beirut, Lebanon
| | - Aline Semaan
- Department of Public Health, Institute of Tropical Medicine, Antwerp, Belgium
| | - Christelle Akl
- Center for Research on Population and Health, American University of Beirut, Beirut, Lebanon
| | - Tarek Trabelsi
- SURVEN Research Laboratory, National Institute of Nutrition and Food Technology, Tunis, Tunisia
| | - Reem Talhouk
- School of Design, Northumbria University, Newcastle upon Tyne, United Kingdom
| | - Houda Ben Gharbia
- SURVEN Research Laboratory, National Institute of Nutrition and Food Technology, Tunis, Tunisia
| | | | - Aya Mourad
- Computer Science Department, American University of Beirut, Beirut, Lebanon
| | | |
Collapse
|
18
|
Raju VB, Hossain D, Sazonov E. Estimation of Plate and Bowl Dimensions for Food Portion Size Assessment in a Wearable Sensor System. IEEE SENSORS JOURNAL 2023; 23:5391-5400. [PMID: 37799776 PMCID: PMC10552861 DOI: 10.1109/jsen.2023.3235956] [Citation(s) in RCA: 2] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Indexed: 10/07/2023]
Abstract
Automatic food portion size estimation (FPSE) with minimal user burden is a challenging task. Most of the existing FPSE methods use fiducial markers and/or virtual models as dimensional references. An alternative approach is to estimate the dimensions of the eating containers prior to estimating the portion size. In this article, we propose a wearable sensor system (the automatic ingestion monitor integrated with a ranging sensor) and a related method for the estimation of dimensions of plates and bowls. The contributions of this study are: 1) the model eliminates the need for fiducial markers; 2) the camera system [automatic ingestion monitor version 2 (AIM-2)] is not restricted in terms of positioning relative to the food item; 3) our model accounts for radial lens distortion caused due to lens aberrations; 4) a ranging sensor directly gives the distance between the sensor and the eating surface; 5) the model is not restricted to circular plates; and 6) the proposed system implements a passive method that can be used for assessment of container dimensions with minimum user interaction. The error rates (mean ± std. dev) for dimension estimation were 2.01% ± 4.10% for plate widths/diameters, 2.75% ± 38.11% for bowl heights, and 4.58% ± 6.78% for bowl diameters.
Collapse
Affiliation(s)
- Viprav B Raju
- Department of Electrical and Computer Engineering, The University of Alabama, Tuscaloosa, AL 35401 USA
| | - Delwar Hossain
- Department of Electrical and Computer Engineering, The University of Alabama, Tuscaloosa, AL 35401 USA
| | - Edward Sazonov
- Department of Electrical and Computer Engineering, The University of Alabama, Tuscaloosa, AL 35401 USA
| |
Collapse
|
19
|
Abstract
This article provides an up-to-date review of technological advances in 3 key areas related to diet monitoring and precision nutrition. First, we review developments in mobile applications, with a focus on food photography and artificial intelligence to facilitate the process of diet monitoring. Second, we review advances in 2 types of wearable and handheld sensors that can potentially be used to fully automate certain aspects of diet logging: physical sensors to detect moments of dietary intake, and chemical sensors to estimate the composition of diets and meals. Finally, we review new programs that can generate personalized/precision nutrition recommendations based on measurements of gut microbiota and continuous glucose monitors with artificial intelligence. The article concludes with a discussion of potential pitfalls of some of these technologies.
Collapse
Affiliation(s)
- Bobak J. Mortazavi
- Department of Computer Science
and Engineering, Texas A&M University, College Station, TX, USA
| | - Ricardo Gutierrez-Osuna
- Department of Computer Science
and Engineering, Texas A&M University, College Station, TX, USA
- Ricardo Gutierrez-Osuna, Ph.D.,
Department of Computer Science and Engineering, Texas A&M
University, College Station, TX 77843-3112, USA.
| |
Collapse
|
20
|
Thomas DM, Kleinberg S, Brown AW, Crow M, Bastian ND, Reisweber N, Lasater R, Kendall T, Shafto P, Blaine R, Smith S, Ruiz D, Morrell C, Clark N. Machine learning modeling practices to support the principles of AI and ethics in nutrition research. Nutr Diabetes 2022; 12:48. [PMID: 36456550 PMCID: PMC9715415 DOI: 10.1038/s41387-022-00226-y] [Citation(s) in RCA: 6] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 10/22/2022] [Revised: 10/28/2022] [Accepted: 11/15/2022] [Indexed: 12/03/2022] Open
Abstract
BACKGROUND Nutrition research is relying more on artificial intelligence and machine learning models to understand, diagnose, predict, and explain data. While artificial intelligence and machine learning models provide powerful modeling tools, failure to use careful and well-thought-out modeling processes can lead to misleading conclusions and concerns surrounding ethics and bias. METHODS Based on our experience as reviewers and journal editors in nutrition and obesity, we identified the most frequently omitted best practices from statistical modeling and how these same practices extend to machine learning models. We next addressed areas required for implementation of machine learning that are not included in commercial software packages. RESULTS Here, we provide a tutorial on best artificial intelligence and machine learning modeling practices that can reduce potential ethical problems with a checklist and guiding principles to aid nutrition researchers in developing, evaluating, and implementing artificial intelligence and machine learning models in nutrition research. CONCLUSION The quality of AI/ML modeling in nutrition research requires iterative and tailored processes to mitigate against potential ethical problems or to predict conclusions that are free of bias.
Collapse
Affiliation(s)
- Diana M. Thomas
- grid.419884.80000 0001 2287 2270Department of Mathematical Sciences, United States Military Academy, West Point, NY 10996 USA
| | - Samantha Kleinberg
- grid.217309.e0000 0001 2180 0654Department of Computer Science, Stevens Institute of Technology, Hoboken, NJ 07030 USA
| | - Andrew W. Brown
- grid.241054.60000 0004 4687 1637Department of Biostatistics, University of Arkansas for Medical Sciences, Little Rock, AR 72205 USA ,grid.488749.eArkansas Children’s Research Institute, Little Rock, AR 72202 USA
| | - Mason Crow
- grid.419884.80000 0001 2287 2270Department of Mathematical Sciences, United States Military Academy, West Point, NY 10996 USA
| | - Nathaniel D. Bastian
- grid.419884.80000 0001 2287 2270Army Cyber Institute, United States Military Academy, West Point, NY 10996 USA
| | - Nicholas Reisweber
- grid.419884.80000 0001 2287 2270Department of Mathematical Sciences, United States Military Academy, West Point, NY 10996 USA
| | - Robert Lasater
- grid.419884.80000 0001 2287 2270Department of Mathematical Sciences, United States Military Academy, West Point, NY 10996 USA
| | - Thomas Kendall
- grid.419884.80000 0001 2287 2270Department of Mathematical Sciences, United States Military Academy, West Point, NY 10996 USA
| | - Patrick Shafto
- grid.430387.b0000 0004 1936 8796Department of Mathematics and Computer Science, Rutgers University, Newark, NJ 07102 USA
| | - Raymond Blaine
- grid.419884.80000 0001 2287 2270Department of Electrical Engineering and Computer Science, United States Military Academy, West Point, NY 10996 USA
| | - Sarah Smith
- grid.419884.80000 0001 2287 2270Department of Electrical Engineering and Computer Science, United States Military Academy, West Point, NY 10996 USA
| | - Daniel Ruiz
- grid.419884.80000 0001 2287 2270Department of Electrical Engineering and Computer Science, United States Military Academy, West Point, NY 10996 USA
| | - Christopher Morrell
- grid.419884.80000 0001 2287 2270Department of Electrical Engineering and Computer Science, United States Military Academy, West Point, NY 10996 USA
| | - Nicholas Clark
- grid.419884.80000 0001 2287 2270Department of Mathematical Sciences, United States Military Academy, West Point, NY 10996 USA
| |
Collapse
|
21
|
Neves PA, Simões J, Costa R, Pimenta L, Gonçalves NJ, Albuquerque C, Cunha C, Zdravevski E, Lameski P, Garcia NM, Pires IM. Thought on Food: A Systematic Review of Current Approaches and Challenges for Food Intake Detection. SENSORS (BASEL, SWITZERLAND) 2022; 22:6443. [PMID: 36080901 PMCID: PMC9460522 DOI: 10.3390/s22176443] [Citation(s) in RCA: 4] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 07/12/2022] [Revised: 08/14/2022] [Accepted: 08/24/2022] [Indexed: 06/15/2023]
Abstract
Nowadays, individuals have very stressful lifestyles, affecting their nutritional habits. In the early stages of life, teenagers begin to exhibit bad habits and inadequate nutrition. Likewise, other people with dementia, Alzheimer's disease, or other conditions may not take food or medicine regularly. Therefore, the ability to monitor could be beneficial for them and for the doctors that can analyze the patterns of eating habits and their correlation with overall health. Many sensors help accurately detect food intake episodes, including electrogastrography, cameras, microphones, and inertial sensors. Accurate detection may provide better control to enable healthy nutrition habits. This paper presents a systematic review of the use of technology for food intake detection, focusing on the different sensors and methodologies used. The search was performed with a Natural Language Processing (NLP) framework that helps screen irrelevant studies while following the PRISMA methodology. It automatically searched and filtered the research studies in different databases, including PubMed, Springer, ACM, IEEE Xplore, MDPI, and Elsevier. Then, the manual analysis selected 30 papers based on the results of the framework for further analysis, which support the interest in using sensors for food intake detection and nutrition assessment. The mainly used sensors are cameras, inertial, and acoustic sensors that handle the recognition of food intake episodes with artificial intelligence techniques. This research identifies the most used sensors and data processing methodologies to detect food intake.
Collapse
Affiliation(s)
- Paulo Alexandre Neves
- School of Technology, Polytechnic Institute of Castelo Branco, 6000-767 Castelo Branco, Portugal
| | - João Simões
- Escola de Ciências e Tecnologia, University of Trás-os-Montes e Alto Douro, Quinta de Prados, 5001-801 Vila Real, Portugal
| | - Ricardo Costa
- Escola de Ciências e Tecnologia, University of Trás-os-Montes e Alto Douro, Quinta de Prados, 5001-801 Vila Real, Portugal
| | - Luís Pimenta
- Escola de Ciências e Tecnologia, University of Trás-os-Montes e Alto Douro, Quinta de Prados, 5001-801 Vila Real, Portugal
| | - Norberto Jorge Gonçalves
- Escola de Ciências e Tecnologia, University of Trás-os-Montes e Alto Douro, Quinta de Prados, 5001-801 Vila Real, Portugal
| | - Carlos Albuquerque
- Health Sciences Research Unit: Nursing (UICISA: E), Nursing School of Coimbra (ESEnfC), 3046-851 Coimbra, Portugal
- Higher School of Health, Polytechnic Institute of Viseu, 3504-510 Viseu, Portugal
- Child Studies Research Center (CIEC), University of Minho, 4710-057 Braga, Portugal
| | - Carlos Cunha
- CISeD—Research Centre in Digital Services, Polytechnic Institute of Viseu, 3504-510 Viseu, Portugal
| | - Eftim Zdravevski
- Faculty of Computer Science and Engineering, University Ss Cyril and Methodius, 1000 Skopje, North Macedonia
| | - Petre Lameski
- Faculty of Computer Science and Engineering, University Ss Cyril and Methodius, 1000 Skopje, North Macedonia
| | - Nuno M. Garcia
- Instituto de Telecomunicações, Universidade da Beira Interior, 6200-001 Covilhã, Portugal
| | - Ivan Miguel Pires
- Instituto de Telecomunicações, Universidade da Beira Interior, 6200-001 Covilhã, Portugal
| |
Collapse
|
22
|
Li X, Holiday S, Cribbet M, Bharadwaj A, White S, Sazonov E, Gan Y. Non-Invasive Screen Exposure Time Assessment Using Wearable Sensor and Object Detection. ANNUAL INTERNATIONAL CONFERENCE OF THE IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. ANNUAL INTERNATIONAL CONFERENCE 2022; 2022:4917-4920. [PMID: 36086530 DOI: 10.1109/embc48229.2022.9871903] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/15/2023]
Abstract
Cumulative screen exposure has been increased due to the explosion of digital technology ownership in the past decade for all people, including children who face exposure related risks such as obesity, eye problems, and disrupted sleep. Screen exposure is linked to physical and mental health risks among both children and adults. Current methods of screen exposure assessment have their limitations, mostly in the prospective of objectiveness, robustness, and invasiveness. In this paper, we propose a novel method to measure screen exposure time using a wearable sensor and computer vision technology. We use a customized, lightweight, wearable senor to capture egocentric images and use deep learning-based object detection module to identify the existence of electronic screens. The duration of screen exposure is further estimated using post-processing technology to filter consecutive frames regarding to the screen usage. Our method is non-invasive and robust, providing an objective and accurate means to screen exposure measurement. We conduct experiments on various environments to identify the existence of three types of screens and duration of screen exposure. The experimental results demonstrate the feasibility of automatically assessing screen time exposure and great potential to be applied in large scale experiments for behavioral study.
Collapse
|
23
|
Rantala E, Balatsas-Lekkas A, Sozer N, Pennanen K. Overview of objective measurement technologies for nutrition research, food-related consumer and marketing research. Trends Food Sci Technol 2022. [DOI: 10.1016/j.tifs.2022.05.006] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/24/2022]
|
24
|
Ghosh T, Sazonov E. A Comparative Study of Deep Learning Algorithms for Detecting Food Intake. ANNUAL INTERNATIONAL CONFERENCE OF THE IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. ANNUAL INTERNATIONAL CONFERENCE 2022; 2022:2993-2996. [PMID: 36085821 DOI: 10.1109/embc48229.2022.9871278] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/15/2023]
Abstract
The choice of appropriate machine learning algorithms is crucial for classification problems. This study compares the performance of state-of-the-art time-series deep learning algorithms for classifying food intake using sensor signals. The sensor signals were collected with the help of a wearable sensor system (the Automatic Ingestion Monitor v2, or AIM-2). AIM-2 used an optical and 3-axis accelerometer sensor to capture temporalis muscle activation. Raw signals from those sensors were used to train five classifiers (multilayer perceptron (MLP), time Convolutional Neural Network (time-CNN), Fully Convolutional Neural Network (FCN), Residual Neural Network (ResNet), and Inception network) to differentiate food intake (eating and drinking) from other activities. Data were collected from 17 pilot subjects over the course of 23 days in free-living conditions. A leave one subject out cross-validation scheme was used for training and testing. Time-CNN, FCN, ResNet, and Inception achieved average balanced classification accuracy of 88.84%, 90.18%, 93.47%, and 92.15%, respectively. The results indicate that ResNet outperforms other state-of-the-art deep learning algorithms for this specific problem.
Collapse
|
25
|
Pan Z, Forjan D, Marden T, Padia J, Ghosh T, Hossain D, Thomas JG, McCrory MA, Sazonov E, Higgins JA. Improvement of Methodology for Manual Energy Intake Estimation From Passive Capture Devices. Front Nutr 2022; 9:877775. [PMID: 35811954 PMCID: PMC9257202 DOI: 10.3389/fnut.2022.877775] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/17/2022] [Accepted: 05/09/2022] [Indexed: 11/25/2022] Open
Abstract
Objective To describe best practices for manual nutritional analyses of data from passive capture wearable devices in free-living conditions. Method 18 participants (10 female) with a mean age of 45 ± 10 years and mean BMI of 34.2 ± 4.6 kg/m2 consumed usual diet for 3 days in a free-living environment while wearing an automated passive capture device. This wearable device facilitates capture of images without manual input from the user. Data from the first nine participants were used by two trained nutritionists to identify sources contributing to inter-nutritionist variance in nutritional analyses. The nutritionists implemented best practices to mitigate these sources of variance in the next nine participants. The three best practices to reduce variance in analysis of energy intake (EI) estimation were: (1) a priori standardized food selection, (2) standardized nutrient database selection, and (3) increased number of images captured around eating episodes. Results Inter-rater repeatability for EI, using intraclass correlation coefficient (ICC), improved by 0.39 from pre-best practices to post-best practices (0.14 vs 0.85, 95% CI, respectively), Bland–Altman analysis indicated strongly improved agreement between nutritionists for limits of agreement (LOA) post-best practices. Conclusion Significant improvement of ICC and LOA for estimation of EI following implementation of best practices demonstrates that these practices improve the reproducibility of dietary analysis from passive capture device images in free-living environments.
Collapse
Affiliation(s)
- Zhaoxing Pan
- Department of Pediatrics, University of Colorado Anschutz Medical Campus, Aurora, CO, United States
- Department of Biostatistics and Informatics, Colorado School of Public Health, University of Colorado Anschutz Medical Campus, Aurora, CO, United States
| | - Dan Forjan
- Colorado Clinical and Translational Sciences Institute, University of Colorado Anschutz Medical Campus, Aurora, CO, United States
- *Correspondence: Dan Forjan,
| | - Tyson Marden
- Colorado Clinical and Translational Sciences Institute, University of Colorado Anschutz Medical Campus, Aurora, CO, United States
| | - Jonathan Padia
- Colorado Clinical and Translational Sciences Institute, University of Colorado Anschutz Medical Campus, Aurora, CO, United States
| | - Tonmoy Ghosh
- Department of Electrical and Computer Engineering (ECE), The University of Alabama, Tuscaloosa, AL, United States
| | - Delwar Hossain
- Department of Electrical and Computer Engineering (ECE), The University of Alabama, Tuscaloosa, AL, United States
| | - J. Graham Thomas
- Department of Psychiatry and Human Behavior, Alpert Medical School of Brown University, Providence, RI, United States
| | - Megan A. McCrory
- Department of Health Sciences, Boston University, Boston, MA, United States
| | - Edward Sazonov
- Department of Electrical and Computer Engineering (ECE), The University of Alabama, Tuscaloosa, AL, United States
| | - Janine A. Higgins
- Department of Pediatrics, University of Colorado Anschutz Medical Campus, Aurora, CO, United States
- Janine A. Higgins,
| |
Collapse
|
26
|
Wang L, Allman-Farinelli M, Yang JA, Taylor JC, Gemming L, Hekler E, Rangan A. Enhancing Nutrition Care Through Real-Time, Sensor-Based Capture of Eating Occasions: A Scoping Review. Front Nutr 2022; 9:852984. [PMID: 35586732 PMCID: PMC9108538 DOI: 10.3389/fnut.2022.852984] [Citation(s) in RCA: 5] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/11/2022] [Accepted: 04/11/2022] [Indexed: 11/24/2022] Open
Abstract
As food intake patterns become less structured, different methods of dietary assessment may be required to capture frequently omitted snacks, smaller meals, and the time of day when they are consumed. Incorporating sensors that passively and objectively detect eating behavior may assist in capturing these eating occasions into dietary assessment methods. The aim of this study was to identify and collate sensor-based technologies that are feasible for dietitians to use to assist with performing dietary assessments in real-world practice settings. A scoping review was conducted using the PRISMA extension for scoping reviews (PRISMA-ScR) framework. Studies were included if they were published between January 2016 and December 2021 and evaluated the performance of sensor-based devices for identifying and recording the time of food intake. Devices from included studies were further evaluated against a set of feasibility criteria to determine whether they could potentially be used to assist dietitians in conducting dietary assessments. The feasibility criteria were, in brief, consisting of an accuracy ≥80%; tested in settings where subjects were free to choose their own foods and activities; social acceptability and comfort; a long battery life; and a relatively rapid detection of an eating episode. Fifty-four studies describing 53 unique devices and 4 device combinations worn on the wrist (n = 18), head (n = 16), neck (n = 9), and other locations (n = 14) were included. Whilst none of the devices strictly met all feasibility criteria currently, continuous refinement and testing of device software and hardware are likely given the rapidly changing nature of this emerging field. The main reasons devices failed to meet the feasibility criteria were: an insufficient or lack of reporting on battery life (91%), the use of a limited number of foods and behaviors to evaluate device performance (63%), and the device being socially unacceptable or uncomfortable to wear for long durations (46%). Until sensor-based dietary assessment tools have been designed into more inconspicuous prototypes and are able to detect most food and beverage consumption throughout the day, their use will not be feasible for dietitians in practice settings.
Collapse
Affiliation(s)
- Leanne Wang
- Charles Perkins Centre, Faculty of Medicine and Health, University of Sydney, Sydney, NSW, Australia
| | - Margaret Allman-Farinelli
- Charles Perkins Centre, Faculty of Medicine and Health, University of Sydney, Sydney, NSW, Australia
| | - Jiue-An Yang
- Department of Population Sciences, Beckman Research Institute, City of Hope, Duarte, CA, United States
| | - Jennifer C. Taylor
- The Design Lab, University of California, San Diego, San Diego, CA, United States
- Herbert Wertheim School of Public Health and Human Longevity Science, University of California, San Diego, San Diego, CA, United States
| | - Luke Gemming
- Charles Perkins Centre, Faculty of Medicine and Health, University of Sydney, Sydney, NSW, Australia
| | - Eric Hekler
- The Design Lab, University of California, San Diego, San Diego, CA, United States
- Herbert Wertheim School of Public Health and Human Longevity Science, University of California, San Diego, San Diego, CA, United States
| | - Anna Rangan
- Charles Perkins Centre, Faculty of Medicine and Health, University of Sydney, Sydney, NSW, Australia
- *Correspondence: Anna Rangan
| |
Collapse
|
27
|
Sharma S, Hoover A. Top-Down Detection of Eating Episodes by Analyzing Large Windows of Wrist Motion Using a Convolutional Neural Network. Bioengineering (Basel) 2022; 9:bioengineering9020070. [PMID: 35200423 PMCID: PMC8869422 DOI: 10.3390/bioengineering9020070] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/15/2021] [Revised: 01/19/2022] [Accepted: 02/07/2022] [Indexed: 11/16/2022] Open
Abstract
In this work, we describe a new method to detect periods of eating by tracking wrist motion during everyday life. Eating uses hand-to-mouth gestures for ingestion, each of which lasts a few seconds. Previous works have detected these gestures individually and then aggregated them to identify meals. The novelty of our approach is that we analyze a much longer window (0.5–15 min) using a convolutional neural network. Longer windows can contain other gestures related to eating, such as cutting or manipulating food, preparing foods for consumption, and resting between ingestion events. The context of these other gestures can improve the detection of periods of eating. We test our methods on the public Clemson all-day dataset, which consists of 354 recordings containing 1063 eating episodes. We found that accuracy at detecting eating increased by 15% in ≥4 min windows compared to ≤15 s windows. Using a 6 min window, we detected 89% of eating episodes, with 1.7 false positives for every true positive (FP/TP). These are the best results achieved to date on this dataset.
Collapse
|
28
|
Das SK, Miki AJ, Blanchard CM, Sazonov E, Gilhooly CH, Dey S, Wolk CB, Khoo CSH, Hill JO, Shook RP. Perspective: Opportunities and Challenges of Technology Tools in Dietary and Activity Assessment: Bridging Stakeholder Viewpoints. Adv Nutr 2022; 13:1-15. [PMID: 34545392 PMCID: PMC8803491 DOI: 10.1093/advances/nmab103] [Citation(s) in RCA: 5] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/13/2021] [Revised: 08/19/2021] [Accepted: 08/25/2021] [Indexed: 12/23/2022] Open
Abstract
The science and tools of measuring energy intake and output in humans have rapidly advanced in the last decade. Engineered devices such as wearables and sensors, software applications, and Web-based tools are now ubiquitous in both research and consumer environments. The assessment of energy expenditure in particular has progressed from reliance on self-report instruments to advanced technologies requiring collaboration across multiple disciplines, from optics to accelerometry. In contrast, assessing energy intake still heavily relies on self-report mechanisms. Although these tools have improved, moving from paper-based to online reporting, considerable room for refinement remains in existing tools, and great opportunities exist for novel, transformational tools, including those using spectroscopy and chemo-sensing. This report reviews the state of the science, and the opportunities and challenges in existing and emerging technologies, from the perspectives of 3 key stakeholders: researchers, users, and developers. Each stakeholder approaches these tools with unique requirements: researchers are concerned with validity, accuracy, data detail and abundance, and ethical use; users with ease of use and privacy; and developers with high adherence and utilization, intellectual property, licensing rights, and monetization. Cross-cutting concerns include frequent updating and integration of the food and nutrient databases on which assessments rely, improving accessibility and reducing disparities in use, and maintaining reliable technical assistance. These contextual challenges are discussed in terms of opportunities and further steps in the direction of personalized health.
Collapse
Affiliation(s)
- Sai Krupa Das
- Jean Mayer USDA Human Nutrition Research Center on Aging at Tufts University, Boston, MA, USA
- Friedman School of Nutrition Science and Policy, Tufts University, Boston, MA, USA
| | - Akari J Miki
- Jean Mayer USDA Human Nutrition Research Center on Aging at Tufts University, Boston, MA, USA
| | - Caroline M Blanchard
- Jean Mayer USDA Human Nutrition Research Center on Aging at Tufts University, Boston, MA, USA
| | - Edward Sazonov
- Department of Electrical and Computer Engineering, University of Alabama, Tuscaloosa, AL, USA
| | - Cheryl H Gilhooly
- Jean Mayer USDA Human Nutrition Research Center on Aging at Tufts University, Boston, MA, USA
- Friedman School of Nutrition Science and Policy, Tufts University, Boston, MA, USA
| | - Sujit Dey
- Department of Electrical and Computer Engineering, University of California, San Diego, La Jolla, CA, USA
| | - Colton B Wolk
- Jean Mayer USDA Human Nutrition Research Center on Aging at Tufts University, Boston, MA, USA
| | - Chor San H Khoo
- Institute for the Advancement of Food and Nutrition Sciences, Washington, DC, USA
| | - James O Hill
- Department of Nutrition Sciences, School of Health Professions, University of Alabama at Birmingham, Birmingham, AL, USA
- Nutrition Obesity Research Center, University of Alabama at Birmingham, Birmingham, AL, USA
| | - Robin P Shook
- Center for Children's Healthy Lifestyles & Nutrition, Children's Mercy Kansas City, Kansas City, MO, USA
- School of Medicine, University of Missouri-Kansas City, Kansas City, MO, USA
| |
Collapse
|
29
|
Ghosh T, Hossain D, Sazonov E. Detection of Food Intake Sensor's Wear Compliance in Free-Living. IEEE SENSORS JOURNAL 2021; 21:27728-27735. [PMID: 35813985 PMCID: PMC9268495 DOI: 10.1109/jsen.2021.3124203] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/01/2023]
Abstract
Objective detection of periods of wear and non-wear is critical for human studies that rely on information from wearable sensors, such as food intake sensors. In this paper, we present a novel method of compliance detection on the example of the Automatic Ingestion Monitor v2 (AIM-2) sensor, containing a tri-axial accelerometer, a still camera, and a chewing sensor. The method was developed and validated using data from a study of 30 participants aged 18-39, each wearing the AIM-2 for two days (a day in pseudo-free-living and a day in free-living). Four types of wear compliance were analyzed: 'normal-wear', 'non-compliant-wear', 'non-wear-carried', and 'non-wear-stationary'. The ground truth of those four types of compliance was obtained by reviewing the images of the egocentric camera. The features for compliance detection were the standard deviation of acceleration, average pitch, and roll angles, and mean square error of two consecutive images. These were used to train three random forest classifiers 1) accelerometer-based, 2) image-based, and 3) combined accelerometer and image-based. Satisfactory wear compliance measurement accuracy was obtained using the combined classifier (89.24%) on leave one subject out cross-validation. The average duration of compliant wear in the study was 9h with a standard deviation of 2h or 70.96% of total on-time. This method can be used to calculate the wear and non-wear time of AIM-2, and potentially be extended to other devices. The study also included assessments of sensor burden and privacy concerns. The survey results suggest recommendations that may be used to increase wear compliance.
Collapse
Affiliation(s)
- Tonmoy Ghosh
- Department of Electrical and Computer Engineering, The University of Alabama, Tuscaloosa, AL 35401 USA
| | - Delwar Hossain
- Department of Electrical and Computer Engineering, The University of Alabama, Tuscaloosa, AL 35401 USA
| | - Edward Sazonov
- Department of Electrical and Computer Engineering, The University of Alabama, Tuscaloosa, AL 35401 USA
| |
Collapse
|
30
|
Turrini A, Catasta G, Censi L, Comendador Azcarraga FJ, D'Addezio L, Ferrari M, Le Donne C, Martone D, Mistura L, Pettinelli A, Piccinelli R, Saba A, Sette S, Barbina D, Guerrera D, Carbone P, Mazzaccara A. A Dietary Assessment Training Course Path: The Italian IV SCAI Study on Children Food Consumption. Front Public Health 2021; 9:590315. [PMID: 33777877 PMCID: PMC7994926 DOI: 10.3389/fpubh.2021.590315] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/01/2020] [Accepted: 01/15/2021] [Indexed: 11/15/2022] Open
Abstract
The eating patterns in a population can be estimated through dietary surveys in which open-ended assessment methods, such as diaries and interviews, or semi-quantitative food frequency questionnaires are administered. A harmonized dietary survey methodology, together with a standardized operational procedure, in conducting the study is crucial to ensure the comparability of the results and the accuracy of information, thus reducing uncertainty and increasing the reliability of the results. Dietary patterns (i) include several target variables (foods, energy and nutrients, other food components), (ii) require several explanatory variables (age, gender, anthropometric measurements, socio-cultural and economic characteristics, lifestyle, preferences, attitudes, beliefs, organization of food-related activities, etc.), and (iii) have impacts in several domains: imbalance diets; acute and chronic exposures affect health, specifically non-communicable diseases; and then sanitary expenditure. On the other hand, food demand has impacts on the food system: production, distribution, and food services system; food wastes and other wastes generated by food-related activities of the households (e.g., packaging disposal) have consequences on the “health of the planet” which in turn can have effects on human health. Harmonization and standardization of measurement methods and procedures in such a complex context require an ad hoc structured information system made by databases (food nomenclatures, portion sizes, food atlas, recipes) and methodological tools (quantification methods, food coding systems, assessment of nutritional status, data processing to extrapolate what we consider validated dietary data). Establishing a community of professionals specialized in dietary data management could lead to build a surveillance system for monitoring eating habits in the short term, thus reducing costs, and to arrange a training re-training system. Creating and maintaining the dietary data managers community is challenging but possible. In this context, the cooperation between the CREA Research Centre for Food and Nutrition and the Italian National Health Institute (ISS) promoted and supported by the Italian Ministry of Health may represent a model of best practice that can ensure a continuous training for the professional community carrying out a nutritional study.
Collapse
Affiliation(s)
- Aida Turrini
- Council for Agricultural Research and Economics, Research Centre for Food and Nutrition, Rome, Italy
| | - Giovina Catasta
- Council for Agricultural Research and Economics, Research Centre for Food and Nutrition, Rome, Italy
| | - Laura Censi
- Council for Agricultural Research and Economics, Research Centre for Food and Nutrition, Rome, Italy
| | | | - Laura D'Addezio
- Council for Agricultural Research and Economics, Research Centre for Food and Nutrition, Rome, Italy
| | - Marika Ferrari
- Council for Agricultural Research and Economics, Research Centre for Food and Nutrition, Rome, Italy
| | - Cinzia Le Donne
- Council for Agricultural Research and Economics, Research Centre for Food and Nutrition, Rome, Italy
| | - Deborah Martone
- Council for Agricultural Research and Economics, Research Centre for Food and Nutrition, Rome, Italy
| | - Lorenza Mistura
- Council for Agricultural Research and Economics, Research Centre for Food and Nutrition, Rome, Italy
| | - Antonella Pettinelli
- Council for Agricultural Research and Economics, Research Centre for Food and Nutrition, Rome, Italy
| | - Raffaela Piccinelli
- Council for Agricultural Research and Economics, Research Centre for Food and Nutrition, Rome, Italy
| | - Anna Saba
- Council for Agricultural Research and Economics, Research Centre for Food and Nutrition, Rome, Italy
| | - Stefania Sette
- Council for Agricultural Research and Economics, Research Centre for Food and Nutrition, Rome, Italy
| | - Donatella Barbina
- Servizio Formazione - Presidenza, National Institute of Health, Rome, Italy
| | - Debora Guerrera
- Servizio Formazione - Presidenza, National Institute of Health, Rome, Italy
| | - Pietro Carbone
- Servizio Formazione - Presidenza, National Institute of Health, Rome, Italy
| | - Alfonso Mazzaccara
- Servizio Formazione - Presidenza, National Institute of Health, Rome, Italy
| |
Collapse
|