Applying Image-Based Food-Recognition Systems on Dietary Assessment: A Systematic Review

Use of digital food records in clinical settings: lessons in translation from a randomised controlled trial

BACKGROUND

Digital food records offer efficiencies in collecting and assessing dietary information remotely; however, research into factors impacting their translation into clinical settings is limited.

METHODS

The study examined factors that may impact the integration of digital food records into clinical dietetic practice by assessing (1) the source and rate of data errors received, (2) the impact of dietitian-adjusted data on dietary variables and (3) the acceptance of use in a complex chronic condition cohort. Adults from specialist clinics enroled in a randomised controlled feasibility trial participated. Participants recorded their dietary intake using a mobile food diary application (Research Food diary, Xyris Software Pty Ltd.); it was analysed via electronic nutrition analysis software (FoodWorks, Xyris Software Pty Ltd.). Records were verified and corrected by a dietitian. Dietary variables assessed before (participant-unadjusted data) and after (dietitian-adjusted data) were compared by the Wilcoxon signed-rank test, Bland-Altman and Passing-Bablok analysis. Surveys and Interviews assessed participants'; acceptance of the mobile application's usability.

RESULTS

Errors appeared in 93% of records. Dietitian-adjusted median data were higher for most variables compared to participant-unadjusted median data (p < 0.05, median changes between 0.0% and 64.7%) of 59 participant records (median age 51 years, interquartile range 38-58). There was poor agreement between participant-unadjusted and dietitian-adjusted data for some dietary variables. Sixty-four percent surveyed (n = 32/50) found the app easy to use, whereas 29 interviews provided insights into facilitators and challenges of use.

CONCLUSIONS

Significant barriers to integrating digital food records into clinical settings exist requiring dietitian adjustment to correct errors which has major implications for estimates of diet quality and intake.

Perspective: Data in personalized nutrition: Bridging biomedical, psycho-behavioral, and food environment approaches for population-wide impact

Personalized Nutrition (PN) represents an approach aimed at delivering tailored dietary recommendations, products or services to support both prevention and treatment of nutrition-related conditions and improve individual health using genetic, phenotypic, medical, nutritional, and other pertinent information. However, current approaches have yielded limited scientific success in improving diets or in mitigating diet-related conditions. In addition, PN currently caters to a specific subgroup of the population rather than having a widespread impact on diet and health at a population level. Addressing these challenges requires integrating traditional biomedical and dietary assessment methods with psycho-behavioral, and novel digital and diagnostic methods for comprehensive data collection, which holds considerable promise in alleviating present PN shortcomings. This comprehensive approach not only allows for deriving personalized goals ("what should be achieved") but also customizing behavioral change processes ("how to bring about change"). We herein outline and discuss the concept of "Adaptive Personalized Nutrition Advice Systems" (APNASs), which blends data from three assessment domains: 1) biomedical/health phenotyping; 2) stable and dynamic behavioral signatures; and 3) food environment data. Personalized goals and behavior change processes are envisaged to no longer be based solely on static data but will adapt dynamically in-time and in-situ based on individual-specific data. To successfully integrate biomedical, behavioral and environmental data for personalized dietary guidance, advanced digital tools (e.g., sensors) and artificial intelligence (AI)-based methods will be essential. In conclusion, the integration of both established and novel static and dynamic assessment paradigms holds great potential for transitioning PN from its current focus on elite nutrition to a widely accessible tool that delivers meaningful health benefits to the general population.

Implementation of Nutrition Labels at the 2022 European Athletics Championships: An Observational Study of the Use and Perceptions of Athletes and Athlete Support Personnel

BACKGROUND/OBJECTIVES

Nutrition labels are an effective tool for providing nutrition information. Additionally, nutrient composition is one of the most dictating factors for athletes' food choices; thus, we aimed to evaluate the use and perceptions regarding the nutrition labels implemented for the meals served at the 2022 European Athletics Championships (EAC).

METHODS

During mealtime at the team restaurants, participants completed an online self-administered questionnaire. We collected 280 questionnaires, 53.8% of the participants were male, most were athletes (78.9%), and 21.1% were athlete support personnel. Likert-type scales and open-ended questions were included to measure the labels' importance, layout, influence on food choices, and participants' understanding of the labels. Mann-Whitney and Kruskal-Wallis tests were used to compare answers.

RESULTS

Almost 40% of the participants used the nutrition labels occasionally (38.8%). Most participants were confident (41.9%) or moderately confident (31.3%) in making food choices because they had labels. Nutrition labels were considered important (41.0%) or very important (28.4%) by most participants, and 91.7% would like to have them in future championships. Athlete support personnel versus athletes (p = 0.037) and participants with dietary restrictions versus participants without (p = 0.028) were more confident in their food choices due to nutrition labels.

CONCLUSIONS

Our results highlight that nutrition labels were helpful for both athletes and athlete support personnel during this EAC and that they should be maintained in future competitions.

Clinical research framework proposal for ketogenic metabolic therapy in glioblastoma

Glioblastoma (GBM) is the most aggressive primary brain tumor in adults, with a universally lethal prognosis despite maximal standard therapies. Here, we present a consensus treatment protocol based on the metabolic requirements of GBM cells for the two major fermentable fuels: glucose and glutamine. Glucose is a source of carbon and ATP synthesis for tumor growth through glycolysis, while glutamine provides nitrogen, carbon, and ATP synthesis through glutaminolysis. As no tumor can grow without anabolic substrates or energy, the simultaneous targeting of glycolysis and glutaminolysis is expected to reduce the proliferation of most if not all GBM cells. Ketogenic metabolic therapy (KMT) leverages diet-drug combinations that inhibit glycolysis, glutaminolysis, and growth signaling while shifting energy metabolism to therapeutic ketosis. The glucose-ketone index (GKI) is a standardized biomarker for assessing biological compliance, ideally via real-time monitoring. KMT aims to increase substrate competition and normalize the tumor microenvironment through GKI-adjusted ketogenic diets, calorie restriction, and fasting, while also targeting glycolytic and glutaminolytic flux using specific metabolic inhibitors. Non-fermentable fuels, such as ketone bodies, fatty acids, or lactate, are comparatively less efficient in supporting the long-term bioenergetic and biosynthetic demands of cancer cell proliferation. The proposed strategy may be implemented as a synergistic metabolic priming baseline in GBM as well as other tumors driven by glycolysis and glutaminolysis, regardless of their residual mitochondrial function. Suggested best practices are provided to guide future KMT research in metabolic oncology, offering a shared, evidence-driven framework for observational and interventional studies.

Wearables and Smartphones for Tracking Modifiable Risk Factors in Metabolic Health: Protocol for a Scoping Review

BACKGROUND

Metabolic diseases, such as cardiovascular diseases and diabetes, contribute significantly to global mortality and disability. Wearable devices and smartphones are increasingly used to track and manage modifiable risk factors associated with metabolic diseases. However, no established guidelines exist on how to derive meaningful signals from these devices, often hampering cross-study comparisons.

OBJECTIVE

This study aims to systematically overview the current empirical literature on how wearables and smartphones are used to track modifiable (physiological and lifestyle) risk factors associated with metabolic diseases.

METHODS

We will conduct a scoping review to overview how wearable and smartphone-based studies measure modifiable risk factors related to metabolic diseases. We will search 5 databases (Scopus, Web of Science, PubMed, Cochrane Central Register of Controlled Trials, and SPORTDiscus) from 2019 to 2024, with search terms related to wearables, smartphones, and modifiable risk factors associated with metabolic diseases. Eligible studies will use smartphones or wearables (worn on the wrist, finger, arm, hip, and chest) to track physiological or lifestyle factors related to metabolic diseases. We will follow the reporting guideline standards from PRISMA-ScR (Preferred Reporting Items for Systematic Reviews and Meta-Analyses Extension for Scoping Reviews) and the JBI (Joanna Briggs Institute) guidance on scoping review methodology. Two reviewers will independently screen articles for inclusion and extract data using a standardized form. The findings will be synthesized and reported qualitatively and quantitatively.

RESULTS

Data collection is expected to begin in November 2024; data analysis in the first quarter of 2025; and submission to a peer-reviewed journal by the second quarter of 2025. We expect to identify the degree to which wearable and smartphone-based studies track modifiable risk factors collectively (versus in isolation), and the consistency and variation in how modifiable risk factors are measured across existing studies.

CONCLUSIONS

Results are expected to inform more standardized guidelines on wearable and smartphone-based measurements, with the goal of aiding cross-study comparison. The final report is planned for submission to a peer-reviewed, indexed journal. This review is among the first to systematically overview the current landscape on how wearables and smartphones measure modifiable risk factors associated with metabolic diseases.

INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID)

PRR1-10.2196/59539.

Advancements in Using AI for Dietary Assessment Based on Food Images: Scoping Review

BACKGROUND

To accurately capture an individual's food intake, dietitians are often required to ask clients about their food frequencies and portions, and they have to rely on the client's memory, which can be burdensome. While taking food photos alongside food records can alleviate user burden and reduce errors in self-reporting, this method still requires trained staff to translate food photos into dietary intake data. Image-assisted dietary assessment (IADA) is an innovative approach that uses computer algorithms to mimic human performance in estimating dietary information from food images. This field has seen continuous improvement through advancements in computer science, particularly in artificial intelligence (AI). However, the technical nature of this field can make it challenging for those without a technical background to understand it completely.

OBJECTIVE

This review aims to fill the gap by providing a current overview of AI's integration into dietary assessment using food images. The content is organized chronologically and presented in an accessible manner for those unfamiliar with AI terminology. In addition, we discuss the systems' strengths and weaknesses and propose enhancements to improve IADA's accuracy and adoption in the nutrition community.

METHODS

This scoping review used PubMed and Google Scholar databases to identify relevant studies. The review focused on computational techniques used in IADA, specifically AI models, devices, and sensors, or digital methods for food recognition and food volume estimation published between 2008 and 2021.

RESULTS

A total of 522 articles were initially identified. On the basis of a rigorous selection process, 84 (16.1%) articles were ultimately included in this review. The selected articles reveal that early systems, developed before 2015, relied on handcrafted machine learning algorithms to manage traditional sequential processes, such as segmentation, food identification, portion estimation, and nutrient calculations. Since 2015, these handcrafted algorithms have been largely replaced by deep learning algorithms for handling the same tasks. More recently, the traditional sequential process has been superseded by advanced algorithms, including multitask convolutional neural networks and generative adversarial networks. Most of the systems were validated for macronutrient and energy estimation, while only a few were capable of estimating micronutrients, such as sodium. Notably, significant advancements have been made in the field of IADA, with efforts focused on replicating humanlike performance.

CONCLUSIONS

This review highlights the progress made by IADA, particularly in the areas of food identification and portion estimation. Advancements in AI techniques have shown great potential to improve the accuracy and efficiency of this field. However, it is crucial to involve dietitians and nutritionists in the development of these systems to ensure they meet the requirements and trust of professionals in the field.

Dietary Patterns of Healthy Underweight Individuals Compared to Normal-BMI Individuals Using Photographic Food Diaries

BACKGROUND

Previously, we found that healthy underweight (HU) subjects, with BMI < 18.5, eat about 12% less food (by calories) each day. It is presently unclear whether this lower intake is associated with them making food choices that provide high satiation and satiety.

METHODS

Using 7-day photographic records of food intake, we analyzed 52 HU and 50 normal-weight participants.

RESULTS

We included 52 HU and 50 normal-weight participants in the final analysis. HU individuals ate 25% fewer calories than normal-weight individuals. Their intake included a higher % of rice (p = 0.0013) and vegetables (p = 0.0006) and a lower % of livestock meat (p = 0.0007), poultry meat (p < 0.0001), and starchy roots (p = 0.0015), compared with the normal-weight population. The percent energy from carbohydrates was significantly higher (p = 0.0234), and the % energy from fat was significantly lower (p < 0.0001) in the HU group, with no difference in the % energy from protein. HU individuals sourced more of their protein from plants. Dietary patterns were grouped into three clusters, with 24 individuals grouped into cluster 1 (87.5% normal-weight population), 28 individuals into cluster 2 (64.3% normal-weight group), and 50 individuals into cluster 3 (78% HU group).

CONCLUSIONS

The HU group ate less overall and had proportionally more rice and vegetables and less poultry and livestock meat, starchy roots, and drinks. With respect to macronutrients, they also ate a greater % carbohydrates and less % fat, and they sourced more of their protein intake from plant sources. HU individuals did not follow a low-carbohydrate lifestyle.

Nutritional composition analysis in food images: an innovative Swin Transformer approach

Accurate recognition of nutritional components in food is crucial for dietary management and health monitoring. Current methods often rely on traditional chemical analysis techniques, which are time-consuming, require destructive sampling, and are not suitable for large-scale or real-time applications. Therefore, there is a pressing need for efficient, non-destructive, and accurate methods to identify and quantify nutrients in food. In this study, we propose a novel deep learning model that integrates EfficientNet, Swin Transformer, and Feature Pyramid Network (FPN) to enhance the accuracy and efficiency of food nutrient recognition. Our model combines the strengths of EfficientNet for feature extraction, Swin Transformer for capturing long-range dependencies, and FPN for multi-scale feature fusion. Experimental results demonstrate that our model significantly outperforms existing methods. On the Nutrition5k dataset, it achieves a Top-1 accuracy of 79.50% and a Mean Absolute Percentage Error (MAPE) for calorie prediction of 14.72%. On the ChinaMartFood109 dataset, the model achieves a Top-1 accuracy of 80.25% and a calorie MAPE of 15.21%. These results highlight the model's robustness and adaptability across diverse food images, providing a reliable and efficient tool for rapid, non-destructive nutrient detection. This advancement supports better dietary management and enhances the understanding of food nutrition, potentially leading to more effective health monitoring applications.

Novel digital-based approach for evaluating wine components' intake: A deep learning model to determine red wine volume in a glass from single-view images

Estimation of wine components' intake (polyphenols, alcohol, etc.) through Food Frequency Questionnaires (FFQs) may be particularly inaccurate. This paper reports the development of a deep learning (DL) method to determine red wine volume from single-view images, along with its application in a consumer study developed via a web service. The DL model demonstrated satisfactory performance not only in a daily lifelike images dataset (mean absolute error = 10 mL), but also in a real images dataset that was generated through the consumer study (mean absolute error = 26 mL). Based on the data reported by the participants in the consumer study (n = 38), average red wine volume in a glass was 114 ± 33 mL, which represents an intake of 137-342 mg of total polyphenols, 11.2 g of alcohol, 0.342 g of sugars, among other components. Therefore, the proposed method constitutes a diet-monitoring tool of substantial utility in the accurate assessment of wine components' intake.

Carbohydrate Estimation Accuracy of Two Commercially Available Smartphone Applications vs Estimation by Individuals With Type 1 Diabetes: A Comparative Study

BACKGROUND

Despite remarkable progress in diabetes technology, most systems still require estimating meal carbohydrate (CHO) content for meal-time insulin delivery. Emerging smartphone applications may obviate this need, but performance data in relation to patient estimates remain scarce.

OBJECTIVE

The objective is to assess the accuracy of two commercial CHO estimation applications, SNAQ and Calorie Mama, and compare their performance with the estimation accuracy of people with type 1 diabetes (T1D).

METHODS

Carbohydrate estimates of 53 individuals with T1D (aged ≥16 years) were compared with those of SNAQ (food recognition + quantification) and Calorie Mama (food recognition + adjustable standard portion size). Twenty-six cooked meals were prepared at the hospital kitchen. Each participant estimated the CHO content of two meals in three different sizes without assistance. Participants then used SNAQ for CHO quantification in one meal and Calorie Mama for the other (all three sizes). Accuracy was the estimate's deviation from ground-truth CHO content (weight multiplied by nutritional facts from recipe database). Furthermore, the applications were rated using the Mars-G questionnaire.

RESULTS

Participants' mean ± standard deviation (SD) absolute error was 21 ± 21.5 g (71 ± 72.7%). Calorie Mama had a mean absolute error of 24 ± 36.5 g (81.2 ± 123.4%). With a mean absolute error of 13.1 ± 11.3 g (44.3 ± 38.2%), SNAQ outperformed the estimation accuracy of patients and Calorie Mama (both P > .05). Error consistency (quantified by the within-participant SD) did not significantly differ between the methods.

CONCLUSIONS

SNAQ may provide effective CHO estimation support for people with T1D, particularly those with large or inconsistent CHO estimation errors. Its impact on glucose control remains to be evaluated.

Empowering Diabetics: Advancements in Smartphone-Based Food Classification, Volume Measurement, and Nutritional Estimation

Diabetes has emerged as a worldwide health crisis, affecting approximately 537 million adults. Maintaining blood glucose requires careful observation of diet, physical activity, and adherence to medications if necessary. Diet monitoring historically involves keeping food diaries; however, this process can be labor-intensive, and recollection of food items may introduce errors. Automated technologies such as food image recognition systems (FIRS) can make use of computer vision and mobile cameras to reduce the burden of keeping diaries and improve diet tracking. These tools provide various levels of diet analysis, and some offer further suggestions for improving the nutritional quality of meals. The current study is a systematic review of mobile computer vision-based approaches for food classification, volume estimation, and nutrient estimation. Relevant articles published over the last two decades are evaluated, and both future directions and issues related to FIRS are explored.

Dietary Intake Assessment Using a Novel, Generic Meal-Based Recall and a 24-Hour Recall: Comparison Study

BACKGROUND

Dietary intake assessment is an integral part of addressing suboptimal dietary intakes. Existing food-based methods are time-consuming and burdensome for users to report the individual foods consumed at each meal. However, ease of use is the most important feature for individuals choosing a nutrition or diet app. Intakes of whole meals can be reported in a manner that is less burdensome than reporting individual foods. No study has developed a method of dietary intake assessment where individuals report their dietary intakes as whole meals rather than individual foods.

OBJECTIVE

This study aims to develop a novel, meal-based method of dietary intake assessment and test its ability to estimate nutrient intakes compared with that of a web-based, 24-hour recall (24HR).

METHODS

Participants completed a web-based, generic meal-based recall. This involved, for each meal type (breakfast, light meal, main meal, snack, and beverage), choosing from a selection of meal images those that most represented their intakes during the previous day. Meal images were based on generic meals from a previous study that were representative of the actual meal intakes in Ireland. Participants also completed a web-based 24HR. Both methods were completed on the same day, 3 hours apart. In a crossover design, participants were randomized in terms of which method they completed first. Then, 2 weeks after the first dietary assessments, participants repeated the process in the reverse order. Estimates of mean daily nutrient intakes and the categorization of individuals according to nutrient-based guidelines (eg, low, adequate, and high) were compared between the 2 methods. P values of less than .05 were considered statistically significant.

RESULTS

In total, 161 participants completed the study. For the 23 nutrient variables compared, the median percentage difference between the 2 methods was 7.6% (IQR 2.6%-13.2%), with P values ranging from <.001 to .97, and out of 23 variables, effect sizes for the differences were small for 19 (83%) variables, moderate for 2 (9%) variables, and large for 2 (9%) variables. Correlation coefficients were statistically significant (P<.05) for 18 (78%) of the 23 variables. Statistically significant correlations ranged from 0.16 to 0.45, with median correlation of 0.32 (IQR 0.25-0.40). When participants were classified according to nutrient-based guidelines, the proportion of individuals who were classified into the same category ranged from 52.8% (85/161) to 84.5% (136/161).

CONCLUSIONS

A generic meal-based method of dietary intake assessment provides estimates of nutrient intake comparable with those provided by a web-based 24HR but with varying levels of agreement among nutrients. Further studies are required to refine and improve the generic recall across a range of nutrients. Future studies will consider user experience including the potential feasibility of incorporating image recognition of whole meals into the generic recall.

AI-based digital image dietary assessment methods compared to humans and ground truth: a systematic review

OBJECTIVE

Human error estimating food intake is a major source of bias in nutrition research. Artificial intelligence (AI) methods may reduce bias, but the overall accuracy of AI estimates is unknown. This study was a systematic review of peer-reviewed journal articles comparing fully automated AI-based (e.g. deep learning) methods of dietary assessment from digital images to human assessors and ground truth (e.g. doubly labelled water).

MATERIALS AND METHODS

Literature was searched through May 2023 in four electronic databases plus reference mining. Eligible articles reported AI estimated volume, energy, or nutrients. Independent investigators screened articles and extracted data. Potential sources of bias were documented in absence of an applicable risk of bias assessment tool.

RESULTS

Database and hand searches identified 14,059 unique publications; fifty-two papers (studies) published from 2010 to 2023 were retained. For food detection and classification, 79% of papers used a convolutional neural network. Common ground truth sources were calculation using nutrient tables (51%) and weighed food (27%). Included papers varied widely in food image databases and results reported, so meta-analytic synthesis could not be conducted. Relative errors were extracted or calculated from 69% of papers. Average overall relative errors (AI vs. ground truth) ranged from 0.10% to 38.3% for calories and 0.09% to 33% for volume, suggesting similar performance. Ranges of relative error were lower when images had single/simple foods.

CONCLUSIONS

Relative errors for volume and calorie estimations suggest that AI methods align with - and have the potential to exceed - accuracy of human estimations. However, variability in food image databases and results reported prevented meta-analytic synthesis. The field can advance by testing AI architectures on a limited number of large-scale food image and nutrition databases that the field determines to be adequate for training and testing and by reporting accuracy of at least absolute and relative error for volume or calorie estimations.

Vision-Based Methods for Food and Fluid Intake Monitoring: A Literature Review

Food and fluid intake monitoring are essential for reducing the risk of dehydration, malnutrition, and obesity. The existing research has been preponderantly focused on dietary monitoring, while fluid intake monitoring, on the other hand, is often neglected. Food and fluid intake monitoring can be based on wearable sensors, environmental sensors, smart containers, and the collaborative use of multiple sensors. Vision-based intake monitoring methods have been widely exploited with the development of visual devices and computer vision algorithms. Vision-based methods provide non-intrusive solutions for monitoring. They have shown promising performance in food/beverage recognition and segmentation, human intake action detection and classification, and food volume/fluid amount estimation. However, occlusion, privacy, computational efficiency, and practicality pose significant challenges. This paper reviews the existing work (253 articles) on vision-based intake (food and fluid) monitoring methods to assess the size and scope of the available literature and identify the current challenges and research gaps. This paper uses tables and graphs to depict the patterns of device selection, viewing angle, tasks, algorithms, experimental settings, and performance of the existing monitoring systems.

Digital Therapeutics: Emerging New Therapy for Nonalcoholic Fatty Liver Disease

The increased prevalence of nonalcoholic fatty liver disease (NAFLD) worldwide is particularly worrisome, as no medication has been approved to treat the disease. Lifestyle modifications aimed at promoting weight loss and weight maintenance remain the current first-line treatment for NAFLD. However, due to the lack of standard and scientific guidance and out-of-hospital supervision, long-term outcomes of lifestyle interventions for patients with NAFLD are often unsatisfactory. In addition, the COVID-19 pandemic aggravated this dilemma. At the same time, digital therapeutics (DTx) are expected to be a new method for the convenient management and treatment of patients with NAFLD and are attracting a great deal of attention. DTx, which provide evidence-based medicine through software programs for remote intervention in preventing, treating, or managing diseases, overcome the drawbacks of traditional treatment. The efficacy of the approach has already been demonstrated for some chronic diseases, but DTx have not been fully developed for NAFLD. This study reviews the concepts, clinical value, and practical applications related to DTx, with an emphasis on recommendations based on unmet needs for NAFLD. A better understanding of the current state will help clinicians and researchers develop high-quality, standardized, and efficient DTx products, with the aim of optimizing the prognosis of patients with NAFLD.