A New Portion Size Estimation Aid for Wedge-Shaped Foods

A Comprehensive Survey of Image-Based Food Recognition and Volume Estimation Methods for Dietary Assessment

Dietary studies showed that dietary problems such as obesity are associated with other chronic diseases, including hypertension, irregular blood sugar levels, and increased risk of heart attacks. The primary cause of these problems is poor lifestyle choices and unhealthy dietary habits, which are manageable using interactive mHealth apps. However, traditional dietary monitoring systems using manual food logging suffer from imprecision, underreporting, time consumption, and low adherence. Recent dietary monitoring systems tackle these challenges by automatic assessment of dietary intake through machine learning methods. This survey discusses the best-performing methodologies that have been developed so far for automatic food recognition and volume estimation. Firstly, the paper presented the rationale of visual-based methods for food recognition. Then, the core of the study is the presentation, discussion, and evaluation of these methods based on popular food image databases. In this context, this study discusses the mobile applications that are implementing these methods for automatic food logging. Our findings indicate that around 66.7% of surveyed studies use visual features from deep neural networks for food recognition. Similarly, all surveyed studies employed a variant of convolutional neural networks (CNN) for ingredient recognition due to recent research interest. Finally, this survey ends with a discussion of potential applications of food image analysis, existing research gaps, and open issues of this research area. Learning from unlabeled image datasets in an unsupervised manner, catastrophic forgetting during continual learning, and improving model transparency using explainable AI are potential areas of interest for future studies.

Evaluation of portion size estimation aids for the Korea National Health and Nutrition Examination Survey

BACKGROUND/OBJECTIVES

This study aimed to improve portion size estimation aids (PSEAs) used in the nutrition survey of the Korea National Health and Nutrition Examination Survey (KNHANES) and validate the accuracy and precision of the newly developed aids.

SUBJECTS/METHODS

We conducted intensive interviews with survey experts in KNHANES and consulted with experts to collect opinions about improvement of PSEAs. Based on the results of the interviews, 5 types of PSEAs (rice bowl, earthen pots, mounds, measuring spoons, and thickness sticks) were newly developed using 3-dimensional (3D) modeling or modification of color or shape. Validation tests were conducted with 96 adults 20 years old or older. For the rice bowl and earthen pots, the participants were asked to select the more similar PSEA in size after being shown the real dishes. For the mounds, measuring spoons, and thickness sticks, the participants were presented with actual plates of food and asked to estimate the given portion sizes using the given PSEAs.

RESULTS

The improved 2-dimensional (2D) picture aid for the rice bowl reflecting the size distortion by angle of view using 3D modeling was perceived more closely to the actual size than the current 2D picture (P < 0.001). The change of the color of 2D pictures and 3D models, the change of shape of the measuring spoons, and the 3-dimensionalization of the 2D mounds had no significant improvement in the subjects' perception.

CONCLUSIONS

The currently used 2D PSEAs need to be fully redesigned using 3D modeling to improve subjects' perception. However, change of color or shape will not be necessary. For amorphous foods, it is suggested that more evaluation be performed before reaching a final conclusion in the use of PSEAs, or alternative ways to improve accuracy of estimation need to be explored.

Developing a Messaging Graphic for Storage Times of Refrigerated Ready to Eat (RTE) Foods for a Consumer Food Safety Health Campaign

This study developed and evaluated risk communication messages for ready to eat (RTE) foods targeted towards consumer storage practices in a food safety health campaign. Concepts were determined from a fractional factorial design of five categories of attributes potentially present in health promotion: title, message, graphic, slogan, and icon. Consumers viewed a subset of concepts and scored how useful the concept was in remembering to throw away RTE foods that were stored too long. Regression analysis determined which combinations of message attributes were most likely to result in using the information to throw out foods, which could help prevent foodborne illness. Findings showed that for this type of information, a graphic is a critical element for the printed schematic. The slogan (i.e., a short statement similar to a jingle or tag-line in a commercial) may be important to consumers, but the icon was not important.

Optimizing portion-size estimation aids: a formative evaluation in Malawi

OBJECTIVE

To investigate preferences for and ease-of-use perceptions of different aspects of printed and digitally displayed photographic portion-size estimation aids (PSEA) in a low-resource setting and to document accuracy of portion-size selections using PSEA with different visual characteristics.

DESIGN

A convergent mixed-methods design and stepwise approach were used to assess characteristics of interest in isolation. Participants served themselves food and water, which were weighed before and after consumption to measure leftovers and quantity consumed. Thirty minutes later, data collectors administered a meal recall using a PSEA and then a semi-structured interview.

SETTING

Blantyre and Chikwawa Districts in the southern region of Malawi.

PARTICIPANTS

Ninety-six women, aged 18-45 years.

RESULTS

Preferences and ease-of-use perceptions favoured photographs rather than drawings of shapes, three and five portion-size options rather than three with four virtual portion-size options, a 45° rather than a 90° photograph angle, and simultaneous rather than sequential presentation of portion-size options. Approximately half to three-quarters of participants found the portion-size options represented appropriate amounts of foods or water consumed. Photographs with three portion sizes resulted in more accurate portion-size selections (closest to measured consumption) than other format and number of portion-size option combinations. A 45° angle and simultaneous presentation were more accurate than a 90° angle and sequential presentation of images.

CONCLUSIONS

Results from testing PSEA visual characteristics separately can be used to generate optimal PSEA, which can improve participants' experiences during meal recalls.

ServAR: An augmented reality tool to guide the serving of food

Background

Accurate estimation of food portion size is a difficult task. Visual cues are important mediators of portion size and therefore technology-based aids may assist consumers when serving and estimating food portions. The current study evaluated the usability and impact on estimation error of standard food servings of a novel augmented reality food serving aid, ServAR.

Methods

Participants were randomised into one of three groups: 1) no information/aid (control); 2) verbal information on standard serving sizes; or 3) ServAR, an aid which overlayed virtual food servings over a plate using a tablet computer. Participants were asked to estimate the standard serving sizes of nine foods (broccoli, carrots, cauliflower, green beans, kidney beans, potato, pasta, rice, and sweetcorn) using validated food replicas. Wilcoxon signed-rank tests compared median served weights of each food to reference standard serving size weights. Percentage error was used to compare the estimation of serving size accuracy between the three groups. All participants also performed a usability test using the ServAR tool to guide the serving of one randomly selected food.

Results

Ninety adults (78.9% female; a mean (95%CI) age 25.8 (24.9–26.7) years; BMI 24.2 (23.2–25.2) kg/m²) completed the study. The median servings were significantly different to the reference portions for five foods in the ServAR group, compared to eight foods in the information only group and seven foods for the control group. The cumulative proportion of total estimations per group within ±10%, ±25% and ±50% of the reference portion was greater for those using ServAR (30.7, 65.2 and 90.7%; respectively), compared to the information only group (19.6, 47.4 and 77.4%) and control group (10.0, 33.7 and 68.9%). Participants generally found the ServAR tool easy to use and agreed that it showed potential to support optimal portion size selection. However, some refinements to the ServAR tool are required to improve the user experience.

Conclusions

Use of the augmented reality tool improved accuracy and consistency of estimating standard serve sizes compared to the information only and control conditions. ServAR demonstrates potential as a practical tool to guide the serving of food. Further evaluation across a broad range of foods, portion sizes and settings is warranted.

Accuracy of hands v. household measures as portion size estimation aids

Accurate estimation of food portion size is critical in dietary studies. Hands are potentially useful as portion size estimation aids; however, their accuracy has not been tested. The aim of the present study was to test the accuracy of a novel portion size estimation method using the width of the fingers as a 'ruler' to measure the dimensions of foods ('finger width method'), as well as fists and thumb or finger tips. These hand measures were also compared with household measures (cups and spoons). A total of sixty-seven participants (70 % female; age 32·7 (sd 13·7) years; BMI 23·2 (sd  3·5) kg/m(2)) attended a 1·5 h session in which they estimated the portion sizes of forty-two pre-weighed foods and liquids. Hand measurements were used in conjunction with geometric formulas to convert estimations to volumes. Volumes determined with hand and household methods were converted to estimated weights using density factors. Estimated weights were compared with true weights, and the percentage difference from the true weight was used to compare accuracy between the hand and household methods. Of geometrically shaped foods and liquids estimated with the finger width method, 80 % were within ±25 % of the true weight of the food, and 13 % were within ±10 %, in contrast to 29 % of those estimated with the household method being within ±25 % of the true weight of the food, and 8 % being within ±10 %. For foods that closely resemble a geometric shape, the finger width method provides a novel and acceptably accurate method of estimating portion size.

An evaluation of portion size estimation aids: precision, ease of use and likelihood of future use

OBJECTIVE

The present study aimed to evaluate the precision, ease of use and likelihood of future use of portion size estimation aids (PSEA).

DESIGN

A range of PSEA were used to estimate the serving sizes of a range of commonly eaten foods and rated for ease of use and likelihood of future usage.

SETTING

For each food, participants selected their preferred PSEA from a range of options including: quantities and measures; reference objects; measuring; and indicators on food packets. These PSEA were used to serve out various foods (e.g. liquid, amorphous, and composite dishes). Ease of use and likelihood of future use were noted. The foods were weighed to determine the precision of each PSEA.

SUBJECTS

Males and females aged 18-64 years (n 120).

RESULTS

The quantities and measures were the most precise PSEA (lowest range of weights for estimated portion sizes). However, participants preferred household measures (e.g. 200 ml disposable cup) - deemed easy to use (median rating of 5), likely to use again in future (all scored either 4 or 5 on a scale from 1='not very likely' to 5='very likely to use again') and precise (narrow range of weights for estimated portion sizes). The majority indicated they would most likely use the PSEA preparing a meal (94 %), particularly dinner (86 %) in the home (89 %; all P<0·001) for amorphous grain foods.

CONCLUSIONS

Household measures may be precise, easy to use and acceptable aids for estimating the appropriate portion size of amorphous grain foods.

Accuracy of food portion size estimation from digital pictures acquired by a chest-worn camera

OBJECTIVE

Accurate estimation of food portion size is of paramount importance in dietary studies. We have developed a small, chest-worn electronic device called eButton which automatically takes pictures of consumed foods for objective dietary assessment. From the acquired pictures, the food portion size can be calculated semi-automatically with the help of computer software. The aim of the present study is to evaluate the accuracy of the calculated food portion size (volumes) from eButton pictures.

DESIGN

Participants wore an eButton during their lunch. The volume of food in each eButton picture was calculated using software. For comparison, three raters estimated the food volume by viewing the same picture. The actual volume was determined by physical measurement using seed displacement.

SETTING

Dining room and offices in a research laboratory.

SUBJECTS

Seven lab member volunteers.

RESULTS

Images of 100 food samples (fifty Western and fifty Asian foods) were collected and each food volume was estimated from these images using software. The mean relative error between the estimated volume and the actual volume over all the samples was -2·8 % (95 % CI -6·8 %, 1·2 %) with sd of 20·4 %. For eighty-five samples, the food volumes determined by computer differed by no more than 30 % from the results of actual physical measurements. When the volume estimates by the computer and raters were compared, the computer estimates showed much less bias and variability.

CONCLUSIONS

From the same eButton pictures, the computer-based method provides more objective and accurate estimates of food volume than the visual estimation method.

Imaged based estimation of food volume using circular referents in dietary assessment

Measuring food volume (portion size) is a critical component in both clinical and research dietary studies. With the wide availability of cell phones and other camera-ready mobile devices, food pictures can be taken, stored or transmitted easily to form an image based dietary record. Although this record enables a more accurate dietary recall, a digital image of food usually cannot be used to estimate portion size directly due to the lack of information about the scale and orientation of the food within the image. The objective of this study is to investigate two novel approaches to provide the missing information, enabling food volume estimation from a single image. Both approaches are based on an elliptical reference pattern, such as the image of a circular pattern (e.g., circular plate) or a projected elliptical spotlight. Using this reference pattern and image processing techniques, the location and orientation of food objects and their volumes are calculated. Experiments were performed to validate our methods using a variety of objects, including regularly shaped objects and food samples.

Food energy content influences food portion size estimation by nutrition students

BACKGROUND

Food portion size estimation involves a complex mental process that may influence food consumption evaluation. Knowing the variables that influence this process can improve the accuracy of dietary assessment. The present study aimed to evaluate the ability of nutrition students to estimate food portions in usual meals and relate food energy content with errors in food portion size estimation.

METHODS

Seventy-eight nutrition students, who had already studied food energy content, participated in this cross-sectional study on the estimation of food portions, organised into four meals. The participants estimated the quantity of each food, in grams or millilitres, with the food in view. Estimation errors were quantified, and their magnitude were evaluated. Estimated quantities (EQ) lower than 90% and higher than 110% of the weighed quantity (WQ) were considered to represent underestimation and overestimation, respectively. Correlation between food energy content and error on estimation was analysed by the Spearman correlation, and comparison between the mean EQ and WQ was accomplished by means of the Wilcoxon signed rank test (P < 0.05).

RESULTS

A low percentage of estimates (18.5%) were considered accurate (+/-10% of the actual weight). The most frequently underestimated food items were cauliflower, lettuce, apple and papaya; the most often overestimated items were milk, margarine and sugar. A significant positive correlation between food energy density and estimation was found (r = 0.8166; P = 0.0002).

CONCLUSIONS

The results obtained in the present study revealed a low percentage of acceptable estimations of food portion size by nutrition students, with trends toward overestimation of high-energy food items and underestimation of low-energy items.

Children’s Preference for Large Portions: Prevalence, Determinants, and Consequences

BACKGROUND

Proliferation of large portions of snack and fast foods parallels dramatic increases in childhood obesity. This study investigates the prevalence, determinants, and consequences of large portions in children's diets.

METHODS

As part of the 2003 Children's Lifestyle and School-performance Study, we surveyed 4,966 children in Nova Scotia regarding their usual portion sizes of french fries, meats, vegetables, and potato chips using three-dimensional graduated food models. We measured heights and weights and assessed dietary intake with the Harvard Youth Adolescent Food Frequency Questionnaire. Diet quality was summarized using the Diet Quality Index International (DQI-I). Parents were surveyed on food habits and socioeconomic background.

STATISTICAL ANALYSIS

We used multilevel regression methods to examine determinants of children's large portion size choice and to evaluate the effect of this selection on energy intake, diet quality, and overweight.

RESULTS

Children reported preference for portions of french fries, meats, and potato chips that are larger and vegetable portions that are smaller than what is recommended. Children from socioeconomically disadvantaged families or who frequently eat while watching television and in fast-food restaurants preferred larger portions of french fries and potato chips. Consequences of consuming large portions of these foods included poor diet quality and increased energy intake. Consuming large portions of vegetables was associated with lower energy intake and better diet quality.

CONCLUSIONS

Successful marketing of large portions of french fries and potato chips may be at the expense of diet quality and appropriate energy intake. Policy regulations and nutrition education emphasizing appropriate portion sizes provide opportunities to prevent overweight and improve future health.