Social media analytics in nutrition research: a rapid review of current usage in investigation of dietary behaviours

Evaluating large language models for health-related text classification tasks with public social media data

OBJECTIVES

Large language models (LLMs) have demonstrated remarkable success in natural language processing (NLP) tasks. This study aimed to evaluate their performances on social media-based health-related text classification tasks.

MATERIALS AND METHODS

We benchmarked 1 Support Vector Machine (SVM), 3 supervised pretrained language models (PLMs), and 2 LLMs-based classifiers across 6 text classification tasks. We developed 3 approaches for leveraging LLMs: employing LLMs as zero-shot classifiers, using LLMs as data annotators, and utilizing LLMs with few-shot examples for data augmentation.

RESULTS

Across all tasks, the mean (SD) F1 score differences for RoBERTa, BERTweet, and SocBERT trained on human-annotated data were 0.24 (±0.10), 0.25 (±0.11), and 0.23 (±0.11), respectively, compared to those trained on the data annotated using GPT3.5, and were 0.16 (±0.07), 0.16 (±0.08), and 0.14 (±0.08) using GPT4, respectively. The GPT3.5 and GPT4 zero-shot classifiers outperformed SVMs in a single task and in 5 out of 6 tasks, respectively. When leveraging LLMs for data augmentation, the RoBERTa models trained on GPT4-augmented data demonstrated superior or comparable performance compared to those trained on human-annotated data alone.

DISCUSSION

The results revealed that using LLM-annotated data only for training supervised classification models was ineffective. However, employing the LLM as a zero-shot classifier exhibited the potential to outperform traditional SVM models and achieved a higher recall than the advanced transformer-based model RoBERTa. Additionally, our results indicated that utilizing GPT3.5 for data augmentation could potentially harm model performance. In contrast, data augmentation with GPT4 demonstrated improved model performances, showcasing the potential of LLMs in reducing the need for extensive training data.

CONCLUSIONS

By leveraging the data augmentation strategy, we can harness the power of LLMs to develop smaller, more effective domain-specific NLP models. Using LLM-annotated data without human guidance for training lightweight supervised classification models is an ineffective strategy. However, LLM, as a zero-shot classifier, shows promise in excluding false negatives and potentially reducing the human effort required for data annotation.

Ifood: Development and usability study of a social media-based applet for dietary monitoring

Background

Dietary monitoring is critical to maintaining human health. Social media platforms are widely used for daily recording and communication for individuals' diets and activities. The textual content shared on social media offers valuable resources for dietary monitoring.

Objective

This study aims to describe the development of iFood, an applet providing personal dietary monitoring based on social media content, and validate its usability, which will enable efficient personal dietary monitoring.

Methods

The process of the development and validation of iFood is divided into four steps: Diet datasets construction, diet record and analysis, diet monitoring applet design, and diet monitoring applet usability assessment. The diet datasets were constructed with the data collected from Weibo, Meishijie, and diet guidelines, which will be used as the basic knowledge for further model training in the phase of diet record and analysis. Then, the friendly user interface was designed to link users with backend functions. Finally, the applet was deployed as a WeChat applet and 10 users from the Beijing Union Medical College have been recruited to validate the usability of iFood.

Results

Three dietary datasets, including User Visual-Textual Dataset, Dietary Information Expansion Dataset, and Diet Recipe Dataset have been constructed. The performance of 4 models for recognizing diet and fusing unimodality data was 40.43%(dictionary-based model), 18.45%(rule-based model), 59.95%(Inception-ResNet-v2), and 51.38% (K-nearest neighbor), respectively. Furthermore, we have designed a user-friendly interface for the iFood applet and conducted a usability assessment, which resulted in an above-average usability score.

Conclusions

iFood is effective for managing individual dietary behaviors through its seamless integration with social media data. This study suggests that future products could utilize social media data to promote healthy lifestyles.

Using Social Media Analysis to Study Population Dietary Behaviours: A Scoping

BACKGROUND

The rapid adoption and sustained use of social media globally has provided researchers with access to unprecedented quantities of low-latency data at minimal costs. This may be of particular interest to nutrition research as food is frequently posted about and discussed on social media platforms. This scoping review investigates the ways in which social media is being used to understand population food consumption, attitudes, and behaviours.

METHODS

The peer-reviewed literature was searched from 2003 to 2021 using four electronic databases.

RESULTS

The review identified 71 eligible studies from 25 countries. Two thirds (n=47) were published within the last five years. The United States had the highest research output (31%, n=22) and Twitter was the most used platform (41%, n=29). A diverse range of dataset sizes were used, with some studies relying on manual techniques to collect and analyse data while others required the use of advanced software technology. Most studies were conducted by disciplines outside health with only two studies (3%) conducted by nutritionists.

CONCLUSION

It appears the development of methodological and ethical frameworks as well as partnerships between experts in nutrition and information technology may be required to advance the field in nutrition research. Moving beyond traditional methods of dietary data collection may prove social media as a useful adjunct to inform recommended dietary practices and food policies. This article is protected by copyright. All rights reserved.