Scientific rigor and credibility in the nutrition research landscape

Unacceptable use of substandard metrics in policy decisions which mandate large reductions in animal-source foods

Many recent very influential reports, including those from the Global Burden of Disease (GBD) Risk Factor Collaborators, the EAT-Lancet Commission on Food, Planet, Health, and the Lancet Countdown on Health and Climate Change, have recommended dramatic reductions or total exclusion of animal-source foods, particularly ruminant products (red meat and dairy), from the human diet. They strongly suggest that these dietary shifts will not only benefit planetary health but also human health. However, as detailed in this perspective, there are grounds for considerable concern in regard to the quality and transparency of the input data, the validity of the assumptions, and the appropriateness of the statistical modelling, used in the calculation of the global health estimates, which underpin the claimed human health benefits. The lessor bioavailability of protein and key micronutrients from plant-source foods versus animal-source foods was not adequately recognised nor addressed in any of these reports. Furthermore, assessments of bias and certainty were either limited or absent. Despite many of these errors and limitations being publically acknowledged by the GBD and the EAT-Lancet authors, no corrections have been applied to the published papers. As a consequence, these reports continue to erroneously influence food policy decisions and international dietary guidelines, such as the World Wildlife Fund's Livewell Diet, and the Nordic Nutrition Recommendations 2023.

Computational reproducibility of Jupyter notebooks from biomedical publications

BACKGROUND

Jupyter notebooks facilitate the bundling of executable code with its documentation and output in one interactive environment, and they represent a popular mechanism to document and share computational workflows, including for research publications. The reproducibility of computational aspects of research is a key component of scientific reproducibility but has not yet been assessed at scale for Jupyter notebooks associated with biomedical publications.

APPROACH

We address computational reproducibility at 2 levels: (i) using fully automated workflows, we analyzed the computational reproducibility of Jupyter notebooks associated with publications indexed in the biomedical literature repository PubMed Central. We identified such notebooks by mining the article's full text, trying to locate them on GitHub, and attempting to rerun them in an environment as close to the original as possible. We documented reproduction success and exceptions and explored relationships between notebook reproducibility and variables related to the notebooks or publications. (ii) This study represents a reproducibility attempt in and of itself, using essentially the same methodology twice on PubMed Central over the course of 2 years, during which the corpus of Jupyter notebooks from articles indexed in PubMed Central has grown in a highly dynamic fashion.

RESULTS

Out of 27,271 Jupyter notebooks from 2,660 GitHub repositories associated with 3,467 publications, 22,578 notebooks were written in Python, including 15,817 that had their dependencies declared in standard requirement files and that we attempted to rerun automatically. For 10,388 of these, all declared dependencies could be installed successfully, and we reran them to assess reproducibility. Of these, 1,203 notebooks ran through without any errors, including 879 that produced results identical to those reported in the original notebook and 324 for which our results differed from the originally reported ones. Running the other notebooks resulted in exceptions.

CONCLUSIONS

We zoom in on common problems and practices, highlight trends, and discuss potential improvements to Jupyter-related workflows associated with biomedical publications.

A call to action in review of the Australian Dietary Guidelines: Impacts of conflicting nutrition information: A mixed methods study

ISSUES ADDRESSED

The overabundance of conflicting nutrition information (CNI) and accompanying confusion and backlash are a public health concern; however, the complexity of responses to CNI has yet to be explored. The following mixed methods study brings depth to the perceptions and behavioural responses to CNI among Australian millennials to better inform successful nutrition guidelines.

METHODS

An explanatory sequential mixed methods design explored the cognitive and behavioural responses to CNI in Australian millennials. Cross-sectional data (n = 204) on CNI exposure, confusion, and backlash was analysed via multivariate ordinal logistic regression. The qualitative phase thematically analysed 18 semi-structured interviews on experiences with and responses to CNI.

RESULTS

Exposure to CNI via social media was positively associated with confusion. Nutrition confusion was positively associated with backlash. Qualitative analysis confirmed social media as a frequent, yet sometimes trusted, source of CNI. In addition, participants revealed using various methods to alleviate backlash while also relying heavily on traditional nutrition information (TNI) to inform dietary choices.

CONCLUSIONS

The methods to alleviate nutrition backlash provide new and innovative ways to tailor nutrition messages for maximum impact. Nutrition promotion initiatives and dietary guidelines should consider the complexity of responses to CNI and modernise interventions across mediums, including social media, with clear and attractive dietary recommendations. SO WHAT?: Results can inform the drafting of the new Australian Dietary Guidelines in 2023 and how they are promoted to the community on an ongoing basis.

Poor Reporting Quality in Basic Nutrition Research: A Case Study Based on a Scoping Review of Recent Folate Research in Mouse Models (2009-2021)

Transparent reporting of nutrition research promotes rigor, reproducibility, and relevance to human nutrition. We performed a scoping review of recent articles reporting dietary folate interventions in mice as a case study to determine the reporting frequency of generic study design items (i.e., sex, strain, and age) and nutrition-specific items (i.e., base diet composition, intervention doses, duration, and exposure verification) in basic nutrition research. We identified 798 original research articles in the EMBASE, Medline, Food Science and Technology Abstracts (FSTA), Global Health, and International Pharmaceutical Abstracts (IPA) databases published between January 2009 and July 2021 in which a dietary folic acid (FA) intervention was used in mice. We identified 312 original peer-reviewed articles including 191 studies in nonpregnant and 126 in pregnant mice. Most studies reported sex (99%), strain (99%), and age (83%). The majority of studies used C57BL/6 (53%) or BALB/c (11%) mice aged 3-9 wk. Nonpregnancy studies were more likely to use only male mice (57%). Dietary FA interventions varied considerably and overlapped: deficiency (0-3 mg/kg), control (0-16 mg/kg), and supplemented (0-50 mg/kg). Only 63% of studies used an open-formula base diet with a declared FA content and 60% of studies verified FA exposure using folate status biomarkers. The duration of intervention ranged from 1 to 104 wk for nonpregnancy studies. The duration of intervention for pregnancy studies was 1-19 wk, occurring variably before pregnancy and/or during pregnancy and/or lactation. Overall, 17% of studies did not report ≥1 generic study design item(s) and 40% did not report ≥1 nutrition-specific study design item(s). The variability and frequent lack of reporting of important generic and nutrition-specific study design details in nutrition studies limit their generalizability, reproducibility, and interpretation. The use of reporting checklists for animal research would enhance reporting quality of key study design and conduct factors in animal-based nutrition research.

Sixteen-week multicentre randomised controlled trial to study the effect of the consumption of an oat beta-glucan-enriched bread versus a whole-grain wheat bread on glycaemic control among persons with pre-diabetes: a study protocol of the CarbHealth study

INTRODUCTION

In 2012, the estimated global prevalence of pre-diabetes was 280 million, and the prevalence is expected to rise to 400 million by 2030. Oat-based foods are a good source of beta-glucans, which have been shown to lower postprandial blood glucose. Studies to evaluate the effectiveness of the long-term intake of beta-glucan-enriched bread as part of a habitual diet among individuals with pre-diabetes are needed. Therefore, we designed a multicentre intervention study in adults with pre-diabetes to investigate the effects of consumption of an oat-derived beta-glucan-enriched bread as part of a normal diet on glycated haemoglobin (HbA1c) in comparison to consumption of whole-grain wheat bread.

METHODS AND ANALYSIS

The CarbHealth trial is a multicentre double-blind randomised controlled 16-week dietary intervention trial in participants 40-70 years of age with a body mass index of ≥27 kg/m² and HbA1c of 35-50 mmol/mol. The study is conducted at four universities located in Norway, Sweden and Germany and uses intervention breads specifically designed for the trial by Nofima AS. The aim is to recruit 250 participants. The primary outcome is the difference in HbA1c between the intervention and the control groups. The main analysis will include intervention group, study centre and baseline HbA1c as independent variables in an analysis of covariance model.

ETHICS AND DISSEMINATION

The study protocol was approved by respective ethical authorities in participating countries. The results of the study will be communicated through publication in international scientific journals and presentations at (inter)national conferences.

TRIAL REGISTRATION NUMBER

NCT04994327.

An Updated Framework for Industry Funding of Food and Nutrition Research: Managing Financial Conflicts and Scientific Integrity

Although the food and beverage industry plays a critical role in advancing food and nutrition science, industry-funded research is subject to intense scrutiny as a result of various perceived and real biases related to funding sources. To address this, the Institute for the Advancement of Food and Nutrition Sciences (IAFNS) Assembly on Scientific Integrity has updated its Guiding Principles for Funding Food Science and Nutrition Research to provide a modernized framework for minimizing bias and promoting integrity in industry-funded research. Existing best practices for managing conflicts and maintaining trust in science, as well as coverage related to conflicts in industry-funded research, were reviewed to inform the development of the updated Guiding Principles. The updated Guiding Principles continue to provide conflict-of-interest guidelines to protect the integrity and credibility of the scientific record. These updates provide clarification, strengthen the guardrails that separate the funding from the science, and reflect the shift within the scientific community toward increased transparency and open science. If the principles are followed as intended, there should be little reason to dispute the results of industry-funded studies, other than to debate the science itself. This article issues a challenge to the research community to strive for just that.

Characterisation, procedures and heritability of acute dietary intake in the Twins UK cohort: an observational study

Background

Estimated food records (EFR) are a common dietary assessment method. This investigation aimed to; (1) define the reporting quality of the EFR, (2) characterise acute dietary intake and eating behaviours, (3) describe diet heritability.

Methods

A total of 1974 one-day EFR were collected from 1858 participants in the TwinsUK cohort between 2012 and 2017. EFR were assessed using a six-point scoring system to determine reporting quality. The frequency and co-occurrence of food items was examined using word clouds and co-occurrence networks. The impact of eating behaviours on weight, BMI and nutrient intake were explored using mixed-effect linear regression models. Finally, diet heritability was estimated using ACE modelling.

Results

We observed that 75% of EFR are of acceptable reporting quality (score > 5). Black tea and semi-skimmed milk were the most consumed items, on an individual basis (respectively 8.27, 6.25%) and paired (0.21%) as co-occurring items. Breakfast consumption had a significantly (p = 5.99 × 10^− 7) greater impact on energy (kcal) (mean 1874.67 (±SD 532.42)) than skipping breakfast (1700.45 (±SD 620.98)), however only length of eating window was significantly associated with body weight (kg) (effect size 0.21 (±SD 0.10), p = 0.05) and BMI (effect size 0.08 (±SD 0.04), p = 0.04) after adjustment for relevant covariates. Lastly, we reported that both length of eating window (h2 = 33%, CI 0.24; 0.41), and breakfast consumption (h2 = 11%, CI 0.02; 0.21) were weakly heritable.

Conclusions

EFR describing acute dietary intake allow for eating behaviour characterisation and can supplement habitual diet intake assessments. Novel findings of heritability warrant further investigation.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12937-022-00763-3.

Governing evidence use in the nutrition policy process: evidence and lessons from the 2020 Canada food guide

Nutrition guideline development is traditionally seen as a mechanism by which evidence is used to inform policy decisions. However, applying evidence in policy is a decidedly complex and politically embedded process, with no single universally agreed-upon body of evidence on which to base decisions, and multiple social concerns to address. Rather than simply calling for “evidence-based policy,” an alternative is to look at the governing features of the evidence use system and reflect on what constitutes improved evidence use from a range of explicitly identified normative concerns. This study evaluated the use of evidence within the Canada Food Guide policy process by applying concepts of the “good governance of evidence” – an approach that incorporates multiple normative principles of scientific and democratic best practice to consider the structure and functioning of evidence advisory systems. The findings indicated that institutionalizing a process for evidence use grounded in democratic and scientific principles can improve evidence use in nutrition policy making.

Metabolism-based therapies for epilepsy: new directions for future cures

OBJECTIVE

Thousands of years after dietary therapy was proposed to treat seizures, how alterations in metabolism relates to epilepsy remains unclear, and metabolism-based therapies are not always effective.

METHODS

We consider the state of the science in metabolism-based therapies for epilepsy across the research lifecycle from basic to translational to clinical studies.

RESULTS

This analysis creates a conceptual framework for creative, rigorous, and transparent research to benefit people with epilepsy through the understanding and modification of metabolism.

INTERPRETATION

Despite intensive past efforts to evaluate metabolism-based therapies for epilepsy, distinct ways of framing a problem offer the chance to engage different mindsets and new (or newly applied) technologies. A comprehensive, creative, and inclusive problem-directed research agenda is needed, with a renewed and stringent adherence to rigor and transparency across all levels of investigation.

A guide for authors and readers of the American Society for Nutrition Journals on the proper use of P values and strategies that promote transparency and improve research reproducibility

Two questions regarding the scientific literature have become grist for public discussion: 1) what place should P values have in reporting the results of studies? 2) How should the perceived difficulty in replicating the results reported in published studies be addressed? We consider these questions to be 2 sides of the same coin; failing to address them can lead to an incomplete or incorrect message being sent to the reader. If P values (which are derived from the estimate of the effect size and a measure of the precision of the estimate of the effect) are used improperly, for example reporting only significant findings, or reporting P values without account for multiple comparisons, or failing to indicate the number of tests performed, the scientific record can be biased. Moreover, if there is a lack of transparency in the conduct of a study and reporting of study results, it will not be possible to repeat a study in a manner that allows inferences from the original study to be reproduced or to design and conduct a different experiment whose aim is to confirm the original study's findings. The goal of this article is to discuss how P values can be used in a manner that is consistent with the scientific method, and to increase transparency and reproducibility in the conduct and analysis of nutrition research.

Evidence of misuse of nonparametric tests in the presence of heteroscedasticity within obesity research

BACKGROUND

Classic nonparametric tests (cNPTs), like Kruskal-Wallis or Mann-Whitney U, are sometimes used to detect differences in central tendency ( i.e., means or medians). However, when the tests' assumptions are violated, such as in the presence of unequal variance and other forms of heteroscedasticity, they are no longer valid for testing differences in central tendency. Yet, sometimes researchers erroneously use cNPTs to account for heteroscedasticity.

OBJECTIVE

To document the appropriateness of cNPT use in obesity literature, characterize studies that use cNPTs, and evaluate the citation and public sharing patterns of these articles.

METHODS

We reviewed obesity studies published in 2017 to determine whether the authors used cNPTs: (1) to correct for heteroscedasticity (invalid); (2) when heteroscedasticity was clearly not present (correct); or (3) when it was unclear whether heteroscedasticity was present (unclear). Open science R packages were used to transparently search literature and extract data on how often papers with errors have been cited in academic literature, read in Mendeley, and disseminated in the media.

RESULTS

We identified nine studies that used a cNPT in the presence of heteroscedasticity (some because of the mistaken rationale that the test corrected for heteroscedasticity), 25 articles that did not explicitly state whether heteroscedasticity was present when a cNPT was used, and only four articles that appropriately reported that heteroscedasticity was not present when a cNPT was used. Errors were found in observational and interventional studies, in human and rodent studies, and only when studies were unregistered. Studies with errors have been cited 113 times, read in Mendeley 123 times, and disseminated in the media 41 times, by the public, scientists, science communicators, and doctors.

CONCLUSIONS

Examples of inappropriate use of cNPTs exist in the obesity literature, and those articles perpetuate the errors via various audiences and dissemination platforms.

Persistent confusion in nutrition and obesity research about the validity of classic nonparametric tests in the presence of heteroscedasticity: evidence of the problem and valid alternatives

The use of classic nonparametric tests (cNPTs), such as the Kruskal-Wallis and Mann-Whitney U tests, in the presence of unequal variance for between-group comparisons of means and medians may lead to marked increases in the rate of falsely rejecting null hypotheses and decreases in statistical power. Yet, this practice remains prevalent in the scientific literature, including nutrition and obesity literature. Some nutrition and obesity studies use a cNPT in the presence of unequal variance (i.e., heteroscedasticity), sometimes because of the mistaken rationale that the test corrects for heteroscedasticity. Herein, we discuss misconceptions of using cNPTs in the presence of heteroscedasticity. We then discuss assumptions, purposes, and limitations of 3 common tests used to test for mean differences between multiple groups, including 2 parametric tests: Fisher's ANOVA and Welch's ANOVA; and 1 cNPT: the Kruskal-Wallis test. To document the impact of heteroscedasticity on the validity of these tests under conditions similar to those used in nutrition and obesity research, we conducted simple simulations and assessed type I error rates (i.e., false positives, defined as incorrectly rejecting the null hypothesis). We demonstrate that type I error rates for Fisher's ANOVA, which does not account for heteroscedasticity, and Kruskal-Wallis, which tests for differences in distributions rather than means, deviated from the expected significance level. Greater deviation from the expected type I error rate was observed as the heterogeneity increased, especially in the presence of an imbalanced sample size. We provide brief tutorial guidance for authors, editors, and reviewers to identify appropriate statistical tests when test assumptions are violated, with a particular focus on cNPTs.

The characteristics and extent of food industry involvement in peer-reviewed research articles from 10 leading nutrition-related journals in 2018

Introduction

There is emerging evidence that food industry involvement in nutrition research may bias research findings and/or research agendas. However, the extent of food industry involvement in nutrition research has not been systematically explored. This study aimed to identify the extent of food industry involvement in peer-reviewed articles from a sample of leading nutrition-related journals, and to examine the extent to which findings from research involving the food industry support industry interests.

Methods

All original research articles published in 2018 in the top 10 most-cited nutrition- and dietetics-related journals were analysed. We evaluated the proportion of articles that disclosed involvement from the food industry, including through author affiliations, funding sources, declarations of interest or other acknowledgments. Principal research findings from articles with food industry involvement, and a random sample of articles without food industry involvement, were categorised according to the extent to which they supported relevant food industry interests.

Results

196/1,461 (13.4%) articles reported food industry involvement. The extent of food industry involvement varied by journal, with The Journal of Nutrition (28.3%) having the highest and Paediatric Obesity (3.8%) having the lowest proportion of industry involvement. Processed food manufacturers were involved in the most articles (77/196, 39.3%). Of articles with food industry involvement, 55.6% reported findings favourable to relevant food industry interests, compared to 9.7% of articles without food industry involvement.

Conclusion

Food industry involvement in peer-reviewed research in leading nutrition-related journals is commonplace. In line with previous literature, this study has shown that a greater proportion of peer-reviewed studies involving the food industry have results that favour relevant food industry interests than peer-reviewed studies without food industry involvement. Given the potential competing interests of the food industry, it is important to explore mechanisms that can safeguard the integrity and public relevance of nutrition research.

Measuring health promotion: translating science into policy

Commonly, it is the end of life when our health is deteriorating, that many will make drastic lifestyle changes to improve their quality of life. However, it is increasingly recognized that bringing good health-promoting behaviors into practice as early in life as possible has the most significant impact across the maximal healthspan. The WHO has brought clarity to health promotion over the last fifteen years, always centering on language relating to a process of enabling people to increase control over, and to improve, their physical, mental and social health. A good healthspan is not just freedom from morbidity and mortality, it is that joie de vivre ("joy of living") that should accompany every day of our lifespan. Therefore, health promotion includes not only the health sector, but also needs individual commitment to achieve that target of a healthspan aligned with the lifespan. This paper explores health promotion and health literacy, and how to design appropriate nutritional studies to characterize contributors to a positive health outcome, the role the human microbiome plays in promoting health and addressing and alleviating morbidity and diseases, and finally how to characterize phenotypic flexibility and a physiologic resilience that we must maintain as our structural and functional systems are bombarded with the insults and perturbations of life.

High impact nutrition and dietetics journals' use of publication procedures to increase research transparency

BACKGROUND

The rigor and integrity of the published research in nutrition studies has come into serious question in recent years. Concerns focus on the use of flexible data analysis practices and selective reporting and the failure of peer review journals to identify and correct these practices. In response, it has been proposed that journals employ editorial procedures designed to improve the transparency of published research.

OBJECTIVE

The present study examines the adoption of editorial procedures designed to improve the reporting of empirical studies in the field of nutrition and dietetics research.

DESIGN

The instructions for authors of 43 journals included in Quartiles 1 and 2 of the Clarivate Analytics' 2018 Journal Citation Report category Nutrition and Dietetics were reviewed. For journals that published original research, conflict of interest disclosure, recommendation of reporting guidelines, registration of clinical trials, registration of other types of studies, encouraging data sharing, and use of the Registered Reports were assessed. For journals that only published reviews, all of the procedures except clinical trial registration were assessed.

RESULTS

Thirty-three journals published original research and 10 published only reviews. Conflict of interest disclosure was required by all 33 original research journals. Use of guidelines, trial registration and encouragement of data sharing were mentioned by 30, 27 and 25 journals, respectively. Registration of other studies was required by eight and none offered Registered Reports as a publication option at the time of the review. All 10 review journals required conflict of interest disclosure, four recommended data sharing and three the use of guidelines. None mentioned the other two procedures.

CONCLUSIONS

While nutrition journals have adopted a number of procedures designed to improve the reporting of research findings, their limited effects likely result from the mechanisms through which they influence analytic flexibility and selective reporting and the extent to which they are properly implemented and enforced by journals.

Improving natural product research translation: From source to clinical trial

While great interest in health effects of natural product (NP) including dietary supplements and foods persists, promising preclinical NP research is not consistently translating into actionable clinical trial (CT) outcomes. Generally considered the gold standard for assessing safety and efficacy, CTs, especially phase III CTs, are costly and require rigorous planning to optimize the value of the information obtained. More effective bridging from NP research to CT was the goal of a September, 2018 transdisciplinary workshop. Participants emphasized that replicability and likelihood of successful translation depend on rigor in experimental design, interpretation, and reporting across the continuum of NP research. Discussions spanned good practices for NP characterization and quality control; use and interpretation of models (computational through in vivo) with strong clinical predictive validity; controls for experimental artefacts, especially for in vitro interrogation of bioactivity and mechanisms of action; rigorous assessment and interpretation of prior research; transparency in all reporting; and prioritization of research questions. Natural product clinical trials prioritized based on rigorous, convergent supporting data and current public health needs are most likely to be informative and ultimately affect public health. Thoughtful, coordinated implementation of these practices should enhance the knowledge gained from future NP research.