The role of The Cochrane Collaboration in support of the WHO Nutrition Guidelines

The future of Cochrane Neonatal

Cochrane Neonatal was first established in 1993, as one of the original review groups of the Cochrane Collaboration. In fact, the origins of Cochrane Neonatal precede the establishment of the collaboration. In the 1980's, the National Perinatal Epidemiology Unit at Oxford, led by Dr. Iain Chalmers, established the "Oxford Database of Perinatal Trials" (ODPT), a register of virtually all randomized controlled trials in perinatal medicine to provide a resource for reviews of the safety and efficacy of interventions used in perinatal care and to foster cooperative and coordinated research efforts in the perinatal field [1]. An effort that was clearly ahead of its time, ODPT comprised four main elements: a register of published reports of trials; a register of unpublished trials; a register of ongoing and planned trials; and data derived from pooled overviews (meta-analyses) of trials. This core effort grew into the creation of the seminal books, "Effective Care in Pregnancy and Childbirth" as well as "Effective Care of the Newborn Infant" [2,3]. As these efforts in perinatal medicine grew, Iain Chalmers thought well beyond perinatal medicine into the creation of a worldwide collaboration that became Cochrane [4]. The mission of the Cochrane Collaboration is to promote evidence-informed health decision-making by producing high-quality, relevant, accessible systematic reviews and other synthesized research evidence (www.cochrane.org). Cochrane Neonatal has continued to be one of the most productive review groups, publishing between 25 tpo to 40 new or updated systematic reviews each year. The impact factor has been steadily increasing over four years and now rivals most of the elite journals in pediatric medicine. Cochrane Neonatal has been a worldwide effort. Currently, there are 404 reviews involving 1206 authors from 52 countries. What has Cochrane done for babies? Reviews from Cochrane Neonatal have informed guidelines and recommendations worldwide. From January 2018 through June 2020, 77 international guidelines cited 221 Cochrane Neonatal reviews. These recommendations have included recommendations of the use of postnatal steroids, inhaled nitric oxide, feeding guidelines for preterm infants and other core aspects of neonatal practice. In addition, Cochrane Reviews has been the impetus for important research, including the large-scale trial of prophylactic indomethacin therapy, a variety of trials of postnatal steroids, trials of emollient ointment and probiotic trials [6]. While justifiably proud of these accomplishments, one needs to examine the future contribution of Cochrane Neonatal to the neonatal community. The future of Cochrane Neonatal is inexorably linked to the future of neonatal research. Obviously, there is no synthesis of trials data if, as a community, we fail to provide the core substrate for that research. As we look at the current trials' environment, fewer randomized controlled trial related to neonates are being published in recent years. A simple search of PubMed, limiting the search to "neonates" and "randomized controlled trials" shows that in the year 2000, 321 randomized controlled trials were published. These peaked five years ago, in 2015, with close to 900 trials being published. However, in 2018, only 791 studies are identified. Does this decrease represent a meaningful change in the neonatal research environment? Quite possibly. There are shifting missions of clinical neonatology at academic medical institutions, at least in the United States, with a focus on business aspects as well as other important competing clinical activities. Quality improvement has taken over as one of the major activities at both private and academic neonatal practices. Clearly, this is a needed improvement. All units at levels need to be dedicated to improving the outcomes of the sick and fragile population we care for. However, this need not be at the expense of formal clinical trials. It is understandable that this approach would be taken. Newer interventions frequently relate to complex systems of care and not the simple single interventions. Even trials that might traditionally have been done as randomized controlled trials, such as the introduction of a new mode of ventilation, are in reality complex challenges to the ability of institutions to create systems to adapt to these new technologies. Cost of doing trials has always been a barrier. The challenging regulatory and ethical environment contributes to these problems as well [7]. Despite these barriers, how does the research agenda of the neonatal community move forward in the 21st Century? We need to reassess how we create and disseminate our research findings. Innovative trial designs will allow us to address complex issues that we may not have tackled with conventional trials. Adaptive designs may allow us to look at potentially life-saving therapies in a way that feel more efficient and more ethical [8]. Clarifying issues such as the use of inhaled nitric oxide in preterm infants would be greatly served if we even knew whether or not there are hypoxemic preterm infant who would benefit from this therapy [9]. Current trials do not suggest so, yet current practice tells us that a significant number of these babies will receive inhaled nitric oxide [10-13]. Adaptive design, such as those done with trials of extracorporeal membrane oxygenation (ECMO), would allow us to quickly assess whether, in fact, these therapies are life-saving and allow us to consider whether or not further trials are needed [14,15]. Our understanding that many interventions involve entire systems approaches does not relegate us only to doing quality improvement work. Cluster designs may allow us to test more complex interventions that have usually been under the purview of quality improvement [16-18]. Cluster trials are well suited for such investigations and can be done with the least interruption to ongoing care. Ultimately, quality improvement is the application of the best evidence available (evidence-based medicine is "what to do" and evidence-based practice is "how to do"). [19,20]. Nascent efforts, such as the statement on "embedding necessary research into culture and health" (the ENRICH statement) call for the conduct of large, efficient pragmatic trials to evaluate neonatal outcomes, as in part called for in the ALPHA Collaboration [21,22]. This statement envisions an international system to identify important research questions by consulting regularly with all stakeholders, including patients, public health professionals, researchers, providers, policy makers, regulators, funders of industry. The ENRICH statement envisions a pathway to enable individuals, educational institutions, hospitals and health-care facilities to confirm their status as research-friendly by integrating an understanding of trials, other research and critical thinking and to teaching learning and culture, as well as an engagement with funders, professional organizations and regulatory bodies and other stake holders to raise awareness of the value of efficient international research to reduce barriers to large international pragmatic trials and other collaborative studies. In the future, if trials are to be done on this scale or trials are prospectively designed to be analyzed together, core outcome measures must be identified and standardized. That clinical trials supply estimates of outcomes that are relevant to patients and their families is critical. In addition, current neonatal research evaluates many different outcomes using multiple measures. A given measure can have multiple widely used definitions. Bronchopulmonary dysplasia (or chronic lung disease just to add to the confusion) quickly comes to mind [23,24]. The use of multiple definitions when attempting to measure the same outcome prevents synthesis of trial results and meta-analysis and hinders efforts to refine our estimates of effects. Towards that end, Webbe and colleagues have set out to develop a core outcome set for neonatal research [25]. Key stakeholders in the neonatal community reviewed multiple outcomes reported in neonatal trials and qualitative studies. Based on consensus, key outcome measures were identified, including survival, sepsis, necrotizing enterocolitis, brain injury on imaging, retinopathy or prematurity, gross motor ability, general cognitive ability, quality of life, adverse events, visual impairment or blindness, hearing impairment or deafness, chronic lung disease/bronchopulmonary dysplasia. Trials registration has to be a continued focus of the neonatal community. Trials registration allows for systematic reviewers to understand whether or not reporting bias has occurred [26]. It also allows for transparent incorporation of these core outcome measures. Ultimately, trials registration should include public reporting of all of these core outcomes and, in the future, access to data on an individual level such that more sophisticated individual patient data meta-analysis could occur. Lastly, there is no reason to see clinical trials and quality improvement as separate or exclusive activities. In fact, in the first NICQ Collaborative, conducted by Vermont Oxford Network, participation in a trial of postnatal steroids was considered part of the quality improvement best practices as opposed to simply choosing an as-of-yet unproven approach to use of this potent drug [27]. What role will Cochrane Neonatal play as we move forward in the 21st Century? As the neonatal community moves forward with its' research agenda, Cochrane Neonatal must not only follow but also lead with innovative approaches to synthesizing research findings. Cochrane Neonatal must continue to work closely with guideline developers. The relationship between systematic review production and guideline development is clearly outlined in reports from the Institute of Medicine [28,29]. Both are essential to guideline development; the systematic review group culling the evidence for the benefits and harms of a given intervention and the guideline group addressing the contextual issues of cost, feasibility, implementation and the values and preferences of individuals and societies. Most national and international guidelines groups now routinely use systematic reviews as the evidence basis for their guidelines and recommendations. Examples of the partnership between Cochrane Neonatal and international guideline development can be seen in our support of the World Health Organization (WHO) guidelines on the use of vitamin A or the soon to be published recommendations from the International Liaison Committee on Resuscitation (ILCOR) on cord management in preterm and term infants [30]. In the future, we need to collaborate early in the guideline development process so that the reviews are fit for purpose and meet the needs of the guideline developers and the end users. Towards this end, all Cochrane Neonatal reviews now contain GRADE assessments of the key clinical findings reported in the systematic review [31]. Addition of these assessments addresses the critical issue of our confidence in the findings. We are most confident in evidence provided by randomized controlled trials but this assessment can be can be downgraded if the studies that reported on the outcome in question had a high risk of bias, indirectness, inconsistency of results, or imprecision, or where there is evidence of reporting bias. Information provided by GRADE assessments is seen as critical in the process of moving from the evidence to formal recommendations [32]. We need to explore complex reviews, such as network (NMA) or multiple treatment comparison (MCT) meta-analyses, to address issues not formally addressed in clinical trials [33]. In conditions where there are multiple effective interventions, it is rare for all possible interventions to have been tested against each other [34]. A solution could be provided by network meta-analysis, which allows for comparing all treatments with each other, even if randomized controlled trials are not available for some treatment comparisons [34]. Network meta-analysis uses both direct (head-to-head) randomized clinical trial (RCT) evidence as well as indirect evidence from RCTs to compare the relative effectiveness of all included interventions [35]. However, Mills and colleagues note that the methodological quality of MTCs may be difficult for clinicians to interpret because the number of interventions evaluated may be large and the methodological approaches may be complex [35]. Cochrane Neonatal must take a role in both the creation of such analyses and the education of the neonatal community regarding the pitfalls of such an approach. The availability of individual patient data will make more sophisticated analyses more available to the community. Although the current crop of individual patient data meta-analyses (including the reviews of elective high frequency ventilation, inhaled nitric oxide and oxygen targets) have not differed substantially from the findings of the trials level reviews (suggesting that, in fact, sick neonates are more alike that unalike), there still will be a large role for individual patient data meta-analysis, at least to end the unfound conclusions that these therapies are effective in various subgroups (be it issues of sex, disease severity, or clinical setting) [36-39]. Future trials should take a lesson from the NeOProM Collaborative [37,39]. Given the difficulty in generating significant sample size and creating funding in any single environment, trials with similar protocols should be conducted in a variety of healthcare settings with an eye towards both study level and individual patient level meta-analysis at the conclusion of those trials, allowing for broader contribution to the trials data, more rapid accrual of sample size, and more precise results. We need to educate the neonatal community regarding the use and abuse of diagnostic tests. Diagnostic tests are a critical component of healthcare but also contribute greatly to the cost of medical care worldwide. These costs include the cost of the tests themselves and the costs of misdiagnosis and treatment of individuals who will not benefit from those treatments. Clinicians may have a limited understanding of diagnostic test accuracy, the ability of a diagnostic test to distinguish between patients with and without the disease or target condition [41,42]. Efforts such as Choosing Wisely have tried to identify these deficiencies [40]. As Cochrane has increased the general literacy of both the medical and general population regarding the interpretation of the results of interventions on various diseases, so should Cochrane move forward and improve the understanding of diagnostic testing. We need to become more efficient at creating and maintaining our reviews. The time spent to produce systematic reviews is far too great. In average, it takes between 2½ to 6½ years to produce a systematic review, requiring intense time input for highly trained and expensive experts. Innovations in the ways in which we produce systematic reviews can make the review process more efficient by outsourcing some of the tasks or crowdsourcing to machine learning. We need to let the crowd and machine learning innovations help us sort the massive amounts of information needed to conduct systematic reviews. It can also allow for "live" updating of critical reviews where the research landscape is quickly changing [43]. Lastly, Cochrane Neonatal must focus more on users of the reviews and not necessarily authors of the reviews. Current Cochrane programming speaks of Cochrane training with an eye towards developing the skills of individuals who will conduct systematic reviews. While this is clearly needed and laudable, the fact of the matter is that most of the community will be "users" of the reviews. Individuals who need to understand how to use and interpret the findings of systematic reviews. These review users include clinicians, guideline developers, policy makers and families. Incorporation of GRADE guidelines has been a huge step in adding transparency to the level of uncertainty we have in our findings. From a family's perspective, we need to overcome the environment of mistrust or misunderstanding of scientific evidence and how we convey what we know, and our uncertainty about what we know, to parents and families.

Towards a new model for producing evidence-based guidelines: a qualitative study of current approaches and opportunities for innovation among Australian guideline developers

Background: Many organisations in Australia undertake systematic reviews to inform development of evidence-based guidelines or would like to do so. However, the substantial resources required to produce systematic reviews limit the feasibility of evidence-based approaches to guideline development. We are working with Australian guideline developers to design, build and test systems that make creating evidence-based guidelines easier and more efficient. Methods: To understand the evidence needs of guideline developers and to inform the development of potential tools and services, we conducted 16 semi-structured interviews with Australian guideline developers. Developers were involved in different types of guidelines, represented both new and established guideline groups, and had access to widely different levels of resources. Results: All guideline developers recognised the importance of having access to timely evidence to support their processes, but were frequently overwhelmed by the scale of this task. Groups developing new guidelines often underestimated the time, expertise and work involved in completing searching and screening. Many were grappling with the challenge of updating and were keen to explore alternatives to the blanket updating of the full guideline. Horizon-scanning and evidence signalling were seen as providing more pragmatic approaches to updating, although some were wary of challenges posed by receiving evidence on a too-frequent basis. Respondents were aware that new technologies, such as machine learning, offered potentially large time and resource savings. Conclusions: As well as the constant challenge of managing financial constraints, Australian guideline developers seeking to develop clinical guidelines face several critical challenges. These include acquiring appropriate methodological expertise, investing in information technology, coping with the proliferation of research output, feasible publication and dissemination options, and keeping guidance up to date.

Consideration of sex and gender in Cochrane reviews of interventions for preventing healthcare-associated infections: a methodology study

BACKGROUND

Healthcare-associated infections (HAIs) are common and increase morbidity, mortality, and healthcare costs. Their control continues to be an unresolved issue worldwide. HAIs epidemiology shows sex/gender differences. Thus the lack of consideration of sex/gender in Cochrane reviews will limit their applicability and capacity to support informed decisions. This study aims to describe the extent to which Cochrane reviews of interventions for preventing HAIs consider sex and gender.

METHODS

Methodology study appraising Cochrane reviews of interventions to prevent HAIs.

SEARCH METHODS

Cochrane Database of Systematic Reviews from 1995 (launch of the journal) to 31 December 2016. Two authors independently extracted data with EPPI-Reviewer 4 software, and independently appraised the sex/gender content of the reviews with the Sex and Gender Appraisal Tool for Systematic Reviews (SGAT-SR).

RESULTS

This study included 113 reviews assessing the effects of interventions for preventing HAIs. 100 reviews (88%) used at least one sex or gender-related term. The terminology used was heterogeneous, being "sex" the term used in more reviews (51%). No review defined neither sex nor gender. Thus we could not assess the definitions provided. Consideration of sex and gender was practically absent in the included reviews; in fact, no review met all the applicable items of the SGAT-SR, and 51 reviews (50%) fulfilled no item. No review provided a complete description of the sex and the gender of the samples of the included studies. Only ten reviews (10%) planned to perform sex- and gender-based analysis and only three (3%) could complete the analysis. The method chosen was always the subgroup analysis based on sex (one review) or gender (two reviews). Three reviews (3%) considered sex or gender-related findings in the conclusions.

CONCLUSION

Consideration of sex and gender in Cochrane reviews of interventions for preventing HAIs was practically absent. This lack of attention to sex and gender reduces the quality of Cochrane reviews, and their applicability for all people: women and men, boys and girls, and people of diverse gender identities. Cochrane should attempt to address the shortfalls detected.

Scope and quality of Cochrane reviews of nutrition interventions: a cross-sectional study

BACKGROUND

All countries face significant challenges from complex manifestations of malnutrition, which affects one in three people globally. Systematic reviews provide ready-to-use syntheses of quality-appraised evidence to inform decision-making for actions. To enhance the utility and quality of future Cochrane nutrition evidence, we described the scope and quality of all nutrition systematic reviews in the Cochrane Database of Systematic Reviews (CDSR).

METHODS

We screened all active CDSR records (31 July 2015) to identify reviews and protocols using pre-specified eligibility criteria and definitions. Duplicate, independent data extraction included criteria for inclusion of studies in completed reviews (PICOS). We assessed methodological quality (AMSTAR), use of GRADE, mapped reviews against 2013 Global Burden of Disease data, and categorised the paradigm (medical, lifestyle and socio-ecological) of the review question. We analysed our results using descriptive statistics.

RESULTS

We screened 8484 records, and included 470 (8%) completed reviews (in 45 Cochrane Review Groups (CRGs)) and 169 (7%) protocols (in 41 CRGs) published by 47 of 53 CRGs with reviews. Most completed reviews were produced by the Pregnancy and Childbirth (n = 73), Neonatal (n = 64), Metabolic and Endocrine Disorders (n = 33), Developmental, Psychosocial and Learning Problems (n = 26), Kidney and Transplant (n = 18) and Heart (n = 18) CRGs. Only 27% (n = 129) of reviews had searches for new studies in 2013 or thereafter. Supplementation/supplement interventions were most common (50%; n = 235; majority with micronutrients; 73%, n = 173), followed by food interventions (20%; n = 95). All reviews included randomised controlled trials; about 5% included other designs; 25% used GRADE; the median AMSTAR score was 9 (interquartile range: 7 to 10), 51% were high (AMSTAR 9-11) and 49% moderate (AMSTAR 5-8) quality. More than 80% framed questions using a medical paradigm. For top causes of years-of-life-lost, most reviews addressed preterm birth, diabetes and ischaemic heart disease; for leading risk factors for disability-adjusted-life-years, most targeted childhood undernutrition and high body mass index.

CONCLUSIONS

Nutrition reviews comprised 8% of active CDSR records, were widely distributed across nearly all CRGs and reflected the double nutrition burden. This analysis presents a comprehensive description of the scope and quality of Cochrane nutrition reviews, and identifies gaps for future activities to support actions to address the nutrition burden, in line with the current nutrition agenda and impetus.

Improving the adaptability of WHO evidence-informed guidelines for nutrition actions: results of a mixed methods evaluation

BACKGROUND

Over the past decade, the World Health Organization (WHO) has implemented a standardized, evidence-informed guideline development process to assure technically sound and policy-relevant guidelines. This study is an independent evaluation of the adaptability of the guidelines produced by the Evidence and Programme Guidance unit, at the Department of Nutrition for Health and Development (NHD). The study systematizes the lessons learned by the NHD group at WHO.

METHODS

We used a mixed methods approach to determine the adaptability of the nutrition guidelines. Adaptability was defined as having two components; methodological quality and implementability of guidelines. Additionally, we gathered recommendations to improve future guideline development in nutrition actions for health and development. Data sources for this evaluation were official documentation and feedback (both qualitative and quantitative) from key stakeholders involved in the development of nutrition guidelines. The qualitative data was collected through a desk review and two waves of semi-structured interviews (n = 12) and was analyzed through axial coding. Guideline adaptability was assessed quantitatively using two standardized instruments completed by key stakeholders. The Appraisal Guideline for Research and Evaluation questionnaire, version II was used to assess guideline quality (n = 6), while implementability was assessed with the electronic version of the GuideLine Implementability Appraisal (n = 7).

RESULTS

The nutrition evidence-informed guideline development process has several strengths, among them are the appropriate management of conflicts of interest of guideline developers and the systematic use of high-quality evidence to inform the recommendations. These features contribute to increase the methodological quality of the guidelines. The key areas for improvement are the limited implementability of the recommendations, the lack of explicit and precise implementation advice in the guidelines and challenges related to collaborative work within interdisciplinary groups.

CONCLUSIONS

Overall, our study found that the nutrition evidence-informed guidelines are of good methodological quality but that the implementability requires improvement. The recommendations to improve guideline adaptability address the guideline content, the dynamics shaping interdisciplinary work, and actions for implementation feasibility. As WHO relies heavily on a standardized procedure to develop guidelines, the lessons learned may be applicable to guideline development across the organization and to other groups developing guidelines.

Options for basing Dietary Reference Intakes (DRIs) on chronic disease endpoints: report from a joint US-/Canadian-sponsored working group

Dietary Reference Intakes (DRIs) are used in Canada and the United States in planning and assessing diets of apparently healthy individuals and population groups. The approaches used to establish DRIs on the basis of classical nutrient deficiencies and/or toxicities have worked well. However, it has proved to be more challenging to base DRI values on chronic disease endpoints; deviations from the traditional framework were often required, and in some cases, DRI values were not established for intakes that affected chronic disease outcomes despite evidence that supported a relation. The increasing proportions of elderly citizens, the growing prevalence of chronic diseases, and the persistently high prevalence of overweight and obesity, which predispose to chronic disease, highlight the importance of understanding the impact of nutrition on chronic disease prevention and control. A multidisciplinary working group sponsored by the Canadian and US government DRI steering committees met from November 2014 to April 2016 to identify options for addressing key scientific challenges encountered in the use of chronic disease endpoints to establish reference values. The working group focused on 3 key questions: 1) What are the important evidentiary challenges for selecting and using chronic disease endpoints in future DRI reviews, 2) what intake-response models can future DRI committees consider when using chronic disease endpoints, and 3) what are the arguments for and against continuing to include chronic disease endpoints in future DRI reviews? This report outlines the range of options identified by the working group for answering these key questions, as well as the strengths and weaknesses of each option.

Evidence from Toxicology: The Most Essential Science for Prevention

BACKGROUND

The most essential goal of medicine and public health is to prevent harm (primum non nocere). This goal is only fully achieved with primary prevention, which requires us to identify and prevent harms prior to human exposure through research and testing that does not involve human subjects. For that reason, public health policies place considerable reliance on nonhuman toxicological studies. However, toxicology as a field has often not produced efficient and timely evidence for decision making in public health. In response to this, the U.S. National Research Council called for the adoption of evidence-based methods and systematic reviews in regulatory decision making. The U.S. Environmental Protection Agency (EPA), the Food and Drug Administration (FDA), and the European Food Safety Agency (EFSA) have recently endorsed these methods in their assessments of safety and risk.

OBJECTIVES

In this commentary we summarize challenges and problems in current practices in toxicology as applied to decision making. We compare these practices with the principles and methods utilized in evidence-based medicine and health care, with emphasis on the record of the Cochrane Collaboration.

DISCUSSION

We propose a stepwise strategy to support the development, validation, and application of evidence-based toxicology (EBT). We discuss current progresses in this field produced by the Office of Health Assessment and Translation (OHAT) of the National Toxicology Program and the Navigation Guide works. We propose that adherence to the Cochrane principles is a fundamental prerequisite for the development and implementation of EBT.

CONCLUSION

The adoption of evidence-based principles and methods will enhance the validity, transparency, efficiency, and acceptance of toxicological evidence, with benefits in terms of reducing delays and costs for all stakeholders (researchers, consumers, regulators, and industry).

CITATION

Mandrioli D, Silbergeld EK. 2016. Evidence from toxicology: the most essential science for prevention. Environ Health Perspect 124:6-11; http://dx.doi.org/10.1289/ehp.1509880.

Considering the value of dietary assessment data in informing nutrition-related health policy

Dietary assessment has long been known to be challenged by measurement error. A substantial amount of literature on methods for determining the effects of error on causal inference has accumulated over the past decades. These methods have unrealized potential for improving the validity of data collected for research studies and national nutritional surveillance, primarily through the NHANES. Recently, the validity of dietary data has been called into question. Arguments against using dietary data to assess diet-health relations or to inform the nutrition policy debate are subject to flaws that fall into 2 broad areas: 1) ignorance or misunderstanding of methodologic issues; and 2) faulty logic in drawing inferences. Nine specific issues are identified in these arguments, indicating insufficient grasp of the methods used for assessing diet and designing nutritional epidemiologic studies. These include a narrow operationalization of validity, failure to properly account for sources of error, and large, unsubstantiated jumps to policy implications. Recent attacks on the inadequacy of 24-h recall-derived data from the NHANES are uninformative regarding effects on estimating risk of health outcomes and on inferences to inform the diet-related health policy debate. Despite errors, for many purposes and in many contexts, these dietary data have proven to be useful in addressing important research and policy questions. Similarly, structured instruments, such as the food frequency questionnaire, which is the mainstay of epidemiologic literature, can provide useful data when errors are measured and considered in analyses.

Use and applications of systematic reviews in public health nutrition

Decisions related to a spectrum of nutrition-related public health and clinical concerns must consider many factors and are best informed by evaluating the totality and quality of the evidence. Systematic review (SR) is a structured process to evaluate, compare, and synthesize relevant evidence for the SR-specific question(s). Applications of SR are exemplified here through the discussion of four case studies: research agendas, nutrient reference intakes, dietary guidance, and practice guidelines. Concerns that SR cannot be effectively applied to nutrition evidence because of the lack of an unexposed comparator and the complex homeostasis in nutrition are discussed. Central to understanding the applicability of SR is its flexibility in defining key inclusion criteria and rigorous elements as appropriate for the SR-specific question(s). Through the reduction of bias and random error by explicit, reproducible, comprehensive, and rigorous examination of all of the evidence, SR informs the scientific judgment needed for sound evidence-based public health nutrition.

From biological to program efficacy: promoting dialogue among the research, policy, and program communities

The biological efficacy of nutritional supplements to complement usual diets in poor populations is well established. This knowledge rests on decades of methodologic research development and, more recently, on codification of methods to compile and interpret results across studies. The challenge now is to develop implementation (delivery) science knowledge and achieve a similar consensus on efficacy criteria for the delivery of these nutrients by public health and other organizations. This requires analysis of the major policy instruments for delivery and well-designed program delivery studies that examine the flow of a nutrient through a program impact pathway. This article discusses the differences between biological and program efficacy, and why elucidating the fidelity of delivery along the program impact pathways is essential for implementing a program efficacy trial and for assessing its internal and external validity. Research on program efficacy is expanding, but there is a lack of adequate frameworks to facilitate the process of harmonizing concepts and vocabulary, which is essential for communication among scientists, policy planners, and program implementers. There is an urgent need to elaborate these frameworks at national and program levels not only for program efficacy studies but also for the broader research agenda to support and improve the science of delivering adequate nutrition to those who need it most.