Methodological Quality Assessment of Review Articles Evaluating Interventions to Improve Microbial Food Safety

Levels of Evidence, Quality Assessment, and Risk of Bias: Evaluating the Internal Validity of Primary Research

Clinical decisions in human and veterinary medicine should be based on the best available evidence. The results of primary research are an important component of that evidence base. Regardless of whether assessing studies for clinical case management, developing clinical practice guidelines, or performing systematic reviews, evidence from primary research should be evaluated for internal validity i.e., whether the results are free from bias (reflect the truth). Three broad approaches to evaluating internal validity are available: evaluating the potential for bias in a body of literature based on the study designs employed (levels of evidence), evaluating whether key study design features associated with the potential for bias were employed (quality assessment), and applying a judgement as to whether design elements of a study were likely to result in biased results given the specific context of the study (risk of bias assessment). The level of evidence framework for assessing internal validity assumes that internal validity can be determined based on the study design alone, and thus makes the strongest assumptions. Risk of bias assessments involve an evaluation of the potential for bias in the context of a specific study, and thus involve the least assumptions about internal validity. Quality assessment sits somewhere between the assumptions of these two. Because risk of bias assessment involves the least assumptions, this approach should be used to assess internal validity where possible. However, risk of bias instruments are not available for all study designs, some clinical questions may be addressed using multiple study designs, and some instruments that include an evaluation of internal validity also include additional components (e.g., evaluation of comprehensiveness of reporting, assessments of feasibility or an evaluation of external validity). Therefore, it may be necessary to embed questions related to risk of bias within existing quality assessment instruments. In this article, we overview the approaches to evaluating internal validity, highlight the current complexities, and propose ideas for approaching assessments of internal validity.

Age Dependence of Antimicrobial Resistance Among Fecal Bacteria in Animals: A Scoping Review

Introduction: A phenomenon of decreasing antimicrobial resistance (AMR) among fecal bacteria as food animals age has been noted in multiple field studies. We conducted a scoping review to summarize the extent, range, and nature of research activity and the data for the following question: "does AMR among enteric/fecal bacteria predictably shift as animals get older?". Methods: This review followed a scoping review methodology framework. Pertinent literature published up until November 2018 for all animals (except humans) was retrieved using keyword searches in two online databases, namely, PubMed® and the Web of Science™ Core Collection, without filtering publication date, geographic location, or language. Data were extracted from the included studies, summarized, and plotted. Study quality was also assessed using the Grades of Recommendation, Assessment, Development, and Evaluation (GRADE) guidelines for all included papers. Results: The publications with detailed relevant data (n = 62) in food animals, poultry, and dogs were identified. These included longitudinal studies (n = 32), cross-sectional studies of different age groups within one food animal production system or small-animal catchment area (n = 16), and experimental or diet trials (n = 14). A decline in host-level prevalence and/or within-host abundance of AMR among fecal bacteria in production beef, dairy cattle, and swine was reported in nearly two-thirds (65%) of the identified studies in different geographic locations from the 1970's to 2018. Mixed results, with AMR abundance among fecal bacteria either increasing or decreasing with age, have been reported in poultry (broiler chicken, layer, and grow-out turkey) and dogs. Conclusions: Quantitative synthesis of the data suggests that the age-dependent AMR phenomenon in cattle and swine is observed irrespective of geographic location and specific production practices. It is unclear whether the phenomenon predates or is related to antimicrobial drug use. However, almost 50% of the identified studies predate recent changes in antimicrobial drug use policy and regulations in food animals in the United States and elsewhere.

A scoping review of the evidence for efficacy of acupuncture in companion animals

Acupuncture has become increasingly popular in veterinary medicine. Within the scientific literature there is debate regarding its efficacy. Due to the complex nature of acupuncture, a scoping review was undertaken to identify and categorize the evidence related to acupuncture in companion animals (dogs, cats, and horses). Our search identified 843 relevant citations. Narrative reviews represented the largest proportion of studies (43%). We identified 179 experimental studies and 175 case reports/case series that examined the efficacy of acupuncture. Dogs were the most common subjects in the experimental trials. The most common indication for use was musculoskeletal conditions, and the most commonly evaluated outcome categories among experimental trials were pain and cardiovascular parameters. The limited number of controlled trials and the breadth of indications for use, outcome categories, and types of acupuncture evaluated present challenges for future systematic reviews or meta-analyses. There is a need for high-quality randomized controlled trials addressing the most common clinical uses of acupuncture, and using consistent and clinically relevant outcomes, to inform conclusions regarding the efficacy of acupuncture in companion animals.

The Development of a Checklist to Enhance Methodological Quality in Intervention Programs

The methodological quality of primary studies is an important issue when performing meta-analyses or systematic reviews. Nevertheless, there are no clear criteria for how methodological quality should be analyzed. Controversies emerge when considering the various theoretical and empirical definitions, especially in relation to three interrelated problems: the lack of representativeness, utility, and feasibility. In this article, we (a) systematize and summarize the available literature about methodological quality in primary studies; (b) propose a specific, parsimonious, 12-items checklist to empirically define the methodological quality of primary studies based on a content validity study; and (c) present an inter-coder reliability study for the resulting 12-items. This paper provides a precise and rigorous description of the development of this checklist, highlighting the clearly specified criteria for the inclusion of items and a substantial inter-coder agreement in the different items. Rather than simply proposing another checklist, however, it then argues that the list constitutes an assessment tool with respect to the representativeness, utility, and feasibility of the most frequent methodological quality items in the literature, one that provides practitioners and researchers with clear criteria for choosing items that may be adequate to their needs. We propose individual methodological features as indicators of quality, arguing that these need to be taken into account when designing, implementing, or evaluating an intervention program. This enhances methodological quality of intervention programs and fosters the cumulative knowledge based on meta-analyses of these interventions. Future development of the checklist is discussed.

Introduction to Preharvest Food Safety

This introductory article provides an overview of preharvest food safety activities and initiatives for the past 15 years. The section on traditional areas of preharvest food safety focuses on significant scientific advancements that are a culmination of collaborative efforts (both public health and agriculture) and significant research results. The highlighted advancements provide the foundation for exploring future preharvest areas and for improving and focusing on more specific intervention/control/prevention strategies. Examples include Escherichia coli and cattle, Salmonella and Campylobacter in poultry, and interventions and prevention and control programs. The section on "nontraditional" preharvest food safety areas brings attention to potential emerging food safety issues and to future food safety research directions. These include organic production, the FDA's Produce Rule (water and manure), genomic sequencing, antimicrobial resistance, and performance metrics. The concluding section emphasizes important themes such as strategic planning, coordination, epidemiology, and the need for understanding food safety production as a continuum. Food safety research, whether at the pre- or postharvest level, will continue to be a fascinating complex web of foodborne pathogens, risk factors, and scientific and policy interactions. Food safety priorities and research must continue to evolve with emerging global issues, emerging technologies, and methods but remain grounded in a multidisciplinary, collaborative, and systematic approach.

Implications of applying methodological shortcuts to expedite systematic reviews: three case studies using systematic reviews from agri-food public health

Background

The rapid review is an approach to synthesizing research evidence when a shorter timeframe is required. The implications of what is lost in terms of rigour, increased bias and accuracy when conducting a rapid review have not yet been elucidated.

Methods

We assessed the potential implications of methodological shortcuts on the outcomes of three completed systematic reviews addressing agri‐food public health topics. For each review, shortcuts were applied individually to assess the impact on the number of relevant studies included and whether omitted studies affected the direction, magnitude or precision of summary estimates from meta‐analyses.

Results

In most instances, the shortcuts resulted in at least one relevant study being omitted from the review. The omission of studies affected 39 of 143 possible meta‐analyses, of which 14 were no longer possible because of insufficient studies (<2). When meta‐analysis was possible, the omission of studies generally resulted in less precise pooled estimates (i.e. wider confidence intervals) that did not differ in direction from the original estimate.

Conclusions

The three case studies demonstrated the risk of missing relevant literature and its impact on summary estimates when methodological shortcuts are applied in rapid reviews. © 2016 The Authors. Research Synthesis Methods Published by John Wiley & Sons Ltd.

Application of a Rapid Knowledge Synthesis and Transfer Approach To Assess the Microbial Safety of Low-Moisture Foods

Low-moisture foods (LMF) are increasingly implicated in outbreaks of foodborne illness, resulting in a significant public health burden. To inform the development of a new Codex Alimentarius code of hygienic practice for LMF, we applied a rapid knowledge synthesis and transfer approach to review global research on the burden of illness, prevalence, and interventions to control nine selected microbial hazards in eight categories of LMF. Knowledge synthesis methods included an integrated scoping review (search strategy, relevance screening and confirmation, and evidence mapping), systematic review (detailed data extraction), and meta-analysis of prevalence data. Knowledge transfer of the results was achieved through multiple reporting formats, including evidence summary cards. We identified 214 unique outbreaks and 204 prevalence and 126 intervention studies. Cereals and grains (n = 142) and Salmonella (n = 278) were the most commonly investigated LMF and microbial hazard categories, respectively. Salmonella was implicated in the most outbreaks (n = 96, 45%), several of which were large and widespread, resulting in the most hospitalizations (n = 895, 89%) and deaths (n = 14, 74%). Salmonella had a consistently low prevalence across all LMF categories (0 to 3%), but the prevalence of other hazards (e.g., Bacillus cereus) was highly variable. A variety of interventions were investigated in small challenge trials. Key knowledge gaps included underreporting of LMF outbreaks, limited reporting of microbial levels in prevalence studies, and a lack of intervention efficacy research under commercial conditions. Summary cards were a useful knowledge transfer format to inform complementary risk ranking activities. This review builds upon previous work in this area by synthesizing a broad range of evidence using a structured, transparent, and integrated approach to provide timely evidence informed inputs into international guidelines.

A guide for developing plain-language and contextual summaries of systematic reviews in agri-food public health

The application of systematic reviews is increasing in the agri-food public health sector to investigate the efficacy of policy-relevant interventions. In order to enhance the uptake and utility of these reviews for decision-making, there is a need to develop summary formats that are written in plain language and incorporate supporting contextual information. The objectives of this study were (1) to develop a guideline for summarizing systematic reviews in one- and three-page formats, and (2) to apply the guideline on two published systematic reviews that investigated the efficacy of vaccination and targeted feed and water additives to reduce Salmonella colonization in broiler chickens. Both summary formats highlight the key systematic review results and implications in plain language. Three-page summaries also incorporated four categories of contextual information (cost, availability, practicality, and other stakeholder considerations) to complement the systematic review findings. We collected contextual information through structured rapid reviews of the peer-reviewed and gray literature and by conducting interviews with 12 topic specialists. The overall utility of the literature searches and interviews depended on the specific intervention topic and contextual category. In general, interviews with topic specialists were the most useful and efficient method of gathering contextual information. Preliminary evaluation with five end-users indicated positive feedback on the summary formats. We estimate that one-page summaries could be developed by trained science-to-policy professionals in 3-5 days, while three-page summaries would require additional resources and time (e.g., 2-4 weeks). Therefore, one-page summaries are more suited for routine development, while three-page summaries could be developed for a more limited number of high-priority reviews. The summary guideline offers a structured and transparent approach to support the utilization of systematic reviews in decision-making in this sector. Future research is necessary to evaluate the utility of these summary formats for a variety of end-users in different contexts.

Introduction to systematic reviews in animal agriculture and veterinary medicine

This article is the first in a series of six articles related to systematic reviews in animal agriculture and veterinary medicine. In this article, we overview the methodology of systematic reviews and provide a discussion of their use. Systematic reviews differ qualitatively from traditional reviews by explicitly defining a specific review question, employing methods to reduce bias in the selection and inclusion of studies that address the review question (including a systematic and specified search strategy, and selection of studies based on explicit eligibility criteria), an assessment of the risk of bias for included studies and objectively summarizing the results qualitatively or quantitatively (i.e. via meta-analysis). Systematic reviews have been widely used to address human healthcare questions and are increasingly being used in veterinary medicine. Systematic reviews can provide veterinarians and other decision-makers with a scientifically defensible summary of the current state of knowledge on a topic without the need for the end-user to read the vast amount of primary research related to that topic.

A scoping review of published research on the relinquishment of companion animals

Globally, large populations of companion animals are relinquished each year. The purpose of this scoping review was to identify all published research investigating companion-animal relinquishment to map out and evaluate research gaps, needs, and opportunities. A comprehensive search strategy was implemented in 4 online databases, identified citations were screened, and relevant articles were procured and characterized. From 6,848 unique citations identified, 192 were confirmed relevant, including 115 primary-research articles and 77 reviews and commentaries. The majority of these articles originated from the United States (131; 68.2%); 74 (38.5%) of them have been published since 2006. Among the primary-research articles, 84 (73.0%) investigated reasons for companion-animal relinquishment. The most commonly studied reasons were aggressive companion-animal behaviors (49; 58.3%); moving, rental, or housing issues (45; 53.6%); and caretaker personal issues (42; 50.0%). Only 17 primary-research articles investigated interventions to prevent companion-animal relinquishment. The quantity of research into reasons for relinquishment highlights an opportunity for future knowledge-synthesis activities in this area, including systematic review and meta-analysis. In comparison, the limited research into interventions identifies a priority for new research.

The application of knowledge synthesis methods in agri-food public health: recent advancements, challenges and opportunities

Knowledge synthesis refers to the integration of findings from individual research studies on a given topic or question into the global knowledge base. The application of knowledge synthesis methods, particularly systematic reviews and meta-analysis, has increased considerably in the agri-food public health sector over the past decade and this trend is expected to continue. The objectives of our review were: (1) to describe the most promising knowledge synthesis methods and their applicability in agri-food public health, and (2) to summarize the recent advancements, challenges, and opportunities in the use of systematic review and meta-analysis methods in this sector. We performed a structured review of knowledge synthesis literature from various disciplines to address the first objective, and used comprehensive insights and experiences in applying these methods in the agri-food public health sector to inform the second objective. We describe five knowledge synthesis methods that can be used to address various agri-food public health questions or topics under different conditions and contexts. Scoping reviews describe the main characteristics and knowledge gaps in a broad research field and can be used to evaluate opportunities for prioritizing focused questions for related systematic reviews. Structured rapid reviews are streamlined systematic reviews conducted within a short timeframe to inform urgent decision-making. Mixed-method and qualitative reviews synthesize diverse sources of contextual knowledge (e.g. socio-cognitive, economic, and feasibility considerations). Systematic reviews are a structured and transparent method used to summarize and synthesize literature on a clearly-defined question, and meta-analysis is the statistical combination of data from multiple individual studies. We briefly describe and discuss key advancements in the use of systematic reviews and meta-analysis, including: risk-of-bias assessments; an overall quality-of-evidence approach; engagement of stakeholders; Bayesian, multivariate, and network meta-analysis; and synthesis of diagnostic test accuracy studies. We also highlight several challenges and opportunities in the conduct of systematic reviews (e.g. inclusion of grey literature, minimizing language bias, and optimizing search strategies) and meta-analysis (e.g. inclusion of observational studies and approaches to address the insufficient reporting of data and significant heterogeneity). Many of these developments have yet to be comprehensively applied and evaluated in an agri-food public health context, and more research is needed in this area. There is a need to strengthen knowledge synthesis capacity and infrastructure at the regional, national, and international levels in this sector to ensure that the best available knowledge is used to inform future decision-making about agri-food public health issues.

Scoping review on search queries and social media for disease surveillance: a chronology of innovation

Background

The threat of a global pandemic posed by outbreaks of influenza H5N1 (1997) and Severe Acute Respiratory Syndrome (SARS, 2002), both diseases of zoonotic origin, provoked interest in improving early warning systems and reinforced the need for combining data from different sources. It led to the use of search query data from search engines such as Google and Yahoo! as an indicator of when and where influenza was occurring. This methodology has subsequently been extended to other diseases and has led to experimentation with new types of social media for disease surveillance.

Objective

The objective of this scoping review was to formally assess the current state of knowledge regarding the use of search queries and social media for disease surveillance in order to inform future work on early detection and more effective mitigation of the effects of foodborne illness.

Methods

Structured scoping review methods were used to identify, characterize, and evaluate all published primary research, expert review, and commentary articles regarding the use of social media in surveillance of infectious diseases from 2002-2011.

Results

Thirty-two primary research articles and 19 reviews and case studies were identified as relevant. Most relevant citations were peer-reviewed journal articles (29/32, 91%) published in 2010-11 (28/32, 88%) and reported use of a Google program for surveillance of influenza. Only four primary research articles investigated social media in the context of foodborne disease or gastroenteritis. Most authors (21/32 articles, 66%) reported that social media-based surveillance had comparable performance when compared to an existing surveillance program. The most commonly reported strengths of social media surveillance programs included their effectiveness (21/32, 66%) and rapid detection of disease (21/32, 66%). The most commonly reported weaknesses were the potential for false positive (16/32, 50%) and false negative (11/32, 34%) results. Most authors (24/32, 75%) recommended that social media programs should primarily be used to support existing surveillance programs.

Conclusions

The use of search queries and social media for disease surveillance are relatively recent phenomena (first reported in 2006). Both the tools themselves and the methodologies for exploiting them are evolving over time. While their accuracy, speed, and cost compare favorably with existing surveillance systems, the primary challenge is to refine the data signal by reducing surrounding noise. Further developments in digital disease surveillance have the potential to improve sensitivity and specificity, passively through advances in machine learning and actively through engagement of users. Adoption, even as supporting systems for existing surveillance, will entail a high level of familiarity with the tools and collaboration across jurisdictions.

The information infrastructure that supports evidence-based veterinary medicine: a comparison with human medicine

In human medicine, the information infrastructure that supports the knowledge translation processes of exchange, synthesis, dissemination, and application of the best clinical intervention research has developed significantly in the past 15 years, facilitating the uptake of research evidence by clinicians as well as the practice of evidence-based medicine. Seven of the key elements of this improved information infrastructure are clinical trial registries, research reporting standards, systematic reviews, organizations that support the production of systematic reviews, the indexing of clinical intervention research in MEDLINE, clinical search filters for MEDLINE, and point-of-care decision support information resources. The objective of this paper is to describe why these elements are important for evidence-based medicine, the key developments and issues related to these seven information infrastructure elements in human medicine, how these 7 elements compare with the corresponding infrastructure elements in veterinary medicine, and how all of these factors affect the translation of clinical intervention research into clinical practice. A focused search of the Ovid MEDLINE database was conducted for English language journal literature published between 2000 and 2010. Two bibliographies were consulted and selected national and international Web sites were searched using Google. The literature reviewed indicates that the information infrastructure supporting evidence-based veterinary medicine practice in all of the 7 elements reviewed is significantly underdeveloped in relation to the corresponding information infrastructure in human medicine. This lack of development creates barriers to the timely translation of veterinary medicine research into clinical practice and also to the conduct of both primary clinical intervention research and synthesis research.

Using the systematic review methodology to evaluate factors that influence the persistence of influenza virus in environmental matrices

Understanding factors that influence persistence of influenza virus in an environment without host animals is critical to appropriate decision-making for issues such as quarantine downtimes, setback distances, and eradication programs in livestock production systems. This systematic review identifies literature describing persistence of influenza virus in environmental samples, i.e., air, water, soil, feces, and fomites. An electronic search of PubMed, CAB, AGRICOLA, Biosis, and Compendex was performed, and citation relevance was determined according to the aim of the review. Quality assessment of relevant studies was performed using criteria from experts in virology, disease ecology, and environmental science. A total of 9,760 abstracts were evaluated, and 40 appeared to report the persistence of influenza virus in environmental samples. Evaluation of full texts revealed that 19 of the 40 studies were suitable for review, as they described virus concentration measured at multiple sampling times, with viruses detectable at least twice. Seven studies reported persistence in air (six published before 1970), seven in water (five published after 1990), two in feces, and three on surfaces. All three fomite and five air studies addressed human influenza virus, and all water and feces studies pertained to avian influenza virus. Outcome measurements were transformed to half-lives, and resultant multivariate mixed linear regression models identified influenza virus surviving longer in water than in air. Temperature was a significant predictor of persistence over all matrices. Salinity and pH were significant predictors of persistence in water conditions. An assessment of the methodological quality review of the included studies revealed significant gaps in reporting critical aspects of study design.

The methodological soundness of literature reviews addressing three potential zoonotic public health issues

Literature reviews are important information sources for multiple stakeholders in zoonotic public health with limited time to keep up with the rapid increase in primary research in this field. However, their validity depends on their methodological soundness. The study purpose was to evaluate the methodological soundness of literature reviews in zoonotic public health. Relevant reviews (n = 132) published between January 2000 and August 2006 were identified on three issues: Mycobacterium avium ssp paratuberculosis as a potential cause of Crohn's disease in humans (30 reviews); antimicrobial use in animals as a risk factor for antimicrobial resistance in human pathogens (36); and the zoonotic potential of transmissible spongiform encephalopathies (66). The zoonotic aspect of the issue was the focus of 59 reviews and a subsection of 73 reviews. Two independent reviewers evaluated reviews using 13 criteria: 10 previously validated in the medical field, and three applicable to aetiology research. No review met more than eight of 13 criteria for methodological soundness; two articles met only one criterion. Two reviews described methods for identifying relevant primary research. In only two and four reviews respectively, authors conducted quantitative syntheses of research evidence or reported summarized measures of effect for the zoonotic risk to humans. Recommendations for future research and economic impact were provided in 64 and 10 reviews respectively. In 14 reviews, conclusions exceeded evidence presented. The various review authors' position on the evidence for the zoonotic association and the zoonotic risk to public health were inconsistent for all three issues. Reviews addressing potential zoonotic public health issues lack structured and transparent methodology preventing the end user from assessing the review's validity. These reviews should adhere to structured scientific principles similar to what is used for primary research articles.