1
|
Wang Z, Lu C, Cui L, Fenfen E, Shang W, Wang Z, Song G, Yang K, Li X. Consumption of ultra-processed foods and multiple health outcomes: An umbrella study of meta-analyses. Food Chem 2024; 434:137460. [PMID: 37722333 DOI: 10.1016/j.foodchem.2023.137460] [Citation(s) in RCA: 6] [Impact Index Per Article: 6.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/02/2023] [Revised: 09/05/2023] [Accepted: 09/10/2023] [Indexed: 09/20/2023]
Abstract
Consumption of ultra-processed foods (UPFs) is associated with various adverse health outcomes, which significantly influence the global disease burden. This umbrella review aimed to fill the knowledge gap and guide public health practices by summarizing the association between UPFs and multiple health outcomes. A total of four databases were systematically searched from inception to December 2022, and 14 eligible systematic reviews (SRs) with meta-analyses (MAs) were identified. The SRs were published in 10 journals from 2020 to 2023, with 54,147-5,750,133 participants and 5-61 studies. The overall corrected covered area (CCA) was corresponded to a slight overlap. The results showed that an increased UPFs consumption is associated with multiple health outcomes (e.g., obesity, diabetes, hypertension, mortality). Only two SRs were "Moderate" regarding the overall methodological quality, while the other twelve were "Low" or "Critically low". Therefore, well-conducted SRs with high-quality prospective cohorts with a particular focus on special populations are needed to verify these findings further.
Collapse
Affiliation(s)
- Ziyi Wang
- Health Technology Assessment Center, Evidence-Based Social Science Research Center, School of Public Health, Lanzhou University, Lanzhou 730000, China; Evidence Based Medicine Center, School of Basic Medical Sciences, Lanzhou University, Lanzhou 730000, China; Key Laboratory of Evidence Based Medicine and Knowledge Translation of Gansu Province, Lanzhou 730000, China
| | - Cuncun Lu
- Institute of Basic Research in Clinical Medicine, China Academy of Chinese Medical Sciences, Beijing 100700, China.
| | - Lu Cui
- Health Technology Assessment Center, Evidence-Based Social Science Research Center, School of Public Health, Lanzhou University, Lanzhou 730000, China; Evidence Based Medicine Center, School of Basic Medical Sciences, Lanzhou University, Lanzhou 730000, China; Key Laboratory of Evidence Based Medicine and Knowledge Translation of Gansu Province, Lanzhou 730000, China
| | - E Fenfen
- Health Technology Assessment Center, Evidence-Based Social Science Research Center, School of Public Health, Lanzhou University, Lanzhou 730000, China; Evidence Based Medicine Center, School of Basic Medical Sciences, Lanzhou University, Lanzhou 730000, China; Key Laboratory of Evidence Based Medicine and Knowledge Translation of Gansu Province, Lanzhou 730000, China
| | - Wenru Shang
- Health Technology Assessment Center, Evidence-Based Social Science Research Center, School of Public Health, Lanzhou University, Lanzhou 730000, China; Evidence Based Medicine Center, School of Basic Medical Sciences, Lanzhou University, Lanzhou 730000, China; Key Laboratory of Evidence Based Medicine and Knowledge Translation of Gansu Province, Lanzhou 730000, China
| | - Zhifei Wang
- Institute of Basic Research in Clinical Medicine, China Academy of Chinese Medical Sciences, Beijing 100700, China
| | - Guihang Song
- Gansu Healthcare Security Administration, Lanzhou 730000, China
| | - Kehu Yang
- Health Technology Assessment Center, Evidence-Based Social Science Research Center, School of Public Health, Lanzhou University, Lanzhou 730000, China; Evidence Based Medicine Center, School of Basic Medical Sciences, Lanzhou University, Lanzhou 730000, China; Key Laboratory of Evidence Based Medicine and Knowledge Translation of Gansu Province, Lanzhou 730000, China.
| | - Xiuxia Li
- Health Technology Assessment Center, Evidence-Based Social Science Research Center, School of Public Health, Lanzhou University, Lanzhou 730000, China; Evidence Based Medicine Center, School of Basic Medical Sciences, Lanzhou University, Lanzhou 730000, China; Key Laboratory of Evidence Based Medicine and Knowledge Translation of Gansu Province, Lanzhou 730000, China.
| |
Collapse
|
2
|
Siedler MR, Harris KN, Rodriguez C, Lewis MH, Semidey-Lamadrid P, Stratton MT, Blacutt M, Hosseini Z, Falck-Ytter Y, Mustafa RA, Sultan S, Dahm P, Morgan RL, Murad MH. Certainty of Evidence Assessment in Systematic Reviews Published by High-Impact Sports Science Journals: A Meta-epidemiological Study. Sports Med 2024; 54:473-484. [PMID: 37776465 DOI: 10.1007/s40279-023-01941-x] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 09/19/2023] [Indexed: 10/02/2023]
Abstract
BACKGROUND Assessing certainty of evidence is a key element of any systematic review. The aim of this meta-epidemiology study was to understand the frequency and ways with which certainty of evidence is assessed in contemporary systematic reviews published in high-impact sports science journals. METHODS We searched PubMed and relevant journal web sites from 1 August 2016 to 11 October 2022 for systematic reviews published in the top-ten highest-impact journals within the 2020 Journal Citation Report for the Sports Sciences category. Pairs of independent reviewers screened items using a priori established criteria. RESULTS Of 1250 eligible documents, 258 (20.6%) assessed the certainty of evidence, defined as using two or more distinct domains to provide an overall rating of the trustworthiness of findings across studies. Nine methods were cited for assessing certainty, with the most common being the Grading of Recommendations Assessment, Development, and Evaluation (GRADE) approach (61.6%). The proportion of systematic reviews assessing certainty of evidence appeared to increase over the 6-year timeframe analyzed. Across all reviews analyzed, a large majority addressed the domains of risk of bias, imprecision, and inconsistency of the results. Other certainty domains including indirectness/applicability were less commonly assessed. DISCUSSION Only one in five recent contemporary systematic reviews in the field of exercise and sports science assessed certainty of evidence. Organizational and institutional education on methods for assessing evidence may help further increase uptake of these methods and improve both the quality and clinical impact of systematic reviews in the field.
Collapse
Affiliation(s)
- Madelin R Siedler
- Evidence Foundation, Cleveland, OH, USA
- Department of Kinesiology and Sport Management, Lubbock, TX, USA
| | - Katie N Harris
- Department of Kinesiology and Sport Management, Lubbock, TX, USA
| | | | - Megan H Lewis
- Department of Health and Kinesiology, Texas A&M University, College Station, TX, USA
| | | | | | - Miguel Blacutt
- Department of Psychology, University of Notre Dame, Notre Dame, IN, USA
| | - Zeinab Hosseini
- College of Kinesiology, University of Saskatchewan, Saskatoon, SK, Canada
| | - Yngve Falck-Ytter
- Evidence Foundation, Cleveland, OH, USA
- Division of Gastroenterology and Hepatology, Case Western Reserve University School of Medicine, Cleveland, OH, USA
| | - Reem A Mustafa
- Evidence Foundation, Cleveland, OH, USA
- Department of Health Research Methods, Evidence, and Impact, McMaster University, Hamilton, ON, Canada
- Department of Internal Medicine, University of Kansas Medical Center, Kansas City, KS, USA
| | - Shahnaz Sultan
- Evidence Foundation, Cleveland, OH, USA
- Division of Gastroenterology, University of Minnesota, Minneapolis, MN, USA
| | - Philipp Dahm
- Evidence Foundation, Cleveland, OH, USA
- Urology, Minneapolis VA Health Care System, Minneapolis, MN, USA
- Department of Urology, University of Minnesota, Minneapolis, MN, USA
| | - Rebecca L Morgan
- Evidence Foundation, Cleveland, OH, USA
- Department of Health Research Methods, Evidence, and Impact, McMaster University, Hamilton, ON, Canada
- Department of Population and Quantitative Health Sciences, School of Medicine, Case Western Reserve University, Cleveland, OH, USA
| | - M Hassan Murad
- Evidence Foundation, Cleveland, OH, USA.
- Evidence-Based Practice Center, Robert D. and Patricia E. Kern Center for the Science of Health Care Delivery, Mayo Clinic, Rochester, MN, USA.
| |
Collapse
|
3
|
Kolaski K, Logan LR, Ioannidis JPA. Guidance to best tools and practices for systematic reviews. Br J Pharmacol 2024; 181:180-210. [PMID: 37282770 DOI: 10.1111/bph.16100] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/26/2023] [Accepted: 04/26/2023] [Indexed: 06/08/2023] Open
Abstract
Data continue to accumulate indicating that many systematic reviews are methodologically flawed, biased, redundant, or uninformative. Some improvements have occurred in recent years based on empirical methods research and standardization of appraisal tools; however, many authors do not routinely or consistently apply these updated methods. In addition, guideline developers, peer reviewers, and journal editors often disregard current methodological standards. Although extensively acknowledged and explored in the methodological literature, most clinicians seem unaware of these issues and may automatically accept evidence syntheses (and clinical practice guidelines based on their conclusions) as trustworthy. A plethora of methods and tools are recommended for the development and evaluation of evidence syntheses. It is important to understand what these are intended to do (and cannot do) and how they can be utilized. Our objective is to distill this sprawling information into a format that is understandable and readily accessible to authors, peer reviewers, and editors. In doing so, we aim to promote appreciation and understanding of the demanding science of evidence synthesis among stakeholders. We focus on well-documented deficiencies in key components of evidence syntheses to elucidate the rationale for current standards. The constructs underlying the tools developed to assess reporting, risk of bias, and methodological quality of evidence syntheses are distinguished from those involved in determining overall certainty of a body of evidence. Another important distinction is made between those tools used by authors to develop their syntheses as opposed to those used to ultimately judge their work. Exemplar methods and research practices are described, complemented by novel pragmatic strategies to improve evidence syntheses. The latter include preferred terminology and a scheme to characterize types of research evidence. We organize best practice resources in a Concise Guide that can be widely adopted and adapted for routine implementation by authors and journals. Appropriate, informed use of these is encouraged, but we caution against their superficial application and emphasize their endorsement does not substitute for in-depth methodological training. By highlighting best practices with their rationale, we hope this guidance will inspire further evolution of methods and tools that can advance the field.
Collapse
Affiliation(s)
- Kat Kolaski
- Departments of Orthopaedic Surgery, Pediatrics, and Neurology, Wake Forest School of Medicine, Winston-Salem, North Carolina, USA
| | - Lynne Romeiser Logan
- Department of Physical Medicine and Rehabilitation, SUNY Upstate Medical University, Syracuse, New York, USA
| | - John P A Ioannidis
- Departments of Medicine, of Epidemiology and Population Health, of Biomedical Data Science, and of Statistics, and Meta-Research Innovation Center at Stanford (METRICS), Stanford University School of Medicine, Stanford, California, USA
| |
Collapse
|
4
|
Kolaski K, Logan LR, Ioannidis JPA. Guidance to best tools and practices for systematic reviews. Acta Anaesthesiol Scand 2023; 67:1148-1177. [PMID: 37288997 DOI: 10.1111/aas.14295] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/26/2023] [Accepted: 04/26/2023] [Indexed: 06/09/2023]
Abstract
Data continue to accumulate indicating that many systematic reviews are methodologically flawed, biased, redundant, or uninformative. Some improvements have occurred in recent years based on empirical methods research and standardization of appraisal tools; however, many authors do not routinely or consistently apply these updated methods. In addition, guideline developers, peer reviewers, and journal editors often disregard current methodological standards. Although extensively acknowledged and explored in the methodological literature, most clinicians seem unaware of these issues and may automatically accept evidence syntheses (and clinical practice guidelines based on their conclusions) as trustworthy. A plethora of methods and tools are recommended for the development and evaluation of evidence syntheses. It is important to understand what these are intended to do (and cannot do) and how they can be utilized. Our objective is to distill this sprawling information into a format that is understandable and readily accessible to authors, peer reviewers, and editors. In doing so, we aim to promote appreciation and understanding of the demanding science of evidence synthesis among stakeholders. We focus on well-documented deficiencies in key components of evidence syntheses to elucidate the rationale for current standards. The constructs underlying the tools developed to assess reporting, risk of bias, and methodological quality of evidence syntheses are distinguished from those involved in determining overall certainty of a body of evidence. Another important distinction is made between those tools used by authors to develop their syntheses as opposed to those used to ultimately judge their work. Exemplar methods and research practices are described, complemented by novel pragmatic strategies to improve evidence syntheses. The latter include preferred terminology and a scheme to characterize types of research evidence. We organize best practice resources in a Concise Guide that can be widely adopted and adapted for routine implementation by authors and journals. Appropriate, informed use of these is encouraged, but we caution against their superficial application and emphasize their endorsement does not substitute for in-depth methodological training. By highlighting best practices with their rationale, we hope this guidance will inspire further evolution of methods and tools that can advance the field.
Collapse
Affiliation(s)
- Kat Kolaski
- Departments of Orthopaedic Surgery, Pediatrics, and Neurology, Wake Forest School of Medicine, Winston-Salem, North Carolina, USA
| | - Lynne Romeiser Logan
- Department of Physical Medicine and Rehabilitation, SUNY Upstate Medical University, Syracuse, New York, USA
| | - John P A Ioannidis
- Departments of Medicine, of Epidemiology and Population Health, of Biomedical Data Science, and of Statistics, and Meta-Research Innovation Center at Stanford (METRICS), Stanford University School of Medicine, Stanford, California, USA
| |
Collapse
|
5
|
Talukdar JR, Steen JP, Goldenberg JZ, Zhang Q, Vernooij RWM, Ge L, Zeraatkar D, Bała MM, Ball GDC, Thabane L, Johnston BC. Saturated fat, the estimated absolute risk and certainty of risk for mortality and major cancer and cardiometabolic outcomes: an overview of systematic reviews. Syst Rev 2023; 12:179. [PMID: 37777760 PMCID: PMC10541715 DOI: 10.1186/s13643-023-02312-3] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 09/15/2022] [Accepted: 08/08/2023] [Indexed: 10/02/2023] Open
Abstract
OBJECTIVE To assess the impact of reducing saturated fat or fatty foods, or replacing saturated fat with unsaturated fat, carbohydrate or protein, on the risk of mortality and major cancer and cardiometabolic outcomes in adults. METHODS We searched MEDLINE, EMBASE, CINAHL, and references of included studies for systematic reviews and meta-analyses (SRMAs) of randomized controlled trials (RCTs) and observational studies in adults published in the past 10 years. Eligible reviews investigated reducing saturated fat or fatty foods or replacing saturated fat with unsaturated fat, carbohydrate or protein, on the risk of cancer and cardiometabolic outcomes and assessed the certainty of evidence for each outcome using, for example, the GRADE (Grading of Recommendations, Assessment, Development, and Evaluations) approach. We assessed the quality of SRMAs using a modified version of AMSTAR-2. Results were summarized as absolute estimates of effect together with the certainty of effects using a narrative synthesis approach. RESULTS We included 17 SRMAs (13 reviews of observational studies with follow-up 1 to 34 years; 4 reviews of RCTs with follow-up 1 to 17 years). The quality of two-thirds of the SRMAs was critically low to moderate; the main limitations included deficient reporting of study selection, absolute effect estimates, sources of funding, and a priori subgroups to explore heterogeneity. Our included reviews reported > 100 estimates of effect across 11 critically important cancer and cardiometabolic outcomes. High quality SRMAs consistently and predominantly reported low to very low certainty evidence that reducing or replacing saturated fat was associated with a very small risk reduction in cancer and cardiometabolic endpoints. The risk reductions where approximately divided, some being statistically significant and some being not statistically significant. However, based on 2 moderate to high quality reviews, we found moderate certainty evidence for a small but important effect that was statistically significant for two outcomes (total mortality events [20 fewer events per 1000 followed] and combined cardiovascular events [16 fewer per 1000 followed]). Conversely, 4 moderate to high quality reviews showed very small effects on total mortality, with 3 of these reviews showing non-statistically significant mortality effects. CONCLUSION Systematic reviews investigating the impact of SFA on mortality and major cancer and cardiometabolic outcomes almost universally suggest very small absolute changes in risk, and the data is based primarily on low and very low certainty evidence. SYSTEMATIC REVIEW REGISTRATION PROSPERO CRD42020172141.
Collapse
Affiliation(s)
- Jhalok Ronjan Talukdar
- Department of Health Research Methods, Evidence, and Impact, McMaster University, Hamilton, ON, Canada
| | - Jeremy P Steen
- Faculty of Health Sciences, McMaster University, Hamilton, ON, Canada
| | - Joshua Z Goldenberg
- Department of Nutrition, College of Agriculture and Life Sciences, Texas A&M University, College Station, TX, USA
- Helfgott Research Institute, National University of Natural Medicine, Portland, OR, USA
| | - Qian Zhang
- School of Nursing, Faculty of Health Sciences, University of Ottawa, Ottawa, ON, Canada
| | - Robin W M Vernooij
- Department of Nephrology and Hypertension, University Medical Center Utrecht, Utrecht, The Netherlands
- Julius Center for Health Sciences and Primary Care, University Medical Center Utrecht, Utrecht University, Utrecht, The Netherlands
| | - Long Ge
- Department of Health Research Methods, Evidence, and Impact, McMaster University, Hamilton, ON, Canada
- Evidence Based Social Science Research Centre, School of Public Health, Lanzhou University, Lanzhou, China
| | - Dena Zeraatkar
- Department of Health Research Methods, Evidence, and Impact, McMaster University, Hamilton, ON, Canada
- Department of Biomedical Informatics, Harvard Medical School, Boston, MA, USA
| | - Małgorzata M Bała
- Department of Hygiene and Dietetics, Chair of Epidemiology and Preventive Medicine, Jagiellonian University Medical College, Krakow, Poland
| | - Geoff D C Ball
- Department of Pediatrics, Faculty of Medicine & Dentistry, College of Health Sciences, University of Alberta, Edmonton, AB, Canada
| | - Lehana Thabane
- Department of Health Research Methods, Evidence, and Impact, McMaster University, Hamilton, ON, Canada
- Faculty of Health Sciences, University of Johannesburg, Johannesburg, South Africa
- Biostatistics Unit, St Joseph's Healthcare-Hamilton, Hamilton, ON, Canada
| | - Bradley C Johnston
- Department of Nutrition, College of Agriculture and Life Sciences, Texas A&M University, College Station, TX, USA.
- Department of Epidemiology and Biostatistics, School of Public Health, Texas A&M University, College Station, TX, USA.
| |
Collapse
|
6
|
Abstract
Data continue to accumulate indicating that many systematic reviews are methodologically flawed, biased, redundant, or uninformative. Some improvements have occurred in recent years based on empirical methods research and standardization of appraisal tools; however, many authors do not routinely or consistently apply these updated methods. In addition, guideline developers, peer reviewers, and journal editors often disregard current methodological standards. Although extensively acknowledged and explored in the methodological literature, most clinicians seem unaware of these issues and may automatically accept evidence syntheses (and clinical practice guidelines based on their conclusions) as trustworthy. A plethora of methods and tools are recommended for the development and evaluation of evidence syntheses. It is important to understand what these are intended to do (and cannot do) and how they can be utilized. Our objective is to distill this sprawling information into a format that is understandable and readily accessible to authors, peer reviewers, and editors. In doing so, we aim to promote appreciation and understanding of the demanding science of evidence synthesis among stakeholders. We focus on well-documented deficiencies in key components of evidence syntheses to elucidate the rationale for current standards. The constructs underlying the tools developed to assess reporting, risk of bias, and methodological quality of evidence syntheses are distinguished from those involved in determining overall certainty of a body of evidence. Another important distinction is made between those tools used by authors to develop their syntheses as opposed to those used to ultimately judge their work. Exemplar methods and research practices are described, complemented by novel pragmatic strategies to improve evidence syntheses. The latter include preferred terminology and a scheme to characterize types of research evidence. We organize best practice resources in a Concise Guide that can be widely adopted and adapted for routine implementation by authors and journals. Appropriate, informed use of these is encouraged, but we caution against their superficial application and emphasize their endorsement does not substitute for in-depth methodological training. By highlighting best practices with their rationale, we hope this guidance will inspire further evolution of methods and tools that can advance the field.
Collapse
Affiliation(s)
- Kat Kolaski
- Departments of Orthopaedic Surgery, Pediatrics, and Neurology, Wake Forest School of Medicine, Winston-Salem, NC, USA
| | - Lynne Romeiser Logan
- Department of Physical Medicine and Rehabilitation, SUNY Upstate Medical University, Syracuse, NY, USA
| | - John P.A. Ioannidis
- Departments of Medicine, of Epidemiology and Population Health, of Biomedical Data Science, and of Statistics, and Meta-Research Innovation Center at Stanford (METRICS), Stanford University School of Medicine, Stanford, CA, USA
| |
Collapse
|
7
|
Kolaski K, Logan LR, Ioannidis JPA. Guidance to best tools and practices for systematic reviews. BMC Infect Dis 2023; 23:383. [PMID: 37286949 DOI: 10.1186/s12879-023-08304-x] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/21/2023] [Accepted: 05/03/2023] [Indexed: 06/09/2023] Open
Abstract
Data continue to accumulate indicating that many systematic reviews are methodologically flawed, biased, redundant, or uninformative. Some improvements have occurred in recent years based on empirical methods research and standardization of appraisal tools; however, many authors do not routinely or consistently apply these updated methods. In addition, guideline developers, peer reviewers, and journal editors often disregard current methodological standards. Although extensively acknowledged and explored in the methodological literature, most clinicians seem unaware of these issues and may automatically accept evidence syntheses (and clinical practice guidelines based on their conclusions) as trustworthy.A plethora of methods and tools are recommended for the development and evaluation of evidence syntheses. It is important to understand what these are intended to do (and cannot do) and how they can be utilized. Our objective is to distill this sprawling information into a format that is understandable and readily accessible to authors, peer reviewers, and editors. In doing so, we aim to promote appreciation and understanding of the demanding science of evidence synthesis among stakeholders. We focus on well-documented deficiencies in key components of evidence syntheses to elucidate the rationale for current standards. The constructs underlying the tools developed to assess reporting, risk of bias, and methodological quality of evidence syntheses are distinguished from those involved in determining overall certainty of a body of evidence. Another important distinction is made between those tools used by authors to develop their syntheses as opposed to those used to ultimately judge their work.Exemplar methods and research practices are described, complemented by novel pragmatic strategies to improve evidence syntheses. The latter include preferred terminology and a scheme to characterize types of research evidence. We organize best practice resources in a Concise Guide that can be widely adopted and adapted for routine implementation by authors and journals. Appropriate, informed use of these is encouraged, but we caution against their superficial application and emphasize their endorsement does not substitute for in-depth methodological training. By highlighting best practices with their rationale, we hope this guidance will inspire further evolution of methods and tools that can advance the field.
Collapse
Affiliation(s)
- Kat Kolaski
- Departments of Orthopaedic Surgery, Pediatrics, and Neurology, Wake Forest School of Medicine, Winston-Salem, NC, USA.
| | - Lynne Romeiser Logan
- Department of Physical Medicine and Rehabilitation, SUNY Upstate Medical University, Syracuse, NY, USA
| | - John P A Ioannidis
- Departments of Medicine, of Epidemiology and Population Health, of Biomedical Data Science, and of Statistics, and Meta-Research Innovation Center at Stanford (METRICS), Stanford University School of Medicine, Stanford, CA, USA
| |
Collapse
|
8
|
Kolaski K, Logan LR, Ioannidis JPA. Guidance to best tools and practices for systematic reviews. Syst Rev 2023; 12:96. [PMID: 37291658 DOI: 10.1186/s13643-023-02255-9] [Citation(s) in RCA: 10] [Impact Index Per Article: 10.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 10/03/2022] [Accepted: 02/19/2023] [Indexed: 06/10/2023] Open
Abstract
Data continue to accumulate indicating that many systematic reviews are methodologically flawed, biased, redundant, or uninformative. Some improvements have occurred in recent years based on empirical methods research and standardization of appraisal tools; however, many authors do not routinely or consistently apply these updated methods. In addition, guideline developers, peer reviewers, and journal editors often disregard current methodological standards. Although extensively acknowledged and explored in the methodological literature, most clinicians seem unaware of these issues and may automatically accept evidence syntheses (and clinical practice guidelines based on their conclusions) as trustworthy.A plethora of methods and tools are recommended for the development and evaluation of evidence syntheses. It is important to understand what these are intended to do (and cannot do) and how they can be utilized. Our objective is to distill this sprawling information into a format that is understandable and readily accessible to authors, peer reviewers, and editors. In doing so, we aim to promote appreciation and understanding of the demanding science of evidence synthesis among stakeholders. We focus on well-documented deficiencies in key components of evidence syntheses to elucidate the rationale for current standards. The constructs underlying the tools developed to assess reporting, risk of bias, and methodological quality of evidence syntheses are distinguished from those involved in determining overall certainty of a body of evidence. Another important distinction is made between those tools used by authors to develop their syntheses as opposed to those used to ultimately judge their work.Exemplar methods and research practices are described, complemented by novel pragmatic strategies to improve evidence syntheses. The latter include preferred terminology and a scheme to characterize types of research evidence. We organize best practice resources in a Concise Guide that can be widely adopted and adapted for routine implementation by authors and journals. Appropriate, informed use of these is encouraged, but we caution against their superficial application and emphasize their endorsement does not substitute for in-depth methodological training. By highlighting best practices with their rationale, we hope this guidance will inspire further evolution of methods and tools that can advance the field.
Collapse
Affiliation(s)
- Kat Kolaski
- Departments of Orthopaedic Surgery, Pediatrics, and Neurology, Wake Forest School of Medicine, Winston-Salem, NC, USA.
| | - Lynne Romeiser Logan
- Department of Physical Medicine and Rehabilitation, SUNY Upstate Medical University, Syracuse, NY, USA
| | - John P A Ioannidis
- Departments of Medicine, of Epidemiology and Population Health, of Biomedical Data Science, and of Statistics, and Meta-Research Innovation Center at Stanford (METRICS), Stanford University School of Medicine, Stanford, CA, USA
| |
Collapse
|
9
|
Kolaski K, Logan LR, Ioannidis JPA. Guidance to Best Tools and Practices for Systematic Reviews. JBJS Rev 2023; 11:01874474-202306000-00009. [PMID: 37285444 DOI: 10.2106/jbjs.rvw.23.00077] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 06/09/2023]
Abstract
» Data continue to accumulate indicating that many systematic reviews are methodologically flawed, biased, redundant, or uninformative. Some improvements have occurred in recent years based on empirical methods research and standardization of appraisal tools; however, many authors do not routinely or consistently apply these updated methods. In addition, guideline developers, peer reviewers, and journal editors often disregard current methodological standards. Although extensively acknowledged and explored in the methodological literature, most clinicians seem unaware of these issues and may automatically accept evidence syntheses (and clinical practice guidelines based on their conclusions) as trustworthy.» A plethora of methods and tools are recommended for the development and evaluation of evidence syntheses. It is important to understand what these are intended to do (and cannot do) and how they can be utilized. Our objective is to distill this sprawling information into a format that is understandable and readily accessible to authors, peer reviewers, and editors. In doing so, we aim to promote appreciation and understanding of the demanding science of evidence synthesis among stakeholders. We focus on well-documented deficiencies in key components of evidence syntheses to elucidate the rationale for current standards. The constructs underlying the tools developed to assess reporting, risk of bias, and methodological quality of evidence syntheses are distinguished from those involved in determining overall certainty of a body of evidence. Another important distinction is made between those tools used by authors to develop their syntheses as opposed to those used to ultimately judge their work.» Exemplar methods and research practices are described, complemented by novel pragmatic strategies to improve evidence syntheses. The latter include preferred terminology and a scheme to characterize types of research evidence. We organize best practice resources in a Concise Guide that can be widely adopted and adapted for routine implementation by authors and journals. Appropriate, informed use of these is encouraged, but we caution against their superficial application and emphasize their endorsement does not substitute for in-depth methodological training. By highlighting best practices with their rationale, we hope this guidance will inspire further evolution of methods and tools that can advance the field.
Collapse
Affiliation(s)
- Kat Kolaski
- Departments of Orthopaedic Surgery, Pediatrics, and Neurology, Wake Forest School of Medicine, Winston-Salem, North Carolina
| | - Lynne Romeiser Logan
- Department of Physical Medicine and Rehabilitation, SUNY Upstate Medical University, Syracuse, New York
| | - John P A Ioannidis
- Departments of Medicine, of Epidemiology and Population Health, of Biomedical Data Science, and of Statistics, and Meta-Research Innovation Center at Stanford (METRICS), Stanford University School of Medicine, Stanford, California
| |
Collapse
|
10
|
Rigutto-Farebrother J, Ahles S, Cade J, Murphy KJ, Plat J, Schwingshackl L, Roche HM, Shyam S, Lachat C, Minihane AM, Weaver C. Perspectives on the application of CONSORT guidelines to randomised controlled trials in nutrition. Eur J Nutr 2023:10.1007/s00394-023-03137-5. [PMID: 37099211 DOI: 10.1007/s00394-023-03137-5] [Citation(s) in RCA: 5] [Impact Index Per Article: 5.0] [Reference Citation Analysis] [Abstract] [Key Words] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/07/2022] [Accepted: 03/21/2023] [Indexed: 04/27/2023]
Abstract
PURPOSE Reporting guidelines facilitate quality and completeness in research reporting. The CONsolidated Standards Of Reporting Trials (CONSORT) statement is widely applied to dietary and nutrition trials but has no extension specific to nutrition. Evidence suggests poor reporting in nutrition research. The Federation of European Nutrition Societies led an initiative to make recommendations for a nutrition extension to the CONSORT statement towards a more robust reporting of the evidence base. METHODS An international working group was formed of nutrition researchers from 14 institutions in 12 different countries and on five continents. Using meetings over a period of one year, we interrogated the CONSORT statement specifically for its application to report nutrition trials. RESULTS We provide a total of 28 new nutrition-specific recommendations or emphasised recommendations for the reporting of the introduction (three), methods (twelve), results (five) and discussion (eight). We also added two additional recommendations that were not allocated under the standard CONSORT headings. CONCLUSION We identify a need to provide guidance in addition to CONSORT to improve the quality and consistency of the reporting and propose key considerations for further development of formal guidelines for the reporting of nutrition trials. Readers are encouraged to engage in this process, provide comments and conduct specific studies to inform further work on the development of reporting guidelines for nutrition trials.
Collapse
Affiliation(s)
- Jessica Rigutto-Farebrother
- Human Nutrition Laboratory, Institute for Food, Nutrition and Health, ETH Zürich, Zürich, Switzerland.
- Laboratory for Nutrition and Metabolic Epigenetics, Institute for Food, Nutrition and Health, LFV E14.1, Schmelzbergstrasse 7, 8092, Zurich, Switzerland.
- Global Center for the Development of the Whole Child, University of Notre Dame, South Bend, USA.
| | - Sanne Ahles
- Department of Nutrition and Movement Sciences, School of Nutrition and Translational Research in Metabolism (NUTRIM), Maastricht University, Maastricht, The Netherlands
- BioActor BV, Maastricht, The Netherlands
| | - Janet Cade
- Nutritional Epidemiology Group, School of Food Science and Nutrition, University of Leeds, Leeds, UK
| | - Karen J Murphy
- Clinical and Health Sciences, Alliance for Research in Exercise, Nutrition and Activity, University of South Australia, Adelaide, Australia
| | - Jogchum Plat
- Department of Nutrition and Movement Sciences, School of Nutrition and Translational Research in Metabolism (NUTRIM), Maastricht University, Maastricht, The Netherlands
| | - Lukas Schwingshackl
- Institute for Evidence in Medicine, Faculty of Medicine, Medical Center-University of Freiburg, University of Freiburg, Freiburg, Germany
| | - Helen M Roche
- Nutrigenomics Research Group, UCD Conway Institute, School of Public Health, Physiotherapy and Sports Science, University College Dublin, Dublin, Ireland
- Institute for Global Food Security, School of Biological Sciences, Queens University Belfast, Belfast, UK
| | - Sangeetha Shyam
- Centre for Translational Research, Institute for Research, Development, and Innovation (IRDI), International Medical University, Kuala Lumpur, Malaysia
- Departament de Bioquímica i Biotecnologia, Centro de Investigación Biomédica en Red de Fisiopatología de la Obesidad y Nutrición (CIBEROBN), Universitat Rovira i Virgili, Unitat de Nutrició Humana, Instituto de Salud Carlos III (ISCIII), Reus, Spain
| | - Carl Lachat
- Department of Food Technology, Safety and Health, Ghent University, Ghent, Belgium
| | - Anne-Marie Minihane
- Nutrition and Preventive Medicine, Norwich Medical School, University of East Anglia (UEA), Norwich, UK
- Norwich Institute of Healthy Ageing, UEA, Norwich, UK
| | - Connie Weaver
- School of Exercise and Nutritional Sciences, San Diego State University, San Diego, USA
| |
Collapse
|
11
|
GRADE Use in Evidence Syntheses Published in High-Impact-Factor Gynecology and Obstetrics Journals: A Methodological Survey. J Clin Med 2023; 12:jcm12020446. [PMID: 36675377 PMCID: PMC9866985 DOI: 10.3390/jcm12020446] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/07/2022] [Revised: 12/24/2022] [Accepted: 12/27/2022] [Indexed: 01/09/2023] Open
Abstract
Objective: To identify and describe the certainty of evidence of gynecology and obstetrics systematic reviews (SRs) using the Grading of Recommendations, Assessment, Development and Evaluation (GRADE) approach. Method: Database searches of SRs using GRADE, published between 1 January 2016 to 31 December 2020, in the 10 "gynecology and obstetrics" journals with the highest impact factor, according to the Journal Citation Report 2019. Selected studies included those SRs using the GRADE approach, used to determine the certainty of evidence. Results: Out of 952 SRs, ninety-six SRs of randomized control trials (RCTs) and/or nonrandomized studies (NRSs) used GRADE. Sixty-seven SRs (7.04%) rated the certainty of evidence for specific outcomes. In total, we identified 946 certainty of evidence outcome ratings (n = 614 RCT ratings), ranging from very-low (42.28%) to low (28.44%), moderate (17.65%), and high (11.63%). High and very low certainty of evidence ratings accounted for 2.16% and 71.60% in the SRs of NRSs, respectively, compared with 16.78% and 26.55% in the SRs of RCTs. In the SRs of RCTs and NRSs, certainty of evidence was mainly downgraded due to imprecision and bias risks. Conclusions: More attention needs to be paid to strengthening GRADE acceptance and building knowledge of GRADE methods in gynecology and obstetrics evidence synthesis.
Collapse
|
12
|
Kolaski K, Romeiser Logan L, Ioannidis JPA. Guidance to best tools and practices for systematic reviews1. J Pediatr Rehabil Med 2023; 16:241-273. [PMID: 37302044 DOI: 10.3233/prm-230019] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 06/12/2023] Open
Abstract
Data continue to accumulate indicating that many systematic reviews are methodologically flawed, biased, redundant, or uninformative. Some improvements have occurred in recent years based on empirical methods research and standardization of appraisal tools; however, many authors do not routinely or consistently apply these updated methods. In addition, guideline developers, peer reviewers, and journal editors often disregard current methodological standards. Although extensively acknowledged and explored in the methodological literature, most clinicians seem unaware of these issues and may automatically accept evidence syntheses (and clinical practice guidelines based on their conclusions) as trustworthy.A plethora of methods and tools are recommended for the development and evaluation of evidence syntheses. It is important to understand what these are intended to do (and cannot do) and how they can be utilized. Our objective is to distill this sprawling information into a format that is understandable and readily accessible to authors, peer reviewers, and editors. In doing so, we aim to promote appreciation and understanding of the demanding science of evidence synthesis among stakeholders. We focus on well-documented deficiencies in key components of evidence syntheses to elucidate the rationale for current standards. The constructs underlying the tools developed to assess reporting, risk of bias, and methodological quality of evidence syntheses are distinguished from those involved in determining overall certainty of a body of evidence. Another important distinction is made between those tools used by authors to develop their syntheses as opposed to those used to ultimately judge their work.Exemplar methods and research practices are described, complemented by novel pragmatic strategies to improve evidence syntheses. The latter include preferred terminology and a scheme to characterize types of research evidence. We organize best practice resources in a Concise Guide that can be widely adopted and adapted for routine implementation by authors and journals. Appropriate, informed use of these is encouraged, but we caution against their superficial application and emphasize their endorsement does not substitute for in-depth methodological training. By highlighting best practices with their rationale, we hope this guidance will inspire further evolution of methods and tools that can advance the field.
Collapse
Affiliation(s)
- Kat Kolaski
- Departments of Orthopaedic Surgery, Pediatrics, and Neurology, Wake Forest School of Medicine, Winston-Salem, NC, USA
| | - Lynne Romeiser Logan
- Department of Physical Medicine and Rehabilitation, SUNY Upstate Medical University, Syracuse, NY, USA
| | - John P A Ioannidis
- Departments of Medicine, of Epidemiology and Population Health, of Biomedical Data Science, and of Statistics, and Meta-Research Innovation Center at Stanford (METRICS), Stanford University School of Medicine, Stanford, CA, USA
| |
Collapse
|
13
|
Schwingshackl L, Bröckelmann N, Beyerbach J, Werner SS, Zähringer J, Schwarzer G, Meerpohl JJ. An Empirical Evaluation of the Impact Scenario of Pooling Bodies of Evidence from Randomized Controlled Trials and Cohort Studies in Nutrition Research. Adv Nutr 2022; 13:1774-1786. [PMID: 35416239 PMCID: PMC9526829 DOI: 10.1093/advances/nmac042] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/19/2021] [Revised: 12/16/2021] [Accepted: 04/08/2022] [Indexed: 01/28/2023] Open
Abstract
Only very few Cochrane nutrition reviews include cohort studies (CSs), but most evidence in nutrition research comes from CSs. We aimed to pool bodies of evidence (BoE) from randomized controlled trials (RCTs) derived from Cochrane reviews with matched BoE from CSs. The Cochrane Database of Systematic Reviews and MEDLINE were searched for systematic reviews (SRs) of RCTs and SRs of CSs. BoE from RCTs were pooled together with BoE from CSs using random-effects and common-effect models. Heterogeneity, 95% prediction intervals, contributed weight of BoE from RCTs to the pooled estimate, and whether integration of BoE from CSs modified the conclusion from BoE of RCTs were evaluated. Overall, 80 diet-disease outcome pairs based on 773 RCTs and 720 CSs were pooled. By pooling BoE from RCTs and CSs with a random-effects model, for 45 (56%) out of 80 diet-disease associations the 95% CI excluded no effect and showed mainly a reduced risk/inverse association. By pooling BoE from RCTs and CSs, median I2 = 46% and the median contributed weight of RCTs to the pooled estimates was 34%. The direction of effect between BoE from RCTs and pooled effect estimates was rarely opposite (n = 17; 21%). The integration of BoE from CSs modified the result (by examining the 95% CI) from BoE of RCTs in 35 (44%) of the 80 diet-disease associations. Our pooling scenario showed that the integration of BoE from CSs modified the conclusion from BoE of RCTs in nearly 50% of the associations, although the direction of effect was mainly concordant between BoE of RCTs and pooled estimates. Our findings provide insights for the potential impact of pooling both BoE in Cochrane nutrition reviews. CSs should be considered for inclusion in future Cochrane nutrition reviews, and we recommend analyzing RCTs and CSs in separate meta-analyses, or, if combined together, with a subgroup analysis.
Collapse
Affiliation(s)
| | - Nils Bröckelmann
- Institute for Evidence in Medicine, Medical Center—University of Freiburg, Faculty of Medicine, University of Freiburg, Freiburg, Germany
| | - Jessica Beyerbach
- Institute for Evidence in Medicine, Medical Center—University of Freiburg, Faculty of Medicine, University of Freiburg, Freiburg, Germany
| | - Sarah S Werner
- Institute for Evidence in Medicine, Medical Center—University of Freiburg, Faculty of Medicine, University of Freiburg, Freiburg, Germany
| | - Jasmin Zähringer
- Institute for Evidence in Medicine, Medical Center—University of Freiburg, Faculty of Medicine, University of Freiburg, Freiburg, Germany
| | - Guido Schwarzer
- Institute of Medical Biometry and Statistics, Faculty of Medicine and Medical Center—University of Freiburg, Freiburg, Germany
| | - Joerg J Meerpohl
- Institute for Evidence in Medicine, Medical Center—University of Freiburg, Faculty of Medicine, University of Freiburg, Freiburg, Germany,Cochrane Germany, Cochrane Germany Foundation, Freiburg, Germany
| |
Collapse
|
14
|
Pooling of cohort studies and RCTs affects GRADE certainty of evidence in nutrition research. J Clin Epidemiol 2022; 147:151-159. [DOI: 10.1016/j.jclinepi.2022.04.013] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/08/2021] [Revised: 03/21/2022] [Accepted: 04/11/2022] [Indexed: 11/19/2022]
|
15
|
Eble J, Harms L, Verbeek J, Morgan RL, Schünemann HJ, Meerpohl JJ, Schwingshackl L. The use of the GRADE dose-response gradient domain in nutrition evidence syntheses varies considerably. J Clin Epidemiol 2022; 146:12-21. [DOI: 10.1016/j.jclinepi.2022.02.009] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/21/2021] [Revised: 01/17/2022] [Accepted: 02/20/2022] [Indexed: 10/19/2022]
|
16
|
Zajac JF, Storman D, Swierz MJ, Koperny M, Weglarz P, Staskiewicz W, Gorecka M, Skuza A, Wach A, Kaluzinska K, Bochenek-Cibor J, Johnston BC, Bala MM. Are systematic reviews addressing nutrition for cancer prevention trustworthy? A systematic survey of quality and risk of bias. Nutr Rev 2021; 80:1558-1567. [PMID: 34921318 PMCID: PMC9086792 DOI: 10.1093/nutrit/nuab093] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/16/2022] Open
Abstract
Context The last 30 years have yielded a vast number of systematic reviews and/or meta-analyses addressing the link between nutrition and cancer risk. Objective The aim of this survey was to assess overall quality and potential for risk of bias in systematic reviews and meta-analyses (SRMAs) that examined the role of nutrition in cancer prevention. Data Sources MEDLINE, Embase, and the Cochrane Library databases were searched (last search performed November 2018). Study Selection Studies identified as SRMAs that investigated a nutritional or dietary intervention or exposure for cancer prevention in the general population or in people at risk of cancer and in which primary studies had a comparison group were eligible for inclusion. Screening, data extraction, and quality assessment were conducted independently by 2 reviewers. Data Extraction Altogether, 101 studies were randomly selected for analysis. The methodological quality and risk of bias were evaluated using the AMSTAR-2 and ROBIS tools, respectively. Results Most SRMAs included observational studies. Less than 10% of SRMAs reported a study protocol, and only 51% of SRMAs assessed the risk of bias in primary studies. Most studies conducted subgroup analyses, but only a few reported tests of interaction or specified subgroups of interest a priori. Overall, according to AMSTAR-2, only 1% of SRMAs were of high quality, while 97% were of critically low quality. Only 3% had a low risk of bias, according to ROBIS. Conclusions This systematic survey revealed substantial limitations with respect to quality and risk of bias of SRMAs. SRMAs examining nutrition and cancer prevention cannot be considered trustworthy, and results should be interpreted with caution. Peer reviewers as well as users of SRMAs should be advised to use the AMSTAR-2 and/or ROBIS instruments to help to determine the overall quality and risk of bias of SRMAs. Systematic Review Registration PROSPERO registration number CRD42019121116.
Collapse
Affiliation(s)
- Joanna F Zajac
- Chair of Epidemiology and Preventive Medicine, Department of Hygiene and Dietetics, Jagiellonian University Medical College, Krakow, Poland
| | - Dawid Storman
- Chair of Epidemiology and Preventive Medicine, Department of Hygiene and Dietetics, Jagiellonian University Medical College, Krakow, Poland
| | - Mateusz J Swierz
- Chair of Epidemiology and Preventive Medicine, Department of Hygiene and Dietetics, Jagiellonian University Medical College, Krakow, Poland
| | - Magdalena Koperny
- Chair of Epidemiology and Preventive Medicine, Department of Epidemiology, Jagiellonian University Medical College, Krakow, Poland
| | - Paulina Weglarz
- Chair of Epidemiology and Preventive Medicine, Department of Hygiene and Dietetics, Jagiellonian University Medical College, Krakow, Poland
| | - Wojciech Staskiewicz
- Students' Scientific Group of Systematic Reviews, Jagiellonian University Medical College, Krakow, Poland
| | - Magdalena Gorecka
- Students' Scientific Group of Systematic Reviews, Jagiellonian University Medical College, Krakow, Poland
| | - Anna Skuza
- Students' Scientific Group of Systematic Reviews, Jagiellonian University Medical College, Krakow, Poland
| | - Adam Wach
- Students' Scientific Group of Systematic Reviews, Jagiellonian University Medical College, Krakow, Poland
| | - Klaudia Kaluzinska
- Students' Scientific Group of Systematic Reviews, Jagiellonian University Medical College, Krakow, Poland
| | | | - Bradley C Johnston
- Department of Nutrition and the Department of Epidemiology and Biostatistics, Texas A&M University, College Station, Texas, USA; and with the Department of Health Research Methods, Evidence, and Impact, McMaster University, Hamilton, Ontario, Canada
| | - Malgorzata M Bala
- Chair of Epidemiology and Preventive Medicine, Department of Hygiene and Dietetics, Jagiellonian University Medical College, Krakow, Poland
| |
Collapse
|
17
|
Schwingshackl L, Balduzzi S, Beyerbach J, Bröckelmann N, Werner SS, Zähringer J, Nagavci B, Meerpohl JJ. Evaluating agreement between bodies of evidence from randomised controlled trials and cohort studies in nutrition research: meta-epidemiological study. BMJ 2021; 374:n1864. [PMID: 34526355 PMCID: PMC8441535 DOI: 10.1136/bmj.n1864] [Citation(s) in RCA: 26] [Impact Index Per Article: 8.7] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Accepted: 07/15/2021] [Indexed: 12/21/2022]
Abstract
OBJECTIVE To evaluate the agreement between diet-disease effect estimates of bodies of evidence from randomised controlled trials and those from cohort studies in nutrition research, and to investigate potential factors for disagreement. DESIGN Meta-epidemiological study. DATA SOURCES Cochrane Database of Systematic Reviews, and Medline. REVIEW METHODS Population, intervention or exposure, comparator, outcome (PI/ECO) elements from a body of evidence from cohort studies (BoE(CS)) were matched with corresponding elements of a body of evidence from randomised controlled trials (BoE(RCT)). Pooled ratio of risk ratios or difference of mean differences across all diet-disease outcome pairs were calculated. Subgroup analyses were conducted to explore factors for disagreement. Heterogeneity was assessed through I2 and τ2. Prediction intervals were calculated to assess the range of possible values for the difference in the results between evidence from randomised controlled trials and evidence from cohort studies in future comparisons. RESULTS 97 diet-disease outcome pairs (that is, matched BoE(RCT) and BoE(CS)) were identified overall. For binary outcomes, the pooled ratio of risk ratios comparing estimates from BoE(RCT) with BoE(CS) was 1.09 (95% confidence interval 1.04 to 1.14; I2=68%; τ2=0.021; 95% prediction interval 0.81 to 1.46). The prediction interval indicated that the difference could be much more substantial, in either direction. We further explored heterogeneity and found that PI/ECO dissimilarities, especially for the comparisons of dietary supplements in randomised controlled trials and nutrient status in cohort studies, explained most of the differences. When the type of intake or exposure between both types of evidence was identical, the estimates were similar. For continuous outcomes, small differences were observed between randomised controlled trials and cohort studies. CONCLUSION On average, the difference in pooled results between estimates from BoE(RCT) and BoE(CS) was small. But wide prediction intervals and some substantial statistical heterogeneity in cohort studies indicate that important differences or potential bias in individual comparisons or studies cannot be excluded. Observed differences were mainly driven by dissimilarities in population, intervention or exposure, comparator, and outcome. These findings could help researchers further understand the integration of such evidence into prospective nutrition evidence syntheses and improve evidence based dietary guidelines.
Collapse
Affiliation(s)
- Lukas Schwingshackl
- Institute for Evidence in Medicine, Medical Centre - University of Freiburg, Faculty of Medicine, University of Freiburg, Freiburg, Germany
| | - Sara Balduzzi
- Institute of Medical Biometry and Statistics, Medical Centre - University of Freiburg, Faculty of Medicine, Freiburg, Germany
| | - Jessica Beyerbach
- Institute for Evidence in Medicine, Medical Centre - University of Freiburg, Faculty of Medicine, University of Freiburg, Freiburg, Germany
| | - Nils Bröckelmann
- Institute for Evidence in Medicine, Medical Centre - University of Freiburg, Faculty of Medicine, University of Freiburg, Freiburg, Germany
| | - Sarah S Werner
- Institute for Evidence in Medicine, Medical Centre - University of Freiburg, Faculty of Medicine, University of Freiburg, Freiburg, Germany
| | - Jasmin Zähringer
- Institute for Evidence in Medicine, Medical Centre - University of Freiburg, Faculty of Medicine, University of Freiburg, Freiburg, Germany
| | - Blin Nagavci
- Institute for Evidence in Medicine, Medical Centre - University of Freiburg, Faculty of Medicine, University of Freiburg, Freiburg, Germany
| | - Joerg J Meerpohl
- Institute for Evidence in Medicine, Medical Centre - University of Freiburg, Faculty of Medicine, University of Freiburg, Freiburg, Germany
- Cochrane Germany, Cochrane Germany Foundation, Freiburg, Germany
| |
Collapse
|
18
|
Schwingshackl L, Schünemann HJ, Meerpohl JJ. Improving the trustworthiness of findings from nutrition evidence syntheses: assessing risk of bias and rating the certainty of evidence. Eur J Nutr 2021; 60:2893-2903. [PMID: 33377996 PMCID: PMC8354882 DOI: 10.1007/s00394-020-02464-1] [Citation(s) in RCA: 39] [Impact Index Per Article: 13.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/22/2020] [Accepted: 12/11/2020] [Indexed: 02/07/2023]
Abstract
Suboptimal diet is recognized as a leading modifiable risk factor for non-communicable diseases. Non-randomized studies (NRSs) with patient relevant outcomes provide many insights into diet-disease relationships. Dietary guidelines are based predominantly on findings from systematic reviews of NRSs-mostly prospective observational studies, despite that these have been repeatedly criticized for yielding potentially less trustworthy results than randomized controlled trials (RCTs). It is assumed that these are a result of bias due to prevalent-user designs, inappropriate comparators, residual confounding, and measurement error. In this article, we aim to highlight the importance of applying risk of bias (RoB) assessments in nutritional studies to improve the credibility of evidence of systematic reviews. First, we discuss the importance and challenges of dietary RCTs and NRSs, and provide reasons for potentially less trustworthy results of dietary studies. We describe currently used tools for RoB assessment (Cochrane RoB, and ROBINS-I), describe the importance of rigorous RoB assessment in dietary studies and provide examples that further the understanding of the key issues to overcome in nutrition research. We then illustrate, by comparing the Grading of Recommendations Assessment, Development, and Evaluation (GRADE) approach with current approaches used by United States Department of Agriculture Dietary Guidelines for Americans, and the World Cancer Research Fund, how to establish trust in dietary recommendations. Our overview shows that the GRADE approach provides more transparency about the single domains for grading the certainty of the evidence and the strength of recommendations. Despite not increasing the certainty of evidence itself, we expect that the rigorous application of the Cochrane RoB and the ROBINS-I tools within systematic reviews of both RCTs and NRSs and their integration within the GRADE approach will strengthen the credibility of dietary recommendations.
Collapse
Affiliation(s)
- Lukas Schwingshackl
- Faculty of Medicine, Institute for Evidence in Medicine, Medical Center, University of Freiburg, Freiburg, Germany.
| | - Holger J Schünemann
- Department of Health Research Methods, Evidence and Impact, Department of Medicine, McMaster University, Hamilton, Canada
| | - Joerg J Meerpohl
- Faculty of Medicine, Institute for Evidence in Medicine, Medical Center, University of Freiburg, Freiburg, Germany
- Cochrane Germany, Cochrane Germany Foundation, Freiburg, Germany
| |
Collapse
|
19
|
Vernooij R, Guyatt GH, Zeraatkar D, Han MA, Valli C, El Dib R, Alonso-Coello P, Bala MM, Johnston BC. Reconciling contrasting guideline recommendations on red and processed meat for health outcomes. J Clin Epidemiol 2021; 138:215-218. [PMID: 34273525 DOI: 10.1016/j.jclinepi.2021.07.008] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/15/2021] [Revised: 07/07/2021] [Accepted: 07/12/2021] [Indexed: 12/12/2022]
Affiliation(s)
- Rwm Vernooij
- Department of Nephrology and Hypertension, University Medical Center Utrecht, Utrecht University, Utrecht, The Netherlands; Julius Center for Health Sciences and Primary Care, University Medical Center Utrecht, Utrecht, The Netherlands
| | - G H Guyatt
- Department of Health Research Methods, Evidence & Impact, McMaster University, Hamilton, Ontario, Canada
| | - D Zeraatkar
- Department of Health Research Methods, Evidence & Impact, McMaster University, Hamilton, Ontario, Canada; Department of Biomedical Informatics, Harvard Medical School, Boston, MA, USA
| | - M A Han
- Department of Preventive Medicine, College of Medicine, Chosun University, Gwangju, Republic of Korea
| | - C Valli
- Iberoamerican Cochrane Centre Barcelona, Biomedical Research Institute San Pau (IIB Sant Pau-CIBERESP), Barcelona, Spain; Department of Paediatrics, Obstetrics,Gynaecology and Preventive Medicine, Universidad Autónoma de Barcelona,Barcelona, Spain
| | - R El Dib
- Institute of Science and Technology, Universidade Estadual Paulista, São José dos Campos, São Paulo, Brazil
| | - P Alonso-Coello
- Iberoamerican Cochrane Centre Barcelona, Biomedical Research Institute San Pau (IIB Sant Pau-CIBERESP), Barcelona, Spain
| | - M M Bala
- Chair of Epidemiology and Preventive Medicine, Department of Hygiene and Dietetics, Jagiellonian University Medical College, Krakow, Poland
| | - B C Johnston
- Department of Health Research Methods, Evidence & Impact, McMaster University, Hamilton, Ontario, Canada; Departments of Nutrition, Epidemiology & Biostatistics, Texas A&M University, College Station, TX, USA.
| |
Collapse
|