|
1
|
Shahabi S, Bagheri Lankarani K, Ezati R, ShahAli S. Reporting and Methodological Quality of Systematic Reviews and Meta-Analyses Evaluating Effects of Extracorporeal Shock Wave Therapy on Tendinopathies: A Scoping Review. J Chiropr Med 2024; 23:136-151. [PMID: 39670204 PMCID: PMC11632692 DOI: 10.1016/j.jcm.2024.08.007] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/30/2023] [Revised: 08/25/2024] [Accepted: 08/26/2024] [Indexed: 12/14/2024] Open
Abstract
Objective The purpose of this study was to evaluate the methodological and reporting quality of the published systematic reviews with or without meta-analyses (SRs/MAs) that looked at the effects of extracorporeal shock wave therapy (ESWT) on tendinopathies and to summarize its effectiveness. Methods A search of PubMed, Scopus, PEDro, Web of Science, Cochrane, Embase, and REHABDATA was conducted. SRs/MAs that assessed the effectiveness of ESWT for treating tendinopathy were included. The methodological and reporting quality of the eligible SRs/MAs were assessed using AMSTAR-2 and the PRISMA checklist. In addition, the ROBIS tool was applied to evaluate the risk of bias (RoB). Results Eighteen SRs/MAs were included. The overall methodological quality was "critically low." Furthermore, the reporting quality of the included reviews according to PRISMA criteria was not optimal. Based on the ROBIS, a total of 16.2% of the studies had a low RoB, 38.9% had an unclear RoB, and 44.4% of the studies were appraised as having a high RoB. Conclusion In this scoping review we found substantial limitations regarding the quality and RoB of SRs/MAs. Therefore, reviewers must consider the AMSTAR-2, PRISMA, and ROBIS tools to improve the quality of future studies.
Collapse
Affiliation(s)
- Saeed Shahabi
- Health Policy Research Center, Institute of Health, Shiraz University of Medical Sciences, Shiraz, Iran
| | - Kamran Bagheri Lankarani
- Health Policy Research Center, Institute of Health, Shiraz University of Medical Sciences, Shiraz, Iran
| | - Rozhin Ezati
- Department of Occupational therapy, School of Rehabilitation Sciences, Iran University of Medical Sciences, Tehran, Iran
| | - Shabnam ShahAli
- Iranian Center of Excellence in Physiotherapy, Rehabilitation Research Center, Department of Physiotherapy, School of Rehabilitation Sciences, Iran University of Medical Sciences, Tehran, Iran
| |
Collapse
|
|
2
|
Johnston BC, Zeraatkar D, Steen J, de Jauregui DRF, Zhu H, Sun M, Cooper M, Maraj M, Prokop-Dorner A, Reyes BC, Valli C, Storman D, Karam G, Zajac J, Ge L, Swierz MJ, Ghosh N, Vernooij RWM, Chang Y, Zhao Y, Thabane L, Guyatt GH, Alonso-Coello P, Hooper L, Bala MM. Saturated fat and human health: a protocol for a methodologically innovative systematic review and meta-analysis to inform public health nutrition guidelines. Syst Rev 2023; 12:39. [PMID: 36918997 PMCID: PMC10012519 DOI: 10.1186/s13643-023-02209-1] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 09/22/2022] [Accepted: 03/01/2023] [Indexed: 03/15/2023] Open
Abstract
BACKGROUND The health effects of dietary fats are a controversial issue on which experts and authoritative organizations have often disagreed. Care providers, guideline developers, policy-makers, and researchers use systematic reviews to advise patients and members of the public on optimal dietary habits, and to formulate public health recommendations and policies. Existing reviews, however, have serious limitations that impede optimal dietary fat recommendations, such as a lack of focus on outcomes important to people, substantial risk of bias (RoB) issues, ignoring absolute estimates of effects together with comprehensive assessments of the certainty of the estimates for all outcomes. OBJECTIVE We therefore propose a methodologically innovative systematic review using direct and indirect evidence on diet and food-based fats (i.e., reduction or replacement of saturated fat with monounsaturated or polyunsaturated fat, or carbohydrates or protein) and the risk of important health outcomes. METHODS We will collaborate with an experienced research librarian to search MEDLINE, EMBASE, CINAHL, and the Cochrane Database of Systematic Reviews (CDSR) for randomized clinical trials (RCTs) addressing saturated fat and our health outcomes of interest. In duplicate, we will screen, extract results from primary studies, assess their RoB, conduct de novo meta-analyses and/or network meta-analysis, assess the impact of missing outcome data on meta-analyses, present absolute effect estimates, and assess the certainty of evidence for each outcome using the GRADE contextualized approach. Our work will inform recommendations on saturated fat based on international standards for reporting systematic reviews and guidelines. CONCLUSION Our systematic review and meta-analysis will provide the most comprehensive and rigorous summary of the evidence addressing the relationship between saturated fat modification for people-important health outcomes. The evidence from this review will be used to inform public health nutrition guidelines. TRIAL REGISTRATION PROSPERO Registration: CRD42023387377 .
Collapse
Affiliation(s)
- Bradley C Johnston
- Department of Nutrition, College of Agriculture and Life Science, Texas A&M University, College Station, TX, USA. .,Department of Epidemiology and Biostatistics, School of Public Health, College Station, TX, USA.
| | - Dena Zeraatkar
- Department of Biomedical Informatics, Harvard Medical School, Boston, MA, USA.,Department of Health Research Methods, Evidence and Impact, McMaster University, Hamilton, ON, Canada
| | - Jeremy Steen
- Faculty of Health Sciences, McMaster University, Hamilton, ON, Canada
| | - Diego Rada Fernandez de Jauregui
- Preventive Medicine and Public Health Department, School of Pharmacy, University of the Basque Country/Euskal Herriko Unibertsitatea (UPV/EHU), Vitoria-Gasteiz, Spain.,Department of Nutrition, Texas A&M University, College Station, TX, USA
| | - Hongfei Zhu
- Evidence Based Social Science Research Centre, School of Public Health, Lanzhou University, Lanzhou, China.,Department of Social Medicine and Health Management, School of Public Health, Lanzhou University, Lanzhou, China
| | - Mingyao Sun
- Evidence Based Nursing Centre, School of Nursing, Lanzhou University, Lanzhou, China
| | - Matthew Cooper
- Department of Medicine, Faculty of Medicine and Dentistry, University of Alberta, Edmonton, AB, Canada
| | - Malgorzata Maraj
- Department of Hygiene and Dietetics, Chair of Epidemiology and Preventive Medicine, Jagiellonian University Medical College, Jagiellonian University Medical College, Krakow, Poland
| | - Anna Prokop-Dorner
- Department of Medical Sociology, Chair of Epidemiology and Preventive Medicine, Jagiellonian University Medical College, Krakow, Poland
| | - Boris Castro Reyes
- Iberoamerican Cochrane Centre, Biomedical Research Institute San Pau (IIB Sant Pau), Barcelona, Spain
| | - Claudia Valli
- Iberoamerican Cochrane Centre, Biomedical Research Institute San Pau (IIB Sant Pau), Barcelona, Spain.,Department of Paediatrics, Obstetrics, Gynaecology and Preventive Medicine, Universidad Autónoma de Barcelona, Barcelona, Spain
| | - Dawid Storman
- Department of Hygiene and Dietetics, Chair of Epidemiology and Preventive Medicine, Faculty of Medicine, Jagiellonian University Medical College, Kraków, Poland.,Systematic Reviews Unit, Jagiellonian University Medical College, Krakow, Poland.,Department of Adult Psychiatry, University Hospital, Krakow, Poland
| | - Giorgio Karam
- Max Rady College of Medicine, University of Manitoba, Winnipeg, MB, Canada
| | - Joanna Zajac
- Department of Hygiene and Dietetics, Chair of Epidemiology and Preventive Medicine, Jagiellonian University Medical College, Jagiellonian University Medical College, Krakow, Poland
| | - Long Ge
- Department of Health Research Methods, Evidence and Impact, McMaster University, Hamilton, ON, Canada.,Evidence Based Social Science Research Centre, School of Public Health, Lanzhou University, Lanzhou, China.,Department of Social Medicine and Health Management, School of Public Health, Lanzhou University, Lanzhou, China
| | - Mateusz J Swierz
- Department of Hygiene and Dietetics, Chair of Epidemiology and Preventive Medicine, Faculty of Medicine, Jagiellonian University Medical College, Kraków, Poland.,Systematic Reviews Unit, Jagiellonian University Medical College, Krakow, Poland
| | - Nirjhar Ghosh
- Department of Nutrition, Texas A&M University, College Station, TX, USA
| | - Robin W M Vernooij
- Department of Nephrology and Hypertension, University Medical Center Utrecht, Utrecht, The Netherlands.,Julius Center for Health Sciences and Primary Care, University Medical Center Utrecht, Utrecht, The Netherlands
| | - Yaping Chang
- Department of Health Research Methods, Evidence and Impact, McMaster University, Hamilton, ON, Canada
| | - Yunli Zhao
- Department of Health Research Methods, Evidence and Impact, McMaster University, Hamilton, ON, Canada.,The Center of Gerontology and Geriatrics (National Clinical Research Center for Geriatrics), West China Hospital, Sichuan University, Chengdu, China
| | - Lehana Thabane
- Department of Health Research Methods, Evidence and Impact, McMaster University, Hamilton, ON, Canada
| | - Gordon H Guyatt
- Department of Health Research Methods, Evidence and Impact, McMaster University, Hamilton, ON, Canada.,Department of Medicine, Faculty of Health Sciences, Hamilton, ON, Canada
| | - Pablo Alonso-Coello
- Iberoamerican Cochrane Centre, Biomedical Research Institute San Pau (IIB Sant Pau), Barcelona, Spain.,CIBER de Epidemiología Y Salud Pública (CIBERESP), Madrid, Spain
| | - Lee Hooper
- Norwich Medical School, University of East Anglia, Norwich Research Park, Norwich, NR4 7TJ, UK
| | - Malgorzata M Bala
- Department of Hygiene and Dietetics, Chair of Epidemiology and Preventive Medicine, Faculty of Medicine, Jagiellonian University Medical College, Kraków, Poland.,Systematic Reviews Unit, Jagiellonian University Medical College, Krakow, Poland
| |
Collapse
|
|
3
|
Storman D, Koperny M, Zając J, Polak M, Weglarz P, Bochenek-Cibor J, Swierz MJ, Staskiewicz W, Gorecka M, Skuza A, Wach AA, Kaluzinska K, Bała MM. Predictors of Higher Quality of Systematic Reviews Addressing Nutrition and Cancer Prevention. INTERNATIONAL JOURNAL OF ENVIRONMENTAL RESEARCH AND PUBLIC HEALTH 2022; 19:ijerph19010506. [PMID: 35010766 PMCID: PMC8744691 DOI: 10.3390/ijerph19010506] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Subscribe] [Scholar Register] [Received: 10/18/2021] [Revised: 12/15/2021] [Accepted: 12/26/2021] [Indexed: 02/04/2023]
Abstract
Systematic reviews/meta-analyses (SR/MAs) are considered a reliable source of information in healthcare. We aimed to explore the association of several characteristics of SR/MAs addressing nutrition in cancer prevention and their quality/risk of bias (using assessments from AMSTAR-2 and ROBIS tools). The analysis included 101 SR/MAs identified in a systematic survey. Associations of each specified characteristic (e.g., information about the protocol, publication year, reported use of GRADE, or other methods for assessing overall certainty of evidence) with the number of AMSTAR-2 not met (‘No’ responses) and the number of ROBIS items met (‘Probably Yes’ or “Yes’ responses) were examined. Poisson regression was used to identify predictors of the number of ‘No’ answers (indicating lower quality) for all AMSTAR-2 items and the number of ‘Yes’ or ‘Probably Yes’ answers (indicating higher quality/lower concern for bias) for all ROBIS items. Logistic regression was used to identify variables associated with at least one domain assessed as ‘low concern for bias’ in the ROBIS tool. In multivariable analysis, SR/MAs not reporting use of any quality/risk of bias assessment instrument for primary studies were associated with a higher number of ‘No’ answers for all AMSTAR-2 items (incidence rate ratio (IRR) 1.26, 95% confidence interval (CI) 1.09–1.45), and a lower number of ‘Yes’ or ‘Probably Yes’ answers for all ROBIS items (IRR 0.76, 95% CI 0.66–0.87). Providing information about the protocol and search for unpublished studies was associated with a lower number of ‘No’ answers (IRR 0.73, 95% CI 0.56–0.97 and IRR 0.75, 95% CI 0.59–0.95, respectively) and a higher number of ‘Yes’ or ‘Probably Yes’ answers (IRR 1.43, 95% CI 1.17–1.74 and IRR 1.28, 95% CI 1.07–1.52, respectively). Not using at least one quality/risk of bias assessment tool for primary studies within an SR/MA was associated with lower odds that a study would be assessed as ‘low concern for bias’ in at least one ROBIS domain (odds ratio 0.061, 95% CI 0.007–0.527). Adherence to methodological standards in the development of SR/MAs was associated with a higher overall quality of SR/MAs addressing nutrition for cancer prevention.
Collapse
Affiliation(s)
- Dawid Storman
- Chair of Epidemiology and Preventive Medicine, Department of Hygiene and Dietetics, Jagiellonian University Medical College, 31-034 Krakow, Poland; (D.S.); (J.Z.); (P.W.); (M.J.S.)
| | - Magdalena Koperny
- Chair of Epidemiology and Preventive Medicine, Department of Epidemiology, Jagiellonian University Medical College, 31-034 Krakow, Poland;
| | - Joanna Zając
- Chair of Epidemiology and Preventive Medicine, Department of Hygiene and Dietetics, Jagiellonian University Medical College, 31-034 Krakow, Poland; (D.S.); (J.Z.); (P.W.); (M.J.S.)
| | - Maciej Polak
- Department of Epidemiology and Population Studies, Institute of Public Health, Faculty of Health Sciences, Jagiellonian University Medical College, 31-034 Krakow, Poland;
| | - Paulina Weglarz
- Chair of Epidemiology and Preventive Medicine, Department of Hygiene and Dietetics, Jagiellonian University Medical College, 31-034 Krakow, Poland; (D.S.); (J.Z.); (P.W.); (M.J.S.)
| | | | - Mateusz J. Swierz
- Chair of Epidemiology and Preventive Medicine, Department of Hygiene and Dietetics, Jagiellonian University Medical College, 31-034 Krakow, Poland; (D.S.); (J.Z.); (P.W.); (M.J.S.)
| | - Wojciech Staskiewicz
- Students’ Scientific Research Group of Systematic Reviews, Jagiellonian University Medical College, 31-034 Krakow, Poland; (W.S.); (M.G.); (A.S.); (A.A.W.); (K.K.)
| | - Magdalena Gorecka
- Students’ Scientific Research Group of Systematic Reviews, Jagiellonian University Medical College, 31-034 Krakow, Poland; (W.S.); (M.G.); (A.S.); (A.A.W.); (K.K.)
| | - Anna Skuza
- Students’ Scientific Research Group of Systematic Reviews, Jagiellonian University Medical College, 31-034 Krakow, Poland; (W.S.); (M.G.); (A.S.); (A.A.W.); (K.K.)
| | - Adam A. Wach
- Students’ Scientific Research Group of Systematic Reviews, Jagiellonian University Medical College, 31-034 Krakow, Poland; (W.S.); (M.G.); (A.S.); (A.A.W.); (K.K.)
| | - Klaudia Kaluzinska
- Students’ Scientific Research Group of Systematic Reviews, Jagiellonian University Medical College, 31-034 Krakow, Poland; (W.S.); (M.G.); (A.S.); (A.A.W.); (K.K.)
| | - Małgorzata M. Bała
- Chair of Epidemiology and Preventive Medicine, Department of Hygiene and Dietetics, Jagiellonian University Medical College, 31-034 Krakow, Poland; (D.S.); (J.Z.); (P.W.); (M.J.S.)
- Correspondence:
| |
Collapse
|
|
4
|
Swierz MJ, Storman D, Zajac J, Koperny M, Weglarz P, Staskiewicz W, Gorecka M, Skuza A, Wach A, Kaluzinska K, Bochenek-Cibor J, Johnston BC, Bala MM. Similarities, reliability and gaps in assessing the quality of conduct of systematic reviews using AMSTAR-2 and ROBIS: systematic survey of nutrition reviews. BMC Med Res Methodol 2021; 21:261. [PMID: 34837960 PMCID: PMC8627612 DOI: 10.1186/s12874-021-01457-w] [Citation(s) in RCA: 17] [Impact Index Per Article: 4.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/26/2021] [Accepted: 11/03/2021] [Indexed: 01/08/2023] Open
Abstract
BACKGROUND AMSTAR-2 ('A Measurement Tool to Assess Systematic Reviews, version 2') and ROBIS ('Risk of Bias in Systematic Reviews') are independent instruments used to assess the quality of conduct of systematic reviews/meta-analyses (SR/MAs). The degree of overlap in methodological constructs together with the reliability and any methodological gaps have not been systematically assessed and summarized in the field of nutrition. METHODS We performed a systematic survey of MEDLINE, EMBASE, and the Cochrane Library for SR/MAs published between January 2010 and November 2018 that examined the effects of any nutritional intervention/exposure for cancer prevention. We followed a systematic review approach including two independent reviewers at each step of the process. For AMSTAR-2 (16 items) and ROBIS (21 items), we assessed the similarities, the inter-rater reliability (IRR) and any methodological limitations of the instruments. Our protocol for the survey was registered in PROSPERO (CRD42019121116). RESULTS We found 4 similar domain constructs based on 11 comparisons from a total of 12 AMSTAR-2 and 14 ROBIS items. Ten comparisons were considered fully overlapping. Based on Gwet's agreement coefficients, six comparisons provided almost perfect (> 0.8), three substantial (> 0.6), and one a moderate level of agreement (> 0.4). While there is considerable overlap in constructs, AMSTAR-2 uniquely addresses explaining the selection of study designs for inclusion, reporting on excluded studies with justification, sources of funding of primary studies, and reviewers' conflict of interest. By contrast, ROBIS uniquely addresses appropriateness and restrictions within eligibility criteria, reducing risk of error in risk of bias (RoB) assessments, completeness of data extracted for analyses, the inclusion of all necessary studies for analyses, and adherence to predefined analysis plan. CONCLUSIONS Among the questions on AMSTAR-2 and ROBIS, 70.3% (26/37 items) address the same or similar methodological constructs. While the IRR of these constructs was moderate to perfect, there are unique methodological constructs that each instrument independently addresses. Notably, both instruments do not address the reporting of absolute estimates of effect or the overall certainty of the evidence, items that are crucial for users' wishing to interpret the importance of SR/MA results.
Collapse
Affiliation(s)
- Mateusz J Swierz
- Chair of Epidemiology and Preventive Medicine, Department of Hygiene and Dietetics, Jagiellonian University Medical College, Kopernika 7 Street 31-034, Krakow, Poland
| | - Dawid Storman
- Chair of Epidemiology and Preventive Medicine, Department of Hygiene and Dietetics, Jagiellonian University Medical College, Kopernika 7 Street 31-034, Krakow, Poland
| | - Joanna Zajac
- Chair of Epidemiology and Preventive Medicine, Department of Hygiene and Dietetics, Jagiellonian University Medical College, Kopernika 7 Street 31-034, Krakow, Poland
| | - Magdalena Koperny
- Chair of Epidemiology and Preventive Medicine, Department of Epidemiology Jagiellonian University Medical College , Kopernika 7 Street 31-034, Krakow, Poland
| | - Paulina Weglarz
- Chair of Epidemiology and Preventive Medicine, Department of Hygiene and Dietetics, Jagiellonian University Medical College, Kopernika 7 Street 31-034, Krakow, Poland
| | - Wojciech Staskiewicz
- Students' Scientific Research Group of Systematic Reviews, Jagiellonian University Medical College, Krakow, Poland
| | - Magdalena Gorecka
- Students' Scientific Research Group of Systematic Reviews, Jagiellonian University Medical College, Krakow, Poland
| | - Anna Skuza
- Students' Scientific Research Group of Systematic Reviews, Jagiellonian University Medical College, Krakow, Poland
| | - Adam Wach
- Students' Scientific Research Group of Systematic Reviews, Jagiellonian University Medical College, Krakow, Poland
| | - Klaudia Kaluzinska
- Students' Scientific Research Group of Systematic Reviews, Jagiellonian University Medical College, Krakow, Poland
| | | | - Bradley C Johnston
- Departments of Nutrition, Epidemiology and Biostatistics, Texas A&M University, College Station, College Station, TX, USA.,Department of Health Research Methods, Evidence, and Impact, McMaster University, Hamilton, Ontario, Canada
| | - Malgorzata M Bala
- Chair of Epidemiology and Preventive Medicine, Department of Hygiene and Dietetics, Jagiellonian University Medical College, Kopernika 7 Street 31-034, Krakow, Poland.
| |
Collapse
|