PRISMA and AMSTAR show systematic reviews on health literacy and cancer screening are of good quality

Methodological, reporting, and evidence quality of systematic reviews of traditional Chinese medicine for ischemic stroke

Objective: The aim of this study is to critically appraise whether published systematic reviews/meta-analyses of traditional Chinese medicine for adults with ischemic stroke are of sufficient quality and to rate the quality of evidence using the Grading of Recommendations, Assessment, Development, and Evaluation approach. Method: A literature search was performed in the Cochrane Library, PubMed, Chinese National Knowledge Infrastructure, and SinoMed databases by March 2022. The inclusion criteria were systematic reviews/meta-analyses of traditional Chinese medicine in adults who suffered from ischemic stroke. A Measurement Tool to Access Systematic Reviews 2 (AMSTAR-2) and Preferred Reporting Items for Systematic Reviews and Meta-Analyses for Abstract (PRISMA-A) statements were used to assess the methodological and reporting quality of the included reviews. The Grading of Recommendations, Assessment, Development, and Evaluation system was utilized to assess each report's evidence level. Results: Of the 1,908 titles and abstracts, 83 reviews met the inclusion criteria. These studies were published between 2005 and 2022. The results of AMSTAR-2 showed that 51.4% of the items were reported, but the registration, reasons for the inclusion of study design, the list of excluded studies, and funding information were ignored in the majority of the reviews. The results of PRISMA-A showed that 33.9% of items were reported, and the information on registration, limitation, and funding was not available in many publications. The assessment of the evidence with the Grading of Recommendations, Assessment, Development, and Evaluation showed that more than half (52/83) of the included studies had either low or very low levels of evidence. Conclusion: The reporting quality in the abstract of systematic reviews/meta-analyses on traditional Chinese medicine for ischemic stroke is poor and does not facilitate timely access to valid information for clinical practitioners. Although the methodological quality is of a medium level, this evidence lacks certainty, especially with a high risk of bias in individual studies.

The impact of funding on the quality and interpretation of systematic reviews of mechanical thrombectomy in stroke patients

BACKGROUND

Funding may impact the quality and findings of systematic reviews (SRs). We aimed to compare the methodological quality of funded and non-funded SRs that investigated the outcomes in ischemic stroke patients undergoing mechanical thrombectomy.

METHODS

We conducted a comprehensive search strategy in different databases, including Ovid Cochrane Central Register of Controlled Trials, Ovid Embase, Ovid Medline (including epub ahead of print, in-process & other non-indexed citations), PubMed, Scopus and Web of Science Core Collection to retrieve all relevant SRs. Random sequence generation matched each funded SR with a non-funded one. A Measurement Tool to Assess Systematic Reviews (AMSTAR)-2 tool was used to assess the bias and quality of the included SRs. We also used uni- and multivariate analysis to perform our analysis, and results were expressed in odds ratio (OR) and 95% confidence interval (CI).

RESULTS

We retrieved 150 articles, which were randomized and matched into 100 SRs, including 50 funded and 50 non-funded studies. By multivariate analysis, we found that including randomized clinical trials (RCTs) (OR: 5.7; 95% CI: 1.8-17.8; p = 0.003) and reporting conflict of interests (OR: 5.2; 95 CI: 1.1-24; p = 0.036) were the only significant differences between funded and non-funded SRs. No significant differences were found regarding the overall confidence for low-quality (OR: 0.54; 95% CI: 0.09-3.2; p = 0.49) and moderate/high-quality SRs (OR: 0.17; 95% CI: 0.02-1.87; p = 0.14).

CONCLUSION

Funded studies tend to include RCTs more often and report conflict of interests with no significant impact on overall confidence.

Impact of industry sponsorship on the quality of systematic reviews of vaccines: a cross-sectional analysis of studies published from 2016 to 2019

BACKGROUND

Systematic reviews (SRs) provide the highest level of evidence and inform evidence-based decision making in health care. Earlier studies found association with industry to be negatively associated with methodological quality of SRs. However, this has not been investigated in SRs on vaccines.

METHODS

We performed a systematic literature search using MEDLINE and EMBASE in March 2020. The results were restricted to those published between 2016 and 2019 with no language restrictions. Study characteristics were extracted by one person and checked by an experienced reviewer. The methodological quality of the SRs was assessed with the AMSTAR 2 tool by multiple reviewers after a calibration exercise was performed. A summary score for each SR was calculated. The Mann-Whitney U test and Fisher's exact test were performed to compare both groups.

RESULTS

Out of 185 SRs that met all inclusion criteria, 27 SRs were industry funded. Those were matched with 30 non-industry funded SRs resulting in a total sample size of 57. The mean AMSTAR 2 summary score across all SRs was 0.49. Overall, the median AMSTAR 2 summary score was higher for the non-industry funded SRs than for the industry-funded SRs (0.62 vs. 0.36; p < .00001). Lower ratings for industry funded SRs were consistent across all but one AMSTAR 2 item, though significantly lower only for three specific items.

CONCLUSION

The methodological quality of SRs in vaccination is comparable to SRs in other fields, while it is still suboptimal. We are not able to provide a satisfactory explanation why industry funded SRs had a lower methodological quality than non-industry funded SRs over recent years. Industry funding is an important indicator of methodological quality for vaccine SRs and should be carefully considered when appraising SR quality.

Adopting AMSTAR 2 critical appraisal tool for systematic reviews: speed of the tool uptake and barriers for its adoption

Background

In 2007, AMSTAR (A MeaSurement Tool to Assess systematic Reviews), a critical appraisal tool for systematic reviews (SRs), was published, and it has since become one of the most widely used instruments for SR appraisal. In September 2017, AMSTAR 2 was published as an updated version of the tool. This mixed-methods study aimed to analyze the extent of the AMSTAR 2 uptake and explore potential barriers to its uptake.

Methods

We analyzed the frequency of AMSTAR or AMSTAR 2 use in articles published in 2018, 2019 and 2020. We surveyed authors who have used AMSTAR but not AMSTAR 2 in the analyzed time frame to identify their reasons and barriers. The inclusion criterion for those authors was that the month of manuscript submission was after September 2017, i.e. after AMSTAR 2 was published.

Results

We included 871 studies. The majority (N = 451; 52%) used AMSTAR 2, while 44% (N = 382) used AMSTAR, 4% (N = 31) used R-AMSTAR and others used a combination of tools. In 2018, 81% of the analyzed studies used AMSTAR, while 16% used AMSTAR 2. In 2019, 52% used AMSTAR, while 44% used AMSTAR 2. Among articles published in 2020, 28% used AMSTAR, while AMSTAR 2 was used by 69%.

An author survey indicated that the authors did not use AMSTAR 2 mostly because they were not aware of it, their protocol was already established, or data collection completed at the time when the new tool was published. Barriers towards AMSTAR 2 use were lack of quantitative assessment, insufficient awareness, length, difficulties with a specific item.

Conclusion

In articles published in 2018-2020, that were submitted to a journal after AMSTAR 2 tool was published, almost half of the authors (44%) still used AMSTAR, the old version of the tool. However, the use of AMSTAR has been declining in each subsequent year. Our survey indicated that editors and peer-reviewers did not ask the authors to use the new version of the tool. Few barriers towards using AMSTAR 2 were identified, and thus it is anticipated that the use of the old version of AMSTAR will continue to decline.

Quality of systematic reviews supporting the 2017 ACC/AHA and 2018 ESC/ESH guidelines for the management of hypertension

OBJECTIVE

To assess the methodological and reporting quality of systematic reviews (SRs) that informed recommendations in the recent American and European hypertension guidelines.

DESIGN AND SETTINGS

Meta-epidemiological study. We identified SRs that were cited for class I recommendations based on Level of Evidence-A in the 2017 American College of Cardiology/American Heart Association (ACC/AHA) and the 2018 European Society of Cardiology/European Society of Hypertension (ESC/ESH) hypertension guidelines.

MAIN OUTCOME MEASURES

Methodological and reporting quality of the SRs was assessed using A Measurement Tool to Assess Systematic Reviews (AMSTAR-2) checklist and Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) checklist, respectively.

RESULTS

A total of 40 SRs was included in the analysis (28 from 2017 ACC/AHA; 22 from 2018 ESC/ESH and 10 were included in both). Based on the AMSTAR-2 assessment, only 7.5% SRs were found to be of high methodological quality, 47.5% were of moderate, each 22.5% were of low and critically low quality. Based on the PRISMA checklist assessment, a mean of 24 items (SD (2.76) were reported appropriately, and only five SRs reported all 27 items appropriately.

CONCLUSION

Methodological and reporting quality of SRs were found to vary considerably. Lack of information on the funding source of included studies, use of a protocol, integration of risk of bias assessments while interpreting findings and reporting of excluded studies were major methodological deficiencies.

Environmental and Socioeconomic Impacts of Shrimp Farming in the Philippines: A Critical Analysis Using PRISMA

Shrimp aquaculture is under pressure to increase its production to meet the growing demand for food from a growing population. In the Philippines, aquaculture has experienced the shift from milkfish to prawn, with its attractive marketable price. This intensification has led to negative and positive impacts, which have raised a range of environmental and socioeconomic problems. This paper reviews the environmental and socioeconomic challenges that the shrimp aquaculture industry faces using the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) method. We examine the gaps and the changes that are required to revitalize the industry. We examine and assess the impacts of shrimp culture on the environment, e.g., shrimp farm management, marine pollution, disease outbreaks, and the social, economic, and climate change impacts. The presence of viral diseases, such as White Spot Syndrome Virus (WSSV), Monodon Baculovirus (MBV), Infectious Hypodermal and Hematopoietic Necrosis Virus (IHHNV), Hepatopancreatic Parvovirus (HPV), and Yellow Head Virus (YHV), have caused approximate losses in the industry of 40,080 mt in 1997, and 51,000 mt in 2014. Recommended strategies and policy changes are considered for the improvement of shrimp aquaculture, including disease management, the adoption of good aquaculture practices, proper environmental monitoring, sustainable practices at the farm level, and priorities for cooperation among the concerned government agencies and local governments, as well as the involvement of state universities and colleges, for better management practices.

Methodological quality and reporting standards in systematic reviews with meta-analysis of physical activity studies: a report from the Strengthening the Evidence in Exercise Sciences Initiative (SEES Initiative)

BACKGROUND

Several resources have been developed (e.g., reporting guidelines) to promote high-standard practices in health research. However, there was no continuous and systematic assessment of recommended practices in published systematic reviews with meta-analysis (SRMAs), which increases the usability of the available resources. Therefore, we aimed to assess the methodological and reporting standards in SRMAs of physical activity studies. This report presents the main results of the SEES Initiative in 2019.

METHODS

Our approach is based on a prospective systematic review methodology to implement post-publication surveillance of research practices in exercise sciences. Briefly, during the year 2019, pre-specified searches were conducted monthly (PubMed/MEDLINE) in journals from the exercise sciences (n = 9) and medicine (n = 5). The assessments were independently conducted by two authors, based on 36 items/practices derived from established statements/tools (PRISMA, AMSTAR 2, ROBIS). To be eligible, SRMAs should summarize studies that had, at least, one arm consisting of physical activity interventions/exposures and one health or behavioral outcome.

RESULTS

Out of 1028 studies assessed for eligibility, 103 SRMAs were included. The minimum adherence was 13/36 items, whereas only one SRMA adhered to all items. Some highly contemplated items included identification of title as SRMA (97.1%) and descriptions of the main outcome in the abstract (95.1%) and risk of bias (RoB) assessment (95.1%). Some poorly contemplated items included publicly available protocol (4.9%), discussion of the results in light of RoB in studies included (32.0%), and data sharing statements (35.9%).

CONCLUSION

In summary, there is a suboptimal adherence to recommended practices on methodological quality and reporting standards in the SRMAs of physical activity intervention/exposure evaluated from the selected journals in 2019, which likely reduce the reproducibility and usefulness of these studies. This incipient evidence from our first 12 months of post-publication surveillance should serve as a call for attention and action for multiple stakeholders (e.g., authors, reviewers, editors, funders, academic institutions) in this important health research field.

Critical problems associated with climate change: a systematic review and meta-analysis of Philippine fisheries research

This paper proposes to analyze the scientific production on climate change and fisheries in the Philippine context. This research theme was chosen considering the continuous increase in scientific studies related to climate change and fisheries and will therefore help in directing researchers on future directions for research to aid in addressing critical issues in the Philippine fisheries. A total of 69 search articles were extracted using the set criteria, indexed in Web of Science, and Scopus, covering the period from 1960 to 2020. After careful screening for eligibility, twenty-seven full-text articles were chosen for qualitative synthesis. Among the literature reviewed, research foci were categorized into four: impacts assessment (56% or 15 studies) followed by management (22% or 6 studies), adaptation (15% or 4 studies), and perception (7% or 2 studies), and main themes were categorized into four: resource management (59%), economy and livelihood (19%), governance and stakeholder participation (11%), and community marginalization (11%). This review contributes to the literature by identifying research potentials and suggesting a prescriptive approach to Philippine fisheries and climate change studies.

Laparoscopic surgery in patients with cystic fibrosis: A systematic review

INTRODUCTION

Laparoscopic surgery may be advantageous for cystic fibrosis (CF) patients because it leads to fewer complications than open surgery. However, it could still lead to pulmonary and cardiovascular complications in CF patients. We aimed to systematically review the use of laparoscopic surgery in CF patients.

METHODS

A systematic review was performed in compliance with PRISMA guidelines. A literature search was performed using PubMed/MEDLINE, ScienceDirect, EMBASE, and Google Scholar, with "cystic fibrosis and laparoscopic surgery" and "cystic fibrosis and minimally invasive surgery" used as the search terms. The Grading of Recommendations Assessment, Development, and Evaluation (GRADE) criteria were applied. The protocol was registered with the PROSPERO register.

RESULTS

Six studies met the predetermined inclusion criteria; accordingly, two studies provided high-quality evidence and four provided moderate-quality evidence. The interrater correlation was convincing (r_s  = .95, P = .02, two-tailed). Therefore, three quantitative studies and three qualitative studies were assessed and evidence-graded in accordance with the GRADE protocol.

CONCLUSION

The benefits of laparoscopic surgical interventions for patients with CF were supported with good evidential value and recognized as a safe and suitable surgical option.

The methodological quality of systematic reviews on the treatment of adult major depression needs improvement according to AMSTAR 2: A cross-sectional study

Background

Several standards have been developed to assess methodological quality of systematic reviews (SR). One widely used tool is the AMSTAR. A recent update - AMSTAR 2 - is a 16 item evaluation tool that enables a detailed assessment of SR that include randomised (RCT) or non-randomised studies (NRS) of healthcare interventions.

Methods

A cross-sectional study of SR on pharmacological or psychological interventions in major depression in adults was conducted. SR published during 2012–2017 were sampled from MEDLINE, EMBASE and the Cochrane Database of SR. Methodological quality was assessed using AMSTAR 2. Potential predictive factors associated with quality were examined.

Results

In rating overall confidence in the results of 60 SR four reviews were rated “high”, two were “moderate”, one was “low” and 53 were “critically low”. The mean AMSTAR 2 percentage score was 45.3% (standard deviation 22.6%) in a wide range from 7.1% to 93.8%. Predictors of higher quality were: type of review (higher quality in Cochrane Reviews), SR including only randomized trials and higher journal impact factor.

Limitations

AMSTAR 2 is not intended to be used for the generation of a percentage score.

Conclusions

According to AMSTAR 2 the overall methodological quality of SR on the treatment of adult major depression needs improvement. Although there is a high need for summarized information in the field of mental health, this work demonstrates the need to critically assess SR before using their findings. Better adherence to established reporting guidelines for SR is needed.

Use of AMSTAR-2 in the methodological assessment of systematic reviews: protocol for a methodological study

Background

Systematic reviews (SRs) with or without meta-analyses (MAs) are widely used in resolving questions in various healthcare areas (such as, traditional Chinese medicine, public health and surgery), and they are the cornerstone of evidence-based healthcare. However, the reliability of SRs is typically influenced by their methodological quality. AMSTAR (A Measurement Tool to Assess Systematic Reviews) and AMSTAR-2 tools can assess the methodological quality of SRs, and the use of AMSTAR has been investigated. However, AMSTAR-2 is now widely used to evaluate the methodological quality of SRs, but the use of AMSTAR-2 for determining the methodological quality of SRs has not yet been investigated and assessed thoroughly. Thus, we designed the present study to investigate the use of AMSTAR-2 in studies that assessed the methodological quality of a sample of SRs with the AMSTAR-2 and provide references to potential users of AMSTAR-2 tool.

Methods

Four commonly used electronic databases including PubMed, EmBase, the Cochrane Library, and Web of Science will be searched following a comprehensive search strategy to identify and retrieve studies that have used AMSTAR-2 tool for evaluating the methodological quality of SRs. Two independent authors will retrieve bibliometric information and methodological data, including all author names, time of publication, and journal names, whether a specific score value was given for each item, and whether overall quality assessment was performed. Descriptive statistical analyses will be used to present the study results, e.g., frequencies and percentages, mean and standard deviation (SD) or median and interquartile range (IQR). In addition, subgroup analyses will be performed to identify the methodological differences (e.g., the reporting of study designs included in SRs) between overviews and methodological studies. The risk ratio (RR) with 95% confidence interval (95% CI) will be calculated to measure the methodological differences. Cytoscape 3.7.1 software tool will be used to construct collaboration network maps. Further, Microsoft Office Excel 2016 and Stata 12.0 will be used to manage and analyze data.

Discussion

The results of this study will identify any gaps in the use of AMSTAR-2 and important bibliometric features, such as active researchers and journals, provide guidance to researchers in various healthcare areas (such as, traditional Chinese medicine and public health) for using AMSTAR-2 tool and help them in developing cooperation and submitting their manuscripts.

Testing the Preliminary Validity of a Multidimensional Framework for Studying the Effects of Cancer Health Literacy on Cancer Screening Behaviors among Diverse Populations

The objective of this study was to evaluate the applicability of a multidimensional framework to explore factors associated with cancer literacy and its effects on receiving cancer screenings among diverse populations. Based on the conceptual framework, we developed and pilot-tested the Multidimensional Cancer Literacy Questionnaire (MCLQ) among 1500 individuals (African Americans, Latinos and Whites) in Louisiana. Exploratory factor analysis was used to identify the MCLQ underlying structure and predominant factors explaining each of the dimensions in the model. A total of 82 items (explaining 67% of the total variance) in the MCLQ were grouped into 20 factors associated with three key dimensions related to cancer literacy. Preliminary validity of the MCLQ was supported: Cronbach alpha for the scale score was 0.89 and internal consistency reliability coefficients for each factor were all above 0.67. The Facilitators Domain included five factors (28 items) that may positively influence individuals to have early-detection cancer screenings. The Barriers Domain included seven factors (26 items) explaining aspects that may negatively influence individuals to have cancer screenings. The Cultural Domain included eight factors (28 items) related to aspects that influence positively or negatively individuals' perceptions regarding cancer as a disease, screenings and treatments. A multidimensional framework to study cancer literacy, including cultural attitudes, beliefs and practices, as well as facilitators and barriers, among diverse populations, will increase understanding of factors influencing individuals' approach to cancer prevention and screening. Results will inform further testing of the multidimensional framework and questionnaire.

Poor Quality in Systematic Reviews on PTSD and EMDR - An Examination of Search Methodology and Reporting

Background: Different user groups regard systematic reviews as reliable and valuable sources for answering research questions. For systematic reviews to fulfill their purpose, methodological quality in all stages are of importance. The studies identified in a systematic search form the basis of the review, thus the search process methodology is important for both performing and reporting the search. The purpose of the present study was to evaluate the quality of non-Cochrane systematic reviews by analyzing how they perform and report the search. This is exemplified by systematic reviews on eye movement desensitization and reprocessing (EMDR), a trauma-focused therapy commonly used for post-traumatic stress disorder (PTSD).

Methods and Results: We examined the method chapters of 20 systematic reviews on the subject, and rated their searches and reporting using relevant elements from the Cochrane Handbook and PRISMA. We found inadequacies in the methods employed for searching and reporting the search strategy, which could have been avoided by greater adherence to guiding documents for performing systematic reviews.

Conclusion: Our findings raise important questions for future debate on the risk of omitting studies, thus impairing the conclusions in a systematic review. For clinical purposes, researchers should investigate if, and how, the search strategy in a systematic review affects the body of knowledge and the results.

Methodological steps used by authors of systematic reviews and meta-analyses of clinical trials: a cross-sectional study

Background

The quality of systematic reviews and meta-analyses (SR/MAs) depends on the extent of the methods used. We investigated the methodological steps used by authors of SR/MAs of clinical trials via an author survey.

Methods

We conducted an email-based cross-sectional study by contacting corresponding authors of SR/MAs that were published in 2015 and 2016 and retrieved through the PubMed database. The 27-item questionnaire was developed to study the methodological steps used by authors when conducting a SR/MA and the demographic characteristics of the respondent. Besides the demographic characteristics, methodological questions regarding the source, extraction and synthesis of data were included.

Results

From 10,292 emails sent, 384 authors responded and were included in the final analysis. Manual searches were carried out by 69.2% of authors, while 87.3% do updated searches, 49.2% search grey literature, 74.9% use the Cochrane tool for risk of bias assessment, 69.8% assign more than two reviewers for data extraction, 20.5% use digital software to extract data from graphs, 57.9% use raw data in the meta-analysis, and 43.8% meta-analyze both adjusted and non-adjusted data. There was a positive correlation of years of experience in conducting of SR/MAs with both searching grey literature (P = 0.0003) and use of adjusted and non-adjusted data (P = 0.006).

Conclusions

Many authors still do not carry out many of the vital methodological steps to be taken when performing any SR/MA. The experience of the authors in SR/MAs is highly correlated with use of the recommended tips for SR/MA conduct. The optimal methodological approach for researchers conducting a SR/MA should be standardized.

Electronic supplementary material

The online version of this article (10.1186/s12874-019-0780-2) contains supplementary material, which is available to authorized users.