Most systematic reviews of high methodological quality on psoriasis interventions are classified as high risk of bias using ROBIS tool

Risk of Bias and Methodological Critical Appraisal in Systematic Reviews of Non- and Micro-Invasive Caries Management for Primary and Permanent Teeth

INTRODUCTION

Pediatric dentistry should rely on evidence-based clinical decisions supported by high-quality, unbiased systematic reviews (SRs). Therefore, the purpose of this study was to systematically evaluate the methodological quality and risk of bias of SRs focused on non- and micro-invasive treatment for caries lesions in primary and permanent teeth.

METHODS

A comprehensive search was conducted in multiple databases, including MEDLINE/PubMed, Scopus, Web of Science, EMBASE, Epistemonikos, and ProQuest, up to March 2023 to identify relevant systematic reviews (SRs) focused on non- and micro-invasive caries treatment. Two independent reviewers extracted data from the included SRs and assessed the methodological quality and risk of bias using the AMSTAR 2 and ROBIS tools, respectively.

RESULTS

A total of 39 SRs were included in the analysis. Among these, 27 SRs (69.2%) were assessed as having critically low methodological quality, 11 SRs (28.2%) were considered to have low methodological quality, and only one SR was rated as high-quality. The primary concern identified was the absence of protocol registration before commencing of the study, observed in 33 SR when using the AMSTAR 2 tool. According to the ROBIS tool, 21 studies (53.8%) were categorized as low risk of bias, 10 (25.6%) as high risk, and eight (20.5%) as unclear risk of bias.

CONCLUSION

Our analysis revealed that SRs focused on non- and micro-invasive treatment for caries in children and adolescents had critically low methodological quality according to the AMSTAR 2 tool but demonstrated a low risk of bias based on the ROBIS tool. These findings highlight the importance of emphasizing prospective protocol registration, transparent reporting of statistical analyses, and addressing potential bias implications within this topic. By addressing these issues, we can enhance the quality of SRs and ensure that clinical decisions rely on unbiased and trustworthy evidence.

Evolution of appraisal tool usage preferences in PROSPERO records: a study of non-Cochrane systematic reviews

OBJECTIVES

This research-on-research substudy uses a data-driven approach to investigate the range of appraisal tools in non-Cochrane systematic reviews and meta-analyses registered in the International Prospective Register of Systematic Reviews (PROSPERO).

STUDY DESIGN AND SETTING

A comprehensive web scraping of all completed non-Cochrane registrations in PROSPERO from February 2011 to December 2017 was performed. The focus was classifying the appraisal tools based on study type, assessment aspects, and research topics.

RESULTS

After analyzing 17,708 complete records, we found a predominant use of methodological quality assessment tools compared to those for reporting quality or risk of bias (RoB). This indicates a greater emphasis on methodological rigor in the studied protocols. Various tools for assessing methodological quality were observed, reflecting the complexity of such evaluations. Instruments designed for evaluating methodological or reporting quality were mainly intended for non-randomized clinical trials or observational studies, unlike RoB tools more commonly used in randomized clinical trials. No distinct trends in tool usage were observed in specific research conditions or domains, suggesting that tool choice is influenced more by study design than research topic.

CONCLUSION

This study provides insights into the preferential use of various assessment tools in conducting non-Cochrane systematic reviews, as evidenced in PROSPERO records. The findings reveal various methodological assessment tools, underscoring their versatility across different study designs and research areas.

The methodological quality assessment of systematic reviews/meta-analyses of chronic prostatitis/chronic pelvic pain syndrome using AMSTAR2

BACKGROUND

This study aimed to assess the methodological quality of the systematic reviews/meta-analyses (SRs/MAs) of chronic prostatitis/chronic pelvic pain syndrome (CP/CPPS) using A Measurement Tool to Assess systematic Reviews (AMSTAR2) and to explore the potential influencing factors.

METHODS

PubMed, EMBASE and Cochrane Library databases were searched for relevant studies. AMSTAR2 was used for evaluating the methodological quality of eligible SRs/MAs. Differences between methodological characteristics of SRs/MAs were compared using chi-square tests. The intra-class correlation coefficient (ICC) was used to assess reviewer agreement in the pre-experiment. Multivariate regression analysis was used to identify potential factors affecting methodological quality.

RESULTS

A total of 45 SRs/MAs were included. After AMSTAR2 evaluation, only two (4.4%) of 45 SRs/MAs were moderate, three (6.7%) were rated as low quality, and the remainder 40 (88.9%) were rated as critically low quality. Among the 16 items of AMSTAR2, item 3 and item 10 had the poorest adherence. Item 4 received the most significant number of "Partial Yes" responses. Univariable analysis indicated that there were significant differences in methodological quality in SRs between different continents (P = 0.027) as well as between preregistered SRs and those that were not (P = 0.004). However, in multivariate analysis, there was no significant association between methodological quality and the following research characteristics: publication year, continent, whether reporting followed Preferred Reporting Items for Systematic Reviews (PRISMA), preregistration, funding support, randomized controlled trials (RCT) enrollment, whether SR was published in the Cochrane Database of Systematic Reviews (CDSR), and whether with meta-analysis. Additionally, subgroup analysis based on interventional SRs/MAs showed that continent was independently associated with the methodological quality of SRs/MAs of CP/CPPS via univariable and multivariate analysis.

CONCLUSIONS

Our study demonstrates that the methodological quality of SRs/MAs of CP/CPPS was generally poor. SRs/MAs of CP/CPPS should adopt the AMSTAR2 to enhance their methodological quality.

The Effectiveness of Tai Chi for Knee Osteoarthritis: An Overview of Systematic Reviews

Background

Knee Osteoarthritis (KOA) has become a serious health issue for elderly patients. Several systematic reviews (SRs) have reported Tai Chi has widely been used in the treatment of KOA. However, the methodology and conclusions of these SRs are controversial. This overview aims to summarize and evaluate the available evidence for the efficacy and safety of Tai Chi for KOA.

Methods

Two independent researchers searched eight databases from the inception to April 30, 2022. The included SRs were assessed respectively by A Measurement Tool to Assess Systematic Reviews (AMSTAR) 2, the Risk of Bias in Systematic Reviews (ROBIS) tool, and the Preferred Reporting Item for Systematic Review and Meta-analysis (PRISMA) statement. The Grading of Recommendations Assessment, Development, and Evaluation (GRADE) tool was used to assess the quality of the included SRs in terms of outcome indicators.

Results

Six SRs were finally included in this overview. The results of methodological quality, reporting quality, and risk of bias of the included SRs were generally unsatisfactory. The limitations were a lack of explaining the reasons for selection, a list of excluded literature, reporting bias assessment, and reporting the potential sources of conflict of interest. In addition, only 1 item was assessed as moderate quality by using the GRADE tool. Limitations were the most common downgraded factors.

Conclusion

Tai Chi is effective as a non-pharmacological intervention in the integrative treatment of KOA. However, the quality of evidence and methodological quality of SRs is generally unsatisfactory, suggesting that these results must be interpreted with caution.

Trial Registration/Protocol Registration

PROSPERO registration number: CRD42022315146.

Quality of systematic reviews on timing of complementary feeding for early childhood allergy prevention

BACKGROUND

Only rigorously prepared analyses can provide the highest level of evidence to inform decision-making. Several recent systematic reviews (SRs) examined the hypothesis that the early introduction of specific allergenic complementary foods (CFs) to infants may lead to a lower incidence of one or more allergic outcomes. However, the methodological rigour and quality of reporting of SRs in this area has not yet been systematically evaluated.

METHODS

We comprehensively searched PubMed, Medline (Ovid), and Web of Science Core Collection on 13th January 2022, using a pre-specified and tested search syntax for SRs with RCT evidence on the early introduction of allergenic CFs as a means for allergy prevention in infants and children. We examined the quality and risk of bias (RoB) using AMSTAR-2 and ROBIS tools, examined adherence to the Preferred Reporting Items for SRs and Meta-Analyses (PRISMA), and checked whether certainty of the evidence was assessed.

RESULTS

Twelve SRs were included. Application of both tools resulted in similar overall judgements in terms of direction and extent for nine of the 12 SRs. Nine SRs were found to be of critically low to low quality according to AMSTAR-2 and to be at high RoB according to ROBIS. One SR received a moderate quality rating (AMSTAR-2) and high RoB rating (ROBIS). However, for two SRs, judgements between AMSTAR-2 and ROBIS were at stark variance. Only two SRs fully adhered to the PRISMA checklist. Six SRs evaluated the certainty of the body of RCT evidence. Several SRs failed to consider unpublished studies either by an explicit a priori exclusion or by inadequate search strategies.

CONCLUSIONS

Well-conducted SRs are important for decision-making and informing guideline development, the quality of their methodology should therefore be considered. The methodological rigour and the reporting quality of SRs on the timing of CF for allergy prevention must be improved.

REGISTRATION

https://osf.io/7cs4b .

Efficacy of Surgical Masks Versus N95 Respirators for the Prevention of COVID-19 in Dental Settings: A Systematic Review

Coronavirus disease 2019 (COVID-19) is an infectious disease caused by severe acute respiratory syndrome coronavirus 2. (SARS-CoV-2). It spreads mainly through saliva droplets or nasal discharge. Dentists are among the professionals with the greatest risk of contracting and transmitting COVID-19. We compared the efficacy of surgical masks versus N95 respirators in preventing COVID-19 infection in dental settings. PubMed, Scopus, Web of Science, and Cochrane Library databases were searched. Search terms corresponded to a predefined PICOS (patient/population, intervention, comparison, and outcomes) question. The risk of bias was evaluated using AMSTAR-2 (A Measurement Tool to Assess Systematic Reviews-2), ROBIS (Risk of Bias in Systematic Reviews), and Health Evidence tools. A total of 191 articles were screened, and nine of them were further evaluated for eligibility, of which five articles (fulfilled the selection criteria) and were included in this study. Two studies concluded that surgical masks could provide equivalent protection to N95 respirators. Another study found that N95 respirators were superior to surgical masks. The fourth study found that better protection can be achieved when using surgical masks by the aerosol source than when the recipient uses an N95 respirator, while the last study concluded that surgical masks or N95 respirators alone do not provide full protection. Thus, according to this systematic review, N95 respirators provide better protection against COVID-19 infection compared to surgical masks.

Mandibular full-arch fixed prostheses supported by three-dental-implants: A protocol of an overview of reviews

INTRODUCTION

To minimize trauma and cost of treatment, oral health practitioners have successfully rehabilitated full arches by supporting the prostheses on four implants. However, there is no consensus whether less than four implants supporting full mandibular arches would provide similar clinical outcomes to other well-established all-on-four alternative.

OBJECTIVE

To identify, summarize, appraise, and compare the clinical outcomes evidence of three-implant fixed full-arch prostheses in completely edentulous mandibular patients.

MATERIALS AND METHODS

This overview of systematic reviews (OoSRs) will include secondary synthesis studies (i.e., systematic reviews with or without a meta-analysis). A three-step search strategy will be conducted in MEDLINE (Ovid), EMBASE (Ovid), Cochrane Database of Systematic Reviews, Scopus, Web of Science (WoS Core Collection), and Google Scholar. Grey literature and a manual search in 12 specialized journals will also be conducted. Three independent reviewers will screen all retrieved articles for eligibility, extract data and assess the methodological quality of the included studies. The results will be presented as tables or narrative synthesis. The studies will be evaluated for risk of bias by ROBIS and methodology quality by the AMSTAR-2 tool. If new primary studies are identified, a meta-analysis will be conducted. Certainty of the evidence will be assessed to answer the following focused research question: In edentulous mandibular patients, what are the implant and prostheses clinical outcomes of three-implant fixed full-arch prostheses compared to other all-on-x solutions?

DISCUSSION

There are some systematic reviews about the use of fixed complete dental prostheses supported by three implants; however, their clinical outcomes related to the other all-on-four plus solutions are conflicting. So, an overview on this topic is required to provide recommendations.

REVIEW REGISTRATION NUMBER

International prospective register of systematic reviews (PROSPERO) ID#: CRD42021262175. National Institute for Health Research (NIHR) and Centre for Reviews and Dissemination, University of York, York, United Kingdom.

Methodological quality and risk of bias in orthodontic systematic reviews using AMSTAR and ROBIS

BACKGROUND

This study aimed to compare the methodological quality and risk bias of orthodontic systematic reviews (SRs) using the AMSTAR (Assessing the Methodological Quality of Systematic Reviews) and ROBIS (Risk of Bias in Systematic Review) tools.

MATERIALS AND METHODS

A search of electronic databases (OVID and Medline) was undertaken to identify orthodontic SRs published in five primary orthodontic journals (January 2015 to December 2018) and the Cochrane Library of Systematic Reviews (January 2000 to January 2018). Full articles were reviewed by two assessors against the eligibility criteria. Methodological quality of each SR was gauged using the AMSTAR tool with a score of 0 or 1 given for each of the 11 items. Cumulative totals were calculated and scores between 4 and 8 represented poor to fair methodological quality and 9 or greater deemed to be good. As per the ROBIS tool, the risk of bias (ROB) for each domain was assessed and the overall ROB was classified as low, high, or unclear.

RESULTS

A total of 91 SRs were included. The median AMSTAR score was 8 (IQR = 3). The methodological quality of 47.3% SRs was rated good. SRs without protocol registration (Coef: -3.00, 95% CI: -3.72, -2.28, P < 0.001) and American continent SRs (Coef: -1.00, 95% CI: -1.72, -0.21, P = 0.007) were associated with lower AMSTAR scores. A total of 56.0% SRs were rated a low ROB, with a lower ROB apparent in multicentre SRs (OR: 0.27, 95% CI: 0.11, 0.64, P = 0.003) and a higher ROB evident in SRs without a registered protocol (OR: 111.81, 95% CI: 22.34, 559.62, P < 0.001). When adjusted for the effect of AMSTAR score on ROB, a higher ROB was associated with SRs without protocol registration (OR: 32.24, 95% CI: 6.03, 172.44, P ≤ 0.001). As the AMSTAR score (per unit) increased, the odds of having a high ROB rating decreased (OR: 0.31, 95% CI: 0.21, 0.45, P ≤ 0.001).

CONCLUSIONS

As the methodological quality rating of orthodontic SRs increases, a reduction in the ROB is evident.

Methodological issues of systematic reviews and meta-analyses in the field of sleep medicine: A meta-epidemiological study

An increasing number of systematic reviews and meta-analyses (SRMAs) have been published in the field of sleep medicine. We evaluated the methodological issues of these SRMAs. A protocol was developed in advance. Three databases were searched from inception to October 2019 for SRMAs published in major academic journals of sleep medicine that assessed healthcare interventions. The AMSTAR 2.0 instrument was used to evaluate the methodological issues and a multivariable regression analysis was conducted to investigate potential measures associated with methodological validity. We identified 163 SRMAs. The median number of missing safeguards of these SRMAs was 7 out of 16 (Interquartile range, IQR: 6-9), and on average, two of these missing safeguards were critical weaknesses. Our regression analysis suggested that SRMAs published in recent years (β = 0.16; 95%CI: 0.08, 0.24; p = 0.002), with the first author from Europe (β = 0.08; 95%CI: 0.02, 0.14; p = 0.013) tend to have higher relative methodological ranks. In conclusion, the methodological validity for current SRMAs in sleep medicine was poor. Further efforts to improve the methodological validity are needed.

Quality Assessment and Risk of Bias of Systematic Reviews of Prophylactic Mesh for Parastomal Hernia Prevention Using AMSTAR and ROBIS Tools

BACKGROUND

Systematic reviews play a crucial role in clinical decision making and resource allocation and are expected to be unbiased and consistent. The aim of this study is a review of systematic reviews on the use of prophylactic mesh to prevent parastomal hernia (PH) formation using ROBIS and AMSTAR tools to assess the risk of bias and methodological quality.

METHODS

We included systematic reviews with or without meta-analysis of which the objective was to assess the use of a prophylactic mesh to prevent PH. A systematic search of the literature in five databases from inception until December 2017 was conducted. For each systematic review, methodologic quality and risk of bias were assessed using the AMSTAR and ROBIS tools, respectively. We estimated the inter-rater reliability for individual domains and for the overall methodological quality and risk of bias using Fleiss' k.

RESULTS

We identified 14 systematic reviews that met the inclusion criteria. Using the AMSTAR scale with a cutoff value, six reviews showed high methodologic quality and eight were of low quality. Using the ROBIS tool, the overall risk of bias was low in 50% of the reviews analyzed. In the remaining studies, the risk of bias was unclear.

CONCLUSIONS

The global evidence in favor of the use of a prophylactic mesh for preventing PH is not uniform regarding quality and risk of bias. Surgeons cannot be equally confident in the results of all systematic reviews published on this topic.

Assessing the methodological quality of systematic reviews of interventions aimed at improving vaccination coverage using AMSTAR and ROBIS checklists

Introduction: Systematic reviews (SRs) are the backbone of evidence-based health care, but no gold standard exists to assess their methodological quality. Although the AMSTAR tool is accepted for analyzing the quality of SRs, the ROBIS instrument was recently developed. This study compared the capacity of both instruments to capture the quality of SRs of interventions for improving vaccination coverage.Methods: We conducted a comprehensive literature search in the Cochrane Library and PubMed. Two reviewers independently screened the search output, assessed study eligibility, and extracted data from eligible SRs; resolving differences through consensus. We conducted Principal Component Analysis (PCA) in Stata 14 to determine similarities and differences between AMSTAR and ROBIS.Results: A total of 2322 records were identified through the search and 75 full-text publications were assessed for eligibility, of which 57 met inclusion criteria. Using AMSTAR, we found 32%, 60% and 9% of SRs to have high, moderate and low quality, respectively. With ROBIS, we judged 74%, 14% and 12% of SRs to have low, unclear and high risk of bias. PCA showed that SRs with low risk of bias in ROBIS clustered together with SRs having high-quality in AMSTAR, and SRs with high risk of bias in ROBIS clustered with low-quality SRs in AMSTAR.Conclusions: Our findings suggest that there is an association between methodological quality and risk of bias in SRs of interventions focused on improving vaccination coverage. Therefore, either AMSTAR or ROBIS checklists can be used to evaluate methodological quality of SRs in vaccinology.

A psychometric study found AMSTAR 2 to be a valid and moderately reliable appraisal tool

OBJECTIVES

The objectives of this study were to determine the interrater reliability (IRR) of assessment of multiple systematic reviews (AMSTAR) 2 for reviews of pharmacological or psychological interventions for the treatment of major depression, to compare it to that of AMSTAR and risk of bias in systematic reviews (ROBIS), and to assess the convergent validity between the appraisal tools.

STUDY DESIGN AND SETTING

Two groups of four raters were each assigned one of two samples of 30 systematic reviews. All eight raters applied AMSTAR 2 to their sample. Each group also applied either AMSTAR or ROBIS. Fleiss' kappa and Gwet's AC₁ were calculated, and agreement between the tools was assessed.

RESULTS

The median kappa values as a measure of IRR indicated a moderate agreement for AMSTAR 2 (median = 0.51), a substantial agreement for AMSTAR (median = 0.62), and a fair agreement for ROBIS (median = 0.27). Validity results showed a positive association for AMSTAR and AMSTAR 2 (r = 0.91) as well as ROBIS and AMSTAR 2 (r = 0.84). For the overall rating, AMSTAR 2 showed a high concordance with ROBIS and a lower concordance with AMSTAR.

CONCLUSION

The IRR of AMSTAR 2 was found to be slightly lower than the IRR of AMSTAR and higher than the IRR of ROBIS. Validity measurements indicate that AMSTAR 2 is closely related to both ROBIS and AMSTAR.

Protocol registration or development may benefit the design, conduct and reporting of dose-response meta-analysis: empirical evidence from a literature survey

Background

To investigate the prevalence of protocol registration (or development) among published dose-response meta-analyses (DRMAs), and whether DRMAs with a protocol are better than those not.

Methods

Three databases were searched for eligible DRMAs. The modified AMSTAR (14 items) and PRISMA checklists (26 items) were used to assess the methodological and reporting quality, with each item assigned 1 point if it met the requirement or 0 if not. We matched (1,2) DRMAs with registered or published protocol to those not, by region and publication years. The summarized quality score and compliance rate of each item were compared between the two groups. Multivariable regression was employed to see if protocol registration or development was associated with total quality score.

Results

We included 529 DRMAs, with 45 (8.51%) completed protocol registration or development. We observed a higher methodological score for DRMAs with protocol than the matched controls (9.47 versus 8.58, P <  0.01); this embodied in 4 out of 14 items of AMSTAR [e.g., Duplicate data extraction (rate difference, RD = 0.17, 95% CI: 0.04, 0.30; P = 0.01). A higher reporting score (cubic transformed) for DRMAs with protocol than the matched controls was also observed (11,875.00 versus 10,229.53, P <  0.01); which embodied in 6 out of 26 items of PRISMA [e.g. Describe methods for publication bias (RD = 0.08, 95% CI: 0.01, 0.14; P = 0.02)]. Regression analysis suggested positive association between protocol registration or development and total reporting score (P = 0.012) while not for methodological score (P = 0.87).

Conclusions

Only a small proportion of DRMAs completed protocol registration or development, and those with protocol were better reported than those not. Protocol registration or development is highly desirable.

Electronic supplementary material

The online version of this article (10.1186/s12874-019-0715-y) contains supplementary material, which is available to authorized users.

Evolution of international collaborative research efforts to develop non-Cochrane systematic reviews

This research-on-research study describes efforts to develop non-Cochrane systematic reviews (SRs) by analyzing demographical and time-course collaborations between international institutions using protocols registered in the International Prospective Register of Systematic Reviews (PROSPERO) or published in scientific journals. We have published an a priori protocol to develop this study. Protocols published in scientific journals were searched using the MEDLINE and Embase databases; the query terms "Systematic review" [Title] AND "protocol" [Title] were searched from February 2011 to December 2017. Protocols registered at PROSPERO during the same period were obtained by web scraping all non-Cochrane records with a Python script. After excluding protocols that had a fulfillment or duplication rate of less than 90%, they were classified as published "only in PROSPERO", "only in journals", or in "journals and PROSPERO". Results of data and metadata extraction using text mining processes were curated by two reviewers. These Datasets and R scripts are freely available to facilitate reproducibility. We obtained 20,814 protocols of non-Cochrane SRs. While "unique protocols" by reviewers' institutions from 60 countries were the most frequent, a median of 6 (2-150) institutions from 130 different countries were involved in the preparation of "collaborative protocols". The highest Ranked countries involved in overall protocol production were the UK, the U.S., Australia, Brazil, China, Canada, the Netherlands, Germany, and Italy. Most protocols were registered only in PROSPERO. However, the number of protocols published in scientific journals (924) or in both PROSPERO and journals (807) has increased over the last three years. Syst Rev and BMJ Open published more than half of the total protocols. While the more productive countries were involved in "unique" and "collaborative protocols", less productive countries only participated in "collaborative protocols" that were mainly published in PROSPERO. Our results suggest that, although most countries were involved in solitary production of protocols for non-Cochrane SRs during the study period, it would be useful to develop new strategies to promote international collaborations, especially with less productive countries.

Reporting bias in the literature on the associations of health-related behaviors and statins with cardiovascular disease and all-cause mortality

Reporting bias in the literature occurs when there is selective revealing or suppression of results, influenced by the direction of findings. We assessed the risk of reporting bias in the epidemiological literature on health-related behavior (tobacco, alcohol, diet, physical activity, and sedentary behavior) and cardiovascular disease mortality and all-cause mortality and provided a comparative assessment of reporting bias between health-related behavior and statin (in primary prevention) meta-analyses. We searched Medline, Embase, Cochrane Methodology Register Database, and Web of Science for systematic reviews synthesizing the associations of health-related behavior and statins with cardiovascular disease mortality and all-cause mortality published between 2010 and 2016. Risk of bias in systematic reviews was assessed using the ROBIS tool. Reporting bias in the literature was evaluated via small-study effect and excess significance tests. We included 49 systematic reviews in our study. The majority of these reviews exhibited a high overall risk of bias, with a higher extent in health-related behavior reviews, relative to statins. We reperformed 111 meta-analyses conducted across these reviews, of which 65% had statistically significant results (P < 0.05). Around 22% of health-related behavior meta-analyses showed small-study effect, as compared to none of statin meta-analyses. Physical activity and the smoking research areas had more than 40% of meta-analyses with small-study effect. We found evidence of excess significance in 26% of health-related behavior meta-analyses, as compared to none of statin meta-analyses. Half of the meta-analyses from physical activity, 26% from diet, 18% from sedentary behavior, 14% for smoking, and 12% from alcohol showed evidence of excess significance bias. These biases may be distorting the body of evidence available by providing inaccurate estimates of preventive effects on cardiovascular and all-cause mortality.

Relationships between abstract features and methodological quality explained variations of social media activity derived from systematic reviews about psoriasis interventions

OBJECTIVES

The aim of this study was to describe the relationship among abstract structure, readability, and completeness, and how these features may influence social media activity and bibliometric results, considering systematic reviews (SRs) about interventions in psoriasis classified by methodological quality.

STUDY DESIGN AND SETTING

Systematic literature searches about psoriasis interventions were undertaken on relevant databases. For each review, methodological quality was evaluated using the assessing the methodological quality of systematic reviews tool. Abstract extension, structure, readability, and quality and completeness of reporting were analyzed. Social media activity, which consider Twitter and Facebook mention counts, as well as Mendeley readers and Google scholar citations were obtained for each article. Analyses were conducted to describe any potential influence of abstract characteristics on review's social media diffusion.

RESULTS

We classified 139 intervention SRs as displaying high/moderate/low methodological quality. We observed that abstract readability of SRs has been maintained high for last 20 years, although there are some differences based on their methodological quality. Free format abstracts were most sensitive to the increase of text readability as compared with more structured abstracts (Introduction, Methods, Results, and Discussion or eight headings), yielding opposite effects on their quality and completeness depending on the methodological quality: a worsening in low quality reviews and an improvement in those of high quality. Both readability indices and preferred reporting items of systematic reviews and meta-analyses for Abstract total scores showed an inverse relationship with social media activity and bibliometric results in high methodological quality reviews but not in those of lower quality.

CONCLUSION

Our results suggest that increasing abstract readability must be specially considered when writing free format summaries of high-quality reviews because this fact correlates with an improvement of their completeness and quality, and this may help to achieve broader social media visibility and article usage.

Making progress with the automation of systematic reviews: principles of the International Collaboration for the Automation of Systematic Reviews (ICASR)

Systematic reviews (SR) are vital to health care, but have become complicated and time-consuming, due to the rapid expansion of evidence to be synthesised. Fortunately, many tasks of systematic reviews have the potential to be automated or may be assisted by automation. Recent advances in natural language processing, text mining and machine learning have produced new algorithms that can accurately mimic human endeavour in systematic review activity, faster and more cheaply. Automation tools need to be able to work together, to exchange data and results. Therefore, we initiated the International Collaboration for the Automation of Systematic Reviews (ICASR), to successfully put all the parts of automation of systematic review production together. The first meeting was held in Vienna in October 2015. We established a set of principles to enable tools to be developed and integrated into toolkits.This paper sets out the principles devised at that meeting, which cover the need for improvement in efficiency of SR tasks, automation across the spectrum of SR tasks, continuous improvement, adherence to high quality standards, flexibility of use and combining components, the need for a collaboration and varied skills, the desire for open source, shared code and evaluation, and a requirement for replicability through rigorous and open evaluation.Automation has a great potential to improve the speed of systematic reviews. Considerable work is already being done on many of the steps involved in a review. The 'Vienna Principles' set out in this paper aim to guide a more coordinated effort which will allow the integration of work by separate teams and build on the experience, code and evaluations done by the many teams working across the globe.

Quality assessment versus risk of bias in systematic reviews: AMSTAR and ROBIS had similar reliability but differed in their construct and applicability

OBJECTIVES

The objective of the study was to assess the inter-rater reliability (IRR) of AMSTAR and ROBIS in judging individual domains and overall methodological quality/risk of bias of systematic reviews, the concurrent validity of the tools, and the time required to apply them.

STUDY DESIGN AND SETTING

This is a cross-sectional study. Five raters independently read 31 systematic reviews and applied AMSTAR and ROBIS. Fleiss' k for multiple raters for individual domains and overall methodological quality/risk of bias was calculated. Similar domains assessed by both tools and final scores were matched to explore the concurrent validity, using the Kendall tau correlation.

RESULTS

IRR ranged from fair to perfect for AMSTAR and from moderate to substantial for ROBIS. Kappa for overall quality/risk of bias was 0.73 (95% confidence interval [CI] 0.65-0.81) for AMSTAR and 0.64 (95% CI 0.54-0.74) for ROBIS. We judged most of the reviews at intermediate quality with AMSTAR (53%), while judgments were split in high (53%) and low (47%) risk of bias with ROBIS. The correlation between judgments on similar domains ranged from moderate to high, while it was fair on the overall judgment (K = 0.35, 95% CI 0.21-0.49). The mean time to complete ROBIS was about double that for AMSTAR.

CONCLUSION

AMSTAR and ROBIS offer similar IRR but differ in their construct and applicability.

Abstract analysis method facilitates filtering low-methodological quality and high-bias risk systematic reviews on psoriasis interventions

BACKGROUND

Article summaries' information and structure may influence researchers/clinicians' decisions to conduct deeper full-text analyses. Specifically, abstracts of systematic reviews (SRs) and meta-analyses (MA) should provide structured summaries for quick assessment. This study explored a method for determining the methodological quality and bias risk of full-text reviews using abstract information alone.

METHODS

Systematic literature searches for SRs and/or MA about psoriasis were undertaken on MEDLINE, EMBASE, and Cochrane database. For each review, quality, abstract-reporting completeness, full-text methodological quality, and bias risk were evaluated using Preferred Reporting Items for Systematic Reviews and Meta-analyses for abstracts (PRISMA-A), Assessing the Methodological Quality of Systematic Reviews (AMSTAR), and ROBIS tools, respectively. Article-, author-, and journal-derived metadata were systematically extracted from eligible studies using a piloted template, and explanatory variables concerning abstract-reporting quality were assessed using univariate and multivariate-regression models. Two classification models concerning SRs' methodological quality and bias risk were developed based on per-item and total PRISMA-A scores and decision-tree algorithms. This work was supported, in part, by project ICI1400136 (JR). No funding was received from any pharmaceutical company.

RESULTS

This study analysed 139 SRs on psoriasis interventions. On average, they featured 56.7% of PRISMA-A items. The mean total PRISMA-A score was significantly higher for high-methodological-quality SRs than for moderate- and low-methodological-quality reviews. SRs with low-bias risk showed higher total PRISMA-A values than reviews with high-bias risk. In the final model, only 'authors per review > 6' (OR: 1.098; 95%CI: 1.012-1.194), 'academic source of funding' (OR: 3.630; 95%CI: 1.788-7.542), and 'PRISMA-endorsed journal' (OR: 4.370; 95%CI: 1.785-10.98) predicted PRISMA-A variability. Reviews with a total PRISMA-A score < 6, lacking identification as SR or MA in the title, and lacking explanation concerning bias risk assessment methods were classified as low-methodological quality. Abstracts with a total PRISMA-A score ≥ 9, including main outcomes results and explanation bias risk assessment method were classified as having low-bias risk.

CONCLUSIONS

The methodological quality and bias risk of SRs may be determined by abstract's quality and completeness analyses. Our proposal aimed to facilitate synthesis of evidence evaluation by clinical professionals lacking methodological skills. External validation is necessary.