Validation of an index of the quality of review articles

Psychometric Properties of Self-Reported Financial Toxicity Measures in Cancer Survivors: An Overview of Systematic Reviews

AIM(S)

To assess the methodological quality and psychometric properties of self-reported financial toxicity measures for cancer survivors, to offer evidence-based guidance for the selection of these measures in clinical practice, and to supply methodological references for the enhancement and development of related measures in the future.

DESIGN

An overview of systematic reviews.

METHODS

Four academic databases were searched to conduct an overview of systematic reviews published from inception to August 2024. The Overview Quality Assessment Questionnaire (OQAQ) was employed to evaluate the methodological quality of the research included. The consensus-based standards for the selection of health measurement instruments checklist (COSMIN) and the Grades of Recommendation Assessment, Development, and Evaluation (GRADE) system were employed to assess the methodological and psychometric quality of the financial toxicity measures included.

RESULTS

Six systematic reviews satisfied the inclusion criteria. The OQAQ results indicated that six studies exhibited significant methodological quality defects, each receiving a score of 3 points. COST-v1, COST-v2, HARDS, ENRICh, and FinTox were classified as level A. Ten measures were classified as level B, lacking evidence to support content validity and internal consistency as '+'. Seven measures were classified at level C, supported by high-quality evidence indicating certain domains as '-'.

CONCLUSION

COST is advised as the best appropriate measurement standard for research and clinical practice across many global contexts. HARDS and ENRICh were advised to select only after thoroughly evaluating the local socio-economic context. FinTox is particularly suggested for the assessment of severe FT. SFDQ and FIT are advised for selection following an evaluation of therapy alternatives and the cancer's location.

IMPLICATIONS FOR THE PATIENT CARE

Healthcare professionals can implement evidence-based measures in clinical practice to effectively assess the financial toxicity experienced by cancer survivors, offer policy-oriented interventions, and enhance patient-reported outcomes.

Several methods for assessing research waste in reviews with a systematic search: a scoping review

Background

Research waste is present in all study designs and can have significant consequences for science, including reducing the reliability of research findings and contributing to the inefficient use of resources. Estimates suggest that as much as 85% of all biomedical research is wasted. However, it is uncertain how avoidable research waste is assessed in specific types of study designs and what methods could be used to examine different aspects of research waste. We aimed to investigate which methods, systematic reviews, scoping reviews, and overviews of reviews discussing research waste, have used to assess avoidable research waste.

Materials and Methods

We published a protocol in the Open Science Framework prospectively (https://osf.io/2fbp4). We searched PubMed and Embase with a 30-year limit (January 1993-August 2023). The concept examined was how research waste and related synonyms (e.g., unnecessary, redundant, duplicate, etc.) were assessed in reviews with a systematic search: systematic, scoping, or overviews of reviews. We extracted data on the method used in the review to examine for research waste and for which study design this method was applied.

Results

The search identified 4,285 records of which 93 reviews with systematic searches were included. The reviews examined a median of 90 (range 10-6,781) studies, where the study designs most commonly included were randomized controlled trials (48%) and systematic reviews (33%). In the last ten years, the number of reports assessing research waste has increased. More than 50% of examined reviews reported evaluating methodological research waste among included studies, typically using tools such as one of Cochrane Risk of Bias tools (n = 8) for randomized controlled trials or AMSTAR 1 or 2 (n = 12) for systematic reviews. One fourth of reviews assessed reporting guideline adherence to e.g., CONSORT (n = 4) for randomized controlled trials or PRISMA (n = 6) for systematic reviews.

Conclusion

Reviews with systematic searches focus on methodological quality and reporting guideline adherence when examining research waste. However, this scoping review revealed that a wide range of tools are used, which may pose difficulties in comparing examinations and performing meta-research. This review aids researchers in selecting methodologies and contributes to the ongoing discourse on optimizing research efficiency.

Effects of task-shifting from primary care physicians to nurses: an overview of systematic reviews

BACKGROUND

Task-shifting from primary care physicians (PCPs) to nurses is a means of overcoming PCP shortages and meeting the needs of patients receiving primary care. The aim of this overview of systematic reviews is to assess the effects of delegation or substitution of PCPs' activities by nurses on patient relevant, clinical, professional and health services-related outcomes.

METHODS

We conducted a systematic literature search for secondary literature in Medline, Embase, Pubmed, the Cochrane Library, and the Cumulative Index of Nursing and Allied Health Literature (CINAHL). We included systematic reviews and meta-analyses that analysed randomised controlled trials (RCTs) and controlled, prospective trials in English and German. Abstracts and full-text articles were screened independently by two reviewers. Full-text articles were assessed using the Overview Quality Assessment Questionnaire. After data extraction a narrative synthesis was performed. We defined patient-relevant outcomes as our primary outcomes.

RESULTS

We included six systematic reviews. The interventions included first contact, history taking and assessment, patient education, review of drug treatment, referrals to GPs and other health professionals, ordering further investigations and ongoing care. Two meta-analyses showed a relative risk reduction of mortality in favour of nurse-led care, whereby the reduction in one analysis was significant. The effect was highest in the group of more highly qualified nurse practitioners (RR 0.19), as opposed to nurse practitioners (RR 0.76) and registered nurses (RR 0.92). Two meta-analyses showed a relative risk reduction in hospital admissions and patient satisfaction. Whereas care conducted by physicians and registered nurses led to the same outcomes, care conducted by nurse practitioners led to better outcomes (RR 0.74). An analysis according to nursing group showed that patients were more satisfied with treatment by registered nurses (SMD 1.37) than with treatment conducted by nurse practitioners and more qualified nurse practitioners (SMD 0.17). In terms of patient-relevant outcomes, no differences were observed between physician-led care and nurse-led care in terms of physical function, quality of life and pain.

CONCLUSION

Nurse-led care is probably as safe or safer than physician-led care in terms of mortality and hospital admissions. However, the impact of nursing staff training has not been sufficiently examined.

Microbial Contamination of Herbal Medicines in Africa, 2000-2024: A Systematic Review

Introduction

Herbal medicine has been a cornerstone of healthcare for centuries, with an estimated 80% of the world's population relying on it. In Africa, herbal medicine is the backbone of rural healthcare, serving 80% to 90% of the population. Despite its widespread use, the safety of herbal medicine raises a significant concern considering the lack of regulation and testing, particularly in Africa. Microbial contamination is a primary safety risk threatening consumer health. In this systematic review, we aimed to synthesise evidence on microbial contamination in herbal medicines across Africa, provide a clear understanding of the problem, and inform effective public health interventions regarding microbial contamination of herbal medicines in Africa.

Method

The systematic review was conducted in accordance with the PRISMA guidelines. A literature search was conducted across PubMed, Web of Science, Science Direct, Scopus, and Google Scholar using appropriate search terms. Eligible studies were selected based on predetermined criteria, and data were extracted and analysed.

Results

The review included fifty eligible studies in Africa, with a combined sample size of 1996, of which 1791 showed microbial contamination. Bacterial contaminants were reported in 98% of studies, with Escherichia coli (62%) being the most reported bacteria, followed by Staphylococcus aureus (57%), and Bacillus spp. (55%). Fungal contaminants were reported in 70% of studies, with Aspergillus spp. (40%) being the most reported, followed by Penicillium spp. (27%) and Candida spp. (26%). Parasitic contaminants were reported in 2% of the studies reviewed. A total of 70 bacterial species, 37 fungal species, and 6 parasite species were identified in this review.

Conclusion

Herbal medicines in Africa pose significant health threats to consumers due to the high prevalence of diverse microbial contaminants and clinically significant pathogens. This emphasises the need for stricter regulations and quality control measures in the production, sale and use of herbal medicines.

Cadaveric Biomechanical Laboratory Research Can Be Quantitatively Scored for Quality With the Biomechanics Objective Basic Science Quality Assessment Tool: The BOBQAT Score

PURPOSE

To develop a quality appraisal tool for the assessment of cadaveric biomechanical laboratory and other basic science biomechanical studies.

METHODS

For item identification and development, a systematic review of the literature was performed. The content validity index (CVI) was used either to include or exclude items. The content validity ratio (CVR) was used to determine content validity. Weighting was performed by each panel member; the final weight was either up- or downgraded to the closest of 5% or 10%. Face validity was scored on a Likert scale ranked from 1 to 7. Test-retest reliability was determined using the Fleiss kappa coefficient. Internal consistency was assessed with Cronbach's alpha. Concurrent criterion validity was assessed against the Quality Appraisal for Cadaveric Studies scale.

RESULTS

The final Biomechanics Objective Basic science Quality Assessment Tool (BOBQAT) score included 15 items and was shown to be valid, reliable, and consistent. Five items had a CVI of 1.0; 10 items had a CVI of 0.875. For weighting, 5 items received a weight of 10%, and 10 items a weight of 5%. CVR was 1.0 for 6 items and 0.75 for 9 items. For face validity, all items achieved a score above 5. For test-retest reliability, almost-perfect test-retest reliability was observed for 10 items, substantial agreement for 4 items, and moderate agreement for 1 item. For internal consistency, Cronbach's alpha was calculated to be 0.71. For concurrent criterion validity, Pearson's product-moment correlation was 0.56 (95% confidence interval [CI] = 0.38-0.70, P = .0001).

CONCLUSIONS

Cadaveric biomechanical and laboratory research can be quantitatively scored for quality based on the inclusion of a clear and answerable purpose, demographics, specimen condition, appropriate bone density, reproducible technique, appropriate outcome measures, appropriate loading conditions, appropriate load magnitude, cyclic loading, sample size calculation, proper statistical analysis, results consistent with methods, limitations considered, conclusions based on results, and disclosure of funding and potential conflicts.

CLINICAL RELEVANCE

Study quality assessments are important to evaluate internal and external validity and reliability and to identify methodological flaws and misleading conclusions. The BOBQAT score will help not only in the critical appraisal of cadaveric biomechanical studies but also in guiding the designs of such research endeavors.

Quality Appraisal in Systematic Literature Reviews of Studies Eliciting Health State Utility Values: Conceptual Considerations

BACKGROUND

The increasing number of studies that generate health state utility values (HSUVs) and the impact of HSUVs on cost-utility analyses make a robust tailored quality appraisal (QA) tool for systematic reviews of these studies necessary.

OBJECTIVE

This study aimed to address conceptual issues regarding QA in systematic reviews of studies eliciting HSUVs by establishing a consensus on the definitions, dimensions and scope of a QA tool specific to this context.

METHODS

A modified Delphi method was used in this study. An international multidisciplinary panel of seven experts was purposively assembled. The experts engaged in two anonymous online survey rounds. After each round, the experts received structured and controlled feedback on the previous phase. Controlled feedback allowed the experts to re-evaluate and adjust their positions based on collective insights. Following these surveys, a virtual face-to-face meeting was held to resolve outstanding issues. Consensus was defined a priori at all stages of the modified Delphi process.

RESULTS

The response rates to the first-round and second-round questionnaires and the virtual consensus meeting were 100%, 86% and 71%, respectively. The entire process culminated in a consensus on the definitions of scientific quality, QA, the three QA dimensions-reporting, relevance and  methodological quality-and the scope of a QA tool specific to studies that elicit HSUVs.

CONCLUSIONS

Achieving this consensus marks a pivotal step towards developing a QA tool specific to systematic reviews of studies eliciting HSUVs. Future research will build on this foundation, identify QA items, signalling questions and response options, and develop a QA tool specific to studies eliciting HSUVs.

Protocol for the development of a reporting guideline for umbrella reviews on epidemiological associations using cross-sectional, case-control and cohort studies: the Preferred Reporting Items for Umbrella Reviews of Cross-sectional, Case-control and Cohort studies (PRIUR-CCC)

INTRODUCTION

Observational studies are fraught with several biases including reverse causation and residual confounding. Overview of reviews of observational studies (ie, umbrella reviews) synthesise systematic reviews with or without meta-analyses of cross-sectional, case-control and cohort studies, and may also aid in the grading of the credibility of reported associations. The number of published umbrella reviews has been increasing. Recently, a reporting guideline for overviews of reviews of healthcare interventions (Preferred Reporting Items for Overviews of Reviews (PRIOR)) was published, but the field lacks reporting guidelines for umbrella reviews of observational studies. Our aim is to develop a reporting guideline for umbrella reviews on cross-sectional, case-control and cohort studies assessing epidemiological associations.

METHODS AND ANALYSIS

We will adhere to established guidance and prepare a PRIOR extension for systematic reviews of cross-sectional, case-control and cohort studies testing epidemiological associations between an exposure and an outcome, namely Preferred Reporting Items for Umbrella Reviews of Cross-sectional, Case-control and Cohort studies (PRIUR-CCC). Step 1 will be the project launch to identify stakeholders. Step 2 will be a literature review of available guidance to conduct umbrella reviews. Step 3 will be an online Delphi study sampling 100 participants among authors and editors of umbrella reviews. Step 4 will encompass the finalisation of PRIUR-CCC statement, including a checklist, a flow diagram, explanation and elaboration document. Deliverables will be (i) identifying stakeholders to involve according to relevant expertise and end-user groups, with an equity, diversity and inclusion lens; (ii) completing a narrative review of methodological guidance on how to conduct umbrella reviews, a narrative review of methodology and reporting in published umbrella reviews and preparing an initial PRIUR-CCC checklist for Delphi study round 1; (iii) preparing a PRIUR-CCC checklist with guidance after Delphi study; (iv) publishing and disseminating PRIUR-CCC statement.

ETHICS AND DISSEMINATION

PRIUR-CCC has been approved by The Ottawa Health Science Network Research Ethics Board and has obtained consent (20220639-01H). Participants to step 3 will give informed consent. PRIUR-CCC steps will be published in a peer-reviewed journal and will guide reporting of umbrella reviews on epidemiological associations.

Development and validation of a risk-of-bias tool for assessing in vitro studies conducted in dentistry: The QUIN

STATEMENT OF PROBLEM

Systematic reviews and meta-analyses have emerged as forerunners of evidence-based dentistry, but assessing the quality of the available research is essential so that it can be applied to clinical practice. While a wide variety of risk-of-bias tools are available, each specifically developed for different study designs, a comprehensive tool exclusively framed to assess the quality of in vitro dental studies is lacking.

PURPOSE

The purpose of this study was to develop and validate a tool to evaluate the quality and risk of bias of in vitro dental studies.

MATERIAL AND METHODS

A Delphi panel was established to conceptualize and develop the Quality Assessment Tool For In Vitro Studies (QUIN Tool). The tool was evaluated by using content validity and reliability testing methods.

RESULTS

The QUIN Tool includes 12 points along with scoring and grading options to allow clinicians to evaluate the quality of in vitro studies. This tool shows good content validity and reliability.

CONCLUSIONS

The QUIN Tool is user-friendly, efficient, and effective for evaluating the risk of bias of in vitro studies.

Mechanical Cardiopulmonary Resuscitation Devices: Evidence Synthesis with an Umbrella Review

AIM

Sudden cardiac arrest is a significant cause of death worldwide. Good quality cardiopulmonary resuscitation increases patients' survival. Manual cardiopulmonary resuscitation is often ineffective as rescuers may experience physical and mental fatigue. Mechanical cardiopulmonary resuscitation devices are designed to address this issue, providing an automated approach for high-quality resuscitation. In the present comprehensive umbrella review we summarize current evidence on mechanical devices.

METHODS

We searched systematic reviews on mechanical devices in MEDLINE/PubMed. Effect estimates were obtained from original reports, including 95% confidence intervals and p values, when applicable and available, focusing on return of spontaneous circulation, survival to discharge or 30 days, survival with good neurological outcome, and resuscitation-related injuries.

RESULTS

From 21 potentially pertinent publications, we shortlisted 10 reviews, each including between 5 and 22 studies. AutoPulse, LUCAS, and LUCAS-2 were among the investigated devices. Most reviews concluded toward mechanical devices being similar or better than manual resuscitation for return of spontaneous circulation and 30-days survival. Regarding survival with good neurological function, some reviews lacked data, while the remaining ones reported similar results or worse outcomes in patients undergoing mechanical resuscitation. Focusing on resuscitation-related injuries, data were limited or conflicting with one review reporting higher rates of injuries with mechanical devices, and two others suggesting similar outcomes.

CONCLUSIONS

Manual and mechanical cardiopulmonary resuscitation appear to be similar in terms of return of spontaneous circulation and short-term survival. Mechanical devices appear to be associated with higher resuscitation-related injuries, while there are conflicting data in terms of survival with good neurological outcomes. A comprehensive and large dedicated randomized trial is urgently needed.

Effects of task shifting from primary care physicians to nurses: a protocol for an overview of systematic reviews

INTRODUCTION

Task-shifting from primary care physicians (PCPs) to nurses is one option to better and more efficiently meet the needs of the population in primary care and to overcome PCP shortages. This protocol outlines an overview of systematic reviews to assess the effects of delegation or substitution by nurses of PCPs' activities regarding clinical, patient-relevant, professional and health services-related outcomes.

METHODS AND ANALYSIS

We will conduct a systematic literature search for secondary literature in PubMed/MEDLINE, EMBASE, CINAHL and Cochrane databases. Systematic reviews, meta-analyses and Health Technology Assessments in German and English comprising randomised controlled trials and prospective controlled trials will be considered for inclusion. Search terms will include Medical Subject Headings combined with free text words. At least one-third of abstracts and full-text articles are reviewed by two independent reviewers. Methodological quality will be assessed using the Overview Quality Assessment Questionnaire. We will only consider reviews if they include controlled trials, if the profession that substituted or delegated tasks was a nurse, if the profession of the control was a PCP, if the assessed intervention was the same in the intervention and control group and if the Overview Quality Assessment Questionnaire score is ≥5. The corrected covered area will be calculated to describe the degree of overlap of studies in the reviews included in the study. We will report the overview according to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses.

ETHICS AND DISSEMINATION

The overview of secondary literature does not require the approval of an Ethics Committee and will be published in a peer-reviewed journal.

PROSPERO REGISTRATION NUMBER

CRD42020183327.

'It depends': what 86 systematic reviews tell us about what strategies to use to support the use of research in clinical practice

BACKGROUND

The gap between research findings and clinical practice is well documented and a range of strategies have been developed to support the implementation of research into clinical practice. The objective of this study was to update and extend two previous reviews of systematic reviews of strategies designed to implement research evidence into clinical practice.

METHODS

We developed a comprehensive systematic literature search strategy based on the terms used in the previous reviews to identify studies that looked explicitly at interventions designed to turn research evidence into practice. The search was performed in June 2022 in four electronic databases: Medline, Embase, Cochrane and Epistemonikos. We searched from January 2010 up to June 2022 and applied no language restrictions. Two independent reviewers appraised the quality of included studies using a quality assessment checklist. To reduce the risk of bias, papers were excluded following discussion between all members of the team. Data were synthesised using descriptive and narrative techniques to identify themes and patterns linked to intervention strategies, targeted behaviours, study settings and study outcomes.

RESULTS

We identified 32 reviews conducted between 2010 and 2022. The reviews are mainly of multi-faceted interventions (n = 20) although there are reviews focusing on single strategies (ICT, educational, reminders, local opinion leaders, audit and feedback, social media and toolkits). The majority of reviews report strategies achieving small impacts (normally on processes of care). There is much less evidence that these strategies have shifted patient outcomes. Furthermore, a lot of nuance lies behind these headline findings, and this is increasingly commented upon in the reviews themselves.

DISCUSSION

Combined with the two previous reviews, 86 systematic reviews of strategies to increase the implementation of research into clinical practice have been identified. We need to shift the emphasis away from isolating individual and multi-faceted interventions to better understanding and building more situated, relational and organisational capability to support the use of research in clinical practice. This will involve drawing on a wider range of research perspectives (including social science) in primary studies and diversifying the types of synthesis undertaken to include approaches such as realist synthesis which facilitate exploration of the context in which strategies are employed.

Interleukin-18 Gene Polymorphisms and Rheumatoid Arthritis Susceptibility: An Umbrella Review of Meta-Analyses

This study systematically analyzes the association between interleukin-18 (IL-18) gene polymorphisms and rheumatoid arthritis (RA) susceptibility. The electronic databases Ovid MEDLINE, Ovid Excerpta Medica Database, and Cochrane Library were searched to identify meta-analyses that included case-control studies reporting IL-18 gene polymorphisms and RA susceptibility. Data were reanalyzed using Review Manager Software 5.1, and Mantel-Haenszel random effects were applied for the five genetic models: allelic, recessive, dominant, homozygote, and heterozygote. The effect size of odds ratios (ORs) and their corresponding 95% confidence interval (CI) were calculated. A total of seven meta-analyses with poor quality were included. The IL-18 polymorphisms -607 A/C, -137 C/G, -920 T/C, and -105 C/A have been reported. With weak evidence, IL-18 -607 A/C polymorphisms were associated with a reduced risk of RA susceptibility using the allele model (OR = 0.76, 95% CI: 0.61 - 0.93, p=0.01), dominant model (OR = 0.67, 95% CI: 0.50 - 0.90, p=0.008), homozygote model (OR = 0.57, 95% CI: 0.35 - 0.91, p=0.02), and heterozygote model (OR = 0.71, 95% CI: 0.54 - 0.93, p=0.01) in the overall population. IL-18 gene polymorphisms and RA susceptibility are affected by ethnicity: With weak evidence, IL-18 -137 C/G polymorphisms were related to reduce RA susceptibility in the Asian population (allele model: OR = 0.59, 95% CI: 0.40 - 0.88, p=0.01; dominant model: OR = 0.57, 95% CI: 0.37 - 0.89, p=0.01; heterozygote model: OR = 0.60, 95% CI: 0.38 - 0.94, p=0.03). IL-18 -607 A/C gene polymorphisms are a protective factor for RA susceptibility in the overall population, and IL-18 -137 C/G gene polymorphisms are a protective factor for RA susceptibility in the Asian population. Further studies are needed to confirm these results owing to the limitations of the included studies.

Methodological review of NMA bias concepts provides groundwork for the development of a list of concepts for potential inclusion in a new risk of bias tool for network meta-analysis (RoB NMA Tool)

INTRODUCTION

Network meta-analyses (NMAs) have gained popularity and grown in number due to their ability to provide estimates of the comparative effectiveness of multiple treatments for the same condition. The aim of this study is to conduct a methodological review to compile a preliminary list of concepts related to bias in NMAs.

METHODS AND ANALYSIS

We included papers that present items related to bias, reporting or methodological quality, papers assessing the quality of NMAs, or method papers. We searched MEDLINE, the Cochrane Library and unpublished literature (up to July 2020). We extracted items related to bias in NMAs. An item was excluded if it related to general systematic review quality or bias and was included in currently available tools such as ROBIS or AMSTAR 2. We reworded items, typically structured as questions, into concepts (i.e. general notions).

RESULTS

One hundred eighty-one articles were assessed in full text and 58 were included. Of these articles, 12 were tools, checklists or journal standards; 13 were guidance documents for NMAs; 27 were studies related to bias or NMA methods; and 6 were papers assessing the quality of NMAs. These studies yielded 99 items of which the majority related to general systematic review quality and biases and were therefore excluded. The 22 items we included were reworded into concepts specific to bias in NMAs.

CONCLUSIONS

A list of 22 concepts was included. This list is not intended to be used to assess biases in NMAs, but to inform the development of items to be included in our tool.

Guidance to best tools and practices for systematic reviews

Data continue to accumulate indicating that many systematic reviews are methodologically flawed, biased, redundant, or uninformative. Some improvements have occurred in recent years based on empirical methods research and standardization of appraisal tools; however, many authors do not routinely or consistently apply these updated methods. In addition, guideline developers, peer reviewers, and journal editors often disregard current methodological standards. Although extensively acknowledged and explored in the methodological literature, most clinicians seem unaware of these issues and may automatically accept evidence syntheses (and clinical practice guidelines based on their conclusions) as trustworthy. A plethora of methods and tools are recommended for the development and evaluation of evidence syntheses. It is important to understand what these are intended to do (and cannot do) and how they can be utilized. Our objective is to distill this sprawling information into a format that is understandable and readily accessible to authors, peer reviewers, and editors. In doing so, we aim to promote appreciation and understanding of the demanding science of evidence synthesis among stakeholders. We focus on well-documented deficiencies in key components of evidence syntheses to elucidate the rationale for current standards. The constructs underlying the tools developed to assess reporting, risk of bias, and methodological quality of evidence syntheses are distinguished from those involved in determining overall certainty of a body of evidence. Another important distinction is made between those tools used by authors to develop their syntheses as opposed to those used to ultimately judge their work. Exemplar methods and research practices are described, complemented by novel pragmatic strategies to improve evidence syntheses. The latter include preferred terminology and a scheme to characterize types of research evidence. We organize best practice resources in a Concise Guide that can be widely adopted and adapted for routine implementation by authors and journals. Appropriate, informed use of these is encouraged, but we caution against their superficial application and emphasize their endorsement does not substitute for in-depth methodological training. By highlighting best practices with their rationale, we hope this guidance will inspire further evolution of methods and tools that can advance the field.

Guidance to best tools and practices for systematic reviews

Data continue to accumulate indicating that many systematic reviews are methodologically flawed, biased, redundant, or uninformative. Some improvements have occurred in recent years based on empirical methods research and standardization of appraisal tools; however, many authors do not routinely or consistently apply these updated methods. In addition, guideline developers, peer reviewers, and journal editors often disregard current methodological standards. Although extensively acknowledged and explored in the methodological literature, most clinicians seem unaware of these issues and may automatically accept evidence syntheses (and clinical practice guidelines based on their conclusions) as trustworthy. A plethora of methods and tools are recommended for the development and evaluation of evidence syntheses. It is important to understand what these are intended to do (and cannot do) and how they can be utilized. Our objective is to distill this sprawling information into a format that is understandable and readily accessible to authors, peer reviewers, and editors. In doing so, we aim to promote appreciation and understanding of the demanding science of evidence synthesis among stakeholders. We focus on well-documented deficiencies in key components of evidence syntheses to elucidate the rationale for current standards. The constructs underlying the tools developed to assess reporting, risk of bias, and methodological quality of evidence syntheses are distinguished from those involved in determining overall certainty of a body of evidence. Another important distinction is made between those tools used by authors to develop their syntheses as opposed to those used to ultimately judge their work. Exemplar methods and research practices are described, complemented by novel pragmatic strategies to improve evidence syntheses. The latter include preferred terminology and a scheme to characterize types of research evidence. We organize best practice resources in a Concise Guide that can be widely adopted and adapted for routine implementation by authors and journals. Appropriate, informed use of these is encouraged, but we caution against their superficial application and emphasize their endorsement does not substitute for in-depth methodological training. By highlighting best practices with their rationale, we hope this guidance will inspire further evolution of methods and tools that can advance the field.

The Use of Questionnaires in Pain Assessment during Orthodontic Treatments: A Narrative Review

Pain is a complex multidimensional feeling combined with sensorial and emotional features. The majority of patients undergoing orthodontic treatment report various degrees of pain, which is perceived as widely variable between individuals, even when the stimulus is the same. Orthodontic pain is considered the main cause of poor-quality outcomes, patients' dissatisfaction, and lack of collaboration up to the interruption of therapy. A deep understanding of pain and how it influences a patient's daily life is fundamental to establishing proper therapeutic procedures and obtaining the correct collaboration. Because of its multifaced and subjective nature, pain is a difficult dimension to measure. The use of questionnaires and their relative rating scales is actually considered the gold standard for pain assessment. Choosing the most appropriate instrument for recording self-reported pain depends on a patient's age and cognitive abilities. Although several such scales have been proposed, and a lot of them are applied, it remains uncertain which of these tools represents the standard and performs the most precise, universal, and predictable task. This review aims to give an overview of the aspects which describe pain, specifically the pain experienced during orthodontic treatment, the main tool to assess self-perceived pain in a better and more efficient way, the different indications for each of them, and their correlated advantages or disadvantages.

Guidance to best tools and practices for systematic reviews

Data continue to accumulate indicating that many systematic reviews are methodologically flawed, biased, redundant, or uninformative. Some improvements have occurred in recent years based on empirical methods research and standardization of appraisal tools; however, many authors do not routinely or consistently apply these updated methods. In addition, guideline developers, peer reviewers, and journal editors often disregard current methodological standards. Although extensively acknowledged and explored in the methodological literature, most clinicians seem unaware of these issues and may automatically accept evidence syntheses (and clinical practice guidelines based on their conclusions) as trustworthy. A plethora of methods and tools are recommended for the development and evaluation of evidence syntheses. It is important to understand what these are intended to do (and cannot do) and how they can be utilized. Our objective is to distill this sprawling information into a format that is understandable and readily accessible to authors, peer reviewers, and editors. In doing so, we aim to promote appreciation and understanding of the demanding science of evidence synthesis among stakeholders. We focus on well-documented deficiencies in key components of evidence syntheses to elucidate the rationale for current standards. The constructs underlying the tools developed to assess reporting, risk of bias, and methodological quality of evidence syntheses are distinguished from those involved in determining overall certainty of a body of evidence. Another important distinction is made between those tools used by authors to develop their syntheses as opposed to those used to ultimately judge their work. Exemplar methods and research practices are described, complemented by novel pragmatic strategies to improve evidence syntheses. The latter include preferred terminology and a scheme to characterize types of research evidence. We organize best practice resources in a Concise Guide that can be widely adopted and adapted for routine implementation by authors and journals. Appropriate, informed use of these is encouraged, but we caution against their superficial application and emphasize their endorsement does not substitute for in-depth methodological training. By highlighting best practices with their rationale, we hope this guidance will inspire further evolution of methods and tools that can advance the field.

Guidance to best tools and practices for systematic reviews

Data continue to accumulate indicating that many systematic reviews are methodologically flawed, biased, redundant, or uninformative. Some improvements have occurred in recent years based on empirical methods research and standardization of appraisal tools; however, many authors do not routinely or consistently apply these updated methods. In addition, guideline developers, peer reviewers, and journal editors often disregard current methodological standards. Although extensively acknowledged and explored in the methodological literature, most clinicians seem unaware of these issues and may automatically accept evidence syntheses (and clinical practice guidelines based on their conclusions) as trustworthy.A plethora of methods and tools are recommended for the development and evaluation of evidence syntheses. It is important to understand what these are intended to do (and cannot do) and how they can be utilized. Our objective is to distill this sprawling information into a format that is understandable and readily accessible to authors, peer reviewers, and editors. In doing so, we aim to promote appreciation and understanding of the demanding science of evidence synthesis among stakeholders. We focus on well-documented deficiencies in key components of evidence syntheses to elucidate the rationale for current standards. The constructs underlying the tools developed to assess reporting, risk of bias, and methodological quality of evidence syntheses are distinguished from those involved in determining overall certainty of a body of evidence. Another important distinction is made between those tools used by authors to develop their syntheses as opposed to those used to ultimately judge their work.Exemplar methods and research practices are described, complemented by novel pragmatic strategies to improve evidence syntheses. The latter include preferred terminology and a scheme to characterize types of research evidence. We organize best practice resources in a Concise Guide that can be widely adopted and adapted for routine implementation by authors and journals. Appropriate, informed use of these is encouraged, but we caution against their superficial application and emphasize their endorsement does not substitute for in-depth methodological training. By highlighting best practices with their rationale, we hope this guidance will inspire further evolution of methods and tools that can advance the field.

Guidance to best tools and practices for systematic reviews

Data continue to accumulate indicating that many systematic reviews are methodologically flawed, biased, redundant, or uninformative. Some improvements have occurred in recent years based on empirical methods research and standardization of appraisal tools; however, many authors do not routinely or consistently apply these updated methods. In addition, guideline developers, peer reviewers, and journal editors often disregard current methodological standards. Although extensively acknowledged and explored in the methodological literature, most clinicians seem unaware of these issues and may automatically accept evidence syntheses (and clinical practice guidelines based on their conclusions) as trustworthy.A plethora of methods and tools are recommended for the development and evaluation of evidence syntheses. It is important to understand what these are intended to do (and cannot do) and how they can be utilized. Our objective is to distill this sprawling information into a format that is understandable and readily accessible to authors, peer reviewers, and editors. In doing so, we aim to promote appreciation and understanding of the demanding science of evidence synthesis among stakeholders. We focus on well-documented deficiencies in key components of evidence syntheses to elucidate the rationale for current standards. The constructs underlying the tools developed to assess reporting, risk of bias, and methodological quality of evidence syntheses are distinguished from those involved in determining overall certainty of a body of evidence. Another important distinction is made between those tools used by authors to develop their syntheses as opposed to those used to ultimately judge their work.Exemplar methods and research practices are described, complemented by novel pragmatic strategies to improve evidence syntheses. The latter include preferred terminology and a scheme to characterize types of research evidence. We organize best practice resources in a Concise Guide that can be widely adopted and adapted for routine implementation by authors and journals. Appropriate, informed use of these is encouraged, but we caution against their superficial application and emphasize their endorsement does not substitute for in-depth methodological training. By highlighting best practices with their rationale, we hope this guidance will inspire further evolution of methods and tools that can advance the field.

Guidance to Best Tools and Practices for Systematic Reviews

» Data continue to accumulate indicating that many systematic reviews are methodologically flawed, biased, redundant, or uninformative. Some improvements have occurred in recent years based on empirical methods research and standardization of appraisal tools; however, many authors do not routinely or consistently apply these updated methods. In addition, guideline developers, peer reviewers, and journal editors often disregard current methodological standards. Although extensively acknowledged and explored in the methodological literature, most clinicians seem unaware of these issues and may automatically accept evidence syntheses (and clinical practice guidelines based on their conclusions) as trustworthy.» A plethora of methods and tools are recommended for the development and evaluation of evidence syntheses. It is important to understand what these are intended to do (and cannot do) and how they can be utilized. Our objective is to distill this sprawling information into a format that is understandable and readily accessible to authors, peer reviewers, and editors. In doing so, we aim to promote appreciation and understanding of the demanding science of evidence synthesis among stakeholders. We focus on well-documented deficiencies in key components of evidence syntheses to elucidate the rationale for current standards. The constructs underlying the tools developed to assess reporting, risk of bias, and methodological quality of evidence syntheses are distinguished from those involved in determining overall certainty of a body of evidence. Another important distinction is made between those tools used by authors to develop their syntheses as opposed to those used to ultimately judge their work.» Exemplar methods and research practices are described, complemented by novel pragmatic strategies to improve evidence syntheses. The latter include preferred terminology and a scheme to characterize types of research evidence. We organize best practice resources in a Concise Guide that can be widely adopted and adapted for routine implementation by authors and journals. Appropriate, informed use of these is encouraged, but we caution against their superficial application and emphasize their endorsement does not substitute for in-depth methodological training. By highlighting best practices with their rationale, we hope this guidance will inspire further evolution of methods and tools that can advance the field.

Spinal Orthoses in the Treatment of Osteoporotic Thoracolumbar Vertebral Fractures in the Elderly: A Systematic Review With Quantitative Quality Assessment

STUDY DESIGN

Systematic review.

OBJECTIVES

Spinal orthoses are frequently used to non-operatively treat osteoporotic vertebral fractures (OVF), despite the available evidence is rare. Previously systematic reviews were carried out, presenting controversial recommendations. The present study aimed to systematic review the recent and current literature on available evidence for the use of orthoses in OVF.

METHODS

A systematic review was conducted using PubMed, Medline, EMBASE and CENTRAL databases. Identified articles including previous systematic reviews were screened and selected by three authors. The results of retrieved articles were presented in a narrative form, quality assessment was performed by two authors using scores according to the study type.

RESULTS

Thirteen studies (n = 5 randomized controlled trials, n = 3 non- randomized controlled trials and n = 5 prospective studies without control group) and eight systematic reviews were analyzed. Studies without comparison group reported improvements in pain, function and quality of life during the follow-up. Studies comparing different types of orthoses favor non-rigid orthoses. In comparison to patients not wearing an orthosis three studies were unable to detect beneficial effects and two studies reported about a significant improvement using an orthosis. In the obtained quality assessment, three studies yielded good to excellent results. Previous reviews detected the low evidence for spinal orthoses but recommended them.

CONCLUSION

Based on the study quality and the affection of included studies in previous systematic reviews a general recommendation for the use of a spinal orthosis when treating OVF is not possible. Currently, no superiority for spinal orthoses in OVF treatment was found.

Health-related quality of life for children using orthokeratology (OK), peripheral lenslet designed (PLD) and single-vision (SV) spectacles: based on Child Health Utility 9 Dimensions (CHU9D)

PURPOSE

This study investigated the health-related quality of life (HRQoL) among children with myopia who used orthokeratology (OK), peripheral lenslet designed (PLD), and single-vision (SV) spectacles.

METHODS

This cross-sectional study was conducted between February of 2021 and August of 2022. It involved 211, 231, and 206 respondents with OK, PLD, and SV spectacle lenses, respectively. The HRQoL was presented as utility values using a general preference-based Child Health Utility-nine Dimensions (CHU9D) questionnaire. Descriptive statistical analysis and nonparametric hypothesis testing were used to compare the HRQoL in the OK, PLD, and SV groups.

RESULTS

Of the 648 respondents, the average utility scores was 0.936 (95 % confidence interval (CI): 0.929-0.943). Children who wore the PLD spectacles had a significantly higher utility scores (0.955, 95 % CI: 0.946-0.963) than those who wore the SV spectacles (0.926, 95 % CI: 0.913-0.939) and the OK lenses (0.925, 95 % CI: 0.913-0.937) (p < 0.01). The PLD spectacle wearers were less likely to be worried, sad, tired, or annoyed than those who wore OK and SV spectacles (P < 0.05). Self-reported improved eyesight and lessened eye pain and discomfort from myopia correction using PLD spectacles had higher utility values (P < 0.05).

CONCLUSIONS

The PLD spectacles had a significantly higher HRQoL than the OK and SV spectacles among children. Having better eyesight and less eye pain/discomfort from myopia correction could improve the HRQoL of children. This data indicates that PLD spectacles may be considered for myopia management in children and adolescents.

Current knowledge on the association between cardiovascular and periodontal disease: an umbrella review

INTRODUCTION

Periodontal disease (PD) and cardiovascular diseases (CVD) are among the most common pathologies in the world and their relationship has long been studied. Both conditions lead to a chronic inflammatory process with degenerative characteristics and their biunivocal correlation is now well established. The aim of this umbrella review on cardiovascular and periodontal disease was to evaluate the real degree of association between these two pathological conditions.

EVIDENCE ACQUISITION

We conducted a comprehensive literature search on PubMed/Medline and in the Cochrane Library for systematic reviews focused on clinical evidence regarding the relationship between PD and CVD. The internal validity of systematic reviews and meta-analyses was formally analyzed using the Overview Quality Assessment Questionnaire (OQAQ) Tool. The umbrella review was planned in accordance with current international recommendations and was described as specified by the PRISMA Guidelines.

EVIDENCE SYNTHESIS

Thirty-one systematic reviews, including 8 meta-analyses for a total of 507 clinical studies and over 3,549,966 patients were included. PD resulted to be associated with a higher risk of developing CVD (acute coronary syndrome, acute myocardial infarction) and cerebrovascular diseases (ischemic stroke); however, if the treatment of periodontitis reduces the risk of CVD events related is yet to be investigated.

CONCLUSIONS

To date, the relationship between CVD and PD provides heterogeneous data. There is an association between PD and CVD, but a causal relationship cannot be established. Further research with properly designed long-term follow-up studies are needed in order to examine various physiopathological aspects of their association.

Cemented vs Uncemented hemiarthroplasties for femoral neck fractures: An overlapping systematic review and evidence appraisal

BACKGROUND

The purpose of our study is to assess the methodology of overlapping systematic reviews related to cemented vs uncemented hip hemiarthroplasties for the treatment of femoral neck fractures to find the study with the best evidence. Also, we assess the gaps in methodology and information to help with direction of future studies.

METHODS

A systematic search was conducted in September 2022 using Pubmed, Embase, and Cochrane Library. Clinical outcome data and characteristics of each study were extracted to see which treatment had better favorability. The outcomes and characteristics extracted from each study includes, first author, search date, publication journal and date, number of studies included, databases, level of evidence, software used, subgroup analyses that were conducted, and heterogeneity with the use of I2 statistics Methodological quality information was extracted from each study using four different methodologic scores (Oxford Levels of Evidence; Assessment of Multiple Systematic Reviews (AMSTAR); Quality of reporting of meta-analyses (QUROM); Oxman and Guyatt. After that, the Jadad decision algorithm was used to identify which studies in our sample contained the best available evidence. Finally, overlap of each systematic review was assessed using Corrected Covered Area (CCA) to look at redundancy and research waste among the systematic reviews published on the topic.

RESULTS

After screening, 12 studies were included in our sample. For the Oxford Levels of Evidence, we found that all the studies were Level I evidence. For the QUORUM assessment, we had 1 study with the highest score of 18. Additionally, we did the Oxman and Guyatt assessment, where we found 4 studies with a maximum score of 6. Finally, we did an AMSTAR assessment and found 2 studies with a score of 9. After conducting the methodological scores; the authors determined that Li. L et al 2021 had the highest quality. In addition, it was found that the CCA found among the primary studies in each systematic review calculated to .22. Any CCA above .15 is considered "very high overlap".

CONCLUSIONS

The best available evidence suggests that Cemented HAs are better at preventing Prosthesis-related complications. Conversely, the best evidence also suggests that Cemented HA also results in longer operative time and increased intraoperative blood loss. When conducting future systematic reviews related to the topic, we ask that authors restrict conducting another systematic review until new evidence emerges so as not to confuse the clinical decision-making of physicians.

Guidance to best tools and practices for systematic reviews1

Data continue to accumulate indicating that many systematic reviews are methodologically flawed, biased, redundant, or uninformative. Some improvements have occurred in recent years based on empirical methods research and standardization of appraisal tools; however, many authors do not routinely or consistently apply these updated methods. In addition, guideline developers, peer reviewers, and journal editors often disregard current methodological standards. Although extensively acknowledged and explored in the methodological literature, most clinicians seem unaware of these issues and may automatically accept evidence syntheses (and clinical practice guidelines based on their conclusions) as trustworthy.A plethora of methods and tools are recommended for the development and evaluation of evidence syntheses. It is important to understand what these are intended to do (and cannot do) and how they can be utilized. Our objective is to distill this sprawling information into a format that is understandable and readily accessible to authors, peer reviewers, and editors. In doing so, we aim to promote appreciation and understanding of the demanding science of evidence synthesis among stakeholders. We focus on well-documented deficiencies in key components of evidence syntheses to elucidate the rationale for current standards. The constructs underlying the tools developed to assess reporting, risk of bias, and methodological quality of evidence syntheses are distinguished from those involved in determining overall certainty of a body of evidence. Another important distinction is made between those tools used by authors to develop their syntheses as opposed to those used to ultimately judge their work.Exemplar methods and research practices are described, complemented by novel pragmatic strategies to improve evidence syntheses. The latter include preferred terminology and a scheme to characterize types of research evidence. We organize best practice resources in a Concise Guide that can be widely adopted and adapted for routine implementation by authors and journals. Appropriate, informed use of these is encouraged, but we caution against their superficial application and emphasize their endorsement does not substitute for in-depth methodological training. By highlighting best practices with their rationale, we hope this guidance will inspire further evolution of methods and tools that can advance the field.

CAT HPPR: a critical appraisal tool to assess the quality of systematic, rapid, and scoping reviews investigating interventions in health promotion and prevention

BACKGROUND

For over three decades researchers have developed critical appraisal tools (CATs) for assessing the scientific quality of research overviews. Most established CATs for reviews in evidence-based medicine and evidence-based public health (EBPH) focus on systematic reviews (SRs) with studies on experimental interventions or exposure included. EBPH- and implementation-oriented organisations and decision-makers, however, often seek access to rapid reviews (RRs) or scoping reviews (ScRs) for rapid evidence synthesis and research field exploration. Until now, no CAT is available to assess the quality of SRs, RRs, and ScRs following a unified approach. We set out to develop such a CAT.

METHODS

The development process of the Critical Appraisal Tool for Health Promotion and Prevention Reviews (CAT HPPR) included six phases: (i) the definition of important review formats and complementary approaches, (ii) the identification of relevant CATs, (iii) prioritisation, selection and adaptation of quality criteria using a consensus approach, (iv) development of the rating system and bilingual guidance documents, (v) engaging with experts in the field for piloting/optimising the CAT, and (vi) approval of the final CAT. We used a pragmatic search approach to identify reporting guidelines/standards (n = 3; e.g. PRISMA, MECIR) as well as guidance documents (n = 17; e.g. for reviews with mixed-methods approach) to develop working definitions for SRs, RRs, ScRs, and other review types (esp. those defined by statistical methods or included data sources).

RESULTS

We successfully identified 14 relevant CATs, predominantly for SRs (e.g. AMSTAR 2), and extracted 46 items. Following consensual discussions 15 individual criteria were included in our CAT and tailored to the review types of interest. The CAT was piloted with 14 different reviews which were eligible to be included in a new German database looking at interventions in health promotion and prevention in different implementation settings.

CONCLUSIONS

The newly developed CAT HPPR follows a unique uniformed approach to assess a set of heterogeneous reviews (e.g. reviews from problem identification to policy evaluations) to assist end-users needs. Feedback of external experts showed general feasibility and satisfaction with the tool. Future studies should further formally test the validity of CAT HPPR using larger sets of reviews.

The impact of funding on the quality and interpretation of systematic reviews of mechanical thrombectomy in stroke patients

BACKGROUND

Funding may impact the quality and findings of systematic reviews (SRs). We aimed to compare the methodological quality of funded and non-funded SRs that investigated the outcomes in ischemic stroke patients undergoing mechanical thrombectomy.

METHODS

We conducted a comprehensive search strategy in different databases, including Ovid Cochrane Central Register of Controlled Trials, Ovid Embase, Ovid Medline (including epub ahead of print, in-process & other non-indexed citations), PubMed, Scopus and Web of Science Core Collection to retrieve all relevant SRs. Random sequence generation matched each funded SR with a non-funded one. A Measurement Tool to Assess Systematic Reviews (AMSTAR)-2 tool was used to assess the bias and quality of the included SRs. We also used uni- and multivariate analysis to perform our analysis, and results were expressed in odds ratio (OR) and 95% confidence interval (CI).

RESULTS

We retrieved 150 articles, which were randomized and matched into 100 SRs, including 50 funded and 50 non-funded studies. By multivariate analysis, we found that including randomized clinical trials (RCTs) (OR: 5.7; 95% CI: 1.8-17.8; p = 0.003) and reporting conflict of interests (OR: 5.2; 95 CI: 1.1-24; p = 0.036) were the only significant differences between funded and non-funded SRs. No significant differences were found regarding the overall confidence for low-quality (OR: 0.54; 95% CI: 0.09-3.2; p = 0.49) and moderate/high-quality SRs (OR: 0.17; 95% CI: 0.02-1.87; p = 0.14).

CONCLUSION

Funded studies tend to include RCTs more often and report conflict of interests with no significant impact on overall confidence.

Studying the Research-Practice Gap in Physical Therapies for Cerebral Palsy: Preliminary Outcomes Based on a Survey of Spanish Clinicians

The purpose of this work is to study the gap between the research evidence and the clinical practice in the physical rehabilitation of people with cerebral palsy. A review process was performed to (1) identify physical therapies to improve postural control in children with cerebral palsy and (2) determine the scientific evidence supporting the effectiveness of those therapies. A Likert-based survey addressing a total of 43 healthcare professionals involved in pediatric physical therapy departments in Spain was carried out. The discussion was mainly supported by studies of level I or II evidence (according to the Oxford scale). The search process yielded 50 studies reporting 16 therapies. A strong positive correlation between the most used treatments and elevated levels of satisfaction was found. Some well-known but not often used techniques, such as hippotherapy, were identified. The treatment with the highest degree of use and satisfaction-neurodevelopment therapy (Bobath)-and some emerging techniques, such as virtual reality, were also identified. The fact that there is a meaningful gap between clinical practice and the scientific evidence was confirmed. The identified gap brings a certain degree of controversy. While some classic and well-known therapies had poor levels of supporting evidence, other relatively new approaches showed promising results.

A review found heterogeneous approaches and insufficient reporting in overviews on adverse events

OBJECTIVES

To investigate reporting and methodological characteristics of overviews on adverse (drug-associated) events (AEs) of pharmacological interventions.

STUDY DESIGN AND SETTING

We searched MEDLINE, Embase, Epistemonikos, and the Cochrane Database of Systematic Reviews from inception to May 17, 2021 for overviews exclusively investigating AEs of pharmacological interventions. We extracted general, reporting, and methodological characteristics and analyzed data descriptively.

RESULTS

We included 27 overviews, 70% of which were published in 2016 or later. The most common nomenclature in the title was "overview" (56%), followed by "umbrella review" (26%). The median number of included systematic reviews (SRs) in each overview was 15 (interquartile range 7-34). Study selection methods were reported in 52%, methods for data extraction in 67%, and methods for critical appraisal in 63% of overviews. An assessment of methodological quality of included SRs was performed in 70% of overviews. Only 22% of overviews reported strategies for dealing with overlapping SRs. An assessment of the certainty of the evidence was performed in 33% of overviews.

CONCLUSION

To ensure methodological rigor, authors of overviews on AEs should follow available guidance for the conduct and reporting of overviews.

How can clinicians choose between conflicting and discordant systematic reviews? A replication study of the Jadad algorithm

INTRODUCTION

The exponential growth of published systematic reviews (SRs) presents challenges for decision makers seeking to answer clinical, public health or policy questions. In 1997, an algorithm was created by Jadad et al. to choose the best SR across multiple. Our study aims to replicate author assessments using the Jadad algorithm to determine: (i) if we chose the same SR as the authors; and (ii) if we reach the same results.

METHODS

We searched MEDLINE, Epistemonikos, and Cochrane Database of SRs. We included any study using the Jadad algorithm. We used consensus building strategies to operationalise the algorithm and to ensure a consistent approach to interpretation.

RESULTS

We identified 21 studies that used the Jadad algorithm to choose one or more SRs. In 62% (13/21) of cases, we were unable to replicate the Jadad assessment and ultimately chose a different SR than the authors. Overall, 18 out of the 21 (86%) independent Jadad assessments agreed in direction of the findings despite 13 having chosen a different SR.

CONCLUSIONS

Our results suggest that the Jadad algorithm is not reproducible between users as there are no prescriptive instructions about how to operationalise the algorithm. In the absence of a validated algorithm, we recommend that healthcare providers, policy makers, patients and researchers address conflicts between review findings by choosing the SR(s) with meta-analysis of RCTs that most closely resemble their clinical, public health, or policy question, are the most recent, comprehensive (i.e. number of included RCTs), and at the lowest risk of bias.

Characteristics and methodological standards across systematic reviews with Meta-analysis of efficacy and/or effectiveness of influenza vaccines: an overview of reviews

BACKGROUND

While systematic reviews (SR) generally suggest that vaccination is an effective way to prevent influenza infection, it is not clear if these conclusions are based on high quality SR methods. As such, we systematically identified, critically appraised, and summarised the characteristics and adherence to methodological standards in SRs with meta-analysis of efficacy/effectiveness of influenza vaccines.

METHODS

We searched MEDLINE, Embase, Scopus, CINAHL, Global Health, and CDSR for English-language SR publications up to July 11, 2022. We summarised the characteristics, adherence to methodological standards and SR quality (AMSTAR 2).

RESULTS

From 11,193 retrieved citations, we included 48 publications (47 SRs). Seventy-five percent were of a critically low quality, 19% of a low quality, 2% of a moderate quality, and 4% of a high quality. Thirteen percent were industry-funded, about 13% co-authored by industry employee(s), and 4% commissioned by an organisation or authority. Only 45% percent reported protocol registration, 6% reported collaboration with a knowledge synthesis librarian/information specialist, and 60% utilised a reporting checklist (e.g. PRISMA).

CONCLUSIONS AND RELEVANCE

SRs with meta-analysis of efficacy/effectiveness of influenza vaccines are mostly of critically low quality and even the more recent reviews did not follow current best SR practices. These findings are significant in view of the controversies that surround influenza vaccines, and the use of SRs in informed decision-making. However, the findings do not justify curtailment or cessation of influenza vaccine use as vaccines continue to offer substantial net public health benefit.HighlightsWe systematically identified, critically appraised, and summarised the characteristics and adherence to methodological standards in 47 systematic reviews with meta-analysis of efficacy/effectiveness of influenza vaccines.13% of the reviews were industry-funded.About 13% of the reviews were co-authored by industry employee(s).4% of the reviews were commissioned by an organisation/authority.45% of the reviews reported protocol registration.6% of the reviews reported collaborating with a knowledge synthesis librarian/information specialist to prepare the search strategy.60% of the reviews reported using the PRISMA (or similar) checklist.75% of the reviews were judged to be of critically low quality; 19% of low quality; 2% of moderate quality; 4% of high quality.

Impact of industry sponsorship on the quality of systematic reviews of vaccines: a cross-sectional analysis of studies published from 2016 to 2019

BACKGROUND

Systematic reviews (SRs) provide the highest level of evidence and inform evidence-based decision making in health care. Earlier studies found association with industry to be negatively associated with methodological quality of SRs. However, this has not been investigated in SRs on vaccines.

METHODS

We performed a systematic literature search using MEDLINE and EMBASE in March 2020. The results were restricted to those published between 2016 and 2019 with no language restrictions. Study characteristics were extracted by one person and checked by an experienced reviewer. The methodological quality of the SRs was assessed with the AMSTAR 2 tool by multiple reviewers after a calibration exercise was performed. A summary score for each SR was calculated. The Mann-Whitney U test and Fisher's exact test were performed to compare both groups.

RESULTS

Out of 185 SRs that met all inclusion criteria, 27 SRs were industry funded. Those were matched with 30 non-industry funded SRs resulting in a total sample size of 57. The mean AMSTAR 2 summary score across all SRs was 0.49. Overall, the median AMSTAR 2 summary score was higher for the non-industry funded SRs than for the industry-funded SRs (0.62 vs. 0.36; p < .00001). Lower ratings for industry funded SRs were consistent across all but one AMSTAR 2 item, though significantly lower only for three specific items.

CONCLUSION

The methodological quality of SRs in vaccination is comparable to SRs in other fields, while it is still suboptimal. We are not able to provide a satisfactory explanation why industry funded SRs had a lower methodological quality than non-industry funded SRs over recent years. Industry funding is an important indicator of methodological quality for vaccine SRs and should be carefully considered when appraising SR quality.

AMSTAR 2 appraisal of systematic reviews and meta-analyses in the field of heart failure from high-impact journals

BACKGROUND

The Measurement Tool to Assess systematic Reviews (AMSTAR) 2 is a critical appraisal tool for systematic reviews (SRs) and meta-analyses (MAs) of interventions. We aimed to perform the first AMSTAR 2-based quality assessment of heart failure-related studies.

METHODS

Eleven high-impact journals were searched from 2009 to 2019. The included studies were assessed on the basis of 16 domains. Seven domains were deemed critical for high-quality studies. On the basis of the performance in these 16 domains with different weights, overall ratings were generated, and the quality was determined to be "high," "moderate," "low," or "critically low."

RESULTS

Eighty-one heart failure-related SRs with MAs were included. Overall, 79 studies were of "critically low quality" and two were of "low quality." These findings were attributed to insufficiency in the following critical domains: a priori protocols (compliance rate, 5%), complete list of exclusions with justification (5%), risk of bias assessment (69%), meta-analysis methodology (78%), and investigation of publication bias (60%).

CONCLUSIONS

The low ratings for these potential high-quality heart failure-related SRs and MAs challenge the discrimination capacity of AMSTAR 2. In addition to identifying certain areas of insufficiency, these findings indicate the need to justify or modify AMSTAR 2's rating rules.

Critical Appraisal of a Systematic Review: A Concise Review

OBJECTIVES

Concise definitive review of how to read and critically appraise a systematic review.

DATA SOURCES

None.

STUDY SELECTION

Current literature describing the conduct, reporting, and appraisal of systematic reviews and meta-analyses.

DATA EXTRACTION

Best practices for conducting, reporting, and appraising systematic review were summarized.

DATA SYNTHESIS

A systematic review is a review of a clearly formulated question that uses systematic and explicit methods to identify, select, and critically appraise relevant original research, and to collect and analyze data from the studies that are included in the review. Critical appraisal methods address both the credibility (quality of conduct) and rate the confidence in the quality of summarized evidence from a systematic review. The A Measurement Tool to Assess Systematic Reviews-2 tool is a widely used practical tool to appraise the conduct of a systematic review. Confidence in estimates of effect is determined by assessing for risk of bias, inconsistency of results, imprecision, indirectness of evidence, and publication bias.

CONCLUSIONS

Systematic reviews are transparent and reproducible summaries of research and conclusions drawn from them are only as credible and reliable as their development process and the studies which form the systematic review. Applying evidence from a systematic review to patient care considers whether the results can be directly applied, whether all important outcomes have been considered, and if the benefits are worth potential harms and costs.

Impact of arterial hypertension and its management strategies on cognitive function and dementia: a comprehensive umbrella review

INTRODUCTION

Cognitive decline and dementia recognize multiple risk factors and pathophysiological mechanisms, often involved simultaneously with complex interactions. Several studies have shown that both arterial hypertension and hypotension are associated with a greater risk of cognitive decline and dementia, but clinical evidence on this point is conflicting. Our aim was to conduct an umbrella review on cognitive function, dementia, and blood pressure, with particular attention to epidemiological, prognostic and therapeutic aspects.

EVIDENCE ACQUISITION

We conducted a dedicated literature search on PubMed for systematic reviews and meta-analyses that focused on arterial pressure, hypertension, hypotension and similar conditions, and cognitive function, cognitive decline and dementia. The internal validity of systematic reviews and meta-analyses was formally analyzed using the OQAQ tool. The umbrella review was planned in accordance with current international recommendations and was described as specified by the PRISMA guidelines.

EVIDENCE SYNTHESIS

Seventeen systematic reviews (including 13 meta-analyses) were included, for a total of 675 clinical studies and over 1 million patients. Hypertension results to be associated with a lower risk of Alzheimer's dementia, greater risk of vascular dementia and greater risk of cognitive decline. Orthostatic hypotension seems to be associated with greater risk of Alzheimer's dementia, vascular dementia and dementia of Parkinson's disease. Therapy with acetylcholinesterase inhibitors produces lower risk of cardiovascular events, greater risk of hypertension and greater risk of bradycardia, while the anti-hypertensive therapy leads to a lower risk of dementia of all types and lower risk of cognitive decline.

CONCLUSIONS

To date, the evidence on the relationship between blood pressure, cognitive decline and dementia provides somewhat heterogeneous data. Further studies are clearly needed, with explicit inclusion criteria as objective as possible, adequate follow-up and precise characterization of implemented cardiovascular and cognitive treatments.

Platelet-Rich Plasma Augmentation of Arthroscopic Rotator Cuff Repair Lowers Retear Rates and Improves Short-Term Postoperative Functional Outcome Scores: A Systematic Review of Meta-Analyses

Purpose

The purpose of this study is to conduct a systematic review of meta-analyses of rotator cuff repair using platelet-rich plasma (PRP) to identify whether PRP improves clinical function and rate of tendon retears. We will (1) conduct a systematic review of the current meta-analyses of rotator cuff repair using platelet-rich plasma available in the literature, (2) evaluate the quality of these meta-analyses using the Preferred Reporting Items for Systematic Review (PRISMA) methodology, (3) identify whether PRP improves clinical function and rate of tendon retears, and develop guidance to improve future studies in this area.

Methods

We carried out a systematic review of previous meta-analyses published on the clinical outcomes of PRP used in the treatment of rotator cuff tears. We performed a comprehensive search of PubMed, Medline, Cochrane, CINAHL (Cumulative Index to Nursing and Allied Health Literature), and Embase databases, using various combinations of the commercial names of each PRP preparation and “rotator cuff” (with its associated terms), looking specifically at human meta-analysis studies involving the repair of the rotator cuff tendon surgically in the English language. Data validity was assessed and collected on clinical outcomes. Following this, a meta-analysis was undertaken.

Results

Thirteen meta-analyses met the inclusion and exclusion criteria. All were considered of similar quality with Oxman-Guyatt index of 9 and PRISMA score of more than 24. A total of 1,800 patients with an average follow up of 12 to 36 months. The use of PRP for arthroscopic rotator cuff tear, when compared with controls, leads to a lower number of retears, improved short-term postoperative scores, and functional outcome. The following postoperative scores were reported: Constant: 12, Simple Shoulder Test: 10, ASES (American Shoulder and Elbow Surgeons): 9, UCLA (University of California, Los Angeles) 11, SANE (Single Assessment Numeric Evaluation) 1, VAS (visual analog scale): 6, and Retears: 13. Subgroup analysis showed that leukocyte content and gel application make no difference in the effectiveness of PRP. VAS score subgroup analysis showed short-term pain relief.

Conclusions

Our study shows that PRP is effective in reducing retears after rotator cuff repair and improving functional outcome scores and reducing short-term pain.

Level of Evidence

Level III, systematic review of Level I-III studies.

Umbrella Reviews: What They Are and Why We Need Them

Evidence in clinical research is accumulating and scientific publications have increased exponentially in the last decade across all disciplines. Available information should be critically assessed. Here, we focus on umbrella reviews, an approach that systematically collects and evaluates information from multiple systematic reviews and meta-analyses. To facilitate the design and the conduct of such a study, we provide a step-by-step guide on how to perform an umbrella review. We also present ways to report the summary findings, we describe various proposed grading criteria, and we discuss potential limitations.

The methodological quality on systematic reviews of surgical randomised controlled trials: A cross-sectional survey

BACKGROUND

Systematic reviews of RCTs have been developed to address end-users' needs and are regarded as the highest level of evidence. Flaws in the design, conduct and analyses of a systematic review can lead to erroneous conclusions and increase the research waste.

OBJECTIVE

We undertook a cross-sectional survey to identify the critical areas of weakness in systematic reviews for surgical interventions by AMSTAR 2.

METHODS

We searched PubMed, EMbase and Cochrane Library to summarize systematic reviews of surgical RCTs published in 2017. The information regarding general characteristics and methodological characteristics were gathered. We conducted descriptive analyses of study characteristics of included systematic reviews and explored the difference among varied methodological quality.

RESULTS

Totally 141 systematic reviews were identified. We found only four reviews (2.8%) were high quality, 3 (2.1%) were moderate quality, 8 (5.7%) were low quality, and the remaining 126 (89.4%) were of critical low quality. The critical weaknesses were lack of pre-registration or published protocols (29.1%), comprehensive literature search (17.7%), lists of excluded studies and reasons for exclusion (19.1%), description of detailed interventions (8.5%), extraction of funding source from included trials (10.6%), and consideration of the risk of bias of included trials when synthesized (16.3%) and interpret (20.6%) the results. Higher methodological quality was only positively associated with Cochrane systematic review.

CONCLUSION

Although two-thirds of included systematic reviews in the field of surgery were published in journals ranking Q1, the methodological quality is suboptimal and needs to be substantially improved. More efforts of multi-disciplinary teams' collaboration, continual education and training, integrally connection between primary studies and systematic review and contributing surgical research to practice should be imperative.

Vertical Transmission of SARS-CoV-2: A Systematic Review of Systematic Reviews

The COVID-19 pandemic has hugely impacted global public health and economy. The COVID-19 has also shown potential impacts on maternal perinatal and neonatal outcomes. This systematic review aimed to summarize the evidence from existing systematic reviews about the effects of SARS-CoV-2 infections on maternal perinatal and neonatal outcomes. We searched PubMed, MEDLINE, Embase, and Web of Science in accordance with PRISMA guidelines, from 1 December 2019 to 7 July 2021, for published review studies that included case reports, primary studies, clinical practice guidelines, overviews, case-control studies, and observational studies. Systematic reviews that reported the plausibility of mother-to-child transmission of COVID-19 (also known as vertical transmission), maternal perinatal and neonatal outcomes, and review studies that addressed the effect of SARS-CoV-2 infection during pregnancy were also included. We identified 947 citations, of which 69 studies were included for further analysis. Most (>70%) of the mother-to-child infection was likely due to environmental exposure, although a significant proportion (about 20%) was attributable to potential vertical transmission of SARS-CoV-2. Further results of the review indicated that the mode of delivery of pregnant women infected with SARS-CoV-2 could not increase or decrease the risk of infection for the newborns (outcomes), thereby emphasizing the significance of breastfeeding. The issue of maternal perinatal and neonatal outcomes with SARS-CoV-2 infection continues to worsen during the ongoing COVID-19 pandemic, increasing maternal and neonatal mortality, stillbirth, ruptured ectopic pregnancies, and maternal depression. Based on this study, we observed increasing rates of cesarean delivery from mothers with SARS-CoV-2 infection. We also found that SARS-CoV-2 could be potentially transmitted vertically during the gestation period. However, more data are needed to further investigate and follow-up, especially with reports of newborns infected with SARS-CoV-2, in order to examine a possible long-term adverse effect.

A systematic overview on the risk effects of psychosocial work characteristics on musculoskeletal disorders, absenteeism, and workplace accidents

The present article provides a systematic overview on the relationship between psychosocial work characteristics and musculoskeletal disorders, absenteeism, and workplace accidents. The study identified and reviewed the findings of 24 systematic reviews or meta-analysis and 6 longitudinal studies. Publications were systematically searched in several databases from 1966 to January 2021. To summarize the level of evidence, a best evidence synthesis was performed, and the quality of included studies was rated. High job demands, high job strain, high effort/reward-imbalance and low social support showed a strong evidence to increase the risk for musculoskeletal disorders. In addition to job demands and job strain, low perceived fairness proved to be a risk factor of absenteeism with strong evidence. Due to the small number of studies, no reliable evidence assessment for workplace accidents was possible. The summarized findings can improve risk assessment methods, by providing a systematic estimation of the potential risk severity of psychosocial work characteristics and assist practitioners in further developing the psychosocial risk assessment.

Methodological review to develop a list of bias items used to assess reviews incorporating network meta-analysis: protocol and rationale

INTRODUCTION

Systematic reviews with network meta-analysis (NMA; ie, multiple treatment comparisons, indirect comparisons) have gained popularity and grown in number due to their ability to provide comparative effectiveness of multiple treatments for the same condition. The methodological review aims to develop a list of items relating to biases in reviews with NMA. Such a list will inform a new tool to assess the risk of bias in NMAs, and potentially other reporting or quality checklists for NMAs which are being updated.

METHODS AND ANALYSIS

We will include articles that present items related to bias, reporting or methodological quality, articles assessing the methodological quality of reviews with NMA, or papers presenting methods for NMAs. We will search Ovid MEDLINE, the Cochrane library and difficult to locate/unpublished literature. Once all items have been extracted, we will combine conceptually similar items, classifying them as referring to bias or to other aspects of quality (eg, reporting). When relevant, reporting items will be reworded into items related to bias in NMA review conclusions, and then reworded as signalling questions.

ETHICS AND DISSEMINATION

No ethics approval was required. We plan to publish the full study open access in a peer-reviewed journal, and disseminate the findings via social media (Twitter, Facebook and author affiliated websites). Patients, healthcare providers and policy-makers need the highest quality evidence to make decisions about which treatments should be used in healthcare practice. Being able to critically appraise the findings of systematic reviews that include NMA is central to informed decision-making in patient care.

Noninvasive cardiovascular imaging for myocardial necrosis, viability, stunning and hibernation: evidence from an umbrella review encompassing 12 systematic reviews, 286 studies, and 201,680 patients

INTRODUCTION

The concomitant presence of myocardial necrosis with myocardial ischemia, stunning or hibernation may complicates appraisal of left ventricular (LV) function and patient management. Several imaging modalities have been proposed for the accurate assessment of myocardial necrosis, viability, stunning and hibernation, with mixed results. We aimed to review the evidence base on myocardial necrosis, stunning and hibernation by conducting an umbrella review (i.e. overview of systematic reviews).

EVIDENCE ACQUISITION

We searched PubMed and The Cochrane Library for meta-analyses focusing on the diagnostic, prognostic, or management appraisal of myocardial necrosis, viability, stunning and hibernation. Diagnostic test accuracy, prognostic yield, and clinical outcomes were systematically abstracted from shortlisted reviews.

EVIDENCE SYNTHESIS

From an initial set of 6069 citations, 12 systematic reviews were finally included, encompassing 286 studies and 201,680 patients. Cardiac magnetic resonance imaging (CMR) had favorable results in 4 reviews that focused on the diagnosis of myocardial stunning or hibernation in patients followed for 6±4 months after coronary revascularization (sensitivity 96% and specificity 91%). Positron emission tomography (PET), single photon emission tomography (SPECT) and CMR in 6 meta-analyses had each a significant and independent prognostic role for the prediction of fatal and non-fatal cardiovascular events in patients with follow-up of 2.8±1.7 years. Finally, 2 reviews with 2.3±1.1 years of follow-up showed moderate quality evidence in favor of coronary revascularization in patients with objective signs of myocardial viability.

CONCLUSIONS

The appraisal of myocardial necrosis and residual viability remains a cornerstone of the modern management of patients with CAD. Current imaging modalities (echocardiography, PET, SPECT and CMR) are widely used. Further trials using contemporary methods are warranted to further clarify the impact of viability assessment on patient management, and the cumulative risk of morbidity and mortality.

Systematic Reviews and Meta-Analysis in Spine Surgery-How Good Are They in Methodological Quality? A Systematic Review

STUDY DESIGN

Systematic review.

OBJECTIVES

To assess the methodological quality of systematic reviews and meta-analyses in spine surgery over the past 2 decades.

MATERIALS AND METHODS

We conducted independent and in duplicate systematic review of the published systematic reviews and meta-analyses between 2000 and 2019 from PubMed Central and Cochrane Database pertaining to spine surgery involving surgical intervention. We searched bibliographies to identify additional relevant studies. Methodological quality was evaluated with AMSTAR score and graded with AMSTAR 2 criteria.

RESULTS

A total of 96 reviews met the eligibility criteria, with mean AMSTAR score of 7.51 (SD = 1.98). Based on AMSTAR 2 criteria, 13.5% (n = 13) and 18.7% (n = 18) of the studies had high and moderate level of confidence of results, respectively, without any critical flaws. A total of 29.1% (n = 28) of the studies had at least 1 critical flaw and 38.5% (n = 37) of the studies had more than 1 critical flaw, so that their results have low and critically low confidence, respectively. Failure to analyze the conflict of interest of authors of primary studies included in review and lack of list of excluded studies with justification were the most common critical flaw. Regression analysis demonstrated that studies with funding and studies published in recent years were significantly associated with higher methodological quality.

CONCLUSION

Despite improvement in methodological quality of systematic reviews and meta-analyses in spine surgery in current decade, a substantial proportion continue to show critical flaws. With increasing number of review articles in spine surgery, stringent measures must be taken to adhere to methodological quality by following PRISMA and AMSTAR guidelines to attain higher standards of evidence in published literature.

How to Write a Systematic Review: A Narrative Review

In recent years, published systematic reviews in the world and in Iran have been increasing. These studies are an important resource to answer evidence-based clinical questions and assist health policy-makers and students who want to identify evidence gaps in published research. Systematic review studies, with or without meta-analysis, synthesize all available evidence from studies focused on the same research question. In this study, the steps for a systematic review such as research question design and identification, the search for qualified published studies, the extraction and synthesis of information that pertain to the research question, and interpretation of the results are presented in details. This will be helpful to all interested researchers.