Poor methodological quality and reporting standards of systematic reviews in burn care management

An overview of systematic reviews on imaging tests for diagnosis of pulmonary embolism applying different network meta-analytic methods

PURPOSE

This study aimed to apply different methods of diagnostic test accuracy network meta-analysis (DTA-NMA) for studies reporting results of five imaging tests for the diagnosis of suspected pulmonary embolism (PE): pulmonary angiography (PA), computed tomography angiography (CTPA), magnetic resonance angiography (MRA), planar ventilation/perfusion (V/Q) scintigraphy and single-photon emission computed tomography ventilation/perfusion (SPECT V/Q).

METHODS

We searched four databases (MEDLINE [via PubMed], Cochrane CENTRAL, Scopus, and Epistemonikos) from inception until June 2, 2022 to identify systematic reviews (SRs) describing diagnostic accuracy of PA, CTPA, MRA, V/Q scan and SPECT V/Q for suspected PE. Study-level data were extracted and pooled using a hierarchical summary receiver operating characteristic (HSROC) meta-regression approach and two DTA-NMA models to compare accuracy estimates of different imaging tests. Risk of bias was assessed using the QUADAS-2 (Quality Assessment of Diagnostic Accuracy Studies-2) tool and certainty of evidence using the GRADE (Grading of Recommendations Assessment, Development and Evaluation) framework.

RESULTS

We identified 13 SRs, synthesizing data from 33 primary studies and for four imaging tests (PA, CTPA, MRA and V/Q scan). The HSROC meta-regression model using PA as the reference standard showed that MRA had the best overall diagnostic performance with sensitivity of 0.93 (95% confidence interval [CI]: 0.76, 1.00) and specificity of 0.94 (95% CI: 0.84, 0.99). However, DTA-NMA models indicated that V/Q scan had the highest sensitivity, while CTPA was most specific.

CONCLUSION

Selecting a different DTA-NMA method to assess multiple diagnostic tests can affect estimates of diagnostic accuracy. There is no established method, but the choice depends on the data and familiarity with Bayesian statistics.

Evaluation of methodological and reporting quality of systematic reviews on conservative non-pharmacological musculoskeletal pain management in children and adolescents: A methodological analysis

BACKGROUND

There are no studies investigating the methodological and report quality of systematic reviews of non-pharmacological interventions for musculoskeletal pain management among children and adolescents.

OBJECTIVE

To evaluate the methodological and reporting quality of systematic reviews on conservative non-pharmacological pain management in children and adolescents with musculoskeletal pain.

METHODS

Searches were conducted on the Cochrane Database of Systematic Reviews, Medline, Embase, and three other databases. Two pairs of reviewers independently assessed each article according to the predetermined selection criteria. We assessed the methodological quality of systematic reviews, using the AMSTAR 2 checklist and the quality of reporting, using PRISMA checklist. Descriptive analysis was used to summarise the characteristics of all included systematic reviews. The percentage of systematic reviews achieving each item from the AMSTAR 2, PRISMA checklist and the overall confidence in the results were described.

RESULTS

We included 17 systematic reviews of conservative non-pharmacological pain management for musculoskeletal pain in children and adolescents. Of the 17 systematic reviews included, nine (53%) were rated as "critically low", seven (41%) were rated as "low", and one (6%) was rated as "high" methodological quality by AMSTAR-2. The reporting quality by items from PRISMA range from 17.6% (95% CI 6.2 to 41) to 100% (95% CI 81.6 to 100).

CONCLUSION

This systematic review of physical interventions in children and adolescents showed overall 'very low' to 'high' methodological quality and usually poor reporting quality.

Health systems research in burn care: an evidence gap map

BACKGROUND

Burn injury is associated with significant mortality and disability. Resilient and responsive health systems are needed for optimal response and care for people who sustain burn injuries. However, the extent of health systems research (HSR) in burn care is unknown. This review aimed to systematically map the global HSR related to burn care.

METHODS

An evidence gap map (EGM) was developed based on the World Health Organization health systems framework. All major medical, health and injury databases were searched. A standard method was used to develop the EGM.

RESULTS

A total of 6586 articles were screened, and the full text of 206 articles was reviewed, of which 106 met the inclusion criteria. Most included studies were cross-sectional (61%) and were conducted in hospitals (71%) with patients (48%) or healthcare providers (29%) as participants. Most studies were conducted in high-income countries, while only 13% were conducted in low-and middle-income countries, accounting for 60% of burns mortality burden globally. The most common health systems areas of focus were service delivery (53%), health workforce (33%) and technology (19%). Studies on health policy, governance and leadership were absent, and there were only 14 qualitative studies.

CONCLUSIONS

Major evidence gaps exist for an integrated health systems response to burns care. There is an inequity between the burden of burn injuries and HSR. Strengthening research capacity will facilitate evidence-informed health systems and policy reforms to sustainably improve access to affordable, equitable and optimal burn care and outcomes.

American Burn Association Strategic Quality Summit 2022: Setting the Direction for the Future

The American Burn Association (ABA) hosted a Burn Care Strategic Quality Summit (SQS) in an ongoing effort to advance the quality of burn care. The goals of the SQS were to discuss and describe characteristics of quality burn care, identify goals for advancing burn care, and develop a roadmap to guide future endeavors while integrating current ABA quality programs. Forty multidisciplinary members attended the two-day event. Prior to the event, they participated in a pre-meeting webinar, reviewed relevant literature, and contemplated statements regarding their vision for improving burn care. At the in-person, professionally facilitated Summit in Chicago, Illinois, in June 2022, participants discussed various elements of quality burn care and shared ideas on future initiatives to advance burn care through small and large group interactive activities. Key outcomes of the SQS included burn-related definitions of quality care, avenues for integration of current ABA quality programs, goals for advancing quality efforts in burn care, and work streams with tasks for a roadmap to guide future burn care quality-related endeavors. Work streams included roadmap development, data strategy, quality program integration, and partners and stakeholders. This paper summarizes the goals and outcomes of the SQS and describes the status of established ABA quality programs as a launching point for futurework.

The impact of funding on the quality and interpretation of systematic reviews of mechanical thrombectomy in stroke patients

BACKGROUND

Funding may impact the quality and findings of systematic reviews (SRs). We aimed to compare the methodological quality of funded and non-funded SRs that investigated the outcomes in ischemic stroke patients undergoing mechanical thrombectomy.

METHODS

We conducted a comprehensive search strategy in different databases, including Ovid Cochrane Central Register of Controlled Trials, Ovid Embase, Ovid Medline (including epub ahead of print, in-process & other non-indexed citations), PubMed, Scopus and Web of Science Core Collection to retrieve all relevant SRs. Random sequence generation matched each funded SR with a non-funded one. A Measurement Tool to Assess Systematic Reviews (AMSTAR)-2 tool was used to assess the bias and quality of the included SRs. We also used uni- and multivariate analysis to perform our analysis, and results were expressed in odds ratio (OR) and 95% confidence interval (CI).

RESULTS

We retrieved 150 articles, which were randomized and matched into 100 SRs, including 50 funded and 50 non-funded studies. By multivariate analysis, we found that including randomized clinical trials (RCTs) (OR: 5.7; 95% CI: 1.8-17.8; p = 0.003) and reporting conflict of interests (OR: 5.2; 95 CI: 1.1-24; p = 0.036) were the only significant differences between funded and non-funded SRs. No significant differences were found regarding the overall confidence for low-quality (OR: 0.54; 95% CI: 0.09-3.2; p = 0.49) and moderate/high-quality SRs (OR: 0.17; 95% CI: 0.02-1.87; p = 0.14).

CONCLUSION

Funded studies tend to include RCTs more often and report conflict of interests with no significant impact on overall confidence.

Impact of industry sponsorship on the quality of systematic reviews of vaccines: a cross-sectional analysis of studies published from 2016 to 2019

BACKGROUND

Systematic reviews (SRs) provide the highest level of evidence and inform evidence-based decision making in health care. Earlier studies found association with industry to be negatively associated with methodological quality of SRs. However, this has not been investigated in SRs on vaccines.

METHODS

We performed a systematic literature search using MEDLINE and EMBASE in March 2020. The results were restricted to those published between 2016 and 2019 with no language restrictions. Study characteristics were extracted by one person and checked by an experienced reviewer. The methodological quality of the SRs was assessed with the AMSTAR 2 tool by multiple reviewers after a calibration exercise was performed. A summary score for each SR was calculated. The Mann-Whitney U test and Fisher's exact test were performed to compare both groups.

RESULTS

Out of 185 SRs that met all inclusion criteria, 27 SRs were industry funded. Those were matched with 30 non-industry funded SRs resulting in a total sample size of 57. The mean AMSTAR 2 summary score across all SRs was 0.49. Overall, the median AMSTAR 2 summary score was higher for the non-industry funded SRs than for the industry-funded SRs (0.62 vs. 0.36; p < .00001). Lower ratings for industry funded SRs were consistent across all but one AMSTAR 2 item, though significantly lower only for three specific items.

CONCLUSION

The methodological quality of SRs in vaccination is comparable to SRs in other fields, while it is still suboptimal. We are not able to provide a satisfactory explanation why industry funded SRs had a lower methodological quality than non-industry funded SRs over recent years. Industry funding is an important indicator of methodological quality for vaccine SRs and should be carefully considered when appraising SR quality.

Methodological assessment and overall confidence in the results of systematic reviews with network meta-analyses in Endodontics

AIM

The aims of the study were to assess the methodological quality of systematic reviews with network meta-analyses (NMAs) in Endodontics using the "A MeaSurement Tool to Assess systematic Reviews" (AMSTAR 2) tool, and to evaluate the overall confidence in the results of the individual reviews included in the analysis.

METHODOLOGY

Systematic reviews with NMAs within the specialty of Endodontics published in English were identified from the PubMed, EbBSCOhost and SCOPUS databases from inception to July 2021. Two reviewers were involved independently in the selection of the reviews, data extraction, methodological quality assessment and overall confidence rating. Disagreements were resolved by discussion between the reviewers to achieve consensus; if disagreements persisted, a third reviewer made the final decision. The methodological quality of the included NMAs was appraised using the AMSTAR 2 checklist, which contains 16 items. The reviewers scored each item - 'Yes' - when the item was fully addressed, 'Partial Yes' - when the item was not fully addressed, or 'No' - when the item was not addressed. The overall confidence in the results of each review was classified as 'High', 'Moderate', 'Low' or 'Critically low' based on the criteria reported by the AMSTAR 2 developers.

RESULTS

Twelve systematic reviews with NMAs were included. All the NMAs adequately reported Item 1 ("Did the research questions and inclusion criteria for the review include the components of PICO?"), Item 8 ("Did the review authors describe the included studies in adequate detail?"), Item 9 ("Did the review authors use a satisfactory technique for assessing the risk of bias (RoB) in individual studies that were included in the review?") and Item 16 ("Did the review authors report any potential sources of conflict of interest, including any funding they received for conducting the review?") , whereas only one NMA reported Item 10 adequately ("Did the review authors report on the sources of funding for the studies included in the review?"). The overall confidence in the results of eight reviews was categorised as "Critically low", one review was "Low", two reviews were "Moderate" and one review was "High".

CONCLUSION

The overall confidence in the results for the majority of systematic reviews with NMAs in Endodontics was judged to be 'Critically low' as their methodological quality was below the necessary standard. AMSTAR 2 and PRISMA for NMA guidelines are available to guide authors to produce high quality systematic reviews with NMAs and for Editors and peer-reviewers when assessing submissions to journals.

Methodological quality assessment criteria for the evaluation of laboratory-based studies included in systematic reviews within the specialty of Endodontology: a development protocol

High quality systematic reviews in the field of Dentistry provide the most definitive overarching evidence for clinicians, guideline developers and healthcare policy makers to judge the foreseeable risks, anticipated benefits, and potential harms of dental treatment. In the process of carrying out a systematic review, it is essential that authors appraise the methodological quality of the primary studies they include, because studies which follow poor methodology will have a potentially serious negative impact on the overall strength of the evidence and the recommendations that can be drawn. In Endodontology, systematic reviews of laboratory studies have used quality assessment criteria developed subjectively by the individual authors as there are no comprehensive, well-structured, and universally accepted criteria that can be used objectively and universally to individual studies included in reviews. Unfortunately, these subjective criteria are likely to be inaccurately-defined, unreliably-applied, inadequately-analysed, unreasonably-biased, defective, and non-repeatable. The aim of the present paper is to outline the process to be followed in the development of comprehensive methodological quality assessment criteria to be used when evaluating laboratory studies that should be included in systematic reviews within Endodontology. The development of new methodological quality assessment criteria for appraising the laboratory-based studies included in systematic reviews within Endodontology will follow a three-stage process. First, a steering committee will be formed by the project leaders to develop a preliminary list of assessment criteria by modifying and adapting those already available, but with the addition of several new items relevant for Endodontology. The initial draft assessment criteria will be reviewed and refined by a Delphi Group (n=40) for their relevance and inclusion using a nine-point Likert scale. Second, the agreed items will then be discussed in an online or face-to-face meeting by a group of experts (n=10) to further refine the assessment criteria. Third, based on the feedback received from the online/face-to-face meeting, the steering committee will revise the quality assessment criteria and subsequently a group of authors will be selected to pilot the new system. Based on the feedback collected, the criteria may be revised further before being approved by the steering committee. The assessment criteria will be published in relevant journals, presented at national and international congresses/meetings, and will be freely available on a dedicated website. The steering committee will update the assessment criteria periodically based on feedback received from end-users.

Traduction franco-canadienne de l’Assessment of Systematic Reviews Revised (AMSTAR 2) : validation transculturelle et fidélité interjuges

Objective: Produce a French-Canadian translation of AMSTAR 2, affirm its content validity, and examine interrater reliability. Methods: Based on Vallerand's methodological approach, we conducted forward and parallel inverse-translations. Subsequently, an expert panel evaluated the translations to create a preliminary experimental French-Canadian version. A second expert panel examined this version and proposed additional modifications. Twenty future health professionals then rated the second experimental version for ambiguity on a scale (from 1 to 7). The principal co-investigators then reviewed the problematic elements and proposed a pre-official version. To ascertain content validity, a final back-translation was conducted resulting in the official version. Four judges evaluated 13 systematic reviews using the official French-Canadian version of AMSTAR 2. The Kappa coefficient was used to evaluate interrater reliability. Results: This rigorous adaptation enabled the development of a Franco-Canadian version of AMSTAR 2. Its application demonstrated low ambiguity (mean 1.15; SD 0.26) as well as good overall interrater reliability (total κ > 0.64) across all items. Conclusion: The French-Canadian version of AMSTAR 2 can now support francophone clinicians, educators, and managers in Canada as they undertake evidence-based practice.

Methodological quality and reporting standards in systematic reviews with meta-analysis of physical activity studies: a report from the Strengthening the Evidence in Exercise Sciences Initiative (SEES Initiative)

BACKGROUND

Several resources have been developed (e.g., reporting guidelines) to promote high-standard practices in health research. However, there was no continuous and systematic assessment of recommended practices in published systematic reviews with meta-analysis (SRMAs), which increases the usability of the available resources. Therefore, we aimed to assess the methodological and reporting standards in SRMAs of physical activity studies. This report presents the main results of the SEES Initiative in 2019.

METHODS

Our approach is based on a prospective systematic review methodology to implement post-publication surveillance of research practices in exercise sciences. Briefly, during the year 2019, pre-specified searches were conducted monthly (PubMed/MEDLINE) in journals from the exercise sciences (n = 9) and medicine (n = 5). The assessments were independently conducted by two authors, based on 36 items/practices derived from established statements/tools (PRISMA, AMSTAR 2, ROBIS). To be eligible, SRMAs should summarize studies that had, at least, one arm consisting of physical activity interventions/exposures and one health or behavioral outcome.

RESULTS

Out of 1028 studies assessed for eligibility, 103 SRMAs were included. The minimum adherence was 13/36 items, whereas only one SRMA adhered to all items. Some highly contemplated items included identification of title as SRMA (97.1%) and descriptions of the main outcome in the abstract (95.1%) and risk of bias (RoB) assessment (95.1%). Some poorly contemplated items included publicly available protocol (4.9%), discussion of the results in light of RoB in studies included (32.0%), and data sharing statements (35.9%).

CONCLUSION

In summary, there is a suboptimal adherence to recommended practices on methodological quality and reporting standards in the SRMAs of physical activity intervention/exposure evaluated from the selected journals in 2019, which likely reduce the reproducibility and usefulness of these studies. This incipient evidence from our first 12 months of post-publication surveillance should serve as a call for attention and action for multiple stakeholders (e.g., authors, reviewers, editors, funders, academic institutions) in this important health research field.

The certainty of the evidence in oral health has not improved according to GRADE: a meta-epidemiological study

OBJECTIVE

The aim of this meta-epidemiological study was to provide an update of the certainty of the evidence in oral health by using the GRADE rating reported in oral health Cochrane systematic reviews (CSR).

STUDY DESIGN AND SETTING

All interventional oral health CSR published between 2003-2021 were sourced. Study characteristics were extracted at the level of the CSR and the outcome/meta-analysis. One-hundred-five CSR were eligible and analysed.

RESULTS

Almost a third of CSR (n=67) were excluded as a GRADE rating was not available. The most prevalent type of primary studies included in the CSR were randomized studies (93.4%) and the most used measure of effect was the risk ratio (67.3%). Overall, the certainty of the evidence according to the GRADE rating for all examined outcomes was very low/low (88%). The two most common reasons for downgrading the confidence in the evidence were study limitations (Risk of bias) and imprecision. The odds of moderate/high vs. low/very low-GRADE rating are higher for the primary compared to the secondary outcomes after adjusting for year and number of trials (OR 2.49; 95% CI: 1.09, 5.65; P=0.02). Per year (2010-2021 period) the odds of moderate/high vs. low/very low-GRADE rating decrease (OR 0.73; 95% CI: 0.60, 0.90; P=0.01), and as the number of trials per comparison increase the odds of moderate/high vs. low/very low GRADE rating increase (OR 1.13; 95% CI: 1.01, 1.25; P=0.001).

CONCLUSIONS

The certainty of the evidence in oral health when assessed with the GRADE rating remains predominantly low or very low.

Methodological quality of systematic reviews comprising clinical practice guidelines for cardiovascular risk assessment and management for noncardiac surgery

BACKGROUND

Cardiac assessment in noncardiac surgery clinical practice guidelines should be supported by the highest-quality evidence such as that offered by systematic reviews. Currently, the methodological and reporting quality of these studies remains unknown.

METHODS

We used PubMed to search for all clinical practice guidelines related to perioperative cardiovascular patients undergoing noncardiac surgery from 2010 to 2021. The included clinical practice guidelines were analysed for all systematic reviews and meta-analyses. The primary objective of this study was to determine reporting and methodological quality using the PRISMA (Preferred Reporting Instrument for Systematic Reviews and Meta-Analyses) and AMSTAR-2 (A Measurement Tool to Assess Systematic Reviews-2) instruments. Our secondary objective was to compare systematic reviews conducted by the Cochrane Collaboration with non-Cochrane studies.

RESULTS

Three clinical practice guidelines were included in our study. Within these, 78 systematic reviews were included. PRISMA completion ranged from 34.8% to 100.0% with a mean of 76.9%. AMSTAR-2 completion ranged from 15.6% to 96.9% with a mean of 58.0%. Fifty-four systematic reviews underpinned a clinical practice guidelines recommendation, of which 25 were rated 'critically low' by AMSTAR-2 appraisal. Cochrane systematic reviews typically performed better than non-Cochrane studies, but were a minority of the included studies (10/78).

CONCLUSION

We found deficiencies in several key areas regarding the methodological and reporting qualities of systematic reviews included in cardiac assessment in noncardiac surgery clinical practice guidelines. As these clinical practice guidelines are instrumental to clinical decision-making and patient care in cardiac assessment in noncardiac surgery, we advocate for improved reporting quality among systematic reviews cited as supportive evidence for these recommendations.

Nanocellulose-based wound dressing for conservative wound management in children with second-degree burns

The initial care of burn wounds and choice of dressing are pivotal to optimally support the healing process. To ensure fast re-epithelialisation within 10-14 days and prevent complications, an optimal healing environment is essential. An innovative dressing based on nanocellulose was used for the treatment of burns in children. Children (0-16 years) with clean, partial-thickness burn wounds, 1 to 10% of the total body surface area were included. Complete re-epithelialisation was achieved within 7-17 days, with 13 patients showing re-epithelialised >95% by day 10. Satisfying results concerning time to re-epithelialisation and material handling were obtained. The possibility to leave the dressing on the wounds for 7 days showed a positive effect in the treatment of children, for whom every hospital visit may cause massive stress reactions. The nanocellulose-based dressing is a promising tool in conservative treatment of burns. Reducing the frequency of dressing changes supports a fast and undisturbed recovery; moreover, the dressing provides an optimal moist healing environment. The time to re-epithelialisation is comparable to frequently used materials, and cost reduction effect can be achieved without loss of quality. Possible pain and distress levels are kept to a minimum; therefore, flexibility and compliance of the patients and their parents are enhanced.

Quality Assessment of Published Systematic Reviews in High Impact Cardiology Journals: Revisiting the Evidence Pyramid

Objective: Systematic reviews are increasingly used as sources of evidence in clinical cardiology guidelines. In the present study, we aimed to assess the quality of published systematic reviews in high impact cardiology journals.

Methods: We searched PubMed for systematic reviews published between 2010 and 2019 in five general cardiology journals with the highest impact factor (according to Clarivate Analytics 2019). We extracted data on eligibility criteria, methodological characteristics, bias assessments, and sources of funding. Further, we assessed the quality of retrieved reviews using the AMSTAR tool.

Results: A total of 352 systematic reviews were assessed. The AMSTAR quality score was low or critically low in 71% (95% CI: 65.7–75.4) of the assessed reviews. Sixty-four reviews (18.2%, 95% CI: 14.5–22.6) registered/published their protocol. Only 221 reviews (62.8%, 95% CI: 57.6–67.7) reported adherence to the EQUATOR checklists, 208 reviews (58.4%, 95% CI: 53.9–64.1) assessed the risk of bias in the included studies, and 177 reviews (52.3%, 95% CI: 45.1–55.5) assessed the risk of publication bias in their primary outcome analysis. The primary outcome was statistically significant in 274 (79.6%, 95% CI: 75.1–83.6) and had statistical heterogeneity in 167 (48.5%, 95% CI: 43.3–53.8) reviews. The use and sources of external funding was not disclosed in 87 reviews (24.7%, 95% CI: 20.5–29.5). Data analysis showed that the existence of publication bias was significantly associated with statistical heterogeneity of the primary outcome and that complex design, larger sample size, and higher AMSTAR quality score were associated with higher citation metrics.

Conclusion: Our analysis uncovered widespread gaps in conducting and reporting systematic reviews in cardiology. These findings highlight the importance of rigorous editorial and peer review policies in systematic review publishing, as well as education of the investigators and clinicians on the synthesis and interpretation of evidence.

The reporting and methodological quality of systematic reviews and meta-analyses of nursing interventions for chronic obstructive pulmonary disease - A systematic review

AIMS

The aim of this review was to evaluate the reporting and methodological quality of systematic reviews and meta-analyses (SRs/MAs) on nursing interventions in patients with chronic obstructive pulmonary disease (COPD) and to determine potential factors that predict high quality.

DESIGN

The review is a quantitative systematic review.

DATA SOURCES

PubMed, Embase and the Cochrane Database of Systematic Reviews.

REVIEW METHODS

A comprehensive literature search was conducted in three databases for SRs/MAs published up to 6 May 2020. The PRISMA statement and AMSTAR checklist were used to evaluate the reporting and methodological quality.

RESULTS

A total of 130 articles published between 1996-2020 from 69 journals were included in this review. Multivariate regression analyses demonstrated that the following factors were related to the higher reporting quality of included articles: having a protocol or registration and being published on the Cochrane Database of Systematic Reviews. Systematic reviews including meta-analyses, number of authors >5, number of pages and having protocol or registration were related to higher methodological quality. A strong linear correlation (r = 0.860) was detected between the scores of PRISMA and AMSTAR.

CONCLUSION

A significant number of systematic reviews and meta-analyses on nursing interventions in patients with COPD show suboptimal reporting and poor methodology quality. The use of PRISMA and AMSTAR guidelines in conducting, reading, reviewing and editing systematic reviews and meta-analyses is recommended to improve the quality of systematic reviews and meta-analyses.

IMPACT

The findings of this review can provide references for health workers and health policy makers to evaluate and apply evidence-based knowledge. Additionally, such high-quality systematic reviews/meta-analyses can guide medical and health practice.

Meta-research publications in dentistry: a review

The present scoping review has the objective of providing an overview of meta-research in dentistry. A search of the PubMed database was performed for the period 11 October 2014 to 10 October 2019. Study selection and data extraction were performed independently by one author; prior to this, a random sample of 10% of the retrieved titles and abstracts were independently screened by two authors, achieving agreement of >80% on eligibility for initial inclusion, corresponding to good agreement. The following information was extracted from the full text of each article: meta-research area of interest; study design; type of studies evaluated in the meta-research; type of methodology used in assessment of the primary research; conflicts of interest reported; sponsorships reported; dental discipline; journal of publication; country of the first author; number of citations; and impact factor. A total of 7800 documents were initially retrieved. After analysis of the title/abstract and the full text of each article, and a snowballing procedure, 155 meta-research studies were identified and included. The 'methods' and 'reporting' meta-research areas were the most prevalent, with 73 (47%) and 61 (40%) studies, respectively. General dentistry, and orthodontics and dentofacial orthopaedics were the dental specialties with the greatest number/proportion of included studies with 45 (29%) and 28 (18%) studies, respectively. These findings may help to prioritize future meta-research in dentistry, consequently avoiding unnessecary investigations, and increasing the value of oral and dental research.

Optimierung der chirurgischen Behandlung Brandverletzter: Auszüge aus der fachübergreifenden S2k-Leitlinie „Behandlung thermischer Verletzungen des Erwachsenen“

Hintergrund Trotz Brandschutzmaßnahmen und Aufklärung erleiden ca. 0,5 Millionen Menschen pro Jahr in Deutschland Verbrennungen. Im Jahr 2018 mussten fast 2000 Schwerstbrandverletzte in deutschen Verbrennungszentren behandelt werden. Brandverletzungen stellen eine schwerwiegende Erkrankung mit hoher Mortalität und langfristigen Folgen für den Patienten dar.

Methodik und Ergebnisse Ziel dieses Übersichtsartikels ist es, eine Zusammenfassung der derzeit gültigen Leitlinie zur Behandlung von Verbrennungen im Erwachsenenalter (https://www.awmf.org/leitlinien/detail/ll/044–001.html) zu präsentieren, wobei der Fokus auf folgenden Aspekten liegt: 1. der Akutbehandlung der Brandwunde am Unfallort, 2. der Entscheidungsfindung zur Verlegung in ein Brandverletztenzentrum, 3. der chirurgischen Wundversorgung mit Debridement und Hauttransplantation und 4. der Nachsorge. Die S2k-Leitlinie basiert auf aktueller Literatur und dem Konsens von 11 an der Behandlung Brandverletzter beteiligter Fachgesellschaften.

Zusammenfassung Dieser Übersichtsartikel enthält gewichtete Empfehlungen zur Behandlung von Brandwunden erwachsener Patienten in Deutschland. Der Artikel soll Ärzten und medizinischem Personal in der Akutsituation vor Ort und in der Klinik die Entscheidung über die notwendige Behandlung erleichtern. Hiermit soll die Behandlungsqualität für Patienten verbessert werden.

Reporting quality in systematic reviews of in vitro studies: a systematic review

Background: Systematic reviews (SRs) and/or meta-analyses of in vitro research have an important role in establishing the foundation for clinical studies. In this study, we aimed to evaluate the reporting quality of SRs of in vitro studies using the PRISMA checklist.Method: Four databases were searched including PubMed, Virtual Health Library (VHL), Web of Science (ISI) and Scopus. The search was limited from 2006 to 2016 to include all SRs and/or meta-analyses (MAs) of pure in vitro studies. The evaluation of reporting quality was done using the PRISMA checklist.Results: Out of 7702 search results, 65 SRs were included and evaluated with the PRISMA checklist. Overall, the mean overall quality score of reported items of the PRISMA checklist was 68%. We have noticed an increasing pattern in the numbers of published SRs of in vitro studies over the last 10 years. In contrast, the reporting quality was not significantly improved over the same period (p = .363). There was a positive but not significant correlation between the overall quality score and the journal impact factor of the included studies.Conclusions: The adherence of SRs of in vitro studies to the PRISMA guidelines was poor. Therefore, we believe that using reporting guidelines and journals paying attention to this fact will improve the quality of SRs of in vitro studies.

Assessment of the abstract reporting of systematic reviews of dose-response meta-analysis: a literature survey

Background

There is an increasing number of published systematic reviews (SR) of dose-response meta-analyses (DRMAs) over the past decades. However, the quality of abstract reporting of these SR-DRMAs remains to be understood. We conducted a literature survey to investigate the abstract reporting of SR-DRMAs.

Methods

Medline, Embase, and Wiley online Library were searched for eligible SR-DRMAs. The reporting quality of SR-DRMAs was assessed by the modified PRISMA-for-Abstract checklist (14 items). We summarized the adherence rate of each item and categorized them as well complied (adhered by 80% or above), moderately complied (50 to 79%), and poorly complied (less than 50%). We used total score to reflect the abstract quality and regression analysis was employed to explore the potential influence factors for it.

Results

We included 529 SR-DRMAs. Eight of 14 items were moderately (3 items) or poorly complied (5 items) while only 6 were well complied by these SR-DRMAs. Most of the SR-DRMAs failed to describe the methods for risk of bias assessment (30.2, 95% CI: 26.4, 34.4%) and the results of bias assessment (48.8, 95% CI: 44.4, 53.1%). Few SR-DRMAs reported the funding (2.3, 95% CI: 1.2, 3.9%) and registration (0.6, 95% CI: 0.1, 1.6%) information in the abstract. Multivariable regression analysis suggested word number of abstracts [> 250 vs. ≤ 250 (estimated ß = 0.31; 95% CI: 0.02, 0.61; P = 0.039)] was positively associated with the abstract reporting quality.

Conclusion

The abstract reporting of SR-DRMAs is suboptimal, substantial effort is needed to improve the reporting. More word number may benefit for the abstract reporting. Given that reporting of abstract largely depends on the reporting and conduct of the SR-DRMA, review authors should also focus on the completeness of SR-DRMA itself.

Electronic supplementary material

The online version of this article (10.1186/s12874-019-0798-5) contains supplementary material, which is available to authorized users.

Exploring reporting quality of systematic reviews and Meta-analyses on nursing interventions in patients with Alzheimer's disease before and after PRISMA introduction

BACKGROUND

Systematic reviews (SRs) and meta-analyses (MAs) are distillation of current best available evidence, but are potentially prone to bias. The bias of SRs and MAs comes from sampling bias, selection bias and within study bias. So, their reporting quality is especially important as it may directly influence their utility for clinicians, nurses, patients and policy makers. The SRs and MAs on nursing interventions in patients with Alzheimer's disease (AD) have been increasingly published over the past decade, but the reporting quality of article has not been evaluated after the introduction of Preferred Reporting Items for Systematic Reviews and Meta Analyses (PRISMA) Statement.

METHODS

According to the inclusion and exclusion criteria, we searched the databases including PubMed, EMBASE and The Cochrane Library from inception through October 16th 2018. Two reviewers independently selected articles and extracted data. The PRISMA checklist was adopted to evaluate reporting quality. Comparisons were made between studies published before (2001-2009) and after (2011-2018) its introduction.

RESULTS

A total of 77 eligible articles, 18 (23.4%) were published before the PRISMA Statement and 59 (76.6%) were published afterwards. There was higher score after publication of the PRISMA Statement than before (20.83 ± 3.78 vs 17.11 ± 4.56, P <  0.05). There was an improvement in the following items after the PRISMA statement was released (P <  0.05): title (item 1, 50.0% vs 74.6%, OR = 3.10, 95CI%: 1.00-9.61), search (item8, 27.8% vs 57.6%,OR = 3.25, 95CI%: 1.14-9.28), study selection (item 9, 44.4% vs 81.4%,OR = 6.28, 95CI%: 1.93-20.37), Data collection process (item 10, 50.0% vs 76.3%,OR = 3.45, 95CI%:1.10-10.84), risk of bias in individual studies (item 12, 50.0% vs 83.1%, OR = 5.78, 95CI%:1.71-19.52), risk of bias across studies (item15, 5.6% vs 28.8%,OR = 3.60, 95CI%:1.04-12.43), study characteristics (item 18, 77.8% vs 98.3%, OR = 28.13, 95CI%:3.35-236.19), risk of bias with studies (item 19, 50.0% vs 83.1%, OR = 5.78, 95CI%:1.71-19.52), results in individual studies (item 20, 72.2% vs 94.9%, OR = 11.09, 95CI%:1.99-61.82), conclusions (item 26, 77.8% vs 98.3%, OR = 28.13, 95CI%:3.35-236.19). After controlling for the confounding factors, there were higher PRISMA score for systematic reviews including meta-analyses, protocol or registration, can't answer of RCT, journal source of SCI (Science Citation Index), manuscript length > 13 page and funding support.

CONCLUSION

Since the publication of the PRISMA Statement, there has been an improvement in the quality of reporting of SRs and MAs on nursing interventions in patients with AD. More endorsement by journals of the report guideline for SRs/MAs may improve articles reporting quality, and the dissemination of reliable evidence to nurses. We recommend authors, readers, reviewers, and editors to become more acquainted with and to more strictly adhere to the PRISMA checklist.

The reporting of safety among drug systematic reviews was poor before the implementation of the PRISMA harms checklist

OBJECTIVES

To examine, through a cross-sectional survey, how well safety information was reported among drug systematic reviews predating PRISMA harms checklist and explore factors associated with better reporting.

STUDY DESIGN AND SETTING

We searched PubMed to identify all systematic reviews published in the Cochrane Database of Systematic Review or the core clinical journals in 2015, one year before the PRISMA harms checklist was published. We randomly selected, in a 1:1 ratio, Cochrane and non-Cochrane systematic reviews assessing drug effects (including both efficacy and safety). We used the PRISMA harms checklist published in 2016 to assess the quality of reporting of drug safety information. Multivariable linear regression analyses were used to explore the association of six prespecified variables with more complete reporting of PRISMA harms items.

RESULTS

We included 120 systematic reviews, including 60 Cochrane and 60 non-Cochrane reviews. Scores on the PRISMA harms checklist (23 items) were low (median 4, [first, third quartile: 2, 6]), with no difference between Cochrane and non-Cochrane reviews (4.5 [2, 7] vs. 4 [2.5, 5]; P = 0.29). Among all eligible reviews, only one item (i.e., state conclusions in coherence with the review findings) was reported adequately (proportion of adherence 81.6%); proportion of reporting for other items ranged from 1.7% to 68.3%. The four essential reporting items from PRISMA harms checklist were also poorly complied (proportion of adherence ranged from 1.7% to 9.2%). Multivariable linear regression analyses found no significant associations between any study characteristic and reporting on the PRISMA harms, likely because of limited variability in scores across studies.

CONCLUSIONS

The reporting of safety information was poor both for Cochrane and non-Cochrane drug systematic reviews predating PRISMA harms checklist. The findings suggested a strong need to use the PRISMA harms checklist for reporting safety among drug systematic reviews.

Defining the process to literature searching in systematic reviews: a literature review of guidance and supporting studies

BACKGROUND

Systematic literature searching is recognised as a critical component of the systematic review process. It involves a systematic search for studies and aims for a transparent report of study identification, leaving readers clear about what was done to identify studies, and how the findings of the review are situated in the relevant evidence. Information specialists and review teams appear to work from a shared and tacit model of the literature search process. How this tacit model has developed and evolved is unclear, and it has not been explicitly examined before. The purpose of this review is to determine if a shared model of the literature searching process can be detected across systematic review guidance documents and, if so, how this process is reported in the guidance and supported by published studies.

METHOD

A literature review. Two types of literature were reviewed: guidance and published studies. Nine guidance documents were identified, including: The Cochrane and Campbell Handbooks. Published studies were identified through 'pearl growing', citation chasing, a search of PubMed using the systematic review methods filter, and the authors' topic knowledge. The relevant sections within each guidance document were then read and re-read, with the aim of determining key methodological stages. Methodological stages were identified and defined. This data was reviewed to identify agreements and areas of unique guidance between guidance documents. Consensus across multiple guidance documents was used to inform selection of 'key stages' in the process of literature searching.

RESULTS

Eight key stages were determined relating specifically to literature searching in systematic reviews. They were: who should literature search, aims and purpose of literature searching, preparation, the search strategy, searching databases, supplementary searching, managing references and reporting the search process.

CONCLUSIONS

Eight key stages to the process of literature searching in systematic reviews were identified. These key stages are consistently reported in the nine guidance documents, suggesting consensus on the key stages of literature searching, and therefore the process of literature searching as a whole, in systematic reviews. Further research to determine the suitability of using the same process of literature searching for all types of systematic review is indicated.

PRISMA and AMSTAR show systematic reviews on health literacy and cancer screening are of good quality

OBJECTIVES

The objectives of this study were to evaluate the reporting and methodological quality of systematic reviews (SRs) on health literacy and cancer screening and to investigate factors that may influence overall quality.

STUDY DESIGN AND SETTING

We reviewed SRs published between 2009 and 2017. We calculated indices to represent the included SRs' adherence to Preferred Reporting Items for Systematic Reviews and Meta-Analysis (PRISMA) and A Measurement Tool to Assess Systematic Reviews (AMSTAR). To assess possible determinants of SRs' quality, we regressed the index scores on year and region of publication, journal impact factor, authors' reported use of PRISMA, and presence of funding statements.

RESULTS

We included 19 SRs, and median index scores were 0.86 for PRISMA (interquartile range [IQR] = 0.11; range = 0.32-1.00) and 0.67 for AMSTAR (IQR = 0.30; range = 0.22-1.00). Methodological and reporting problems pertained to protocol registration or publication, number of raters used, gray literature searches, excluded article lists, and unintegrated discussions of risk of bias and efficacy. Only journal impact factor was statistically significantly associated (positively) with PRISMA and AMSTAR index scores.

CONCLUSION

The quality of SRs on health literacy and cancer screening was generally good. Systematic reviewers should register or publish their protocols, include PRISMA and AMSTAR checklists when submitting SRs to journals, and self-evaluate their SRs before submission.

A survey of prevalence of narrative and systematic reviews in five major medical journals

Background

Systematic reviews may provide less biased evidence than narrative reviews because they observe a strict methodology, similarly to primary studies. Hence, for clinical research questions, systematic reviews should be the study design of choice. It would be important to evaluate the prevalence and characteristics of narrative and systematic reviews published in prominent medical journals. Researchers and clinicians give great value to articles published in such scientific journals. This study sought to evaluate the prevalence and characteristics of narrative and systematic reviews in the five highest-ranked general medical journals and investigate the associations among type of review, number of citations, and impact factor (IF).

Methods

We surveyed the five highest-ranked medical journals (The New England Journal of Medicine, The Lancet, The Journal of the American Medical Association, The BMJ, and Annals of Internal Medicine) for narrative and systematic reviews published between June 2015 and June 2016. We independently selected and extracted the data from the reviews by strictly following the pre-determined eligibility criteria (Systematic and narrative reviews that focused on the management of diseases). We conducted regression analyses to investigate the associations among review type, number of citations, and IF. We also descriptively reported narrative reviews containing some methodology that might be reproducible.

Results

Two hundred seventy-five reviews were included: 75 (27%) systematic; 126 (46%) narrative with some methodology reported, and 74 (27%) narrative reviews. In comparison to systematic reviews, narrative reviews were more frequently published in journals with higher IF (risk ratio [RR] = 1.114 (95% CI 1.080 to 1.149). Systematic reviews received more citations than narrative reviews (group formed by narrative and narrative with some methodology reported (RR = 0.985 95% CI 0.978 to 0.991).

Conclusions

Non-systematic evidence is the most prevalent type of evidence in reviews published in the five highest-ranked general medical journals. Narrative reviews were more frequently published in journals with higher IF. We recommend that journals limit their space for narrative information, and to address clinical research questions, these journals consider publishing systematic evidence exclusively.

Electronic supplementary material

The online version of this article (10.1186/s12874-017-0453-y) contains supplementary material, which is available to authorized users.

Evaluations of the uptake and impact of the Preferred Reporting Items for Systematic reviews and Meta-Analyses (PRISMA) Statement and extensions: a scoping review

BACKGROUND

The PRISMA Statement is a reporting guideline designed to improve transparency of systematic reviews (SRs) and meta-analyses. Seven extensions to the PRISMA Statement have been published to address the reporting of different types or aspects of SRs, and another eight are in development. We performed a scoping review to map the research that has been conducted to evaluate the uptake and impact of the PRISMA Statement and extensions. We also synthesised studies evaluating how well SRs published after the PRISMA Statement was disseminated adhere to its recommendations.

METHODS

We searched for meta-research studies indexed in MEDLINE® from inception to 31 July 2017, which investigated some component of the PRISMA Statement or extensions (e.g. SR adherence to PRISMA, journal endorsement of PRISMA). One author screened all records and classified the types of evidence available in the studies. We pooled data on SR adherence to individual PRISMA items across all SRs in the included studies and across SRs published after 2009 (the year PRISMA was disseminated).

RESULTS

We included 100 meta-research studies. The most common type of evidence available was data on SR adherence to the PRISMA Statement, which has been evaluated in 57 studies that have assessed 6487 SRs. The pooled results of these studies suggest that reporting of many items in the PRISMA Statement is suboptimal, even in the 2382 SRs published after 2009 (where nine items were adhered to by fewer than 67% of SRs). Few meta-research studies have evaluated the adherence of SRs to the PRISMA extensions or strategies to increase adherence to the PRISMA Statement and extensions.

CONCLUSIONS

Many studies have evaluated how well SRs adhere to the PRISMA Statement, and the pooled result of these suggest that reporting of many items is suboptimal. An update of the PRISMA Statement, along with a toolkit of strategies to help journals endorse and implement the updated guideline, may improve the transparency of SRs.

Poor methodological quality and reporting standards of systematic reviews in burn care management

The methodological and reporting quality of burn-specific systematic reviews has not been established. The aim of this study was to evaluate the methodological quality of systematic reviews in burn care management. Computerised searches were performed in Ovid MEDLINE, Ovid EMBASE and The Cochrane Library through to February 2016 for systematic reviews relevant to burn care using medical subject and free-text terms such as 'burn', 'systematic review' or 'meta-analysis'. Additional studies were identified by hand-searching five discipline-specific journals. Two authors independently screened papers, extracted and evaluated methodological quality using the 11-item A Measurement Tool to Assess Systematic Reviews (AMSTAR) tool and reporting quality using the 27-item Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) checklist. Characteristics of systematic reviews associated with methodological and reporting quality were identified. Descriptive statistics and linear regression identified features associated with improved methodological quality. A total of 60 systematic reviews met the inclusion criteria. Six of the 11 AMSTAR items reporting on 'a priori' design, duplicate study selection, grey literature, included/excluded studies, publication bias and conflict of interest were reported in less than 50% of the systematic reviews. Of the 27 items listed for PRISMA, 13 items reporting on introduction, methods, results and the discussion were addressed in less than 50% of systematic reviews. Multivariable analyses showed that systematic reviews associated with higher methodological or reporting quality incorporated a meta-analysis (AMSTAR regression coefficient 2.1; 95% CI: 1.1, 3.1; PRISMA regression coefficient 6·3; 95% CI: 3·8, 8·7) were published in the Cochrane library (AMSTAR regression coefficient 2·9; 95% CI: 1·6, 4·2; PRISMA regression coefficient 6·1; 95% CI: 3·1, 9·2) and included a randomised control trial (AMSTAR regression coefficient 1·4; 95%CI: 0·4, 2·4; PRISMA regression coefficient 3·4; 95% CI: 0·9, 5·8). The methodological and reporting quality of systematic reviews in burn care requires further improvement with stricter adherence by authors to the PRISMA checklist and AMSTAR tool.

The influence of the team in conducting a systematic review

There is an increasing body of research documenting flaws in many published systematic reviews' methodological and reporting conduct. When good systematic review practice is questioned, attention is rarely turned to the composition of the team that conducted the systematic review. This commentary highlights a number of relevant articles indicating how the composition of the review team could jeopardise the integrity of the systematic review study and its conclusions. Key biases require closer attention such as sponsorship bias and researcher allegiance, but there may also be less obvious affiliations in teams conducting secondary evidence-syntheses. The importance of transparency and disclosure are now firmly on the agenda for clinical trials and primary research, but the meta-biases that systematic reviews may be at risk from now require further scrutiny.

Review of reviews on exposures to synthetic organic chemicals and children's neurodevelopment: Methodological and interpretation challenges

Environmental epidemiology data are becoming increasingly important in public health decision making, which commonly incorporates a systematic review of multiple studies. This review addresses two fundamental questions: What is the quality of available reviews on associations between exposure to synthetic organic chemicals and neurodevelopmental outcomes? What is the value (e.g., quality and consistency) of the underlying literature? Published reviews on associations between synthetic organic environmental chemical exposures and neurodevelopmental outcomes in children were systematically evaluated. Seventy-four relevant reviews were identified, and these were evaluated with respect to four methodological characteristics: (1) systematic inclusion/exclusion criteria and reproducible methods for search and retrieval of studies; (2) structured evaluation of underlying data quality; (3) systematic assessment of consistency across specific exposure-outcome associations; and (4) evaluation of reporting/publication bias. None of the 74 reviews fully met the criteria for all four methodological characteristics. Only four reviews met two criteria, and six reviews fulfilled only one criterion. Perhaps more importantly, the higher quality reviews were not able to meet all of the criteria owing to the shortcomings of underlying studies, which lacked comparability in terms of specific research question of interest, overall design, exposure assessment, outcome ascertainment, and analytic methods. Thus, even the most thoughtful and rigorous review may be of limited value if the underlying literature includes investigations that address different hypotheses and are beset by methodological inconsistencies and limitations. Issues identified in this review of reviews illustrate considerable challenges that are facing assessments of epidemiological evidence.