Innovations in data collection, management, and archiving for systematic reviews

Exercises for the prevention of non-specific low back pain

OBJECTIVES

This is a protocol for a Cochrane Review (intervention). The objectives are as follows: To evaluate the effects of exercise alone or exercise plus education compared with inactive control or education alone to prevent non-specific LBP.

Stroke Recurrence Rate and Risk Factors Among Stroke Survivors in Sub-Saharan Africa: A Systematic Review

Purpose

Evidence supporting secondary stroke in sub-Saharan Africa is scarce. This study describes the incidence of stroke recurrence and associated risk factors in sub-Saharan Africa.

Methods and Materials

Scientific databases were systematically searched from January 2000 to December 2022 for population-based observational studies, case-control or cohort studies of recurrent stroke involving adults aged 18 years and above in sub-Saharan Africa (SSA). We assessed the quality of the eligible studies using the Critical Appraisal Skills Program (CASP) checklist for observational studies.

Results

Six studies met the inclusion criteria and were included in this study. Stroke recurrence rates in SSA ranged from 9.4% to 25%. Majority of the studies were conducted from Western Africa and showed that stroke recurrence rates are high within sub-Saharan Africa ranging from 2% to 25%. The known stroke risk factors such as hypertension, chronic alcohol consumption, etc., remained the leading causes of stroke recurrence. The studies reported a higher mortality rate ranging from 20.5 -23% among those with recurrent strokes compared to primary strokes.

Conclusion

This systematic review is an update and summary of the available literature on stroke recurrence within sub-Saharan Africa. Further studies are warranted to assess the outcomes and burden of stroke recurrence in SSA.

Predictors of Intra-Operative and Post-Operative Pain Associated with Routine Dental Procedures in Children: A Systematic Review and Meta-Analysis

Background: Understanding predictors of pain associated with paediatric dental procedures could play an important role in preventing loss of cooperation, which often leads to the procedure having to be performed under general anaesthesia. Aim: We aimed to identify predictors of intra-operative and post-operative pain associated with routine dental procedures in children. Materials and Methods: A systematic review of observational studies was performed using electronic searches on MEDLINE, EMBASE, PsycINFO, Global Health via OVID, PubMed, Scopus, and SciELO. The NIH Quality Assessment Tool for Observational Cohort and Cross-Sectional Studies was used to evaluate the quality of the included studies, which were meta-analysed to estimate the impact of dental procedures and anxiety on children's pain perception. A meta-regression analysis was also performed to determine the relative effect of predictors on children's pain perception measured as mean differences on a visual analogue scale (VAS). Results: The search identified 532 articles; 53 were retrieved for full-text screening; 6 studies were included in the review; and 4 were eligible for the meta-analysis. The meta-analysis showed the types of procedures that predicted intra-operative pain, with dental extractions being the most painful (Mean VAS Difference [MD] 46.51 mm, 95% confidence interval [CI] 40.40 to 52.62 mm). The meta-regression showed that pain scores for dental extractions were significantly higher than polishing (the least painful procedure (reference category)) by VAS MD = 23.80 mm (95% CI 5.13-42.46 mm, p-value = 0.012). It also showed that highly anxious children reported significantly higher pain scores during dental procedures by a 12.31 mm MD VAS score (95% CI 5.23-19.40 mm, p-value = 0.001) compared to those with low anxiety levels. Conclusions: This systematic review demonstrates that the strongest predictors of intra-operative pain associated with paediatric dental procedures are dental extractions followed by drilling. Children with high anxiety also reported more pain for similar procedures. Tailoring interventions to reduce pain associated with paediatric dental procedures should be a priority for future research, as reducing pain can impact compliance and could reduce the need for general anaesthesia in dental treatment.

Data extraction and comparison for complex systematic reviews: a step-by-step guideline and an implementation example using open-source software

BACKGROUND

Data extraction (DE) is a challenging step in systematic reviews (SRs). Complex SRs can involve multiple interventions and/or outcomes and encompass multiple research questions. Attempts have been made to clarify DE aspects focusing on the subsequent meta-analysis; there are, however, no guidelines for DE in complex SRs. Comparing datasets extracted independently by pairs of reviewers to detect discrepancies is also cumbersome, especially when the number of extracted variables and/or studies is colossal. This work aims to provide a set of practical steps to help SR teams design and build DE tools and compare extracted data for complex SRs.

METHODS

We provided a 10-step guideline, from determining data items and structure to data comparison, to help identify discrepancies and solve data disagreements between reviewers. The steps were organised into three phases: planning and building the database and data manipulation. Each step was described and illustrated with examples, and relevant references were provided for further guidance. A demonstration example was presented to illustrate the application of Epi Info and R in the database building and data manipulation phases. The proposed guideline was also summarised and compared with previous DE guidelines.

RESULTS

The steps of this guideline are described generally without focusing on a particular software application or meta-analysis technique. We emphasised determining the organisational data structure and highlighted its role in the subsequent steps of database building. In addition to the minimal programming skills needed, creating relational databases and data validation features of Epi info can be utilised to build DE tools for complex SRs. However, two R libraries are needed to facilitate data comparison and solve discrepancies.

CONCLUSIONS

We hope adopting this guideline can help review teams construct DE tools that suit their complex review projects. Although Epi Info depends on proprietary software for data storage, it can still be a potential alternative to other commercial DE software for completing complex reviews.

Outcomes of patient education in adult oncologic patients receiving oral anticancer agents: a systematic review protocol

BACKGROUND

A large variety of oral anticancer agents have become available and while at first glance these therapies appear to provide only benefits, patients have expressed their need for educational interventions and raised safety issues. Although both patients and providers have recognized patient education's importance, and an interplay with safety has been acknowledged, no systematic reviews of the literature that summarize all of the current evidence related to patient education's outcomes for patients who receive oral anticancer agents have been performed to date. Accordingly, this systematic review will attempt to fill the gap in the literature as well as to map (1) contents, (2) methodologies, (3) settings, (4) timing/duration, and (5) healthcare professionals involved.

METHODS

This protocol is being reported in accordance with the Preferred Reporting Items for Systematic Reviews and Meta-Analyses guidelines. A systematic review will be performed. Studies that targeted eligible adult patients (≥ 18 years old) in hospital, outpatient, and home settings, and reported patient education's outcomes for those taking oral anticancer agents will be included. Searches will be conducted in PubMed/MEDLINE, CINAHL, Embase, and Scopus, and gray literature will be also sought. Two researchers will screen the search results independently and blindly in two phases: (1) title/abstract screening and (2) full-text screening using the Rayyan AI platform. An electronic data extraction form will be implemented and piloted, and then, two trained data extractors will extract the data cooperatively. Thereafter, a quality appraisal will be conducted using the Critical Appraisal Tools from The Joanna Briggs Institute. The results will be analyzed, grouped, clustered into categories, and discussed until a consensus is reached. Emerging evidence will be synthesized narratively and reported in accordance with the synthesis without meta-analysis guidelines.

DISCUSSION

The systematic review's results will be relevant to (1) policymakers and management at an institutional level, and (2) for clinical practice, in an evidence-based paradigm, potentially leading to a quality improvement with respect to safety and patient satisfaction.

SYSTEMATIC REVIEW REGISTRATION

PROSPERO CRD42022341797.

Impact of molecular diagnostic tests on diagnostic and treatment delays in tuberculosis: a systematic review and meta-analysis

BACKGROUND

Countries with high TB burden have expanded access to molecular diagnostic tests. However, their impact on reducing delays in TB diagnosis and treatment has not been assessed. Our primary aim was to summarize the quantitative evidence on the impact of nucleic acid amplification tests (NAAT) on diagnostic and treatment delays compared to that of the standard of care for drug-sensitive and drug-resistant tuberculosis (DS-TB and DR-TB).

METHODS

We searched MEDLINE, EMBASE, Web of Science, and the Global Health databases (from their inception to October 12, 2020) and extracted time delay data for each test. We then analysed the diagnostic and treatment initiation delay separately for DS-TB and DR-TB by comparing smear vs Xpert for DS-TB and culture drug sensitivity testing (DST) vs line probe assay (LPA) for DR-TB. We conducted random effects meta-analyses of differences of the medians to quantify the difference in diagnostic and treatment initiation delay, and we investigated heterogeneity in effect estimates based on the period the test was used in, empiric treatment rate, HIV prevalence, healthcare level, and study design. We also evaluated methodological differences in assessing time delays.

RESULTS

A total of 45 studies were included in this review (DS = 26; DR = 20). We found considerable heterogeneity in the definition and reporting of time delays across the studies. For DS-TB, the use of Xpert reduced diagnostic delay by 1.79 days (95% CI - 0.27 to 3.85) and treatment initiation delay by 2.55 days (95% CI 0.54-4.56) in comparison to sputum microscopy. For DR-TB, use of LPAs reduced diagnostic delay by 40.09 days (95% CI 26.82-53.37) and treatment initiation delay by 45.32 days (95% CI 30.27-60.37) in comparison to any culture DST methods.

CONCLUSIONS

Our findings indicate that the use of World Health Organization recommended diagnostics for TB reduced delays in diagnosing and initiating TB treatment. Future studies evaluating performance and impact of diagnostics should consider reporting time delay estimates based on the standardized reporting framework.

Prevalence and Incidence of Dry Eye and Meibomian Gland Dysfunction in the United States: A Systematic Review and Meta-analysis

Importance

Dry eye is a common clinical manifestation, a leading cause of eye clinic visits, and a significant societal and personal economic burden in the United States. Meibomian gland dysfunction (MGD) is a major cause of evaporative dry eye.

Objective

To conduct a systematic review and meta-analysis to obtain updated estimates of the prevalence and incidence of dry eye and MGD in the United States.

Data Sources

Ovid MEDLINE and Embase.

Study Selection

A search conducted on August 16, 2021, identified studies published between January 1, 2010, and August 16, 2021, with no restrictions regarding participant age or language of publication. Case reports, case series, case-control studies, and interventional studies were excluded.

Data Extraction and Synthesis

The conduct of review followed a protocol registered on PROSPERO (CRD42021256934). PRISMA guidelines were followed for reporting. Joanna Briggs Institute and Newcastle Ottawa Scale tools were used to assess risk of bias. Data extraction was conducted by 1 reviewer and verified by another for accuracy. Prevalence of dry eye and MGD were combined in separate meta-analyses using random-effects models.

Main Outcomes and Measures

Prevalence and incidence of dry eye and MGD in the United States. Summary estimates from meta-analysis of dry eye and MGD prevalence with 95% CI and 95% prediction intervals (95% PI).

Results

Thirteen studies were included in the systematic review. Dry eye prevalence was reported by 10 studies, dry eye incidence by 2 studies, and MGD prevalence by 3 studies. Meta-analysis estimated a dry eye prevalence of 8.1% (95% CI, 4.9%-13.1%; 95% PI, 0%-98.9%; 3 studies; 9 808 758 participants) and MGD prevalence of 21.2% (95% CI, 7.2%-48.3%; 95% PI, 0%-100%; 3 studies; 19 648 participants). Dry eye incidence was 3.5% in a population 18 years and older and 7.8% in a population aged 68 years and older. No studies reported MGD incidence.

Conclusions and Relevance

This systematic review and meta-analysis demonstrated uncertainty about the prevalence and incidence of dry eye and MGD in the United States. Population-based epidemiological studies that use consistent and validated definitions of dry eye and MGD are needed for higher-certainty estimates of dry eye and MGD prevalence and incidence in the United States.

Diagnostic accuracy of rapid one-step PCR assays for detection of herpes Simplex virus -1 and -2 in cerebrospinal fluid: A systematic Review and meta-analysis

BACKGROUND

Rapid and accurate diagnosis of HSV-1 and -2 (HSV1/2) in cerebrospinal fluid (CSF) is important for patient management.

OBJECTIVES

Summarize the diagnostic accuracy of commercial rapid sample-to-answer PCR assays (results in <90 minutes, without a separate nucleic acid extraction step) for HSV1/2 detection in CSF.

DATA SOURCES

Four databases (MEDLINE, EMBASE, Scopus and CENTRAL) and five conference abstract datasets from January 2012 to March 2022.

STUDY ELIGIBILITY CRITERIA

Diagnostic accuracy studies of FilmArray Meningitis-Encephalitis Panel™ and Simplexa™ HSV 1&2 Direct Kit compared to a PCR reference standard were included. Eligible studies provided sufficient data for the construction of a standard diagnostic accuracy two-by-two table.

PARTICIPANTS

Patients with suspected meningitis and/or encephalitis.

ASSESSMENT OF RISK OF BIAS

Two investigators independently extracted data, rated risk of bias and assessed quality using QUADAS-2.

METHODS

Accuracy estimates were pooled using Bayesian random effects models.

RESULTS

Thirty-one studies were included (27 FilmArray; 4 Simplexa), comprising 9,924 samples, with 95 HSV-1 and 247 HSV-2 infections. Pooled FilmArray sensitivities were 84.3% (95% credible interval 72.3%-93.0%) and 92.9% (95%CrI, 82.0%-98.5%) for HSV-1 and HSV-2, respectively; specificities were 99.8% (95%CrI, 99.6%-99.9%) and 99.9% (95%CrI, 99.9%-100%). Pooled Simplexa sensitivities were 97.1% (95%CrI, 88.1%-99.6%) and 97.9% (95%CrI, 89.6%-99.9%), respectively; specificities were 98.9% (95%CrI, 96.8%-99.7%) and 98.9% (95%CrI, 97.1%-99.7%). Pooled FilmArray sensitivities favored industry-sponsored studies by 10.0 and 13.0 percentage points for HSV-1 and HSV-2, respectively. Incomplete reporting frequently led to unclear risk of bias. Several FilmArray studies did not fully report true negative data leading to their exclusion.

CONCLUSION

Our results suggest Simplexa is accurate for HSV1/2 detection in CSF. Moderate FilmArray sensitivity for HSV-1 suggests additional testing and/or repeat CSF sampling is required for suspected HSV encephalitis when the HSV-1 result is negative. Low prevalence of HSV-1 infections limited summary estimates' precision. Underreporting of covariates limited exploration of heterogeneity.

Effectiveness of interventions for dry eye: a protocol for an overview of systematic reviews

INTRODUCTION

Dry eye is a leading cause of ocular morbidity and economic and societal burden for patients and healthcare systems. There are several treatment options available for dry eye and high-quality systematic reviews synthesise the evidence for their effectiveness and potential harms.

METHODS AND ANALYSIS

We will search the Cochrane Eyes and Vision US satellite (CEV@US) database of eyes and vision systematic reviews for systematic reviews on interventions for dry eye. CEV@US conducted an initial search of PubMed and Embase to populate the CEV@US database of eyes and vision systematic reviews in 2007, which was updated most recently in August 2021. We will search the database for systematic reviews published since 1 January 2016 because systematic reviews more than 5 years are unlikely to be up to date. We will consider Cochrane and non-Cochrane systematic reviews eligible for inclusion. Two authors will independently screen articles. We will include studies that evaluate interventions for dry eye and/or meibomian gland dysfunction with no restriction on types of participants or review language. We will select reliable systematic reviews (ie, those meeting pre-established methodological criteria) for inclusion, assessed by one investigator and verified by a second investigator. We will extract ratings of the certainty of evidence from within each review. We will report the degree of overlap for systematic reviews that answer similar questions and include overlapping primary studies. We will present results of the overview in alignment with guidelines in the Cochrane Handbook of Systematic Reviews of Interventions (Online Chapter 5: Overviews of Reviews), the Preferred Reporting Items for Systematic Reviews and Meta-Analyses statement, and an overview of reviews quality and transparency checklist. The anticipated start and completion dates for this overview are 1 May 2021 and 30 April 2022, respectively.

ETHICS AND DISSEMINATION

This overview will not require the approval of an Ethics Committee because it will use published studies. We will publish results in a peer-reviewed journal.

PROSPERO REGISTRATION NUMBER

CRD42021279880.

Assessing author willingness to enter study information into structured data templates as part of the manuscript submission process: A pilot study

Background

Environmental health and other researchers can benefit from automated or semi-automated summaries of data within published studies as summarizing study methods and results is time and resource intensive. Automated summaries can be designed to identify and extract details of interest pertaining to the study design, population, testing agent/intervention, or outcome (etc.). Much of the data reported across existing publications lack unified structure, standardization and machine-readable formats or may be presented in complex tables which serve as barriers that impede the development of automated data extraction methodologies.

As full automation of data extraction seems unlikely soon, encouraging investigators to submit structured summaries of methods and results in standardized formats with meta-data tagging of content may be of value during the publication process. This would produce machine-readable content to facilitate automated data extraction, establish sharable data repositories, help make research data FAIR, and could improve reporting quality.

Objectives

A pilot study was conducted to assess the feasibility of asking participants to summarize study methods and results using a structured, web-based data extraction model as a potential workflow that could be implemented during the manuscript submission process.

Methods

Eight participants entered study details and data into the Health Assessment Workplace Collaborative (HAWC). Participants were surveyed after the extraction exercise to ascertain 1) whether this extraction exercise will impact their conducting and reporting of future research, 2) the ease of data extraction, including which fields were easiest and relatively more problematic to extract and 3) the amount of time taken to perform data extractions and other related tasks. Investigators then presented participants the potential benefits of providing structured data in the format they were extracting. After this, participants were surveyed about 1) their willingness to provide structured data during the publication process and 2) whether they felt the potential application of structured data entry approaches and their implementation during the journal submission process should continue to be further explored.

Conclusions

Routine provision of structured data that summarizes key information from research studies could reduce the amount of effort required for reusing that data in the future, such as in systematic reviews or agency scientific assessments. Our pilot study suggests that directly asking authors to provide that data, via structured templates, may be a viable approach to achieving this: participants were willing to do so, and the overall process was not prohibitively arduous. We also found some support for the hypothesis that use of study templates may have halo benefits in improving the conduct and completeness of reporting of future research. While limitations in the generalizability of our findings mean that the conditions of success of templates cannot be assumed, further research into how such templates might be designed and implemented does seem to have enough chance of success that it ought to be undertaken.

Measurement properties of smartphone applications for the measurement of neck range of motion: a systematic review and meta analyses

Background

Smartphone applications offer an accessible and practical option to measure neck range of motion (ROM) and are becoming more commonly used in clinical practice. We assessed the validity, reliability, and responsiveness of smartphone applications (apps) to measure neck ROM in people with and without neck pain.

Methods

A comprehensive electronic search strategy of the main electronic databases was conducted from inception until June 2021. The identified studies investigated apps which measured neck ROM, and evaluated their validity, reliability, or responsiveness, in adult participants with neck pain or asymptomatic individuals. Two independent reviewers determined eligibility and risk of bias following COSMIN guidelines. The quality of evidence was assessed according to the GRADE approach.

Results

Eleven studies, with a total of 376 participants were included. Three types of apps were identified: clinometer apps, compass apps, and other apps of ‘adequate’ to ‘doubtful’ risk of bias. A meta-analysis revealed ‘good’ to ‘excellent’ intra-rater and inter-rater reliability across the three types of apps. The overall validity was rated from ‘moderate’ to ‘very high’ across all apps. The level of evidence was rated as ‘low’ to ‘very low’.

Conclusion

Smartphone applications showed sufficient intra-rater reliability, inter-rater reliability, and validity to measure neck ROM in people with and without neck pain. However, the quality of evidence and the confidence in the findings are low. High-quality research with large sample sizes is needed to further provide evidence to support the measurement properties of smartphone applications for the assessment of neck ROM.

Study registration

Following indications of Prisma-P guidelines, this protocol was registered in PROSPERO on 1/05/2021 with the number CRD42021239501.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12891-022-05066-6.

Reporting of methods to prepare, pilot and perform data extraction in systematic reviews: analysis of a sample of 152 Cochrane and non-Cochrane reviews

BACKGROUND

Previous research on data extraction methods in systematic reviews has focused on single aspects of the process. We aimed to provide a deeper insight into these methods by analysing a current sample of reviews.

METHODS

We included systematic reviews of health interventions in humans published in English. We analysed 75 Cochrane reviews from May and June 2020 and a random sample of non-Cochrane reviews published in the same period and retrieved from Medline. We linked reviews with protocols and study registrations. We collected information on preparing, piloting, and performing data extraction and on use of software to assist review conduct (automation tools). Data were extracted by one author, with 20% extracted in duplicate. Data were analysed descriptively.

RESULTS

Of the 152 included reviews, 77 reported use of a standardized extraction form (51%); 42 provided information on the type of form used (28%); 24 on piloting (16%); 58 on what data was collected (38%); 133 on the extraction method (88%); 107 on resolving disagreements (70%); 103 on methods to obtain additional data or information (68%); 52 on procedures to avoid data errors (34%); and 47 on methods to deal with multiple study reports (31%). Items were more frequently reported in Cochrane than non-Cochrane reviews. The data extraction form used was published in 10 reviews (7%). Use of software was rarely reported except for statistical analysis software and use of RevMan and GRADEpro GDT in Cochrane reviews. Covidence was the most frequent automation tool used: 18 reviews used it for study selection (12%) and 9 for data extraction (6%).

CONCLUSIONS

Reporting of data extraction methods in systematic reviews is limited, especially in non-Cochrane reviews. This includes core items of data extraction such as methods used to manage disagreements. Few reviews currently use software to assist data extraction and review conduct. Our results can serve as a baseline to assess the uptake of such tools in future analyses.

Quality appraisal of systematic reviews of interventions for children with cerebral palsy reveals critically low confidence

AIM

To evaluate the methodological quality of recent systematic reviews of interventions for children with cerebral palsy in order to determine the level of confidence in the reviews' conclusions.

METHOD

A comprehensive search of 22 databases identified eligible systematic reviews with and without meta-analysis published worldwide from 2015 to 2019. We independently extracted data and used A Measurement Tool to Assess Systematic Reviews-2 (AMSTAR-2) to appraise methodological quality.

RESULTS

Eighty-three systematic reviews met strict eligibility criteria. Most were from Europe and Latin America and reported on rehabilitative interventions. AMSTAR-2 appraisal found critically low confidence in 88% (n=73) because of multiple and varied deficiencies. Only 7% (n=6) had no AMSTAR-2 critical domain deficiency. The number of systematic reviews increased fivefold from 2015 to 2019; however, quality did not improve over time.

INTERPRETATION

Most of these systematic reviews are considered unreliable according to AMSTAR-2. Current recommendations for treating children with CP based on these flawed systematic reviews need re-evaluation. Findings are comparable to reports from other areas of medicine, despite the general perception that systematic reviews are high-level evidence. The required use of current widely accepted guidance for conducting and reporting systematic reviews by authors, peer reviewers, and editors is critical to ensure reliable, unbiased, and transparent systematic reviews. What this paper adds Confidence was critically low in the conclusions of 88% of systematic reviews about interventions for children with cerebral palsy (CP). Quality issues in the sample were not limited to systematic reviews of non-randomized trials, or to those about certain populations of CP or interventions. The inclusion of meta-analysis did not improve the level of confidence in these systematic reviews. Numbers of systematic reviews on this topic increased over the 5 search years but their methodological quality did not improve.

Adherence to Telemonitoring by Electronic Patient-Reported Outcome Measures in Patients with Chronic Diseases: A Systematic Review

Background: Effective telemonitoring is possible through repetitive collection of electronic patient-reported outcome measures (ePROMs) in patients with chronic diseases. Low adherence to telemonitoring may have a negative impact on the effectiveness, but it is unknown which factors are associated with adherence to telemonitoring by ePROMs. The objective was to identify factors associated with adherence to telemonitoring by ePROMs in patients with chronic diseases. Methods: A systematic literature search was conducted in PubMed, Embase, PsycINFO and the Cochrane Library up to 8 June 2021. Eligibility criteria were: (1) interventional and cohort studies, (2) patients with a chronic disease, (3) repetitive ePROMs being used for telemonitoring, and (4) the study quantitatively investigating factors associated with adherence to telemonitoring by ePROMs. The Cochrane risk of bias tool and the risk of bias in nonrandomized studies of interventions were used to assess the risk of bias. An evidence synthesis was performed assigning to the results a strong, moderate, weak, inconclusive or an inconsistent level of evidence. Results: Five studies were included, one randomized controlled trial, two prospective uncontrolled studies and two retrospective cohort studies. A total of 15 factors potentially associated with adherence to telemonitoring by ePROMs were identified in the predominate studies of low quality. We found moderate-level evidence that sex is not associated with adherence. Some studies showed associations of the remaining factors with adherence, but the overall results were inconsistent or inconclusive. Conclusions: None of the 15 studied factors had conclusive evidence to be associated with adherence. Sex was, with moderate strength, not associated with adherence. The results were conflicting or indecisive, mainly due to the low number and low quality of studies. To optimize adherence to telemonitoring with ePROMs, mixed-method studies are needed.

Efficacy of various prescribed vitamin D supplementation regimens on 25-hydroxyvitamin D serum levels in long-term care

Objective:

The aims of this study were to examine the efficacy among various vitamin D supplementation regimens on serum 25-hydroxyvitamin D (25(OH)D) concentrations and determine the minimal dose rate required to achieve sufficient serum concentrations (≥75 nmol/l) among older adults in long-term care (LTC).

Design:

A 1-year medical history was abstracted from medical records, and a one-time blood draw to measure serum 25(OH)D concentrations was obtained. Individuals were stratified into vitamin D-supplemented and non-supplemented groups. The supplemented group was further categorised into four treatment forms: single-ingredient vitamin D_2or3, multivitamin, Ca with vitamin D or combination of the three, and by daily prescribed doses: 0–9·9, 10–19·9, 20–49·9, 50–99·9 and >100 μg/d.

Setting:

Five LTC communities in Austin, Texas.

Participants:

One hundred seventy-three older (≥65 years) adults.

Results:

Of the participants, 62% received a vitamin D supplement and 55% had insufficient (≤75 nmol/l) 25(OH)D serum concentrations. Individuals receiving single-ingredient vitamin D_2or3 supplementation received the highest daily vitamin D mean dose (72·5 μg/d), while combination of forms was the most frequent treatment (44%) with the highest mean serum concentration (108 nmol/l). All supplementation doses were successful at reaching sufficient serum concentrations, except those<20 μg/d. Using a prediction model, it was observed that 0·025 μg/d of vitamin D supplementation resulted in a 0·008 nmol/l increase in serum 25(OH)D concentrations.

Conclusions:

Based on the predictive equation, results suggest that supplementation of 37·5 μg/d of vitamin D_2or3 or combination of vitamin D is most likely to achieve sufficient serum 25(OH)D concentrations in older adults in LTC.

Development, testing and use of data extraction forms in systematic reviews: a review of methodological guidance

Background

Data extraction forms link systematic reviews with primary research and provide the foundation for appraising, analysing, summarising and interpreting a body of evidence. This makes their development, pilot testing and use a crucial part of the systematic reviews process. Several studies have shown that data extraction errors are frequent in systematic reviews, especially regarding outcome data.

Methods

We reviewed guidance on the development and pilot testing of data extraction forms and the data extraction process. We reviewed four types of sources: 1) methodological handbooks of systematic review organisations (SRO); 2) textbooks on conducting systematic reviews; 3) method documents from health technology assessment (HTA) agencies and 4) journal articles. HTA documents were retrieved in February 2019 and database searches conducted in December 2019. One author extracted the recommendations and a second author checked them for accuracy. Results are presented descriptively.

Results

Our analysis includes recommendations from 25 documents: 4 SRO handbooks, 11 textbooks, 5 HTA method documents and 5 journal articles. Across these sources the most common recommendations on form development are to use customized or adapted standardised extraction forms (14/25); provide detailed instructions on their use (10/25); ensure clear and consistent coding and response options (9/25); plan in advance which data are needed (9/25); obtain additional data if required (8/25); and link multiple reports of the same study (8/25). The most frequent recommendations on piloting extractions forms are that forms should be piloted on a sample of studies (18/25); and that data extractors should be trained in the use of the forms (7/25). The most frequent recommendations on data extraction are that extraction should be conducted by at least two people (17/25); that independent parallel extraction should be used (11/25); and that procedures to resolve disagreements between data extractors should be in place (14/25).

Conclusions

Overall, our results suggest a lack of comprehensiveness of recommendations. This may be particularly problematic for less experienced reviewers. Limitations of our method are the scoping nature of the review and that we did not analyse internal documents of health technology agencies.

Reliability of the Evidence Addressing Treatment of Corneal Diseases: A Summary of Systematic Reviews

Importance

Patient care should be informed by clinical practice guidelines, which in turn should be informed by evidence from reliable systematic reviews. The American Academy of Ophthalmology is updating its Preferred Practice Patterns (PPPs) for the management of the following 6 corneal diseases: bacterial keratitis, blepharitis, conjunctivitis, corneal ectasia, corneal edema and opacification, and dry eye syndrome.

Objective

To summarize the reliability of the existing systematic reviews addressing interventions for corneal diseases.

Data Source

The Cochrane Eyes and Vision US Satellite database.

Study Selection

In this study of published systematic reviews from 1997 to 2017 (median, 2014), the Cochrane Eyes and Vision US Satellite database was searched for systematic reviews evaluating interventions for the management of any corneal disease, combining eyes and vision keywords and controlled vocabulary terms with a validated search filter.

Data Extraction and Synthesis

The study classified systematic reviews as reliable when each of the following 5 criteria were met: the systematic review specified eligibility criteria for inclusion of studies, conducted a comprehensive literature search for studies, assessed risk of bias of the individual included studies, used appropriate methods for quantitative syntheses (meta-analysis) (only assessed if meta-analysis was performed), and had conclusions that were supported by the results of the systematic review. They were classified as unreliable if at least 1 criterion was not met.

Main Outcomes and Measures

The proportion of systematic reviews that were reliable and the reasons for unreliability.

Results

This study identified 98 systematic reviews that addressed interventions for 15 corneal diseases. Thirty-three of 98 systematic reviews (34%) were classified as unreliable. The most frequent reasons for unreliability were that the systematic review did not conduct a comprehensive literature search for studies (22 of 33 [67%]), did not assess risk of bias of the individual included studies (13 of 33 [39%]), and did not use appropriate methods for quantitative syntheses (meta-analysis) (12 of 17 systematic reviews that conducted a quantitative synthesis [71%]). Sixty-five of 98 systematic reviews (66%) were classified as reliable. Forty-two of the 65 reliable systematic reviews (65%) addressed corneal diseases relevant to the 2018 American Academy of Ophthalmology PPPs; 33 of these 42 systematic reviews (79%) are cited in the 2018 PPPs.

Conclusions and Relevance

One in 3 systematic reviews addressing interventions for corneal diseases are unreliable and thus were not used to inform PPP recommendations. Careful adherence by systematic reviewers and journal editors to well-established best practices regarding systematic review conduct and reporting might help make future systematic reviews in eyes and vision more reliable.

Outcome choice and definition in systematic reviews leads to few eligible studies included in meta-analyses: a case study

BACKGROUND

There is broad recognition of the importance of evidence in informing clinical decisions. When information from all studies included in a systematic review ("review") does not contribute to a meta-analysis, decision-makers can be frustrated. Our objectives were to use the field of eyes and vision as a case study and examine the extent to which authors of Cochrane reviews conducted meta-analyses for their review's pre-specified main outcome domain and the reasons that some otherwise eligible studies were not incorporated into meta-analyses.

METHODS

We examined all completed systematic reviews published by Cochrane Eyes and Vision, as of August 11, 2017. We extracted information about each review's outcomes and, using an algorithm, categorized one outcome as its "main" outcome. We calculated the percentage of included studies incorporated into meta-analyses for any outcome and for the main outcome. We examined reasons for non-inclusion of studies into the meta-analysis for the main outcome.

RESULTS

We identified 175 completed reviews, of which 125 reviews included two or more studies. Across these 125 reviews, the median proportions of studies incorporated into at least one meta-analysis for any outcome and for the main outcome were 74% (interquartile range [IQR] 0-100%) and 28% (IQR 0-71%), respectively. Fifty-one reviews (41%) could not conduct a meta-analysis for the main outcome, mostly because fewer than two included studies measured the outcome (21/51 reviews) or the specific measurements for the outcome were inconsistent (16/51 reviews).

CONCLUSIONS

Outcome choice during systematic reviews can lead to few eligible studies included in meta-analyses. Core outcome sets and improved reporting of outcomes can help solve some of these problems.

Adjudication rather than experience of data abstraction matters more in reducing errors in abstracting data in systematic reviews

BACKGROUND

During systematic reviews, "data abstraction" refers to the process of collecting data from reports of studies. The data abstractors' level of experience may affect the accuracy of data abstracted. Using data from a randomized crossover trial in which different data abstraction approaches were compared, we examined the association between abstractors' level of experience and accuracy of data abstraction.

METHODS

We classified abstractors as "more experienced" if they had authored three or more published systematic reviews, and "less experienced" otherwise. Each abstractor abstracted data related to study design, baseline characteristics, and outcomes/results from six articles. We considered two types of errors: incorrect abstraction and errors of omission. We estimated the proportion of errors by level of experience using a binomial generalized linear mixed model.

RESULTS

We used data from 25 less experienced and 25 more experienced data abstractors. Overall error proportions were similar for less experienced abstractors (21%) and more experienced abstractors (19%). Compared with less experienced abstractors, more experienced abstractors had a lower odds of errors for data items related to outcomes/results (adjusted odds ratio [OR] = 0.53; 95% CI, 0.34-0.82) and potentially for data items related to study design (adjusted OR = 0.83; 95% CI, 0.64-1.09) but a potentially higher odds of errors for items related to baseline characteristics (adjusted OR = 1.42; 95% CI, 0.97-2.06).

CONCLUSION

Experience of data abstraction matters little. Errors are reduced by adjudication but still remain high for data items related to outcomes/results.

The Systematic Review Data Repository (SRDR): descriptive characteristics of publicly available data and opportunities for research

BACKGROUND

Conducting systematic reviews ("reviews") requires a great deal of effort and resources. Making data extracted during reviews available publicly could offer many benefits, including reducing unnecessary duplication of effort, standardizing data, supporting analyses to address secondary research questions, and facilitating methodologic research. Funded by the US Agency for Healthcare Research and Quality (AHRQ), the Systematic Review Data Repository (SRDR) is a free, web-based, open-source, data management and archival platform for reviews. Our specific objectives in this paper are to describe (1) the current extent of usage of SRDR and (2) the characteristics of all projects with publicly available data on the SRDR website.

METHODS

We examined all projects with data made publicly available through SRDR as of November 12, 2019. We extracted information about the characteristics of these projects. Two investigators extracted and verified the data.

RESULTS

SRDR has had 2552 individual user accounts belonging to users from 80 countries. Since SRDR's launch in 2012, data have been made available publicly for 152 of the 735 projects in SRDR (21%), at a rate of 24.5 projects per year, on average. Most projects are in clinical fields (144/152 projects; 95%); most have evaluated interventions (therapeutic or preventive) (109/152; 72%). The most frequent health areas addressed are mental and behavioral disorders (31/152; 20%) and diseases of the eye and ocular adnexa (23/152; 15%). Two-thirds of the projects (104/152; 67%) were funded by AHRQ, and one-sixth (23/152; 15%) are Cochrane reviews. The 152 projects each address a median of 3 research questions (IQR 1-5) and include a median of 70 studies (IQR 20-130).

CONCLUSIONS

Until we arrive at a future in which the systematic review and broader research communities are comfortable with the accuracy of automated data extraction, re-use of data extracted by humans has the potential to help reduce redundancy and costs. The 152 projects with publicly available data through SRDR, and the more than 15,000 studies therein, are freely available to researchers and the general public who might be working on similar reviews or updates of reviews or who want access to the data for decision-making, meta-research, or other purposes.

Mapping of reporting guidance for systematic reviews and meta-analyses generated a comprehensive item bank for future reporting guidelines

OBJECTIVES

The aim of the study was to generate a comprehensive bank of systematic review (SR) reporting items to inform an update of the Preferred Reporting Items for Systematic reviews and Meta-Analyses (PRISMA) 2009 statement.

METHODS

We searched the Enhancing the QUAlity and Transparency Of health Research Network library in May 2019 to identify all reporting guidelines for SRs that were published after 2009, regardless of the scope of the guideline. We also conducted a selective review of four guidance manuals for SRs, three tools for assessing the risk of bias in SRs, six meta-research studies evaluating the reporting quality of SRs using a tailored checklist, and five reporting guidelines for other study designs. One author screened and selected sources for inclusion, extracted reporting guidance from sources, and mapped guidance against the PRISMA 2009 checklist items.

RESULTS

We included 60 sources providing guidance on reporting of SRs and meta-analyses. From these, we collated a list of 221 unique reporting items. Items were categorized into title (four items), abstract (10 items), introduction (12 items), methods (111 items), results (61 items), discussion (12 items), funding and conflicts of interest (four items), administrative information (three items), and data availability (four items). This exercise generated 175 reporting items that could be added to the guidance in the PRISMA 2009 statement.

CONCLUSION

Generation of a comprehensive item bank through review and mapping of the literature facilitates identification of missing items and those needing modification, which may not otherwise be identified by the guideline development team or from other activities commonly used to develop reporting guidelines.

Identification and Description of Reliable Evidence for 2016 American Academy of Ophthalmology Preferred Practice Pattern Guidelines for Cataract in the Adult Eye

Importance

Trustworthy clinical practice guidelines require reliable systematic reviews of the evidence to support recommendations. Since 2016, the American Academy of Ophthalmology (AAO) has partnered with Cochrane Eyes and Vision US Satellite to update their guidelines, the Preferred Practice Patterns (PPP).

Objective

To describe experiences and findings related to identifying reliable systematic reviews that support topics likely to be addressed in the 2016 update of the 2011 AAO PPP guidelines on cataract in the adult eye.

Design, Setting, and Participants

Cross-sectional study. Systematic reviews on the management of cataract were searched for in an established database. Each relevant systematic review was mapped to 1 or more of the 24 management categories listed under the Management section of the table of contents of the 2011 AAO PPP guidelines. Data were extracted to determine the reliability of each systematic review using prespecified criteria, and the reliable systematic reviews were examined to find whether they were referenced in the 2016 AAO PPP guidelines. For comparison, we assessed whether the reliable systematic reviews published before February 2010 the last search date of the 2011 AAO PPP guidelines were referenced in the 2011 AAO PPP guidelines. Cochrane Eyes and Vision US Satellite did not provide systematic reviews to the AAO during the development of the 2011 AAO PPP guidelines.

Main Outcomes and Measures

Systematic review reliability was defined by reporting eligibility criteria, performing a comprehensive literature search, assessing methodologic quality of included studies, using appropriate methods for meta-analysis, and basing conclusions on review findings.

Results

From 99 systematic reviews on management of cataract, 46 (46%) were classified as reliable. No evidence that a comprehensive search had been conducted was the most common reason a review was classified as unreliable. All 46 reliable systematic reviews were cited in the 2016 AAO PPP guidelines, and 8 of 15 available reliable reviews (53%) were cited in the 2011 PPP guidelines.

Conclusions and Relevance

The partnership between Cochrane Eyes and Vision US Satellite and the AAO provides the AAO access to an evidence base of relevant and reliable systematic reviews, thereby supporting robust and efficient clinical practice guidelines development to improve the quality of eye care.

Researching the Appropriateness of Care in the Complementary and Integrative Health Professions Part 5: Using Patient Records: Selection, Protection, and Abstraction

OBJECTIVES

The purpose of this paper is to describe the 4-step process (consent, selection, protection, and abstraction) of acquiring a large sample of chiropractic patient records from multiple practices and subsequent data abstraction.

METHODS

From April 2017 to December 2017, RAND acquired patient records from 99 chiropractic practices across the United States. The records included patients enrolled in a survey e-study (prospective sample) and a random sample of all clinic patients (retrospective sample) with chronic back or neck pain. Clinic staff were trained to collect the sample, scan, and transfer the records. We designed an online data collection tool for abstraction. Protocols were instituted to protect patient confidentiality. Doctors of chiropractic were selected and trained as abstractors, and a system was established to monitor data collection.

RESULTS

In compliance with data protection protocols, 3603 patient records were scanned, including 1475 in the prospective sample and 2128 in the random sample. A total of 1716 patients (prospective sample) consented to having their records scanned, but only 1475 could be retrieved. Of records scanned, 19% were unusable owing to illegibility, no care during the period of interest, or poor scanning. The abstractor interrater reliability for appropriateness of care decisions was fair to moderate (κ .38-.48).

CONCLUSION

The acquisition, handling, and abstraction of a large sample of chiropractic records was a complex task with challenges that necessitated adapting planned approaches. Of the records abstracted, many revealed incomplete provider documentation regarding the details of and rationale for care. Better documentation and more standardized record keeping would facilitate future research using patient records.

Registration Practices of Randomized Clinical Trials in Rhinosinusitis: A Cross-sectional Review

Importance

Randomized clinical trials (RCTs) play an important role in the development of clinical practice guidelines and in clinical decision making. Little is known about the registration practices of RCTs in the diagnosis and treatment of rhinosinusitis.

Objectives

The primary outcome was the frequency of reported RCT registry numbers by authors of rhinosinusitis RCTs. A secondary outcome was the rates of selective reporting bias in RCTs that were prospectively registered. A tertiary outcome end point was the frequency of publication of RCTs registered on ClinicalTrials.gov.

Evidence Review

Our sample was derived from a PubMed (MEDLINE) search performed on October 31, 2017, using the keywords "sinusitis OR rhinosinusitis OR rhinitis," and filtering by date (January 1, 2015, through October 31, 2017) and study type (RCT). Studies that were considered an RCT were included for the primary outcome analysis. All RCTs that were registered prior to or during patient enrollment were included for secondary outcome analysis. For the tertiary outcome, a search was performed on ClinicalTrials.gov on September 4, 2018, using the keywords "sinusitis OR rhinosinusitis," and filtering by date (January 1, 2013, through October 31, 2015). All analysis took place between October 29, 2018, and October 31, 2018.

Findings

A total of 179 RCTs were analyzed for our primary outcome: 94 (52.5%) included a registration number in their publication, and 70 (39.1%) were included for secondary outcome analysis of rates of selective reporting bias. Of these 70 RCTs, 22 (31%) were found to have at least 1 major discrepancy between trial registration and publication. For the tertiary outcome, 52 completed clinical trials were identified on ClinicalTrials.gov, of which 21 (40%) had listed publication in the registry.

Conclusions and Relevance

We found that published rhinosinusitis RCTs frequently do not include a trial registration number and that registered RCTs frequently are not published. Furthermore, RCTs that are registered often display selective reporting of their outcomes and frequently favor positive results. We recommend strict adherence to RCT registration policies and the enforcement of accurate reporting to help strengthen the evidence behind clinical decision making.

Reproducibility and replicability of systematic reviews

Irreproducibility of research causes a major concern in academia. This concern affects all study designs regardless of scientific fields. Without testing the reproducibility and replicability it is almost impossible to repeat the research and to gain the same or similar results. In addition, irreproducibility limits the translation of research findings into practice where the same results are expected. To find the solutions, the Interacademy Partnership for Health gathered academics from established networks of science, medicine and engineering around a table to introduce seven strategies that can enhance the reproducibility: pre-registration, open methods, open data, collaboration, automation, reporting guidelines, and post-publication reviews. The current editorial discusses the generalisability and practicality of these strategies to systematic reviews and claims that systematic reviews have even a greater potential than other research designs to lead the movement toward the reproducibility of research. Moreover, I discuss the potential of reproducibility, on the other hand, to upgrade the systematic review from review to research. Furthermore, there are references to the successful and ongoing practices from collaborative efforts around the world to encourage the systematic reviewers, the journal editors and publishers, the organizations linked to evidence synthesis, and the funders and policy makers to facilitate this movement and to gain the public trust in research.

Cochrane Eyes and Vision: a perspective introducing Cochrane Corner in Eye

In 1972, Archie Cochrane wrote "It is surely a great criticism of our profession that we have not organised a critical summary, by specialty or subspecialty, adapted periodically, of all relevant randomised controlled trials". The Cochrane Collaboration arose in response to Archie Cochrane's challenge. Cochrane Eyes and Vision aims to prepare and promote access to systematic reviews of interventions for preventing or treating eye conditions and/or visual impairment, and helping people adjust to visual impairment or blindness. To identify all relevant randomised controlled trials, Cochrane Eyes and Vision has a team of information specialists who develop search strategies to identify studies for inclusion in Cochrane reviews. Since 1997 we have published 266 protocols, 193 new reviews and 158 updated reviews. The majority of these are reviews of intervention effectiveness; three reviews are diagnostic test accuracy reviews. Overall 18% of reviews contain no trials, highlighting a potential evidence gap. We provide training, education and guidance to systematic review authors and work with clinical and patient partners to prioritise and disseminate reviews. In addition, Cochrane Eyes and Vision US satellite carries out critical methodologic research addressing topics relevant to producing high-quality reviews. We are partnering with the journal Eye to publish commentaries on selected Cochrane systematic review findings. This partnership will allow us to make high-quality evidence available to ophthalmologists and other practitioners, researchers, policy makers and patients.

Features and functioning of Data Abstraction Assistant, a software application for data abstraction during systematic reviews

INTRODUCTION

During systematic reviews, data abstraction is labor- and time-intensive and error-prone. Existing data abstraction systems do not track specific locations and contexts of abstracted information. To address this limitation, we developed a software application, the Data Abstraction Assistant (DAA) and surveyed early users about their experience using DAA.

FEATURES OF DAA

We designed DAA to encompass three essential features: (1) a platform for indicating the source of abstracted information, (2) compatibility with a variety of data abstraction systems, and (3) user-friendliness.

HOW DAA FUNCTIONS

DAA (1) converts source documents from PDF to HTML format (to enable tracking of source of abstracted information), (2) transmits the HTML to the data abstraction system, and (3) displays the HTML in an area adjacent to the data abstraction form in the data abstraction system. The data abstractor can mark locations on the HTML that DAA associates with items on the data abstraction form.

EXPERIENCES OF EARLY USERS OF DAA

When we surveyed 52 early users of DAA, 83% reported that using DAA was either very or somewhat easy; 71% are very or somewhat likely to use DAA in the future; and 87% are very or somewhat likely to recommend that others use DAA in the future.

DISCUSSION

DAA, a user-friendly software for linking abstracted data with their exact source, is likely to be a very useful tool in the toolbox of systematic reviewers. DAA facilitates verification of abstracted data and provides an audit trail that is crucial for reproducible research.

Standards for design and measurement would make clinical research reproducible and usable

We find standards useful in everyday life and in science, although we do not always follow them. Adopting new standards can be expensive, so there may be a strong incentive to maintain the status quo rather than adopt new standards. The scientific community has many standards encompassing both doing clinical research and reporting it, including standards for design and measurement. Although existing research standards have improved both research and its reporting, we need to unify existing standards and to fill the gaps between steps throughout the research process. Existing gaps include implementation of standards and links between standards for study registration (to know about all studies undertaken), study protocols (to identify the preplanned study design and methods), data collection (to assess outcomes that are important and comparable across studies), dissemination of findings (to know the results of previous studies), data sharing (to make best use of existing data), and evidence synthesis (to draw appropriate conclusions from the body of evidence). The scientific community must work together to harmonize existing standards, to ensure that standards are kept up to date, to check that standards are followed, and to develop standards where they are still needed. A unified system of standards will make our work more reproducible.

Reproducible research practices are underused in systematic reviews of biomedical interventions

OBJECTIVES

To evaluate how often reproducible research practices, which allow others to recreate the findings of studies, given the original data, are used in systematic reviews (SRs) of biomedical research.

STUDY DESIGN AND SETTING

We evaluated a random sample of SRs indexed in MEDLINE during February 2014, which focused on a therapeutic intervention and reported at least one meta-analysis. Data on reproducible research practices in each SR were extracted using a 26-item form by one author, with a 20% random sample extracted in duplicate. We explored whether the use of reproducible research practices was associated with an SR being a Cochrane review, as well as with the reported use of the Preferred Reporting Items for Systematic Reviews and Meta-Analyses statement.

RESULTS

We evaluated 110 SRs of therapeutic interventions, 78 (71%) of which were non-Cochrane SRs. Across the SRs, there were 2,139 meta-analytic effects (including subgroup meta-analytic effects and sensitivity analyses), 1,551 (73%) of which were reported in sufficient detail to recreate them. Systematic reviewers reported the data needed to recreate all meta-analytic effects in 72 (65%) SRs only. This percentage was higher in Cochrane than in non-Cochrane SRs (30/32 [94%] vs. 42/78 [54%]; risk ratio 1.74, 95% confidence interval 1.39-2.18). Systematic reviewers who reported imputing, algebraically manipulating, or obtaining some data from the study author/sponsor infrequently stated which specific data were handled in this way. Only 33 (30%) SRs mentioned access to data sets and statistical code used to perform analyses.

CONCLUSION

Reproducible research practices are underused in SRs of biomedical interventions. Adoption of such practices facilitates identification of errors and allows the SR data to be reanalyzed.

Cefepime Efficacy and Safety in Children: A Systematic Review and Meta-analysis

BACKGROUND

Cefepime is a fourth-generation cephalosporin antibiotic used to treat a variety of infections. The US Food and Drug Administration approved its use in certain types of infections among pediatric patients, and yet there have been mixed data about its efficacy and safety in this population.

OBJECTIVE

The objective of this review is to compare efficacy and all-cause mortality of cefepime to other clinically indicated antibiotics among children.

METHODS

We conducted a systematic search of MEDLINE, EMBASE, CENTRAL, LILACS, and clinicaltrials.gov databases through February 8, 2016. We included randomized controlled trials comparing cefepime to other clinical antibiotics, placebo, or no treatment in children aged 0-19 years in the inpatient setting with clinical signs of infection. The primary outcome of interest was all-cause mortality. The secondary outcomes were success rate, treatment failure, and incidence of adverse events. Study quality was assessed using the Cochrane Risk of Bias Assessment Tool.

RESULTS

Seventeen studies met the inclusion criteria. There was a total of 1,285 participants included, 624 participants in the cefepime arm and 661 in the comparison arm. A random effects meta-analysis for all-cause mortality showed no difference in rates of mortality between cefepime and comparator antibiotics with a mortality risk ratio of 0.88 (95% CI: 0.71-1.08). For the secondary outcomes of success rate and treatment failure, a random effects model meta-analysis conducted of the studies showed no difference in rate between cefepime and comparator antibiotics with an overall risk ratio of 0.98 (95% CI: 0.92-1.05) and 1.04 (95% CI: 0.91-1.19), respectively. Adverse events were not statistically assessed given widespread heterogeneity. Overall, the studies had unclear risk of bias and were limited by high heterogeneity and methodological flaws.

CONCLUSION

The efficacy and safety of cefepime in pediatric patients remain unclear despite the inclusion of newer trials since the last index systematic review conducted a decade ago.

Evaluations of the uptake and impact of the Preferred Reporting Items for Systematic reviews and Meta-Analyses (PRISMA) Statement and extensions: a scoping review

BACKGROUND

The PRISMA Statement is a reporting guideline designed to improve transparency of systematic reviews (SRs) and meta-analyses. Seven extensions to the PRISMA Statement have been published to address the reporting of different types or aspects of SRs, and another eight are in development. We performed a scoping review to map the research that has been conducted to evaluate the uptake and impact of the PRISMA Statement and extensions. We also synthesised studies evaluating how well SRs published after the PRISMA Statement was disseminated adhere to its recommendations.

METHODS

We searched for meta-research studies indexed in MEDLINE® from inception to 31 July 2017, which investigated some component of the PRISMA Statement or extensions (e.g. SR adherence to PRISMA, journal endorsement of PRISMA). One author screened all records and classified the types of evidence available in the studies. We pooled data on SR adherence to individual PRISMA items across all SRs in the included studies and across SRs published after 2009 (the year PRISMA was disseminated).

RESULTS

We included 100 meta-research studies. The most common type of evidence available was data on SR adherence to the PRISMA Statement, which has been evaluated in 57 studies that have assessed 6487 SRs. The pooled results of these studies suggest that reporting of many items in the PRISMA Statement is suboptimal, even in the 2382 SRs published after 2009 (where nine items were adhered to by fewer than 67% of SRs). Few meta-research studies have evaluated the adherence of SRs to the PRISMA extensions or strategies to increase adherence to the PRISMA Statement and extensions.

CONCLUSIONS

Many studies have evaluated how well SRs adhere to the PRISMA Statement, and the pooled result of these suggest that reporting of many items is suboptimal. An update of the PRISMA Statement, along with a toolkit of strategies to help journals endorse and implement the updated guideline, may improve the transparency of SRs.

Inconsistent selection of outcomes and measurement devices found in shoulder arthroplasty research: An analysis of studies on ClinicalTrials.gov

Introduction

Recent evidence suggests a lack of standardization of shoulder arthroplasty outcomes. This issue is a limiting factor in systematic reviews. Core outcome set (COS) methodology could address this problem by delineating a minimum set of outcomes for measurement in all shoulder arthroplasty trials.

Methods

A ClinicalTrials.gov search yielded 114 results. Eligible trials were coded on the following characteristics: study status, study type, arthroplasty type, sample size, measured outcomes, outcome measurement device, specific metric of measurement, method of aggregation, outcome classification, and adverse events.

Results

Sixty-six trials underwent data abstraction and data synthesis. Following abstraction, 383 shoulder arthroplasty outcomes were organized into 11 outcome domains. The most commonly reported outcomes were shoulder outcome score (n = 58), pain (n = 33), and quality of life (n = 15). The most common measurement devices were the Constant-Murley Shoulder Outcome Score (n = 38) and American Shoulder and Elbow Surgeons Shoulder Score (n = 33). Temporal patterns of outcome use was also found.

Conclusion

Our study suggests the need for greater standardization of outcomes and instruments. The lack of consistency across trials indicates that developing a core outcome set for shoulder arthroplasty trials would be worthwhile. Such standardization would allow for more effective comparison across studies in systematic reviews, while at the same time consider important outcomes that may be underrepresented otherwise. This review of outcomes provides an evidence-based foundation for the development of a COS for shoulder arthroplasty.

Study-based registers of randomized controlled trials: Starting a systematic review with data extraction or meta-analysis

Introduction: Despite years of use of study-based registers for storing reports of randomized controlled trials (RCTs), the methodology used in developing such registers/databases has not been documented. Such registers are integral to the process of scientific reviewing. We document and discuss methodological aspects of the development and use of study-based registers. Although the content is focused on the study-based register of randomized/controlled clinical trials, this work applies to developers of databases of all sorts of studies related to the human, animals, cells, genes, and molecules. Methods: We describe necessity, rationale, and steps for the development, utilization and maintenance of study-based registers as well as the challenges and gains for the organizations supporting systematic reviews of the published and unpublished literature. Conclusion: The ultimate goal of having a study-based register is to facilitate efficient production of systematic reviews providing rapid, yet accurate, evidence for the decision-makers. We argue that moving towards study-based registers is an inevitable welcome direction and that infrastructures are ready for such movement.

The quality of systematic reviews about interventions for refractive error can be improved: a review of systematic reviews

BACKGROUND

Systematic reviews should inform American Academy of Ophthalmology (AAO) Preferred Practice Pattern® (PPP) guidelines. The quality of systematic reviews related to the forthcoming Preferred Practice Pattern® guideline (PPP) Refractive Errors & Refractive Surgery is unknown. We sought to identify reliable systematic reviews to assist the AAO Refractive Errors & Refractive Surgery PPP.

METHODS

Systematic reviews were eligible if they evaluated the effectiveness or safety of interventions included in the 2012 PPP Refractive Errors & Refractive Surgery. To identify potentially eligible systematic reviews, we searched the Cochrane Eyes and Vision United States Satellite database of systematic reviews. Two authors identified eligible reviews and abstracted information about the characteristics and quality of the reviews independently using the Systematic Review Data Repository. We classified systematic reviews as "reliable" when they (1) defined criteria for the selection of studies, (2) conducted comprehensive literature searches for eligible studies, (3) assessed the methodological quality (risk of bias) of the included studies, (4) used appropriate methods for meta-analyses (which we assessed only when meta-analyses were reported), (5) presented conclusions that were supported by the evidence provided in the review.

RESULTS

We identified 124 systematic reviews related to refractive error; 39 met our eligibility criteria, of which we classified 11 to be reliable. Systematic reviews classified as unreliable did not define the criteria for selecting studies (5; 13%), did not assess methodological rigor (10; 26%), did not conduct comprehensive searches (17; 44%), or used inappropriate quantitative methods (3; 8%). The 11 reliable reviews were published between 2002 and 2016. They included 0 to 23 studies (median = 9) and analyzed 0 to 4696 participants (median = 666). Seven reliable reviews (64%) assessed surgical interventions.

CONCLUSIONS

Most systematic reviews of interventions for refractive error are low methodological quality. Following widely accepted guidance, such as Cochrane or Institute of Medicine standards for conducting systematic reviews, would contribute to improved patient care and inform future research.

Comparison of Clinical Trial and Systematic Review Outcomes for the 4 Most Prevalent Eye Diseases

Importance

Suboptimal overlap in outcomes reported in clinical trials and systematic reviews compromises efforts to compare and summarize results across these studies.

Objectives

To examine the most frequent outcomes used in trials and reviews of the 4 most prevalent eye diseases (age-related macular degeneration [AMD], cataract, diabetic retinopathy [DR], and glaucoma) and the overlap between outcomes in the reviews and the trials included in the reviews.

Design, Setting, and Participants

This cross-sectional study examined all Cochrane reviews that addressed AMD, cataract, DR, and glaucoma; were published as of July 20, 2016; and included at least 1 trial and the trials included in the reviews. For each disease, a pair of clinical experts independently classified all outcomes and resolved discrepancies. Outcomes (outcome domains) were then compared separately for each disease.

Main Outcomes and Measures

Proportion of review outcomes also reported in trials and vice versa.

Results

This study included 56 reviews that comprised 414 trials. Although the median number of outcomes per trial and per review was the same (n = 5) for each disease, the trials included a greater number of outcomes overall than did the reviews, ranging from 2.9 times greater (89 vs 30 outcomes for glaucoma) to 4.9 times greater (107 vs 22 outcomes for AMD). Most review outcomes, ranging from 14 of 19 outcomes (73.7%) (for DR) to 27 of 29 outcomes (93.1%) (for cataract), were also reported in the trials. For trial outcomes, however, the proportion also named in reviews was low, ranging from 19 of 107 outcomes (17.8%) (for AMD) to 24 of 89 outcomes (27.0%) (for glaucoma). Only 1 outcome (visual acuity) was consistently reported in greater than half the trials and greater than half the reviews.

Conclusions and Relevance

Although most review outcomes were reported in the trials, most trial outcomes were not reported in the reviews. The current analysis focused on outcome domains, which might underestimate the problem of inconsistent outcomes. Other important elements of an outcome (ie, specific measurement, specific metric, method of aggregation, and time points) might have differed even though the domains overlapped. Inconsistency in trial outcomes may impede research synthesis and indicates the need for disease-specific core outcome sets in ophthalmology.

Traditional and complementary medicine used with curative intent in childhood cancer: A systematic review

Traditional and complementary medicine (T&CM) strategies are widely utilized in pediatric oncology, with many families reporting T&CM use with the intention to cure cancer. Study of T&CM agents presents many challenges, as a heterogeneous group of agents and techniques are used for a variety of different purpose in many different oncologic conditions. We present a systematic review of the literature examining published reports in which T&CM agents are used with an intention of cure. Twenty-two reports were identified, with most reports being of poor quality. Novel paradigms are likely needed to further investigate T&CM agents.

Choice of initial antiretroviral drugs and treatment outcomes among HIV-infected patients in sub-Saharan Africa: systematic review and meta-analysis of observational studies

BACKGROUND

The effectiveness of antiretroviral therapy (ART) depends on the choice of regimens during initiation. Most evidences from developed countries indicated that there is difference between efavirenz (EFV) and nevirapine (NVP). However, the evidences are limited in resource poor countries particularly in Africa. Thus, this systematic review and meta-analysis was carried out to summarize reported long-term treatment outcomes among people on first line therapy in sub-Saharan Africa.

METHODS

Observational studies that reported odds ratio, relative risk, hazard ratio, or standardized incidence ratio to compare risk of treatment failure among HIV/AIDS patients who initiated ART with EFV versus NVP were systematically searched. Searches were conducted using the MEDLINE database within PubMed, Google Scholar, HINARI, and Research Gates between 2007 and 2016. Information was extracted using standardized form. Pooled risk ratios (RR) and 95% confidence intervals (CI) were calculated using random-effect, generic inverse variance method.

RESULT

A total of 6394 articles were identified, of which, 29 were eligible for review and abstraction in sub-Saharan Africa. Seventeen articles were used for the meta-analysis. Of a total of 121,092 independent study participants, 76,719 (63.36%) were females. Of these, 40,480 (33.43%) initiated with NVP containing regimen. Two studies did not report the median CD4 cell counts at initiation. Patients who have low CD4 cell counts initiated with EFV containing regimen. The pooled effect size indicated that treatment failure was reduced by 15%, 0.85 (95%CI: 0.75-0.98), and non-nucleoside reverse transcriptase inhibitor (NNRTI) switch was reduced by 43%, 0.57 (95%CI: 0.37-0.89).

CONCLUSION

The risk of treatment failure and NNRTI switch were lower in patients who initiated with EFV than NVP-containing regimen. The review suggests that initiation of patients with EFV-containing regimen will reduce treatment failure and NNRTI switch.

Methodological quality of systematic reviews referenced in clinical practice guidelines for the treatment of opioid use disorder

INTRODUCTION

With efforts to combat opioid use disorder, there is an increased interest in clinical practice guidelines (CPGs) for opioid use disorder treatments. No literature exists examining the quality of systematic reviews used in opioid use disorder CPGs. This study aims to describe the methodological quality and reporting clarity of systematic reviews (SRs) used to create CPGs for opioid use disorder.

METHODS

From June to July 2016 guideline clearinghouses and medical literature databases were searched for relevant CPGs used in the treatment of opioid use disorder. Included CPGs must have been recognized by a national organization. SRs from the reference section of each CPG was scored by using AMSTAR (a measurement tool to assess the methodological quality of systematic reviews) tool and PRISMA (preferred reporting items for systematic reviews and meta-analyses) checklist.

RESULTS

Seventeen CPGs from 2006-2016 were included in the review. From these, 57 unique SRs were extracted. SRS comprised 0.28% to 17.92% of all references found in the CPGs. All SRs obtained moderate or high methodological quality score on the AMSTAR tool. All reviews met at least 70% of PRISMA criteria. In PRISMA, underperforming areas included accurate title labeling, protocol registration, and risk of bias. Underperforming areas in AMSTAR included conflicts of interest, funding, and publication bias. A positive correlation was found between AMSTAR and PRISMA scores (r = .79).

CONCLUSION

Although the SRs in the CPGs were of good quality, there are still areas for improvement. Systematic reviewers should consult PRISMA and AMSTAR when conducting and reporting reviews. It is important for CPG developers to consider methodological quality as a factor when developing CPG recommendations, recognizing that the quality of systematic reviews underpinning guidelines does not necessarily correspond to the quality of the guideline itself.

Selective outcome reporting in obesity clinical trials: a cross-sectional review

Selective outcome reporting is a form of bias resulting from discrepancies between outcomes presented in a trial's registration and the published report. We investigate this selective bias in obesity clinical trials. A PubMed search was conducted to identify randomized controlled trials (RCTs) published in four obesity journals from 2013 to 2015. Primary, secondary and tertiary outcomes were recorded for each trial and compared to pre-specified outcomes in each trial's registration. Of the 392 identified articles, 142 were included in the final analysis; 22 (15%) RCTs demonstrated major outcome discrepancies between registration and publication: No primary outcomes were demoted to a secondary or tertiary outcome; 14 (36.84%) primary outcomes were omitted; 14 (36.84%) primary outcomes were added: 5 (13.16%) secondary outcomes were upgraded to primary outcomes; and timing of assessment for a primary outcome changed 5 (13.16%) times. Out of the 63 prospectively registered studies, 53 had no discrepancies. A total of 76 of the studies (29.80%) were unregistered or did not have an associated registration number. Our results suggest that selective outcome reporting may be a concern in obesity clinical trials. As selective outcome reporting may distort clinical findings and limit outcomes in systematic reviews, we encourage trialists and journal editors to work towards solutions to mitigate this issue.

Systematic review: Outcome reporting bias is a problem in high impact factor neurology journals

BACKGROUND

Selective outcome reporting is a significant methodological concern. Comparisons between the outcomes reported in clinical trial registrations and those later published allow investigators to understand the extent of selection bias among trialists. We examined the possibility of selective outcome reporting in randomized controlled trials (RCTs) published in neurology journals.

METHODS

We searched PubMed for randomized controlled trials from Jan 1, 2010 -Dec 31, 2015 published in the top 3 impact factor neurology journals. These articles were screened according to specific inclusion criteria. Each author individually extracted data from trials following a standardized protocol. A second author verified each extracted element and discrepancies were resolved. Consistency between registered and published outcomes was evaluated and correlations between discrepancies and funding, journal, and temporal trends were examined.

RESULTS

180 trials were included for analysis. 10 (6%) primary outcomes were demoted, 38 (21%) primary outcomes were omitted from the publication, and 61 (34%) unregistered primary outcomes were added to the published report. There were 18 (10%) cases of secondary outcomes being upgraded to primary outcomes in the publication, and there were 53 (29%) changes in timing of assessment. Of 82 (46%) major discrepancies with reported p-values, 54 (66.0%) favored publication of statistically significant results.

CONCLUSION

Across trials, we found 180 major discrepancies. 66% of major discrepancies with a reported p-value (n = 82) favored statistically significant results. These results suggest a need within neurology to provide more consistent and timely registration of outcomes.

Evidence of selective reporting bias in hematology journals: A systematic review

Introduction

Selective reporting bias occurs when chance or selective outcome reporting rather than the intervention contributes to group differences. The prevailing concern about selective reporting bias is the possibility of results being modified towards specific conclusions. In this study, we evaluate randomized controlled trials (RCTs) published in hematology journals, a group in which selective outcome reporting has not yet been explored.

Methods

Our primary goal was to examine discrepancies between the reported primary and secondary outcomes in registered and published RCTs concerning hematological malignancies reported in hematology journals with a high impact factor. The secondary goals were to address whether outcome reporting discrepancies favored statistically significant outcomes, whether a pattern existed between the funding source and likelihood of outcome reporting bias, and whether temporal trends were present in outcome reporting bias. For trials with major outcome discrepancies, we contacted trialists to determine reasons for these discrepancies. Trials published between January 1, 2010 and December 31, 2015 in Blood; British Journal of Haematology; American Journal of Hematology; Leukemia; and Haematologica were included.

Results

Of 499 RCTs screened, 109 RCTs were included. Our analysis revealed 118 major discrepancies and 629 total discrepancies. Among the 118 discrepancies, 30 (25.4%) primary outcomes were demoted, 47 (39.8%) primary outcomes were omitted, and 30 (25.4%) primary outcomes were added. Three (2.5%) secondary outcomes were upgraded to a primary outcome. The timing of assessment for a primary outcome changed eight (6.8%) times. Thirty-one major discrepancies were published with a P-value and twenty-five (80.6%) favored statistical significance. A majority of authors whom we contacted cited a pre-planned subgroup analysis as a reason for outcome changes.

Conclusion

Our results suggest that outcome changes occur frequently in hematology trials. Because RCTs ultimately underpin clinical judgment and guide policy implementation, selective reporting could pose a threat to medical decision making.

Multiple outcomes and analyses in clinical trials create challenges for interpretation and research synthesis

OBJECTIVE

To identify variations in outcomes and results across reports of randomized clinical trials (RCTs).

STUDY DESIGN AND SETTING

Eligible RCTs examined gabapentin for neuropathic pain and quetiapine for bipolar depression, reported in public (e.g., journal articles) and nonpublic (e.g., clinical study reports) sources by 2015. We prespecified outcome domains. From each source, we collected "outcomes" (i.e., domain, measure, metric, method of aggregation, and time point); "treatment effect" (i.e., outcome plus the methods of analysis [e.g., how missing data were handled]); and results (i.e., numerical contrasts of treatment and comparison groups). We assessed whether results included sufficient information for meta-analysis.

RESULTS

We found 21 gabapentin (68 public, 6 nonpublic reports) and seven quetiapine RCTs (46 public, 4 nonpublic reports). For four (gabapentin) and seven (quetiapine) prespecified outcome domains, RCTs reported 214 and 81 outcomes by varying four elements. RCTs assessed 605 and 188 treatment effects by varying the analysis of those outcomes. RCTs reported 1,230 and 661 meta-analyzable results, 305 (25%) and 109 (16%) in public reports.

CONCLUSION

RCTs included hundreds of outcomes and results; a small proportion were in public reports. Trialists and meta-analysts may cherry-pick what they report from multiple sources of RCT information.

How to Conduct and Interpret Systematic Reviews and Meta-Analyses

Systematic reviews with or without meta-analyses serve a key purpose in critically and objectively synthesizing all available evidence regarding a focused clinical question and can inform clinical practice and clinical guidelines. Performing a rigorous systematic review is multi-step process, which includes (a) identifying a well-defined focused clinically relevant question, (b) developing a detailed review protocol with strict inclusion and exclusion criteria, (c) systematic literature search of multiple databases and unpublished data, in consultation with a medical librarian, (d) meticulous study identification and (e) systematic data abstraction, by at least two sets of investigators independently, (f) risk of bias assessment, and (g) thoughtful quantitative synthesis through meta-analysis where relevant. Besides informing guidelines, credible systematic reviews and quality of evidence assessment can help identify key knowledge gaps for future studies.