Contacting of authors modified crucial outcomes of systematic reviews but was poorly reported, not systematic, and produced conflicting results

Reproducibility of Exercise Interventions in Randomized Controlled Trials for the Treatment of Rotator Cuff-Related Shoulder Pain: A Systematic Review

OBJECTIVE

To determine the reproducibility of exercise therapy interventions in randomized controlled trials for rotator cuff-related shoulder pain (RCRSP).

DATA SOURCES

Data sources included Medline, Cumulative Index to Nursing and Allied Health Literature, Cochrane Central Register of Controlled Trials, and SPORTDiscus from studies published from database inception to April 23, 2022.

STUDY SELECTION

Randomized controlled trials studying the use of exercise therapy for RCRSP.

DATA EXTRACTION

Two reviewers extracted exercise reporting details from all studies using the Template for Intervention Description and Replication (TIDieR) and the modified Consensus on Exercise Reporting Template (CERT). The same 2 reviewers assessed risk of bias of all studies using Cochrane Risk of Bias Tool version 2.0.

DATA SYNTHESIS

For 104 studies meeting inclusion criteria, the average number of items reported on the TIDieR was 5.27 (SD 2.50, range 1-12 out of 12) and 5.09 (SD 4.01, range 0-16 out of 16) on the CERT. Improved reporting over time was seen on both the TIDieR and CERT dating back to 1993 and through April 23, 2022. When comparing groups of studies published before and after the TIDieR (2014) and CERT (2016) were established, a statistically significant increase in median scores was noted on the TIDieR (P=.02) but not the CERT (P=.31). Quality of exercise therapy reporting was highest in studies with "low risk" of bias, and lowest in studies with "high risk" of bias on the RoB-2.

CONCLUSION

Overall exercise reporting in trials for RCRSP is incomplete despite the development of the TIDieR and CERT checklists. This has implications for translating evidence into practice.

Systematic reviewers used various approaches to data extraction and expressed several research needs: a survey

OBJECTIVE

Data extraction is a prerequisite for analyzing, summarizing, and interpreting evidence in systematic reviews. Yet guidance is limited, and little is known about current approaches. We surveyed systematic reviewers on their current approaches to data extraction, opinions on methods, and research needs.

STUDY DESIGN AND SETTING

We developed a 29-question online survey and distributed it through relevant organizations, social media, and personal networks in 2022. Closed questions were evaluated using descriptive statistics, and open questions were analyzed using content analysis.

RESULTS

162 reviewers participated. Use of adapted (65%) or newly developed extraction forms (62%) was common. Generic forms were rarely used (14%). Spreadsheet software was the most popular extraction tool (83%). Piloting was reported by 74% of respondents and included a variety of approaches. Independent and duplicate extraction was considered the most appropriate approach to data collection (64%). About half of respondents agreed that blank forms and/or raw data should be published. Suggested research gaps were the effects of different methods on error rates (60%) and the use of data extraction support tools (46%).

CONCLUSION

Systematic reviewers used varying approaches to pilot data extraction. Methods to reduce errors and use of support tools such as (semi-)automation tools are top research gaps.

Effectiveness, quality and implementation of pain, sedation, delirium, and iatrogenic withdrawal syndrome algorithms in pediatric intensive care: a systematic review and meta-analysis

Background

Pain, sedation, delirium, and iatrogenic withdrawal syndrome are conditions that often coexist, algorithms can be used to assist healthcare professionals in decision making. However, a comprehensive review is lacking. This systematic review aimed to assess the effectiveness, quality, and implementation of algorithms for the management of pain, sedation, delirium, and iatrogenic withdrawal syndrome in all pediatric intensive care settings.

Methods

A literature search was conducted on November 29, 2022, in PubMed, Embase, CINAHL and Cochrane Library, ProQuest Dissertations & Theses, and Google Scholar to identify algorithms implemented in pediatric intensive care and published since 2005. Three reviewers independently screened the records for inclusion, verified and extracted data. Included studies were assessed for risk of bias using the JBI checklists, and algorithm quality was assessed using the PROFILE tool (higher % = higher quality). Meta-analyses were performed to compare algorithms to usual care on various outcomes (length of stay, duration and cumulative dose of analgesics and sedatives, length of mechanical ventilation, and incidence of withdrawal).

Results

From 6,779 records, 32 studies, including 28 algorithms, were included. The majority of algorithms (68%) focused on sedation in combination with other conditions. Risk of bias was low in 28 studies. The average overall quality score of the algorithm was 54%, with 11 (39%) scoring as high quality. Four algorithms used clinical practice guidelines during development. The use of algorithms was found to be effective in reducing length of stay (intensive care and hospital), length of mechanical ventilation, duration of analgesic and sedative medications, cumulative dose of analgesics and sedatives, and incidence of withdrawal. Implementation strategies included education and distribution of materials (95%). Supportive determinants of algorithm implementation included leadership support and buy-in, staff training, and integration into electronic health records. The fidelity to algorithm varied from 8.2% to 100%.

Conclusions

The review suggests that algorithm-based management of pain, sedation and withdrawal is more effective than usual care in pediatric intensive care settings. There is a need for more rigorous use of evidence in the development of algorithms and the provision of details on the implementation process.

Systematic Review Registration

https://www.crd.york.ac.uk/prospero/display_record.php?ID=CRD42021276053, PROSPERO [CRD42021276053].

Prediction Models for Future High-Need High-Cost Healthcare Use: a Systematic Review

BACKGROUND

In an effort to improve both quality of care and cost-effectiveness, various care-management programmes have been developed for high-need high-cost (HNHC) patients. Early identification of patients at risk of becoming HNHC (i.e. case finding) is crucial to a programme's success. We aim to systematically identify prediction models predicting future HNHC healthcare use in adults, to describe their predictive performance and to assess their applicability.

METHODS

Ovid MEDLINE® All, EMBASE, CINAHL, Web of Science and Google Scholar were systematically searched from inception through January 31, 2021. Risk of bias and methodological quality assessment was performed through the Prediction model Risk Of Bias Assessment Tool (PROBAST).

RESULTS

Of 5890 studies, 60 studies met inclusion criteria. Within these studies, 313 unique models were presented using a median development cohort size of 20,248 patients (IQR 5601-174,242). Predictors were derived from a combination of data sources, most often claims data (n = 37; 62%) and patient survey data (n = 29; 48%). Most studies (n = 36; 60%) estimated patients' risk to become part of some top percentage of the cost distribution (top-1-20%) within a mean time horizon of 16 months (range 12-60). Five studies (8%) predicted HNHC persistence over multiple years. Model validation was performed in 45 studies (76%). Model performance in terms of both calibration and discrimination was reported in 14 studies (23%). Overall risk of bias was rated as 'high' in 40 studies (67%), mostly due to a 'high' risk of bias in the subdomain 'Analysis' (n = 37; 62%).

DISCUSSION

This is the first systematic review (PROSPERO CRD42020164734) of non-proprietary prognostic models predicting HNHC healthcare use. Meta-analysis was not possible due to heterogeneity. Most identified models estimated a patient's risk to incur high healthcare expenditure during the subsequent year. However, case-finding strategies for HNHC care-management programmes are best informed by a model predicting HNHC persistence. Therefore, future studies should not only focus on validating and extending existing models, but also concentrate on clinical usefulness.

A new method for testing reproducibility in systematic reviews was developed, but needs more testing

BACKGROUND

To develop and test an approach to test reproducibility of SRs.

METHODS

Case study. We have developed an approach to test reproducibility retrospectively while focusing on the whole conduct of an SR instead of single steps of it. We replicated the literature searches and drew a 25% random sample followed by study selection, data extraction, and risk of bias (ROB) assessments performed by two reviewers independently. These results were compared narratively with the original review.

RESULTS

We were not able to fully reproduce the original search resulting in minor differences in the number of citations retrieved. The biggest disagreements were found in study selection. The most difficult section to be reproduced was the RoB assessment due to the lack of reporting clear criteria to support the judgement of RoB ratings, although agreement was still found to be satisfactory.

CONCLUSION

Our approach as well as other approaches needs to undergo testing and comparison in the future as the area of testing for reproducibility of SRs is still in its infancy.

Managing overlap of primary study results across systematic reviews: practical considerations for authors of overviews of reviews

BACKGROUND

Overviews often identify and synthesise a large number of systematic reviews on the same topic, which is likely to lead to overlap (i.e. duplication) in primary studies across the reviews. Using a primary study result multiple times in the same analysis overstates its sample size and number of events, falsely leading to greater precision in the analysis. This paper aims to: (a) describe types of overlapping data that arise from the same primary studies reported across multiple reviews, (b) describe methods to identify and explain overlap of primary study data, and (c) present six case studies illustrating different approaches to manage overlap.

METHODS

We first updated the search in PubMed for methods from the MOoR framework relating to overlap of primary studies. One author screened the studies titles and abstracts, and any full-text articles retrieved, extracted methods data relating to overlap of primary studies and mapped it to the overlap methods from the MOoR framework. We also describe six case studies as examples of overviews that use specific overlap methods across the steps in the conduct of an overview. For each case study, we discuss potential methodological implications in terms of limitations, efficiency, usability, and resource use.

RESULTS

Nine methods studies were found and mapped to the methods identified by the MOoR framework to address overlap. Overlap methods were mapped across four steps in the conduct of an overview - the eligibility criteria step, the data extraction step, the assessment of risk of bias step, and the synthesis step. Our overview case studies used multiple methods to reduce overlap at different steps in the conduct of an overview.

CONCLUSIONS

Our study underlines that there is currently no standard methodological approach to deal with overlap in primary studies across reviews. The level of complexity when dealing with overlap can vary depending on the yield, trends and patterns of the included literature and the scope of the overview question. Choosing a method might be dependent on the number of included reviews and their primary studies. Gaps in evaluation of methods to address overlap were found and further investigation in this area is needed.

Frequency of receiving requested data for a systematic review and associated factors: A cross-sectional study

This study aimed to assess the frequency of receiving requested data for a systematic review and associated factors. We contacted the authors of studies in need of additional data via e-mail. The primary outcome was the success in receiving the requested data according to the time until receipt. We estimated the hazard ratio (HR) and 95% confidence interval (CI) for success in each variable compared to the reference category, with weighted Cox proportional hazards models using Stata (version 14.2). Out of 164 studies contacted, 110 replied (67.1%), and 51 sent requested data (31.1%). Median time to receive a response or withdraw contact was 36.0 days (interquartile range: 17.5, 142.5). Higher success ratio was observed in studies published as scientific papers (HR = 3.01, 95% CI = [1.18, 7.70]), in more than one publication (HR = 2.00, 95% CI = [1.14, 3.51]), and contacted by personal e-mail (HR = 2.85, 95% CI = [1.34, 6.07]). Three or more contact attempts led to lower success ratio (HR = 0.19, 95% CI = [0.11, 0.35]) than one or two. Requesting data for a systematic review was time-consuming and effective in three out of ten studies. Fewer contacts were more successful than insisting.

Author queries via email text elicited high response and took less reviewer time than data forms - a randomised study within a review

OBJECTIVE

To compare two strategies for requesting additional information for systematic reviews (SR) from study authors.

STUDY DESIGN AND SETTING

Randomised study within a SR of hospital volume-outcome relationships in total knee arthroplasty. We sent personalized email requests for additional information to study authors as either email text ("Email" group) or attachment with self-developed, personalised data request forms ("Attachment" group). The primary outcome was the response rate, the secondary outcomes were the data completeness rate and the reviewer time invested in author contact.

RESULTS

Of 57 study authors, 29 were randomised to the Email group and 28 to the Attachment group. The response rate was 93% for Email and 75% for Attachment (odds ratio 4.5, 95% confidence interval [0.9-24.0]). Complete data were provided by 55% (Email) vs. 36% (Attachment) of authors (odds ratio 2.2 [0.8-6.4]). The mean reviewer time was shorter in the Email (mean ± standard deviation of 20.2±14.4 minutes/author) than the Attachment group (31.8±14.4 minutes/author) with a mean difference of 11.6 [4.1-19.1] minutes/author.

CONCLUSION

Personalised email requests elicited high response but only moderate data completeness rates regardless of the method (email text or attachment). Email requests as text took less reviewer time than creating attachments.