Rapid evidence synthesis approach for limits on the search date: How rapid could it be?

Accelerating evidence synthesis for safety assessment through ClinicalTrials.gov platform: a feasibility study

BACKGROUND

Standard systematic review can be labor-intensive and time-consuming meaning that it can be difficult to provide timely evidence when there is an urgent public health emergency such as a pandemic. The ClinicalTrials.gov provides a promising way to accelerate evidence production.

METHODS

We conducted a search on PubMed to gather systematic reviews containing a minimum of 5 studies focused on safety aspects derived from randomized controlled trials (RCTs) of pharmacological interventions, aiming to establish a real-world dataset. The registration information of each trial from eligible reviews was further collected and verified. The meta-analytic data were then re-analyzed by using 1) the full meta-analytic data with all trials and 2) emulated rapid data with trials that had been registered and posted results on ClinicalTrials.gov, under the same synthesis methods. The effect estimates of the full meta-analysis and rapid meta-analysis were then compared.

RESULTS

The real-world dataset comprises 558 meta-analyses. Among them, 56 (10.0%) meta-analyses included RCTs that were not registered in ClinicalTrials.gov. For the remaining 502 meta-analyses, the median percentage of RCTs registered within each meta-analysis is 70.1% (interquartile range: 33.3% to 88.9%). Under a 20% bias threshold, rapid meta-analyses conducted through ClinicalTrials.gov achieved accurate point estimates ranging from 77.4% (using the MH model) to 83.1% (using the GLMM model); 91.0% to 95.3% of these analyses accurately predicted the direction of effects.

CONCLUSIONS

Utilizing the ClinicalTrials.gov platform for safety assessment with a minimum of 5 RCTs holds significant potential for accelerating evidence synthesis to support urgent decision-making.

Rapid review: A review of methods and recommendations based on current evidence

Rapid review (RR) could accelerate the traditional systematic review (SR) process by simplifying or omitting steps using various shortcuts. With the increasing popularity of RR, numerous shortcuts had emerged, but there was no consensus on how to choose the most appropriate ones. This study conducted a literature search in PubMed from inception to December 21, 2023, using terms such as "rapid review" "rapid assessment" "rapid systematic review" and "rapid evaluation". We also scanned the reference lists and performed citation tracking of included impact studies to obtain more included studies. We conducted a narrative synthesis of all RR approaches, shortcuts and studies assessing their effectiveness at each stage of RRs. Based on the current evidence, we provided recommendations on utilizing certain shortcuts in RRs. Ultimately, we identified 185 studies focusing on summarizing RR approaches and shortcuts, or evaluating their impact. There was relatively sufficient evidence to support the use of the following shortcuts in RRs: limiting studies to those published in English-language; conducting abbreviated database searches (e.g., only searching PubMed/MEDLINE, Embase, and CENTRAL); omitting retrieval of grey literature; restricting the search timeframe to the recent 20 years for medical intervention and the recent 15 years for reviewing diagnostic test accuracy; conducting a single screening by an experienced screener. To some extent, the above shortcuts were also applicable to SRs. This study provided a reference for future RR researchers in selecting shortcuts, and it also presented a potential research topic for methodologists.

Assessing the Competence of Nursing Students in Clinical Practice: The Clinical Preceptors' Perspective

Nursing students' integration of theoretical knowledge and practical abilities is facilitated by their practice of nursing skills in a clinical environment. A key role of preceptors is to assess the learning goals that nursing students must meet while participating in clinical practice. Consequently, the purpose of this study was to explore the current evidence in relation to competency assessment and assessment approaches, and the willingness of preceptors for assessing nursing students' competency in a clinical setting. The scoping review used the five-stage methodological framework that was developed by Arksey and O'Malley, as well as the Preferred Reporting Items for Systematic Reviews and Meta-Analyses Extension for Scoping Reviews. Relevant studies were searched by applying a comprehensive literature search strategy up to April 2024 across the following databases: CINAHL, OVID MEDLINE, EMBASE, and PUBMED. A total of 11,297 studies published between 2000 and April 2024 were revealed, and 38 were eligible for inclusion, which the research team categorised into three main themes: definitions of competence, tools for assessing competence and preceptors' and mentors' viewpoints in relation to the assessment of nursing students' competence. This review established that there are a multitude of quantitative instruments available to assess clinical competence; however, a lack of consistency among assessment instruments and approaches between countries and higher education institutions is prevalent. Existing research evidence suggests that the preceptors carried out the assessment process clinically and they found difficulties in documenting assessment. The assessing of nursing students' competency and the complexity of assessment is a concern for educators and mentors worldwide. The main concern centers around issues such as the interpretation of competence and complex measurement tools.

What factors affect early mobilisation following hip fracture surgery: a scoping review

OBJECTIVES

Identify and evaluate factors affecting early mobilisation on the day following hip fracture surgery.

DESIGN

Mixed methods, scoping review.

DATA SOURCES

MEDLINE, AMED, CINAHL, APA PsycINFO, APA PsycArticles, ISRCTN, Clinical Trials registry and grey literature accessed in November 2022 with publication dates between 2001 and November 2022.

ELIGIBILITY CRITERIA

English language publications that:1. Include patient populations who sustain a fragility hip fracture managed surgically2. Include patient populations who are mobilised out of bed on the day following their hip fracture surgery3. Report factors which influence the ability to undergo early mobility postsurgery DATA EXTRACTION AND SYNTHESIS: One reviewer screened all titles and abstracts for inclusion. Two reviewers performed data extraction and quality assessments using the relevant Critical Appraisal Skills Programme tools and the Mixed Methods Appraisal Tool.

RESULTS

3337 papers were identified, of which 23 studies were eligible for review, representing 210 811 patients. The heterogeneity in the types of study included, the definition of early mobilisation and the outcome measures used precluded meta-analysis. 13 factors were identified as having an effect on whether people were mobilised on day 1 post-hip fracture surgery, grouped into 5 principal themes: (1) healthcare setting or worker-related factors, (2) patient psychological factors, (3) acute patient health factors, (4) non-acute patient health factors and (5) surgical factors.

CONCLUSIONS

There was a paucity of robust research investigating day 1 mobilisation post-hip fracture surgery.Each of the five factors identified is potentially modifiable through service improvement change and innovation strategies. There is an opportunity to explore how service provision change could be implemented to improve outcomes for all patients following hip fracture surgery demonstrating the clinical and cost benefits of these changes against the cost of delivering the change.

What are the best methods for rapid reviews of the research evidence? A systematic review of reviews and primary studies

Rapid review methodology aims to facilitate faster conduct of systematic reviews to meet the needs of the decision-maker, while also maintaining quality and credibility. This systematic review aimed to determine the impact of different methodological shortcuts for undertaking rapid reviews on the risk of bias (RoB) of the results of the review. Review stages for which reviews and primary studies were sought included the preparation of a protocol, question formulation, inclusion criteria, searching, selection, data extraction, RoB assessment, synthesis, and reporting. We searched 11 electronic databases in April 2022, and conducted some supplementary searching. Reviewers worked in pairs to screen, select, extract data, and assess the RoB of included reviews and studies. We included 15 systematic reviews, 7 scoping reviews, and 65 primary studies. We found that several commonly used shortcuts in rapid reviews are likely to increase the RoB in the results. These include restrictions based on publication date, use of a single electronic database as a source of studies, and use of a single reviewer for screening titles and abstracts, selecting studies based on the full-text, and for extracting data. Authors of rapid reviews should be transparent in reporting their use of these shortcuts and acknowledge the possibility of them causing bias in the results. This review also highlights shortcuts that can save time without increasing the risk of bias. Further research is needed for both systematic and rapid reviews on faster methods for accurate data extraction and RoB assessment, and on development of more precise search strategies.

Basic analgesic use in randomised trials assessing local and regional analgesic interventions for mastectomy: a critical appraisal and clinical implications

Regional analgesia is a core component of an optimal multimodal analgesia technique. Several advanced regional analgesic techniques have been evaluated for mastectomy; however, the optimal choice remains unclear. Many randomised clinical trials (RCTs) evaluating various local/regional analgesic techniques do not include basic analgesics (i.e. paracetamol, non-steroidal anti-inflammatory drugs, cyclooxygenase-2 specific inhibitors, and dexamethasone) which precludes objective evaluation of their efficacy. The aim of this scoping review was to assess the use of basic analgesics in RCTs evaluating efficacy of local and regional analgesic techniques in patients undergoing mastectomy. PubMed was searched to identify relevant articles from January 1, 2010 to May 31, 2023. The key finding of this study is that almost 90% (n=82/92) of the RCTs evaluating local/regional analgesic techniques in patients undergoing mastectomy did not administer well accepted basic analgesics in the comparator groups. Consequently, the conclusions of the RCTs assessing local/regional analgesic techniques for mastectomy should be interpreted with caution. Also, clinical guidelines based on meta-analyses of these RCTs could be inadequate or inappropriate.

Limits in the search date for rapid reviews of diagnostic test accuracy studies

Limiting the search date is a common approach utilised in therapeutic/interventional rapid reviews. Yet the accuracy of pooled estimates is unknown when applied to rapid reviews of diagnostic test accuracy studies. Data from all systematic reviews of diagnostic test accuracy studies published in the Cochrane Database of Systematic Reviews, until February 2022 were collected. Meta-analyses with at least five studies were included to emulate rapid reviews by limiting the search to the recent 1, 2, 3, 4, 5, 10, 15, 20, 25, 30, 35 and 40 years. The magnitude of the pooled area under the curve (AUC), sensitivity and specificity for the full meta-analysis and the rapid reviews were compared. A total of 846 diagnostic meta-analyses were included. When the search date was limited to the recent 10 and 15 years, more than 75% and 80% of meta-analyses presented less than 5% difference between the pooled AUC, sensitivity and specificity of the full meta-analysis and the rapid review. There was little gain in the precision of the pooled estimates when the emulated rapid reviews included more than 15 years in the search. Rapid reviews restricted by search date are a valid and reliable approach for diagnostic test accuracy studies. Robust evidence can be achieved by restricting the search date to the recent 10-15 years. Future studies need to examine the reduction in workload and time to finish the rapid reviews under different search date limits.

2023 American Society of Anesthesiologists Practice Guidelines for Monitoring and Antagonism of Neuromuscular Blockade: A Report by the American Society of Anesthesiologists Task Force on Neuromuscular Blockade

These practice guidelines provide evidence-based recommendations on the management of neuromuscular monitoring and antagonism of neuromuscular blocking agents during and after general anesthesia. The guidance focuses primarily on the type and site of monitoring and the process of antagonizing neuromuscular blockade to reduce residual neuromuscular blockade.

An empirical comparison of the harmful effects for randomized controlled trials and non-randomized studies of interventions

Introduction: Randomized controlled trials (RCTs) are the gold standard to evaluate the efficacy of interventions (e.g., drugs and vaccines), yet the sample size of RCTs is often limited for safety assessment. Non-randomized studies of interventions (NRSIs) had been proposed as an important alternative source for safety assessment. In this study, we aimed to investigate whether there is any difference between RCTs and NRSIs in the evaluation of adverse events. Methods: We used the dataset of systematic reviews with at least one meta-analysis including both RCTs and NRSIs and collected the 2 × 2 table information (i.e., numbers of cases and sample sizes in intervention and control groups) of each study in the meta-analysis. We matched RCTs and NRSIs by their sample sizes (ratio: 0.85/1 to 1/0.85) within a meta-analysis. We estimated the ratio of the odds ratios (RORs) of an NRSI against an RCT in each pair and used the inverse variance as the weight to combine the natural logarithm of ROR (lnROR). Results: We included systematic reviews with 178 meta analyses, from which we confirmed 119 pairs of RCTs and NRSIs. The pooled ROR of NRSIs compared to that of RCTs was estimated to be 0.96 (95% confidence interval: 0.87 and 1.07). Similar results were obtained with different sample size subgroups and treatment subgroups. With the increase in sample size, the difference in ROR between RCTs and NRSIs decreased, although not significantly. Discussion: There was no substantial difference in the effects between RCTs and NRSIs in safety assessment when they have similar sample sizes. Evidence from NRSIs might be considered a supplement to RCTs for safety assessment.

Data reproducibility issues and their potential impact on conclusions from evidence syntheses of randomized controlled trials in sleep medicine

In this study, we examined the data reproducibility issues in systematic reviews in sleep medicine. We searched for systematic reviews of randomized controlled trials published in sleep medicine journals. The metadata in meta-analyses among the eligible systematic reviews were collected. The original sources of the data were reviewed to see if the components used in the meta-analyses were correctly extracted or estimated. The impacts of the data reproducibility issues were investigated. We identified 48 systematic reviews with 244 meta-analyses of continuous outcomes and 54 of binary outcomes. Our results suggest that for continuous outcomes, 20.03% of the data used in meta-analyses cannot be reproduced at the trial level, and 43.44% of the data cannot be reproduced at the meta-analysis level. For binary outcomes, the proportions were 14.14% and 40.74%. In total, 83.33% of the data cannot be reproduced at the systematic review level. Our further analysis suggested that these reproducibility issues would lead to as much as 6.52% of the available meta-analyses changing the direction of the effects, and 9.78% changing the significance of the P-values. Sleep medicine systematic reviews and meta-analyses face serious issues in terms of data reproducibility, and further efforts are urgently needed to improve this situation.