The strong focus on positive results in abstracts may cause bias in systematic reviews: a case study on abstract reporting bias

PROTOCOL: Breakfast consumption, anthropometry, and nutrition-related outcomes in adolescents from low- and middle-income countries: Protocol for a systematic review and meta-analysis

Objectives This is the protocol for a Campbell systematic review. The objectives are as follows: The aim of this systematic review is to examine the scientific evidence available from low- and middle-income countries on the association of breakfast consumption habits and anthropometry/adiposity- and nutrition-related outcomes in adolescents aged 10-19 years old.

Methods, strategies, and incentives to increase response to mental health surveys among adolescents: a systematic review

BACKGROUND

This systematic review aimed to identify effective methods to increase adolescents' response to surveys about mental health and substance use, to improve the quality of survey information.

METHODS

We followed a protocol and searched for studies that compared different survey delivery modes to adolescents. Eligible studies reported response rates, mental health score variation per survey mode and participant variations in mental health scores. We searched CENTRAL, PsycINFO, MEDLINE and Scopus in May 2022, and conducted citation searches in June 2022. Two reviewers independently undertook study selection, data extraction, and risk of bias assessments. Following the assessment of heterogeneity, some studies were pooled using meta-analysis.

RESULTS

Fifteen studies were identified, reporting six comparisons related to survey methods and strategies. Results indicate that response rates do not differ between survey modes (e.g., web versus paper-and-pencil) delivered in classroom settings. However, web surveys may yield higher response rates outside classroom settings. The largest effects on response rates were achieved using unconditional monetary incentives and obtaining passive parental consent. Survey mode influenced mental health scores in certain comparisons.

CONCLUSIONS

Despite the mixed quality of the studies, the low volume for some comparisons and the limit to studies in high income countries, several effective methods and strategies to improve adolescents' response rates to mental health surveys were identified.

Screening for in vitro systematic reviews: a comparison of screening methods and training of a machine learning classifier

OBJECTIVE

Existing strategies to identify relevant studies for systematic review may not perform equally well across research domains. We compare four approaches based on either human or automated screening of either title and abstract or full text, and report the training of a machine learning algorithm to identify in vitro studies from bibliographic records.

METHODS

We used a systematic review of oxygen-glucose deprivation (OGD) in PC-12 cells to compare approaches. For human screening, two reviewers independently screened studies based on title and abstract or full text, with disagreements reconciled by a third. For automated screening, we applied text mining to either title and abstract or full text. We trained a machine learning algorithm with decisions from 2000 randomly selected PubMed Central records enriched with a dataset of known in vitro studies.

RESULTS

Full-text approaches performed best, with human (sensitivity: 0.990, specificity: 1.000 and precision: 0.994) outperforming text mining (sensitivity: 0.972, specificity: 0.980 and precision: 0.764). For title and abstract, text mining (sensitivity: 0.890, specificity: 0.995 and precision: 0.922) outperformed human screening (sensitivity: 0.862, specificity: 0.998 and precision: 0.975). At our target sensitivity of 95% the algorithm performed with specificity of 0.850 and precision of 0.700.

CONCLUSION

In this in vitro systematic review, human screening based on title and abstract erroneously excluded 14% of relevant studies, perhaps because title and abstract provide an incomplete description of methods used. Our algorithm might be used as a first selection phase in in vitro systematic reviews to limit the extent of full text screening required.

The Race to Retain Healthcare Workers: A Systematic Review on Factors that Impact Retention of Nurses and Physicians in Hospitals

The shortage of healthcare workers is a growing problem across the globe. Nurses and physicians, in particular, are vulnerable as a result of the COVID-19 pandemic. Understanding why they might leave is imperative for improving retention. This systematic review explores both the prevalence of nurses and physicians who are intent on leaving their position at hospitals in European countries and the main determinants influencing job retention among nurses and physicians of their respective position in a hospital setting in both European and non-European countries. A comprehensive search was fulfilled within 3 electronic databases on June 3rd 2021. In total 345 articles met the inclusion criteria. The determinants were categorized into 6 themes: personal characteristics, job demands, employment services, working conditions, work relationships, and organizational culture. The main determinants for job retention were job satisfaction, career development and work-life balance. European and non-European countries showed similarities and differences in determinants influencing retention. Identifying these factors supports the development of multifactorial interventions, which can aid the formulation of medical strategies and help to maximize retention.

Salivary noncoding RNA in the diagnosis of pancreatic cancer: Systematic review and meta-analysis

BACKGROUND

Pancreatic cancer is considered one of the most deadly malignancies, primarily because of its diagnostic challenges. We performed a systematic review and diagnostic meta-analysis to evaluate the diagnostic value of noncoding salivary RNAs in pancreatic cancer diagnosis.

METHODS

Our investigation involved pertinent studies published in PubMed, Scopus, Web of Science, LIVIVO, Ovid and also the Google Scholar search engine. Specificity and sensitivity were calculated, as were positive and negative likelihood ratios (PLR and NLR), and the diagnostic odds ratio (DOR). The summary receiver-operating characteristics and area under the curve were plotted and assessed.

RESULTS

This meta-analysis and systematic review involved and examined five studies that contained 145 study units with a total of 2731 subjects (1465 pancreatic cancer patients versus 1266 noncancer controls). The pooled specificity, sensitivity, NLR, PLR and DOR were 0.783 (95% CI: 0.759-0.805), 0.829 (95% CI: 0.809-0.848), 0.309 (95% CI: 0.279-0.343), 3.386 (95% CI: 2.956-3.879) and 18.403 (95% CI: 14.753-22.954), respectively, with the area under the curve (AUC) equal to 0.882. Subgroup analyses were conducted based on the saliva type (unstimulated and stimulated), mean age of patients, sample size, type of control, serum carbohydrate antigen 19-9 (CA19-9) level and type of salivary noncoding RNA (microRNA (miRNA) and long noncoding RNA (lncRNA)).

CONCLUSIONS

The results of our systematic review and meta-analysis suggest that noncoding RNA biomarkers in the stimulated saliva could be a promising approach for accurate pancreatic cancer diagnosis in the early stages.

Salivary biomarkers in breast cancer diagnosis: A systematic review and diagnostic meta-analysis

BACKGROUND

Salivary diagnostics and their utility as a nonaggressive approach for breast cancer diagnosis have been extensively studied in recent years. This meta-analysis assesses the diagnostic value of salivary biomarkers in differentiating between patients with breast cancer and controls.

METHODS

We conducted a meta-analysis and systematic review of studies related to salivary diagnostics published in PubMed, EMBASE, Scopus, Ovid, Science Direct, Web of Science (WOS), and Google Scholar. The articles were chosen utilizing inclusion and exclusion criteria, as well as assessing their quality. Specificity and sensitivity, along with negative and positive likelihood ratios (NLR and PLR) and diagnostic odds ratio (DOR), were calculated based on random- or fixed-effects model. Area under the curve (AUC) and summary receiver-operating characteristic (SROC) were plotted and evaluated, and Fagan's Nomogram was evaluated for clinical utility.

RESULTS

Our systematic review and meta-analysis included 14 papers containing 121 study units with 8639 adult subjects (4149 breast cancer patients and 4490 controls without cancer). The pooled specificity and sensitivity were 0.727 (95% CI: 0.713-0.740) and 0.717 (95% CI: 0.703-0.730), respectively. The pooled NLR and PLR were 0.396 (95% CI: 0.364-0.432) and 2.597 (95% CI: 2.389-2.824), respectively. The pooled DOR was 7.837 (95% CI: 6.624-9.277), with the AUC equal to 0.801. The Fagan's nomogram showed post-test probabilities of 28% and 72% for negative and positive outcomes, respectively. We also conducted subgroup analyses to determine specificity, sensitivity, DOR, PLR, and NLR based on the mean age of patients (≤52 or >52 years old), saliva type (stimulated and unstimulated saliva), biomarker measurement method (mass spectrometry [MS] and non-MS measurement methods), sample size (≤55 or >55), biomarker type (proteomics, metabolomics, transcriptomics and proteomics, and reagent-free biophotonic), and nations.

CONCLUSION

Saliva, as a noninvasive biomarker, has the potential to accurately differentiate breast cancer patients from healthy controls.

Avoiding searching for outcomes called for additional search strategies: A study of Cochrane review searches

OBJECTIVE

A search strategy for a systematic review that use the PICO-model as framework, should include the population, the intervention(s), and the type(s) of study design. According to existing guidelines outcome should generally be excluded from the search strategy unless the search is multistranded. However, a recent study found that approximately 10% (51) of recent Cochrane reviews on interventions included outcomes in their literature search strategies. This study aims to analyze the alternatives to including outcome in a search strategy, by analyzing these recent Cochrane reviews.

STUDY DESIGN

This study analyses the 51 Cochrane reviews that included outcomes in their literature search strategies and analyzes the results of alternative search strategies that follow current recommendations.

RESULTS

Despite a small study sample of 51 reviews, the results show that many of the reviews excluded some of the recommended elements due to very broadly defined elements (e.g., all interventions or all people). Furthermore, excluding outcome from the search strategy is followed by an enormous increase in the number of retrieved records making it unmanageable to screen, if using a single stranded search strategy.

CONCLUSION

Recommendations for search strategies in difficult cases are called for.

Effect of spin in the abstract of a randomised controlled trial on physiotherapists' perception of treatment benefit: a randomised controlled trial

OBJECTIVE

To assess the effect of spin in the abstract of a randomised controlled trial (RCT) on physiotherapists' perception of treatment benefit evaluated in the trial.

DESIGN

Parallel-group RCT.

SETTING

Physiotherapy departments in hospitals and clinics in India.

PARTICIPANTS

Physiotherapists working in clinical settings.

INTERVENTIONS

We selected one abstract with high level of spin published in one of the core journals of physiotherapy and created two versions of the abstract, that is, with and without spin. We randomly assigned physiotherapists working in clinical settings (N=128) to read one version of the selected abstract, with or without spin. Participants were blinded to the study design, objectives and randomisation.

MAIN OUTCOME MEASURES

Physiotherapists' interpretation of beneficial effect of the experimental treatment (0-10 scale) reported in the abstract. The secondary outcomes were clinicians' perception of methodological rigour and the study importance, their interest in reading the full text, and their interest in running another trial evaluating this treatment.

RESULTS

We found a medium reduction in confidence of beneficial effect of the experimental treatment among physiotherapists who read the abstract without spin (mean score 4.3±2.8) compared with those who read the abstract with spin (mean score 6.14±2.6). The mean difference in scores between abstracts with and without spin was 1.8 (95% CI 0.8 to 2.8; p<0.001). For other outcomes measures studied there was no statistically significant effect.

CONCLUSIONS

Removal of spin in the abstract of RCT reporting statistically non-significant results have medium effect in improving physiotherapists' accuracy of interpretation of study results. Spin contributes to clinicians' positive perception about the benefit of experimental intervention tested in the trial despite the evidence showing no superiority of experimental intervention.

TRIAL REGISTRATION NUMBER

CTRI/2020/02/023557.

The Effect of Yoga on Health-Related Fitness among Patients with Type 2 Diabetes Mellitus: A Systematic Review and Meta-Analysis

BACKGROUND

There is a need for a type of physical activity that could address the challenging cycle of physical inactivity, impaired health-related fitness, and type 2 diabetes mellitus (T2DM) conditions. Yoga could be one type of exercise to overcome the barriers to adhere to regular physical activity. The current study aimed to systematically review the effect of yoga on health-related fitness, including cardiorespiratory fitness, muscle strength, body composition, balance, and flexibility, among patients with T2DM.

METHODS

We systematically searched four databases and two registries (Pubmed, Scopus, Cochrane, Embase, WHO-ITCRP, and Clinicaltrials.gov) in September 2021, following a registered protocol on PROSPERO (CRD42022276225). Study inclusion criteria were T2DM patients with or without complication, yoga intervention as a single component or as a complement compared to other kinds of exercise or an inactive control, health-related fitness, and a randomized, controlled trial or quasi-experimental with control group design. The ROBINS-I tool and ROB 2.0 tool were used to assess the risk of bias in the included studies. A vote-counting analysis and meta-analysis computed using random effects' models were conducted.

RESULTS

A total of 10 records from 3 quasi-experimental and 7 randomized, controlled trials with 815 participants in total were included. The meta-analysis favored yoga groups compared to inactive controls in improving muscle strength by 3.42 (95% confidence interval 2.42 to 4.43), repetitions of chair stand test, and improving cardiorespiratory fitness by 6.6% (95% confidence interval 0.4 to 12.8) improvement of baseline forced vital capacity. The quality of evidence for both outcomes was low.

CONCLUSION

Low-quality evidence favored yoga in improving health-related fitness, particularly muscle strength and cardiorespiratory fitness, among patients with T2DM.

FUNDING

All authors in this systematic review received no specific grant from any funding agency in the public, commercial, or not-for-profit sectors.

Sex/gender differences in the human autistic brains: A systematic review of 20 years of neuroimaging research

Our current understanding of autism is largely based on clinical experiences and research involving male individuals given the male-predominance in prevalence and the under-inclusion of female individuals due to small samples, co-occurring conditions, or simply being missed for diagnosis. There is a significantly biased 'male lens' in this field with autistic females insufficiently understood. We therefore conducted a systematic review to examine how sex and gender modulate brain structure and function in autistic individuals. Findings from the past 20 years are yet to converge on specific brain regions/networks with consistent sex/gender-modulating effects. Despite at least three well-powered studies identifying specific patterns of significant sex/gender-modulation of autism-control differences, many other studies are likely underpowered, suggesting a critical need for future investigation into sex/gender-based heterogeneity with better-powered designs. Future research should also formally investigate the effects of gender, beyond biological sex, which is mostly absent in the current literature. Understanding the roles of sex and gender in the development of autism is an imperative step to extend beyond the 'male lens' in this field.

Around ten percent of most recent Cochrane reviews included outcomes in their literature search strategy and were associated with potentially exaggerated results: A research-on-research study

OBJECTIVES

To assess the proportion of the recent Cochrane reviews that included outcomes in their literature search strategy, how often they acknowledged these limitations, and how qualitatively different the results of outcomes included and not included in the search strategy were.

DESIGN AND SETTING

We identified all the Cochrane reviews of the interventions published in 2020 that used a search strategy connecting outcome terms with "AND." Reviews were defined as acknowledging the limitations of searching for outcomes if they mentioned them in the discussion. We compared the characteristics of outcomes included and not included in the search strategy.

RESULTS

Of the 523 Cochrane reviews published in 2020, 51 (9.8%) included outcomes in their search strategy. Only one review acknowledged it as a limitation. Forty-seven (92%) assessed outcomes not included in the search strategy. Outcomes included in the search strategies tended to include a larger number of studies and show their effects in favor of the intervention.

CONCLUSIONS

Around ten percent of the recent Cochrane reviews included outcomes in their search, which may have resulted in more outcomes significantly in favor of the intervention. Reviewers should be more explicit in acknowledging the potential implications of searching for outcomes.

Systematic review and meta-analysis efficacy and safety of immune checkpoint inhibitors in advanced melanoma patients with anti-PD-1 progression: a systematic review and meta-analysis

BACKGROUND

More than half of melanoma patients taking first-line anti-PD-1 therapy either express transient or no response at all. The efficacy and safety of secondary treatments for these patients are still not well established. Here, we evaluate the efficacy and safety of different melanoma FDA-approved ICI modalities used in post-anti-PD-1 refractory settings.

MATERIALS AND METHODS

We searched the PubMed database and the ASCO meetings library for studies on advanced melanoma patients with cancer progression on anti-PD-1 therapy and were then treated with ipilimumab, nivolumab/ipilimumab combination, or retreated with anti-PD-1. Primary and secondary endpoints were efficacy and toxicity, respectively. Pooled estimates for each treatment group were obtained using a random or fixed effects model according to detected heterogeneity.

RESULTS

Fourteen studies, of which 10 on ipilimumab, 2 on anti-PD-1 treatment, and 6 on combination therapies, were included, involving a total of 1460 patients. Twelve studies reported objective response rates (ORRs) and nine of them reported immune-related adverse events (irAEs). As for ORR, patients experienced a response that was inferior compared to the same therapy in treatment -naïve patients, with combination therapy having the best ORR of a pooled 23.08% (95% CI: 16.75% to 30.03%), followed by ipilimumab with a pooled ORR of 8.19% (95% CI: 5.78% to 10.92%). Survival data were also inferior in the ipilimumab cohort (mOS: 5.1 to 7.4 months) compared to ipilimumab in anti-PD-1 naive patients. As for grade 3/4 irAE occurrence, the ipilimumab cohort showed an estimate of 43.77% (95% CI 22.55% to 66.19%).

CONCLUSION

Our findings provide the best current evidence that patients who progress on anti-PD-1 can still respond to different ICI modalities (ipilimumab with or without nivolumab, and retreatment or continuation beyond progression with anti-PD-1) with tolerable grade 3/4 irAEs. However, more prospective clinical trials are needed to confirm these results.

Vaccination for Human Papillomavirus: an historic and bibliometric study

A systematic literature review was conducted to describe in a historical perspective the evolution of studies concerning HPV vaccination. The search identified 794 articles of which 568 were included. The first article was published in 2001, and the maximum annual number of publications was reached in 2014. The average number of authors per paper was 8.8. Papers originated from 49 different countries, with the USA accounted for the maximum number of publications (n = 217). Efficacy (46.5%) and safety (31.0%) were the most prevalent objectives. Clinical trials constituted the largest group of methods (37.9%). Chronological trends did not reveal any lasting curve-crossings, indicating that the priority topics have remained the same. The geographical origin of these studies raises questions about the transposability of the results to populations where HPV vaccination has been studied only a little. This study could help guide future research to less-studied research objectives, particularly for vaccines.

fiddle: a tool to combat publication bias by getting research out of the file drawer and into the scientific community

Statistically significant findings are more likely to be published than non-significant or null findings, leaving scientists and healthcare personnel to make decisions based on distorted scientific evidence. Continuously expanding ´file drawers' of unpublished data from well-designed experiments waste resources creates problems for researchers, the scientific community and the public. There is limited awareness of the negative impact that publication bias and selective reporting have on the scientific literature. Alternative publication formats have recently been introduced that make it easier to publish research that is difficult to publish in traditional peer reviewed journals. These include micropublications, data repositories, data journals, preprints, publishing platforms, and journals focusing on null or neutral results. While these alternative formats have the potential to reduce publication bias, many scientists are unaware that these formats exist and don't know how to use them. Our open source file drawer data liberation effort (fiddle) tool (RRID:SCR_017327 available at: http://s-quest.bihealth.org/fiddle/) is a match-making Shiny app designed to help biomedical researchers to identify the most appropriate publication format for their data. Users can search for a publication format that meets their needs, compare and contrast different publication formats, and find links to publishing platforms. This tool will assist scientists in getting otherwise inaccessible, hidden data out of the file drawer into the scientific community and literature. We briefly highlight essential details that should be included to ensure reporting quality, which will allow others to use and benefit from research published in these new formats.

MyCites: a proposal to mark and report inaccurate citations in scholarly publications

Background

Inaccurate citations are erroneous quotations or instances of paraphrasing of previously published material that mislead readers about the claims of the cited source. They are often unaddressed due to underreporting, the inability of peer reviewers and editors to detect them, and editors' reluctance to publish corrections about them. In this paper, we propose a new tool that could be used to tackle their circulation.

Methods

We provide a review of available data about inaccurate citations and analytically explore current ways of reporting and dealing with these inaccuracies. Consequently, we make a distinction between publication (i.e., first occurrence) and circulation (i.e., reuse) of inaccurate citations. Sloppy reading of published items, literature ambiguity and insufficient quality control in the editorial process are identified as factors that contribute to the publication of inaccurate citations. However, reiteration or copy-pasting without checking the validity of citations, paralleled with lack of resources/motivation to report/correct inaccurate citations contribute to their circulation.

Results and discussion

We propose the development of an online annotation tool called "MyCites" as means with which to mark and map inaccurate citations. This tool allows ORCID users to annotate citations and alert authors (of the cited and citing articles) and also editors of journals where inaccurate citations are published. Each marked citation would travel with the digital version of the document (persistent identifiers) and be visible on websites that host peer-reviewed articles (journals' websites, Pubmed, etc.). In the future development of MyCites, challenges such as the conditions of correct/incorrect-ness and parties that should adjudicate that, and, the issue of dealing with incorrect reports need to be addressed.

Using the full PICO model as a search tool for systematic reviews resulted in lower recall for some PICO elements

OBJECTIVES

The use of the four-part PICO model to facilitate search strategy development for a precise answer is recommended for structuring searches for systematic reviews. Existing guidelines generally recommend that a search strategy should include the population, intervention(s), and types of study design. Consequently, comparison and outcome are not recommended as a part of the search strategy. There is evidence that comparison and particularly outcome is not represented in enough detail, but this needs to be confirmed.

STUDY DESIGN AND SETTING

The present study examines the presence of PICO elements in the records in two commonly used databases for health sciences research: Embase and PubMed. We examine the field of upper GI and pancreatic diseases as well as the field of pregnancy and childbirth by extracting the included studies as well as the related PICO elements from a random selection of Cochrane reviews within these two areas.

RESULTS

We find that the PICO elements C and O had a lower retrieval potential across the two Cochrane groups and databases also when combining text words and subject headings. In particular, we find a lower retrieval when searching for both primary and secondary outcomes.

CONCLUSION

Our results support the existing recommendation not to search for outcomes.