How to critically appraise the clinical literature

Reporting quality and risk of bias of systematic reviews of ultra-processed foods: a methodological study

A dramatic shift in the global food system is occurring with the rapid growth of ultra-processed foods (UPFs) consumption, which poses potentially serious health risks. Systematic review (SR) method has been used to summarise the association between UPF consumption and multiple health outcomes; however, a suboptimal-quality SR may mislead the decision-making in clinical practices and health policies. Therefore, a methodological review was conducted to identify the areas that can be improved regarding the risk of bias and reporting quality of relevant SRs. Systematic searches to collect SRs with meta-analyses of UPFs were performed using four databases from their inception to April 14, 2023. The risk of bias and reporting quality were evaluated using ROBIS and PRISMA 2020, respectively. The key characteristics of the included SRs were summarised descriptively. Excel 2019 and R 4.2.3 were used to analyse the data and draw graphs. Finally, 16 relevant SRs written in English and published between 2020 and 2023 in 12 academic journals were included. Only one SR was rated as low risk of bias, and the others were rated as higher risk of bias mainly because the risk of bias in the original studies was not explicitly addressed when synthesising the evidence. The reporting was required to be advanced significantly, involving amendments of registration and protocol, data and analytic code statement, and lists of excluded studies with justifications. The reviews' results could improve the quality, strengthen future relevant SRs' robustness, and further underpin the evidence base for supporting clinical decisions and health policies.

Methodological quality and risk of bias of meta-analyses of pharmacy services: A systematic review

BACKGROUND

A suboptimal meta-analysis with misleading conclusions, frequently published in the healthcare journals, can compromise decision making in clinical practice.

OBJECTIVE

To evaluate the reporting quality, methodological quality, and risk of bias of meta-analyses of pharmacy services.

METHODS

Systematic searches to identify all the meta-analyses reporting the effect of pharmacy services were performed in PubMed, Scopus, and Web of Science. The reporting quality, the methodological quality, and the risk of bias of the included meta-analyses were evaluated using PRISMA checklist, R-AMSTAR, and ROBIS, respectively.

RESULTS

A total of 109 meta-analyses were eligible for the study. The heterogeneity, the quality of evidence, and the quality analyses were poorly reported on authors' conclusions (14.3%, 14.7%, and 17.4%, respectively). The median scores of PRISMA and R-AMSTAR tolls were 24 (IQR 21.75-25), and 30 (IQR 27-32.5), respectively. Additionally, most of the studies were considered as high risk of bias (n = 83, 76.1%). No association between the date of publication and guideline compliance exists. PRISMA score was higher in studies published in high impact factor journals (rho = 0.313; p = 0.002), in articles that reported the quality of evidence obtained (p = 0.018), and in those that stated the need for future studies in their conclusions (p = 0.011). R-AMSTAR score was higher in studies published in high impact factor journals (rho = 0.338; p = 0.001), in those which reported the quality of evidence (p = 0.002), and in articles that described the quality analyses in their conclusions (p = 0.046). An association between the risk of bias and the recognition of the need for further studies in their conclusions (p = 0.041) was also found.

CONCLUSION

The rapid increase of the meta-analyses of pharmacy services was not associated with higher quality. Mechanistic meta-analyses with poor conclusions are commonly published. Quality of the analyses, strength of evidence, heterogeneity, and absence of confrontation with current guidelines are rarely considered when synthetizing evidence and making recommendations.

Mapping the characteristics of meta-analyses of pharmacy services: a systematic review

BACKGROUND

Suboptimal meta-analyses with misleading conclusions are frequently published in the health areas, and they can compromise decision making in clinical practice.

AIM OF THE REVIEW

This systematic review aimed to map the characteristics of published meta-analyses of pharmacy services and their association with the study conclusions.

METHOD

We searched electronic databases (PubMed, Scopus, and Web of Science) to identify published meta-analyses of pharmacy services up to January 2019. Components of meta-analyses were extracted (i.e. studies' metadata; methods used in the systematic review; description of the statistical model used for the meta-analysis; main results; conflict of interest and funding source). The methodological quality was evaluated using the R-AMSTAR tool.

RESULTS

A total of 85 meta-analyses were included, with 2016 as the median publication year. Overall, the methodological quality of meta-analyses of pharmacy services was considered suboptimal. Only one-third of authors registered a protocol; complete search strategy and raw data were provided by 55.3% and 9.4% of studies, respectively. Evidence strength (GRADE) was evaluated in only 19.2% of studies. PRISMA and Cochrane recommendations were stated to be followed in 60% and 27.4% of articles, respectively. Around half of studies performed sensitivity analysis, however, the prediction interval was presented by only one meta-analysis. Studies that favoured the pharmacists' interventions poorly discussed the methodological quality and heterogeneity of primary trials.

CONCLUSION

Poor conduction and reporting were observed in meta-analyses of pharmacy services, especially in those that favoured the pharmacist's interventions. Reproducibility and transparency should be rigorously ensured by journal editors and peer-reviewers.

Characteristics, Trends, and Quality of Systematic Review and Meta-Analysis in General Radiology between 2007 and 2015

RATIONALE AND OBJECTIVES

To evaluate the trends, characteristics, and quality of systematic review and meta-analysis in general radiology journals.

MATERIALS AND METHODS

We performed a PubMed search to identify systematic reviews and meta-analyses that had been carried out in the field of radiology between 2007 and 2015. The following data were extracted: journal, impact factor, type of research, year of publication, radiological subspecialty, imaging modalities used, number of authors, affiliated department of the first and corresponding authors, presence of a radiologist and a statistician among the authors, discordance between the first and corresponding authors, funding, country of first author, methodological quality, methods used for quality assessment, and statistics.

RESULTS

Ultimately, we included 210 articles from nine general radiology journals. The European Journal of Radiology was the most common journal represented (47 of 210; 22.4%). Meta-analyses (n = 177; 84.3%) were published about five times more than systematic reviews without meta-analysis (n = 33; 15.7%). Radiology of the gastrointestinal tract was the most commonly represented subspecialty (n = 49, 23.3%). The first authors were most frequently located in China (n = 64; 30.3%). In terms of modality, magnetic resonance imaging was used most often (n = 59; 28.1%). The number of authors tended to progressively increase over time, and the ratio of discordance between the first and corresponding authors also increased significantly, as did the proportion of research that has received funding from an external source. The mean AMSTAR assessment score improved over time (5.87/11 in 2007-2009, 7.11/11 in 2010-2012, and 7.49/11 in 2013-2015). In this regard, the journal Radiology had the highest score (7.59/11).

CONCLUSIONS

The quantity and quality of radiological meta-analyses have significantly increased over the past 9 years; however, specific weak areas remain, providing the opportunity for quality improvement.

Another Time, Another Space: The Evolution of the Virtual Journal Club

Virtual journal clubs (VJCs) provide a standardized, easily accessible forum for evidence-based discussion. The new virtual reality setting in which journal clubs and other online education events now take place offers great advantages and new opportunities for radiologists in academic medicine and private practice. VJCs continue to evolve, largely due to many emerging technologies and platforms. VJCs will continue to play an increasingly important role in medical education, interdisciplinary interaction, and multi-institutional collaboration. In this article, we discuss how to conduct and lead a critical review of medical literature in the setting of a virtual or traditional journal club. We discuss the current applications of VJCs in medical and graduate medical education and continued lifelong learning. We also explain the advantages and disadvantages of VJCs over traditional venues. Finally, the reader will be given the tools to successfully implement and run a VJC.

The Science of Quality Improvement

Scientific rigor should be consistently applied to quality improvement (QI) research to ensure that healthcare interventions improve quality and patient safety before widespread implementation. This article provides an overview of the various study designs that can be used for QI research depending on the stage of investigation, scope of the QI intervention, constraints on the researchers and intervention being studied, and evidence needed to support widespread implementation. The most commonly used designs in QI studies are quasi-experimental designs. Randomized controlled trials and cluster randomized trials are typically reserved for large-scale research projects evaluating the effectiveness of QI interventions that may be implemented broadly, have more than a minimal impact on patients, or are costly. Systematic reviews of QI studies will play an important role in providing overviews of evidence supporting particular QI interventions or methods of achieving change. We also review the general requirements for developing quality measures for reimbursement, public reporting, and pay-for-performance initiatives. A critical part of the testing process for quality measures includes assessment of feasibility, reliability, validity, and unintended consequences. Finally, publication and critical appraisal of QI work is discussed as an essential component to generating evidence supporting QI initiatives in radiology.

Teaching Critical Thinking in Graduate Medical Education: Lessons Learned in Diagnostic Radiology

The 2014 Institute of Medicine report, Graduate Medical Education that Meets the Nation's Health Needs, challenged the current graduate medical training process and encouraged new opportunities to redefine the fundamental skills and abilities of the physician workforce. This workforce should be skilled in critically evaluating the current systems to improve care delivery and health. To meet these goals, current challenges, motivations, and educational models at the medical school and graduate medical education levels related to formal training in nonclinical aspects of medicine, especially critical thinking, are reviewed. Our diagnostic radiology training program is presented as a "case study" to frame the review.

Cancer-related fatigue: appraising evidence-based guidelines for screening, assessment and management

PURPOSE

There is inconsistent management of cancer-related fatigue (CRF) by health professionals worldwide. This research aims to identify the most appropriate guidelines for the management of cancer-related fatigue.

METHODS

A systematic search of international literature identified evidence-based clinical practice guidelines for CRF. Four reviewers independently appraised the highest quality guidelines using the AGREE-II instrument and National Heath and Medical Research Council (NHMRC) guideline standards.

RESULTS

Five guidelines met the inclusion criteria. Of these, the 2015 Canadian Association of Psychosocial Oncology (CAPO) CRF guidelines and the 2014 American Society of Clinical Oncology (ASCO) fatigue guidelines for cancer survivors were selected for in-depth appraisal. The CAPO guideline scored higher than the ASCO for five domains of the AGREE-II. For one domain, the differences were statistically significant (p ≤ 0.05). The CAPO guideline met 37 of 47 NHMRC mandatory guideline standards and the ASCO guideline met 20. The difference in the proportion of standards met was statistically significant for one domain (p ≤ 0.05). Both guidelines had low scores for applicability and implementation.

CONCLUSIONS

Currently, the CAPO guideline for cancer-related fatigue has the strongest evidence for use. To enhance implementation, further strategies for guideline dissemination and application are needed.