Optimal search strategies for identifying sound clinical prediction studies in EMBASE

Applying 'cultural humility' to occupational therapy practice: a scoping review protocol

INTRODUCTION

Cultural humility is becoming increasingly important in healthcare delivery. Recognition of power imbalances between clients and healthcare providers is critical to enhancing cross-cultural interactions in healthcare delivery. While cultural humility has been broadly examined in healthcare, knowledge gaps exist regarding its application in occupational therapy (OT) practice. This scoping review protocol aims to: (1) describe the extent and nature of the published health literature on cultural humility, including concepts, descriptions and definitions and practice recommendations, (2) map the findings from objective one to OT practice using the Canadian Practice Process Framework (CPPF), and (3) conduct a consultation exercise to confirm the CPPF mapping and generate recommendations for the practice of cultural humility in OT.

METHODS AND ANALYSIS

We will search Ovid Medline, Ovid Embase, Ovid PsycINFO, Ebsco CINAHL Plus, ProQuest ASSIA, ProQuest Sociological Abstracts, ProQuest ERIC, WHO Global Index Medicus, and Web of Science databases. Published health-related literature on cultural humility will be included. There will be no restrictions on population or article type. Following deduplication on Endnote, the search results will undergo title, abstract, and full-text review by two reviewers working independently on Covidence. Extracted data will include descriptors of the article, context, population, and cultural humility. After descriptive extraction, data describing cultural humility-related content will be descriptively and interpretively analysed using an inductive thematic synthesis approach. The data will also be mapped to OT practice through deductive coding using the CPPF. Occupational therapists and clients will be consulted to further critique, interpret and validate the mapping and generate practice recommendations.

ETHICS AND DISSEMINATION

Ethics approval was not required for this scoping review protocol. We will disseminate the findings, which can enhance understanding of cultural humility in OT, facilitate cross-cultural encounters between occupational therapists and clients and improve care outcomes through publications and presentations.

Search methods for prognostic factor systematic reviews: a methodologic investigation

Objective

This study retroactively investigated the search used in a 2019 review by Hayden et al., one of the first systematic reviews of prognostic factors that was published in the Cochrane Library. The review was designed to address recognized weaknesses in reviews of prognosis by using multiple supplementary search methods in addition to traditional electronic database searching.

Methods

The authors used four approaches to comprehensively assess aspects of systematic review literature searching for prognostic factor studies: (1) comparison of search recall of broad versus focused electronic search strategies, (2) linking of search methods of origin for eligible studies, (3) analysis of impact of supplementary search methods on meta-analysis conclusions, and (4) analysis of prognosis filter performance.

Results

The review's focused electronic search strategy resulted in a 91% reduction in recall, compared to a broader version. Had the team relied on the focused search strategy without using supplementary search methods, they would have missed 23 of 58 eligible studies that were indexed in MEDLINE; additionally, the number of included studies in 2 of the review's primary outcome meta-analyses would have changed. Using a broader strategy without supplementary searches would still have missed 5 studies. The prognosis filter used in the review demonstrated the highest sensitivity of any of the filters tested.

Conclusions

Our study results support recommendations for supplementary search methods made by prominent systematic review methodologists. Leaving out any supplemental search methods would have resulted in missed studies, and these omissions would not have been prevented by using a broader search strategy or any of the other prognosis filters tested.

Optimizing a literature surveillance strategy to retrieve sound overall prognosis and risk assessment model papers

OBJECTIVE

Our aim was to develop an efficient search strategy for prognostic studies and clinical prediction guides (CPGs), optimally balancing sensitivity and precision while independent of MeSH terms, as relying on them may miss the most current literature.

MATERIALS AND METHODS

We combined 2 Hedges-based search strategies, modified to remove MeSH terms for overall prognostic studies and CPGs, and ran the search on 269 journals. We read abstracts from a random subset of retrieved references until ≥ 20 per journal were reviewed and classified them as positive when fulfilling standardized quality criteria, thereby assembling a standard dataset used to calibrate the search strategy. We determined performance characteristics of our new search strategy against the Hedges standard and performance characteristics of published search strategies against the standard dataset.

RESULTS

Our search strategy retrieved 16 089 references from 269 journals during our study period. One hundred fifty-four journals yielded ≥ 20 references and ≥ 1 prognostic study or CPG. Against the Hedges standard, the new search strategy had sensitivity/specificity/precision/accuracy of 84%/80%/2%/80%, respectively. Existing published strategies tested against our standard dataset had sensitivities of 36%-94% and precision of 5%-10%.

DISCUSSION

We developed a new search strategy to identify overall prognosis studies and CPGs independent of MeSH terms. These studies are important for medical decision-making, as they identify specific populations and individuals who may benefit from interventions.

CONCLUSION

Our results may benefit literature surveillance and clinical guideline efforts, as our search strategy performs as well as published search strategies while capturing literature at the time of publication.

Risk prediction models for survival after heart transplantation: A systematic review

Risk prediction scores have been developed to predict survival following heart transplantation (HT). Our objective was to systematically review the model characteristics and performance for all available scores that predict survival after HT. Ovid Medline and Epub Ahead of Print and In-Process & Other Non-Indexed Citations, Ovid Embase, Cochrane Database of Systematic Reviews, and Cochrane Central Register of Controlled Clinical Trials were searched to December 2018. Eligible articles reported a score to predict mortality following HT. Of the 5392 studies screened, 21 studies were included that derived and/or validated 16 scores. Seven (44%) scores were validated in external cohorts and 8 (50%) assessed model performance. Overall model discrimination ranged from poor to moderate (C-statistic/area under the receiver operating characteristics 0.54-0.77). The IMPACT score was the most widely validated, was well calibrated in two large registries, and was best at discriminating 3-month survival (C-statistic 0.76). Most scores did not perform particularly well in any cohort in which they were assessed. This review shows that there are insufficient data to recommend the use of one model over the others for prediction of post-HT outcomes.

Search strategies to identify observational studies in MEDLINE and Embase

BACKGROUND

Systematic reviews are essential for decision-making. Systematic reviews on observational studies help answer research questions on aetiology, risk, prognosis, and frequency of rare outcomes or complications. However, identifying observational studies as part of systematic reviews efficiently is challenging due to poor and inconsistent indexing in literature databases. Search strategies that include a methodological filter focusing on study design of observational studies might be useful for improving the precision of the search performance.

OBJECTIVES

To assess the sensitivity and precision of a search strategy with a methodological filter to identify observational studies in MEDLINE and Embase.

SEARCH METHODS

We searched MEDLINE (1946 to April 2018), Embase (1974 to April 2018), CINAHL (1937 to April 2018), the Cochrane Library (1992 to April 2018), Google Scholar and Open Grey in April 2018, and scanned reference lists of articles.

SELECTION CRITERIA

Studies using a relative recall approach, i.e. comparing sensitivity or precision of a search strategy containing a methodological filter to identify observational studies in MEDLINE and Embase against a reference standard, or studies that compared two or more methodological filters.

DATA COLLECTION AND ANALYSIS

Two review authors independently screened articles, extracted relevant information and assessed the quality of the search strategies using the InterTASC Information Specialists' Sub-Group (ISSG) Search Filter Appraisal Checklist.

MAIN RESULTS

We identified two eligible studies reporting 18 methodological filters. All methodological filters in these two studies were developed using terms from the reference standard records.The first study evaluated six filters for retrieving observational studies of surgical interventions. The study reported on six filters: one Precision Terms Filter (comprising terms with higher precision while maximum sensitivity was maintained) and one Specificity Terms Filter (comprising terms with higher specificity while maximum sensitivity was maintained), both of which were adapted for MEDLINE, for Embase, and for combined MEDLINE/Embase searches. The study reported one reference standard consisting of 217 articles from one systematic review of which 83.9% of the included studies were case seriesThe second study reported on 12 filters for retrieving comparative non-randomised studies (cNRSs) including cohort, case-control, and cross-sectional studies. This study reported on 12 filters using four different approaches: Fixed method A (comprising of a fixed set of controlled vocabulary (CV) words), Fixed method B (comprising a fixed set of CV words and text words (TW)), Progressive method (CV) (a random choice of study design-related CV terms), and Progressive method (CV or TW) (a random choice of study design-related CV terms, and title and abstracts-based TWs). The study reported four reference standards consisting of 89 cNRSs from four systematic reviews.The six methodological filters developed from the first study reported sensitivity of 99.5% to 100% and precision of 16.7% to 21.1%. The Specificity Terms Filter for combined MEDLINE/Embase was preferred because it had higher precision and equal sensitivity to the Precision Terms Filter. The 12 filters from the second study reported lower sensitivity (48% to 100%) and much lower precision (0.09% to 4.47%). The Progressive method (CV or TW) had the highest sensitivity.There were methodological limitations in both included studies. The first study used one surgical intervention-focused systematic review thus limiting the generalizability of findings. The second study used four systematic reviews but with less than 100 studies. The external validation was performed only on Specificity Terms Filter from the first study Both studies were published 10 years ago and labelling and indexing of observational studies has changed since then.

AUTHORS' CONCLUSIONS

We found 18 methodological filters across two eligible studies. Search strategies from the first study had higher sensitivity and precision, underwent external validation and targeted observational studies. Search strategies from the second study had lower sensitivity and precision, focused on cNRSs, and were not validated externally. Given this limited and heterogeneous evidence, and its methodological limitations, further research and better indexation are needed.

Design Characteristics Influence Performance of Clinical Prediction Rules in Validation: A Meta-Epidemiological Study

BACKGROUND

Many new clinical prediction rules are derived and validated. But the design and reporting quality of clinical prediction research has been less than optimal. We aimed to assess whether design characteristics of validation studies were associated with the overestimation of clinical prediction rules' performance. We also aimed to evaluate whether validation studies clearly reported important methodological characteristics.

METHODS

Electronic databases were searched for systematic reviews of clinical prediction rule studies published between 2006 and 2010. Data were extracted from the eligible validation studies included in the systematic reviews. A meta-analytic meta-epidemiological approach was used to assess the influence of design characteristics on predictive performance. From each validation study, it was assessed whether 7 design and 7 reporting characteristics were properly described.

RESULTS

A total of 287 validation studies of clinical prediction rule were collected from 15 systematic reviews (31 meta-analyses). Validation studies using case-control design produced a summary diagnostic odds ratio (DOR) 2.2 times (95% CI: 1.2-4.3) larger than validation studies using cohort design and unclear design. When differential verification was used, the summary DOR was overestimated by twofold (95% CI: 1.2 -3.1) compared to complete, partial and unclear verification. The summary RDOR of validation studies with inadequate sample size was 1.9 (95% CI: 1.2 -3.1) compared to studies with adequate sample size. Study site, reliability, and clinical prediction rule was adequately described in 10.1%, 9.4%, and 7.0% of validation studies respectively.

CONCLUSION

Validation studies with design shortcomings may overestimate the performance of clinical prediction rules. The quality of reporting among studies validating clinical prediction rules needs to be improved.

Validation and impact analysis of prognostic clinical prediction rules for low back pain is needed: a systematic review

OBJECTIVES

To identify prognostic forms of clinical prediction rules (CPRs) related to the nonsurgical management of adults with low back pain (LBP) and to evaluate their current stage of development.

STUDY DESIGN AND SETTING

Systematic review using a sensitive search strategy across seven databases with hand searching and citation tracking.

RESULTS

A total of 10,005 records were screened for eligibility with 35 studies included in the review. The included studies report on the development of 30 prognostic LBP CPRs. Most of the identified CPRs are in their initial phase of development. Three CPRs were found to have undergone validation--the Cassandra rule for predicting long-term significant functional limitations and the five-item and two-item Flynn manipulation CPRs for predicting a favorable functional prognosis in patients being treated with lumbopelvic manipulation. No studies were identified that investigated whether the implementation of a CPR resulted in beneficial patient outcomes or improved resource efficiencies.

CONCLUSION

Most of the identified prognostic CPRs for LBP are in the initial phase of development and are consequently not recommended for direct application in clinical practice at this time. The body of evidence provides emergent confidence in the limited predictive performance of the Cassandra rule and the five-item Flynn manipulation CPR in comparable clinical settings and patient populations.

Diagnostic clinical prediction rules for specific subtypes of low back pain: a systematic review

STUDY DESIGN

Systematic review.

OBJECTIVES

To identify diagnostic clinical prediction rules (CPRs) for low back pain (LBP) and to assess their readiness for clinical application.

BACKGROUND

Significant research has been invested into the development of CPRs that may assist in the meaningful subgrouping of patients with LBP. To date, very little is known about diagnostic forms of CPRs for LBP, which relate to the present status or classification of an individual, and whether they have been developed sufficiently to enable their application in clinical practice.

METHODS

A sensitive electronic search strategy using 7 databases was combined with hand searching and citation tracking to identify eligible studies. Two independent reviewers identified relevant studies for inclusion using a 2-stage selection process. The quality appraisal of included studies was conducted by 2 independent raters using the Quality Assessment of Diagnostic Accuracy Studies-2 and checklists composed of accepted methodological standards for the development of CPRs.

RESULTS

Of 10 014 studies screened for eligibility, the search identified that 13 diagnostic CPRs for LBP have been derived. Among those, 1 tool for identifying lumbar spinal stenosis and 2 tools for identifying inflammatory back pain have undergone validation. No impact analysis studies were identified.

CONCLUSION

Most diagnostic CPRs for LBP are in their initial development phase and cannot be recommended for use in clinical practice at this time. Validation and impact analysis of the diagnostic CPRs identified in this review are warranted, particularly for those tools that meet an identified unmet need of clinicians who manage patients with LBP.

LEVEL OF EVIDENCE

Diagnosis, level 2a-.

Transparent Reporting of a multivariable prediction model for Individual Prognosis or Diagnosis (TRIPOD): explanation and elaboration

The TRIPOD (Transparent Reporting of a multivariable prediction model for Individual Prognosis Or Diagnosis) Statement includes a 22-item checklist, which aims to improve the reporting of studies developing, validating, or updating a prediction model, whether for diagnostic or prognostic purposes. The TRIPOD Statement aims to improve the transparency of the reporting of a prediction model study regardless of the study methods used. This explanation and elaboration document describes the rationale; clarifies the meaning of each item; and discusses why transparent reporting is important, with a view to assessing risk of bias and clinical usefulness of the prediction model. Each checklist item of the TRIPOD Statement is explained in detail and accompanied by published examples of good reporting. The document also provides a valuable reference of issues to consider when designing, conducting, and analyzing prediction model studies. To aid the editorial process and help peer reviewers and, ultimately, readers and systematic reviewers of prediction model studies, it is recommended that authors include a completed checklist in their submission. The TRIPOD checklist can also be downloaded from www.tripod-statement.org.

Examination of the Clinical Queries and Systematic Review “hedges” in EMBASE and MEDLINE

Introduction – This investigation sought to determine whether the methodological search filters in place as Clinical Queries limits in OvidSP EMBASE and OvidSP MEDLINE had been modified from those written by Haynes et al. and whether the translations of these in PubMed and EBSCO MEDLINE were reliable. The translated National Library of Medicine (NLM) Systematic Reviews hedges in place in OvidSP MEDLINE and EBSCO MEDLINE were also examined. Methods – Search queries were run using the Clinical Queries and Systematic Reviews hedges incorporated into OvidSP EMBASE, OvidSP MEDLINE, PubMed, and EBSCO MEDLINE to determine the reliability of these limits in comparison with the published hedge search strings. Results – Five of the OvidSP EMBASE Clinical Queries hedges produced results that were different from the published search strings. Three of the EBSCO MEDLINE and five of the PubMed translated Clinical Queries hedges yielded markedly different results (>10% difference) than those obtained using the OvidSP MEDLINE hedge counterparts. The OvidSP MEDLINE Systematic Reviews subject subset hedge was found to have a major error, which has been corrected. Discussion – Translations of hedges to appropriate syntax for other database platforms may result in significantly different search results. The platform searched should ideally be the one for which the hedges were written and tested. Regardless, the hedges in place may not be the same as the published hedge search strings. Quality control testing is needed to ensure that the hedges in place as limits are the same as those that have been published.

A generalised model for individualising a treatment recommendation based on group-level evidence from randomised clinical trials

OBJECTIVES

Randomised controlled trials report group-level treatment effects. However, an individual patient confronting a treatment decision needs to know whether that person's expected treatment benefit will exceed the expected treatment harm. We describe a flexible model for individualising a treatment decision. It individualises group-level results from randomised trials using clinical prediction guides.

METHODS

We constructed models that estimate the size of individualised absolute risk reduction (ARR) for the target outcome that is required to offset individualised absolute risk increase (ARI) for the treatment harm. Inputs to the model include estimates for the individualised predicted absolute treatment benefit and harm, and the relative value assigned by the patient to harm/benefit. A decision rule recommends treatment when the predicted benefit exceeds the predicted harm, value-adjusted. We also derived expressions for the maximum treatment harm, or the maximum relative value for harm/benefit, above which treatment would not be recommended.

RESULTS

For the simpler model, including one kind of benefit and one kind of harm, the individualised ARR required to justify treatment was expressed as required ARRtarget(i)=ARIharm(i) × RVharm/target(i). A complex model was also developed, applicable to treatments causing multiple kinds of benefits and/or harms. We demonstrated the applicability of the models to treatments tested in superiority trials (either placebo or active control, either fixed harm or variable harm) and non-inferiority trials.

CONCLUSIONS

Individualised treatment recommendations can be derived using a model that applies clinical prediction guides to the results of randomised trials in order to identify which individual patients are likely to derive a clinically important benefit from the treatment. The resulting individualised prediction-based recommendations require validation by comparison with strategies of treat all or treat none.

Optimized retrieval of primary care clinical prediction rules from MEDLINE to establish a Web-based register

OBJECTIVES

Identifying clinical prediction rules (CPRs) for primary care from electronic databases is difficult. This study aims to identify a search filter to optimize retrieval of these to establish a register of CPRs for the Cochrane Primary Health Care field.

STUDY DESIGN AND SETTING

Thirty primary care journals were manually searched for CPRs. This was compared with electronic search filters using alternative methodologies: (1) textword searching; (2) proximity searching; (3) inclusion terms using specific phrases and truncation; (4) exclusion terms; and (5) combinations of methodologies.

RESULTS

We manually searched 6,344 articles, revealing 41 CPRs. Across the 45 search filters, sensitivities ranged from 12% to 98%, whereas specificities ranged from 43% to 100%. There was generally a trade-off between the sensitivity and specificity of each filter (i.e., the number of CPRs and total number of articles retrieved). Combining textword searching with the inclusion terms (using specific phrases) resulted in the highest sensitivity (98%) but lower specificity (59%) than other methods. The associated precision (2%) and accuracy (60%) were also low.

CONCLUSION

The novel use of combining textword searching with inclusion terms was considered the most appropriate for updating a register of primary care CPRs where sensitivity has to be optimized.

So many databases, such little clarity: Searching the literature for the topic aboriginal

OBJECTIVE

To describe the scope, content, and organization of commonly used medical databases and search strategies, using a search of the topic aboriginal to illustrate the various ways the topic is covered in each of the databases.

DESIGN

Comparison of literature searches.

METHOD

Seven common medical databases were searched using all the MeSH terms that are permutations of aboriginal. A secondary analysis using the "remove duplicates" function in Ovid was done to identify articles specific to each database.

MAIN OUTCOME MEASURES

Number of articles found by each search.

RESULTS

Searching by MeSH terms often produces very different information from that found when searching by text word. A unique term, such as Ojibway, is best found with a text word search. A more general term, such as Aborigines, is best searched by subject using a MeSH term. Many databases can be searched through Ovid and might all use different MeSH terms for the same reference. PubMed default searches that use MeSH terms and text words simultaneously often produce very large numbers of articles. In searching for North American aboriginal using MeSH terms, MEDLINE and PubMed produced the most references, followed by Healthstar. Calculating distinct "all aboriginal" references in EMBASE, Healthstar, and PsycINFO indicated that MEDLINE produced nearly all the articles found in Healthstar. In fact, MEDLINE alone produced 88% of the articles found in MEDLINE and EMBASE and 79% of the articles found in MEDLINE and PsycINFO.

CONCLUSION

Although several researchers and medical librarians have noted that MEDLINE and EMBASE are quite distinct databases, suggesting both need to be searched for a complete search, we did not find that to be the case for the topic aboriginal. The results of this study demonstrate that using MEDLINE produces the most extensive coverage of literature on the topic aboriginal. To fully capture the complete body of available literature on other subjects might require searches of many databases, depending on the topic.

Developing search strategies for clinical practice guidelines in SUMSearch and Google Scholar and assessing their retrieval performance

Background

Information overload, increasing time constraints, and inappropriate search strategies complicate the detection of clinical practice guidelines (CPGs). The aim of this study was to provide clinicians with recommendations for search strategies to efficiently identify relevant CPGs in SUMSearch and Google Scholar.

Methods

We compared the retrieval efficiency (retrieval performance) of search strategies to identify CPGs in SUMSearch and Google Scholar. For this purpose, a two-term GLAD (GuideLine And Disease) strategy was developed, combining a defined CPG term with a specific disease term (MeSH term). We used three different CPG terms and nine MeSH terms for nine selected diseases to identify the most efficient GLAD strategy for each search engine. The retrievals for the nine diseases were pooled. To compare GLAD strategies, we used a manual review of all retrievals as a reference standard. The CPGs detected had to fulfil predefined criteria, e.g., the inclusion of therapeutic recommendations. Retrieval performance was evaluated by calculating so-called diagnostic parameters (sensitivity, specificity, and "Number Needed to Read" [NNR]) for search strategies.

Results

The search yielded a total of 2830 retrievals; 987 (34.9%) in Google Scholar and 1843 (65.1%) in SUMSearch. Altogether, we found 119 unique and relevant guidelines for nine diseases (reference standard). Overall, the GLAD strategies showed a better retrieval performance in SUMSearch than in Google Scholar. The performance pattern between search engines was similar: search strategies including the term "guideline" yielded the highest sensitivity (SUMSearch: 81.5%; Google Scholar: 31.9%), and search strategies including the term "practice guideline" yielded the highest specificity (SUMSearch: 89.5%; Google Scholar: 95.7%), and the lowest NNR (SUMSearch: 7.0; Google Scholar: 9.3).

Conclusion

SUMSearch is a useful tool to swiftly gain an overview of available CPGs. Its retrieval performance is superior to that of Google Scholar, where a search is more time consuming, as substantially more retrievals have to be reviewed to detect one relevant CPG. In both search engines, the CPG term "guideline" should be used to obtain a comprehensive overview of CPGs, and the term "practice guideline" should be used if a less time consuming approach for the detection of CPGs is desired.

Synthesizing medical evidence: systematic reviews and metaanalyses

Systematic reviews and metaanalyses have become increasingly popular ways of summarizing, and sometimes extending, existing medical knowledge. In this review the authors summarize current methods of performing meta-analyses, including the following: formulating a research question; performing a structured literature search and a search for trials not published in the formal medical literature; summarizing and, where appropriate, combining results from several trials; and reporting and presenting results. Topics such as cumulative and Bayesian metaanalysis and metaregression are also addressed. References to textbooks, articles, and Internet resources are also provided. The goal is to assist readers who wish to perform their own metaanalysis or to interpret critically a published example.