Evaluation of methodological search filters--a review

The optimal approach for retrieving systematic reviews was achieved when searching MEDLINE and Epistemonikos in addition to reference checking: a methodological validation study

BACKGROUND

Systematic reviews (SRs) are used to inform clinical practice guidelines and healthcare decision making by synthesising the results of primary studies. Efficiently retrieving as many relevant SRs as possible is challenging with a minimum number of databases, as there is currently no guidance on how to do this optimally. In a previous study, we determined which individual databases contain the most SRs, and which combination of databases retrieved the most SRs. In this study, we aimed to validate those previous results by using a different, larger, and more recent set of SRs.

METHODS

We obtained a set of 100 Overviews of Reviews that included a total of 2276 SRs. SR inclusion was assessed in MEDLINE, Embase, and Epistemonikos. The mean inclusion rates (% of included SRs) and corresponding 95% confidence intervals were calculated for each database individually, as well as for combinations of MEDLINE with each other database and reference checking. Features of SRs not identified by the best database combination were reviewed qualitatively.

RESULTS

Inclusion rates of SRs were similar in all three databases (mean inclusion rates in % with 95% confidence intervals: 94.3 [93.9-94.8] for MEDLINE, 94.4 [94.0-94.9] for Embase, and 94.4 [93.9-94.9] for Epistemonikos). Adding reference checking to MEDLINE increased the inclusion rate to 95.5 [95.1-96.0]. The best combination of two databases plus reference checking consisted of MEDLINE and Epistemonikos (98.1 [97.7-98.5]). Among the 44/2276 SRs not identified by this combination, 34 were published in journals from China, four were other journal publications, three were health agency reports, two were dissertations, and one was a preprint. When discounting the journal publications from China, the SR inclusion rate in the recommended combination (MEDLINE, Epistemonikos and reference checking) was even higher than in the previous study (99.6 vs. 99.2%).

CONCLUSIONS

A combination of databases and reference checking was the best approach to searching for biomedical SRs. MEDLINE and Epistemonikos, complemented by checking the references of the included studies, was the most efficient and produced the highest recall. However, our results point to the presence of geographical bias, because some publications in journals from China were not identified.

STUDY REGISTRATION

https://doi.org/10.17605/OSF.IO/R5EAS (Open Science Framework).

Development and validation of a geographic search filter for MEDLINE (PubMed) to identify studies about Germany

While geographic search filters exist, few of them are validated and there are currently none that focus on Germany. We aimed to develop and validate a highly sensitive geographic search filter for MEDLINE (PubMed) that identifies studies about Germany. First, using the relative recall method, we created a gold standard set of studies about Germany, dividing it into 'development' and 'testing' sets. Next, candidate search terms were identified using (i) term frequency analyses in the 'development set' and a random set of MEDLINE records; and (ii) a list of German geographic locations, compiled by our team. Then, we iteratively created the filter, evaluating it against the 'development' and 'testing' sets. To validate the filter, we conducted a number of case studies (CSs) and a simulation study. For this validation we used systematic reviews (SRs) that had included studies about Germany but did not restrict their search strategy geographically. When applying the filter to the original search strategies of the 17 SRs eligible for CSs, the median precision was 2.64% (interquartile range [IQR]: 1.34%-6.88%) versus 0.16% (IQR: 0.10%-0.49%) without the filter. The median number-needed-to-read (NNR) decreased from 625 (IQR: 211-1042) to 38 (IQR: 15-76). The filter achieved 100% sensitivity in 13 CSs, 85.71% in 2 CSs and 87.50% and 80% in the remaining 2 CSs. In a simulation study, the filter demonstrated an overall sensitivity of 97.19% and NNR of 42. The filter reliably identifies studies about Germany, enhancing screening efficiency and can be applied in evidence syntheses focusing on Germany.

Interventions for people with perceptual disorders after stroke: the PIONEER scoping review, Cochrane systematic review and priority setting project

Background

Stroke often affects recognition and interpretation of information from our senses, resulting in perceptual disorders. Evidence to inform treatment is unclear.

Objective

To determine the breadth and effectiveness of interventions for stroke-related perceptual disorders and identify priority research questions.

Methods

We undertook a scoping review and then Cochrane systematic review. Definitions, outcome prioritisation, data interpretation and research prioritisation were coproduced with people who had perceptual disorders post stroke and healthcare professionals. We systematically searched electronic databases (including MEDLINE, EMBASE, inception to August 2021) and grey literature. We included studies (any design) of interventions for people with hearing, smell, somatosensation, taste, touch or visual perception disorders following stroke. Abstracts and full texts were independently dual reviewed. Data were tabulated, synthesised narratively and mapped by availability, sense and interventions. Research quality was not evaluated. Our Cochrane review synthesised the randomised controlled trial data, evaluated risk of bias (including randomisation, blinding, reporting) and meta-analysed intervention comparisons (vs. controls or no treatment) using RevMan 5.4. We judged certainty of evidence using grading of recommendations, assessment, development and evaluation. Activities of daily living after treatment was our primary outcome. Extended activities of daily living, quality of life, mental health and psychological well-being perceptual functional and adverse event data were also extracted.

Results

We included 80 studies (n = 893): case studies (36/80) and randomised controlled trials (22/80). No stroke survivor or family stakeholder involvement was reported. Studies addressed visual (42.5%, 34/80), somatosensation (35%, 28/80), auditory (8.7%, 7/80) and tactile (7.5%, 6/80) perceptual disorders; some studies focused on 'mixed perceptual disorders' (6.2%, 5/80 such as taste-smell disorders). We identified 93 pharmacological, non-invasive brain stimulation or rehabilitation (restitution, substitution, compensation or mixed) interventions. Details were limited. Studies commonly measured perceptual (75%, 60/80), motor-sensorimotor (40%, 32/80) activities of daily living (22.5%, 18/80) or sensory function (15%, 12/80) outcomes.

Cochrane systematic review

We included 18 randomised controlled trials (n = 541) addressing tactile (3 randomised controlled trials; n = 70), somatosensory (7 randomised controlled trials; n = 196), visual (7 randomised controlled trials; n = 225) and mixed tactile-somatosensory (1 randomised controlled trial; n = 50) disorders. None addressed hearing, taste or smell disorders. One non-invasive brain stimulation, one compensation, 25 restitution and 4 mixed interventions were described. Risk of bias was low for random sequence generation (13/18), attrition (14/18) and outcome reporting (16/18). Perception was the most commonly measured outcome (11 randomised controlled trials); only 7 randomised controlled trials measured activities of daily living. Limited data provided insufficient evidence to determine the effectiveness of any intervention. Confidence in the evidence was low-very low. Our clinical (n = 4) and lived experience (n = 5) experts contributed throughout the project, coproducing a list of clinical implications and research priorities. Top research priorities included exploring the impact of, assessment of, and interventions for post-stroke perceptual disorders.

Limitations

Results are limited by the small number of studies identified and the small sample sizes, with a high proportion of single-participant studies. There was limited description of the perceptual disorders and intervention(s) evaluated. Few studies measured outcomes relating to functional impacts. There was limited investigation of hearing, smell, taste and touch perception disorders.

Conclusion

Evidence informing interventions for perceptual disorders after stroke is limited for all senses.

Future work

Further research, including high-quality randomised controlled trials, to inform clinical practice are required.

Study registration

This study is registered as PROSPERO CRD42019160270.

Funding

This award was funded by the National Institute for Health and Care Research (NIHR) Health Technology Assessment programme (NIHR award ref: NIHR128829) and is published in full in Health Technology Assessment; Vol. 28, No. 69. See the NIHR Funding and Awards Website for further award information.

Development and validation of search filters to retrieve medication discontinuation articles in Medline and Embase

BACKGROUND

Medication discontinuation studies explore the outcomes of stopping a medication compared to continuing it. Comprehensively identifying medication discontinuation articles in bibliographic databases remains challenging due to variability in terminology.

OBJECTIVES

To develop and validate search filters to retrieve medication discontinuation articles in Medline and Embase.

METHODS

We identified medication discontinuation articles in a convenience sample of systematic reviews. We used primary articles to create two reference sets for Medline and Embase, respectively. The reference sets were equally divided by randomization in development sets and validation sets. Terms relevant for discontinuation were identified by term frequency analysis in development sets and combined to develop two search filters that maximized relative recalls. The filters were validated against validation sets. Relative recalls were calculated with their 95% confidences intervals (95% CI).

RESULTS

We included 316 articles for Medline and 407 articles for Embase, from 15 systematic reviews. The Medline optimized search filter combined 7 terms. The Embase optimized search filter combined 8 terms. The relative recalls were respectively 92% (95% CI: 87-96) and 91% (95% CI: 86-94).

CONCLUSIONS

We developed two search filters for retrieving medication discontinuation articles in Medline and Embase. Further research is needed to estimate precision and specificity of the filters.

Validated geographic search filters for bibliographic databases: a scoping review protocol

OBJECTIVE

The purpose of this scoping review is to identify validated geographic search filters and report their methodology and performance measures.

INTRODUCTION

Data on specific geographic areas can be required for evidence syntheses topics, such as the investigation of regional inequalities in health care or to answer context-specific epidemiological questions. Search filters are useful tools for reviewers aiming to identify publications with common characteristics in bibliographic databases. Geographic search filters limit the literature search results to a specific geographic feature (eg, a country or region).

INCLUSION CRITERIA

We will include reports on validated geographic search filters that aim to identify research evidence about a defined geographic area (eg, a country/region or a group of countries/regions).

METHODS

This review will be conducted in accordance with JBI methodology for scoping reviews. The literature search will be conducted in PubMed and Embase. The InterTASC Information Specialists' Sub-Group Search Filter resource and Google Scholar will also be searched. Reports published in any language, from database inception to the present, will be considered for inclusion. Two researchers will independently screen the title, abstract, and full text of the search results. A third reviewer will be consulted in the event of any disagreements. The data extraction will include study characteristics, basic characteristics of the geographical search filter (eg, country/region), and the methods used to develop and validate the search filter. The extracted data will be summarized narratively and presented in a table.

REVIEW REGISTRATION

Open Science Framework https://osf.io/5czhs.

Search strategies (filters) to identify systematic reviews in MEDLINE and Embase

BACKGROUND

Bibliographic databases provide access to an international body of scientific literature in health and medical sciences. Systematic reviews are an important source of evidence for clinicians, researchers, consumers, and policymakers as they address a specific health-related question and use explicit methods to identify, appraise and synthesize evidence from which conclusions can be drawn and decisions made. Methodological search filters help database end-users search the literature effectively with different levels of sensitivity and specificity. These filters have been developed for various study designs and have been found to be particularly useful for intervention studies. Other filters have been developed for finding systematic reviews. Considering the variety and number of available search filters for systematic reviews, there is a need for a review of them in order to provide evidence about their retrieval properties at the time they were developed.

OBJECTIVES

To review systematically empirical studies that report the development, evaluation, or comparison of search filters to retrieve reports of systematic reviews in MEDLINE and Embase.

SEARCH METHODS

We searched the following databases from inception to January 2023: MEDLINE, Embase, PsycINFO; Library, Information Science & Technology Abstracts (LISTA) and Science Citation Index (Web of Science).

SELECTION CRITERIA

We included studies if one of their primary objectives is the development, evaluation, or comparison of a search filter that could be used to retrieve systematic reviews on MEDLINE, Embase, or both.

DATA COLLECTION AND ANALYSIS

Two review authors independently extracted data using a pre-specified and piloted data extraction form using InterTASC Information Specialist Subgroup (ISSG) Search Filter Evaluation Checklist.

MAIN RESULTS

We identified eight studies that developed filters for MEDLINE and three studies that developed filters for Embase. Most studies are very old and some were limited to systematic reviews in specific clinical areas. Six included studies reported the sensitivity of their developed filter. Seven studies reported precision and six studies reported specificity. Only one study reported the number needed to read and positive predictive value. None of the filters were designed to differentiate systematic reviews on the basis of their methodological quality. For MEDLINE, all filters showed similar sensitivity and precision, and one filter showed higher levels of specificity. For Embase, filters showed variable sensitivity and precision, with limited study reports that may affect accuracy assessments. The report of these studies had some limitations, and the assessments of their accuracy may suffer from indirectness, considering that they were mostly developed before the release of the PRISMA 2009 statement or due to their limited scope in the selection of systematic review topics. Search filters for MEDLINE Three studies produced filters with sensitivity > 90% with variable degrees of precision, and only one of them was developed and validated in a gold-standard database, which allowed the calculation of specificity. The other two search filters had lower levels of sensitivity. One of these produced a filter with higher levels of specificity (> 90%). All filters showed similar sensitivity and precision in the external validation, except for one which was not externally validated and another one which was conceptually derived and only externally validated. Search filters for Embase We identified three studies that developed filters for this database. One of these studies developed filters with variable sensitivity and precision, including highly sensitive strategies (> 90%); however, it was not externally validated. The other study produced a filter with a lower sensitivity (72.7%) but high specificity (99.1%) with a similar performance in the external validation.

AUTHORS' CONCLUSIONS

Studies reporting the development, evaluation, or comparison of search filters to retrieve reports of systematic reviews in MEDLINE showed similar sensitivity and precision, with one filter showing higher levels of specificity. For Embase, filters showed variable sensitivity and precision, with limited information about how the filter was produced, which leaves us uncertain about their performance assessments. Newer filters had limitations in their methods or scope, including very focused subject topics for their gold standards, limiting their applicability across other topics. Our findings highlight that consensus guidance on the conduct of search filters and standardized reporting of search filters are needed, as we found highly heterogeneous development methods, accuracy assessments and outcome selection. New strategies adaptable across interfaces could enhance their usability. Moreover, the performance of existing filters needs to be evaluated in light of the impact of reporting guidelines, including the PRISMA 2009, on how systematic reviews are reported. Finally, future filter developments should also consider comparing the filters against a common reference set to establish comparative performance and assess the quality of systematic reviews retrieved by strategies.

Development of a validated search filter for Ovid Embase for degenerative cervical myelopathy

BACKGROUND

Degenerative cervical myelopathy (DCM) is a recently proposed umbrella term for symptomatic cervical spinal cord compression secondary to degeneration of the spine. Currently literature searching for DCM is challenged by the inconsistent uptake of the term 'DCM' with many overlapping keywords and numerous synonyms.

OBJECTIVES

Here, we adapt our previous Ovid medline search filter for the Ovid embase database, to support comprehensive literature searching. Both embase and medline are recommended as a minimum for systematic reviews.

METHODS

References contained within embase identified in our prior study formed a 'development gold standard' reference database (N = 220). The search filter was adapted for embase and checked against the reference database. The filter was then validated against the 'validation gold standard'.

RESULTS

A direct translation was not possible, as medline indexing for DCM and the keywords search field were not available in embase. We also used the 'focus' function to improve precision. The resulting search filter has 100% sensitivity in testing.

DISCUSSION AND CONCLUSION

We have developed a validated search filter capable of retrieving DCM references in embase with high sensitivity. In the absence of consistent terminology and indexing, this will support more efficient and robust evidence synthesis in the field.

Improving the Understanding and Managing of the Quality of Life of Patients With Lung Cancer With Electronic Patient-Reported Outcome Measures: Scoping Review

BACKGROUND

Electronic patient-reported outcome measures (ePROMs) are essential to clinical practice and research. The growth of eHealth technologies has provided unprecedented opportunities to collect information systematically through ePROMs. Although they are widely used in scientific research, more evidence is needed to determine their use and implementation in daily clinical practice. For example, when diagnosed, patients with lung cancer are at an advanced stage of the disease. This entails tremendous burden because of high mortality and losses in the different dimensions of the human being. In this case, monitoring symptoms and other outcomes help improve the patient's quality of life.

OBJECTIVE

ePROMs offered unprecedented opportunities to collect information systematically. Our goal was to demonstrate that ePROMs are more useful in controlling patient symptoms, lung cancer, and overall survival than their alternatives, such as nonelectronic PROMs.

METHODS

This exploratory review considered articles published between 2017 and 2022 identified through searches of PubMed, Scopus, Cochrane, CINAHL, and PsycINFO. We found 5097 articles; after removing the duplicates, we reduced them to 3315. After reading the summary, we were left with 56. Finally, after applying the exclusion criteria, we reviewed 12. The 5-step framework by Arksey and O'Malley was used to refine the initial search results with the following research questions: Do ePROMs help physician-patient communication? To what extent do they improve decision-making? Are institutions and their digitization policies barriers to or facilitators of this process? and What else is needed for routine implementation?

RESULTS

This review included 12 articles. We found that ePROMs are an integrative and facilitative communication tool, highlighting their importance in the relationship between palliative care and medical oncology. ePROMs help assess patient symptoms and functionality more accurately and facilitate clinical decision-making. In addition, it allows for more precise predictions of overall patient survival and the adverse effects of their treatments. The main institutional obstacles are the initial investment, which can be costly, and the data protection policy. However, enablers included better funding through the development of telemedicine, support from institutional leaders to overcome resistance to change, and transparent policies to ensure the safe and secure use of ePROMs.

CONCLUSIONS

Routine collection of remote ePROMs is an effective and valuable strategy for providing real-time clinical feedback. In addition, it provides satisfaction to patients and professionals. Optimizing ePROMs in patients with lung cancer leads to a more accurate view of health outcomes and ensures quality patient follow-up. It also allows us to stratify patients based on their morbidity, creating specific follow-ups for their needs. However, data privacy and security are concerns when using ePROMs to ensure compliance with local entities. At least four barriers were identified: cost, complex programming within health systems, safety, and social and health literacy.

Non-pharmacologic interventions for improving cancer-related fatigue (CRF): A systematic review of systematic reviews and pooled meta-analysis

INTRODUCTION

Literature encloses numerous systematic reviews (SRs) on nonpharmacologic interventions for improving cancer-related fatigue (CRF). The effect of these interventions remains controversial, and the available SRs have not been synthesized yet. We conducted a systematic synthesis of SRs and meta-analysis to determine the effect of nonpharmacologic interventions on CRF in adults.

MATERIAL AND METHODS

We systematically searched 4 databases. The effect sizes (standard mean difference) were quantitatively pooled using a random-effects model. Chi-squared (Q) and I-square statistics (I²) tested the heterogeneity.

RESULTS

We selected 28 SRs, including 35 eligible meta-analyses. The pooled effect size (standard mean difference, 95% CI) was -0.67 (-1.16, -0.18). The subgroup analysis by types of interventions showed a significant effect in all the investigated approaches (complementary integrative medicine, physical exercise, self-management/e-health interventions).

CONCLUSIONS

There is evidence that nonpharmacologic interventions are associated with CRF reduction. Future research should focus on testing these interventions on specific population clusters and trajectories.

PROSPERO REGISTRATION

CRD42020194258.

A Systematic Review of Systematic Reviews and a Pooled Meta-Analysis on Complementary and Integrative Medicine for Improving Cancer-Related Fatigue

PURPOSE

Evidence supporting complementary and integrative medicine (CIM) for improving cancer-related fatigue (CRF) is still fragmented. This study therefore critically appraised all the systematic reviews (SRs) regarding the effectiveness of CIM in mitigating CRF in adults.

METHODS

A systematic review of SRs and a meta-analysis were conducted in 4 databases. The effect sizes of the included SRs were quantitatively pooled (standardized mean difference [SMD]; 95% CI) using a random-effects model. Heterogeneity was tested by using χ² (Q) tests and I² statistics.

FINDINGS

Twenty-two SRs met the inclusion criteria, and results from 20 SRs underwent meta-analysis. The pooled significant estimate of fatigue reduction was as follows: SMD, -0.50; 95% CI, -0.67 to -0.32; P < 0.001. The subgroup analysis based on the type of CIM intervention revealed that the approach showing higher effects in reducing fatigue thus far is acupuncture: SMD, -0.99; 95% CI = -1.37 to -0.62, P < 0.001; I² = 84%. CIM therapies showed a significant reduction of fatigue in patients with breast cancer: SMD, -0.46; 95% CI, -0.69 to -0.23; P < 0.001; I² = 82%.

IMPLICATIONS

CIM interventions showed effectiveness in reducing CRF. Subgroup analysis suggested some potential influencing, such as tumor type and specific CIM therapy factors, that require in-depth assessment in future research. Study protocol registration: PROSPERO CRD42020194254.

The ScHARR LMIC filter: Adapting a low- and middle-income countries geographic search filter to identify studies on preterm birth prevention and management

Search filters are used to find evidence on specific subjects. Performance of filters can be varied and may need adapting to meet the needs of research topics. There are limited geographic search filters available, and only one pertaining to low- and middle-income countries (LMICs). When searching for literature on preterm birth prevention and management in LMICs for a research project at the School of Health and Related Research (ScHARR), we made use of the Cochrane Effective Practice and Organisation of Care (EPOC) LMIC geographic search filter for the databases; Ovid MEDLINE, Ovid Embase, Cochrane Library. During screening following a broad scoping search in Ovid MEDLINE, it was found that the EPOC LMIC filter did not identify a relevant study. Adaptations were made to the LMIC geographic search filter to maximise retrieval and identify the missing study. Institution was included as a search field, and the search terms high burden or countdown countries were added. The filter was translated for the databases; Ovid Embase, Cochrane Library, Ovid PsycINFO, and CINAHL via EBSCO. The adapted ScHARR LMIC filter is a non-validated 1st generation filter which increases the sensitivity of the EPOC LMIC search filter. Validating the filter would confirm its retrieval performance and benefit information professionals, researchers, and health professionals. We recommend that the ScHARR LMIC filter is used to improve sensitivity of the Cochrane EPOC LMIC filter and reduce the risk of missing relevant studies.

Psychometric Properties of Lower Limb Somatosensory Function and Body Awareness Outcome Measures in Children with Upper Motor Neuron Lesions: A Systematic Review

PURPOSE

A systematic review of the psychometric properties and feasibility of outcome measures assessing lower limb somatosensory function and body awareness in children with upper motor neuron lesion.

METHODS

We followed the COnsensus-based Standards for the selection of health Measurement INstruments guidelines. Two raters independently judged the quality and risk of bias of each study. Data synthesis was performed, and aspects of feasibility were extracted.

RESULTS

Twelve studies investigated eleven somatosensory function measures quantifying four modalities and eight body awareness measures quantifying two modalities. The best evidence synthesis was very low to low for somatosensory function modalities and low for body awareness modalities. Few feasibility aspects were reported (e.g., the percentage or minimum age of participants able to perform the tests).

CONCLUSION

Current evidence on the psychometric characteristics of somatosensory function and body awareness outcome measures are relatively sparse. Further research on psychometric properties and practical application is needed.

Development and evaluation of a search filter to identify prognostic factor studies in Ovid MEDLINE

Background

Systematic reviews (SRs) are valuable resources as they address specific clinical questions by summarizing all existing relevant studies. However, finding all information to include in systematic reviews can be challenging. Methodological search filters have been developed to find articles related to specific clinical questions. To our knowledge, no filter exists for finding studies on the role of prognostic factor (PF). We aimed to develop and evaluate a search filter to identify PF studies in Ovid MEDLINE that has maximum sensitivity.

Methods

We followed current recommendations for the development of a search filter by first identifying a reference set of PF studies included in relevant systematic reviews on the topic, and by selecting search terms using a word frequency analysis complemented with an expert panel discussion. We evaluated filter performance using the relative recall methodology.

Results

We constructed a reference set of 73 studies included in six systematic reviews from a larger sample. After completing a word frequency analysis using the reference set studies, we compiled a list of 80 of the frequent methodological terms. This list of terms was evaluated by the Delphi panel for inclusion in the filter, resulting in a final set of 8 appropriate terms. The consecutive connection of these terms with the Boolean operator OR produced the filter. We then evaluated the filter using the relative recall method against the reference set, comparing the references included in the SRs with our new search using the filter. The overall sensitivity of the filter was calculated to be 95%, while the overall specificity was 41%. The precision of the filter varied considerably, ranging from 0.36 to 17%. The NNR (number needed to read) value varied largely from 6 to 278. The time saved by using the filter ranged from 13–70%.

Conclusions

We developed a search filter for OVID-Medline with acceptable performance that could be used in systematic reviews of PF studies. Using this filter could save as much as 40% of the title and abstract screening task. The specificity of the filter could be improved by defining additional terms to be included, although it is important to evaluate any modification to guarantee the filter is still highly sensitive.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12874-022-01595-9.

Conducting Systematic Reviews of the Quality and Psychometric Properties of Health-Related Measurement Instruments: Finding the Right Tool for the Job

Many health-related measurement instruments have been developed to measure psychological constructs and whilst several instruments are usually available for a particular study, finding the right tool for the job is important. Systematic reviews of measurement properties of instruments have long been identified as a valuable strategy to ensure that we select the right tool to assess mental health. There are many important steps and procedures to guide these types of systematic reviews to find the "best fit" and this paper summarizes some of these key processes and steps. The selection of instrument(s) to use should be made considering the most recent comprehensive review of the quality of the outcome measurement instrument based on unbiased assessment of its psychometric properties, responsiveness, and generalizability of results. Researchers planning to conduct a systematic review of health-related measurement instruments should design the review beforehand using standardized frameworks. Conducting systematic reviews of the quality and psychometric properties of health-related measurement instruments is important to ensure we choose the best tool for the research question and target population.

Patient-based benefit-risk assessment of medicines: development, refinement, and validation of a content search strategy to retrieve relevant studies

Introduction:

Poor indexing and inconsistent use of terms and keywords may prevent efficient retrieval of studies on the patient-based benefit-risk assessment (BRA) of medicines. We aimed to develop and validate an objectively derived content search strategy containing generic search terms that can be adapted for any search for evidence on patient-based BRA of medicines for any therapeutic area.

Methods:

We used a robust multistep process to develop and validate the content search strategy: (1) we developed a bank of search terms derived from screening studies on patient-based BRA of medicines in various therapeutic areas, (2) we refined the proposed content search strategy through an iterative process of testing sensitivity and precision of search terms, and (3) we validated the final search strategy in PubMed by firstly using multiple sclerosis as a case condition and secondly computing its relative performance versus a published systematic review on patient-based BRA of medicines in rheumatoid arthritis.

Results:

We conceptualized a final search strategy to retrieve studies on patient-based BRA containing generic search terms grouped into two domains, namely the patient and the BRA of medicines (sensitivity 84%, specificity 99.4%, precision 20.7%). The relative performance of the content search strategy was 85.7% compared with a search from a published systematic review of patient preferences in the treatment of rheumatoid arthritis. We also developed a more extended filter, with a relative performance of 93.3% when compared with a search from a published systematic review of patient preferences in lung cancer.

Determinants in Stakeholder Opinions About Telemedicine in Palliative Care: A Scoping Review

Background: The use of Telemedicine is growing, and its application in palliative medicine may facilitate patient care and be a solution to the growing pressures on hospital services in these pandemic times. Aim: The main objective of this review is to describe the current use of telemedicine in palliative care and assess stakeholders' views on the initiatives that have been implemented worldwide regarding digital service standards. Materials and Methods: Articles published between 2010 and 2020 were identified through PubMed, SCOPUS, Web of Science, and Google Scholar searches. We used Arksey and O'Malley's five-step framework to delimit and guide the initial search results. Results: The search identified 291 articles, of which 16 are included in this review. The selected studies were sufficiently detailed to allow their evaluation and answer our research questions. In addition, Telemedicine was used for patient and caregiver support and professional education. Conclusions: The use of Telemedicine for patient and caregiver support and professional education. Telemedicine empowers patients and increases their functional capacity. The imperative need to dictate implementation policies and ethical issues are some of the pending questions. In countries where a Telemedicine project has been initiated, it is valued as a good option for continuity of care, but all those involved would like face-to-face contact first, even if it is not mandatory.

Effects from physical exercise on reduced cancer-related fatigue: a systematic review of systematic reviews and meta-analysis

BACKGROUND

Cancer-related fatigue is one of the most prevalent and distressing symptoms among cancer patients, resulting in a great cancer research challenge. Numerous systematic reviews of physical training interventions have been conducted to find the most effective approach. However, evidence remains fragmented, and in which cancer population physical training is more effective than other populations is still unclear. Thus, this study critically appraised systematic reviews and meta-analyses on physical training to reduce adults' cancer-related fatigue.

METHODS

A systematic review of systematic reviews and meta-analysis (PROSPERO: CRD42020189049), assessing the efficacy of exercise training for reducing cancer-related fatigue in adults, was conducted in PubMed, CINAHL, Cochrane Database of Systematic Reviews, and the Database of Abstracts of Reviews of Effects, and Pedro. The selected studies (standardized mean difference, SMD; 95%CI), was quantitatively pooled using a random-effects model. Heterogeneity was tested using chi-squared (Q) and I-square statistics (I²).

RESULTS

Of 1438 identified articles, 11 met the inclusion criteria, and ten were meta-analyzed. The results yielded a positive effect of physical training on fatigue in all cancer populations, SMD = -0.33 (-0.43, -0.23). Subgroup analysis based on tumor localization showed a slightly higher physical training effect on fatigue in adults with breast cancer, SMD = -0.36 (-0.57, -0.15), and prostate cancer SMD = -0.34 (-0.45, -.0.22).

CONCLUSIONS

Our analysis demonstrated some potential improvement in cancer-related fatigue in adult patients undergoing physical training during and after cancer treatments, particularly in patients with breast or prostate cancer.

Translation and validation of PubMed and Embase search filters for identification of systematic reviews, intervention studies, and observational studies in the field of first aid

Objective

The aim of this project was to validate search filters for systematic reviews, intervention studies, and observational studies translated from Ovid MEDLINE and Embase syntax and used for searches in PubMed and Embase.com during the development of evidence summaries supporting first aid guidelines. We aimed to achieve a balance among recall, specificity, precision, and number needed to read (NNR).

Methods

Reference gold standards were constructed per study type derived from existing evidence summaries. Search filter performance was assessed through retrospective searches and measurement of relative recall, specificity, precision, and NNR when using the translated search filters. Where necessary, search filters were optimized. Adapted filters were validated in separate validation gold standards.

Results

Search filters for systematic reviews and observational studies reached recall of ≥85% in both PubMed and Embase. Corresponding specificities for systematic review filters were ≥96% in both databases, with a precision of 9.7% (NNR 10) in PubMed and 5.4% (NNR 19) in Embase. For observational study filters, specificity, precision, and NNR were 68%, 2%, and 51 in PubMed and 47%, 0.8%, and 123 in Embase, respectively. These filters were considered sufficiently effective. Search filters for intervention studies reached a recall of 85% and 83% in PubMed and Embase, respectively. Optimization led to recall of ≥95% with specificity, precision, and NNR of 49%, 1.3%, and 79 in PubMed and 56%, 0.74%, and 136 in Embase, respectively.

Conclusions

We report validated filters to search for systematic reviews, observational studies, and intervention studies in guideline projects in PubMed and Embase.com.

Development of an efficient search filter to retrieve systematic reviews from PubMed

Objective:

Locating systematic reviews is essential for clinicians and researchers when creating or updating reviews and for decision-making in health care. This study aimed to develop a search filter for retrieving systematic reviews that improves upon the performance of the PubMed systematic review search filter.

Methods:

Search terms were identified from abstracts of reviews published in Cochrane Database of Systematic Reviews and the titles of articles indexed as systematic reviews in PubMed. Both the precision of the candidate terms and the number of systematic reviews retrieved from PubMed were evaluated after excluding the subset of articles retrieved by the PubMed systematic review filter. Terms that achieved a precision greater than 70% and relevant publication types indexed with MeSH terms were included in the filter search strategy.

Results:

The search strategy used in our filter added specific terms not included in PubMed's systematic review filter and achieved a 61.3% increase in the number of retrieved articles that are potential systematic reviews. Moreover, it achieved an average precision that is likely greater than 80%.

Conclusions:

The developed search filter will enable users to identify more systematic reviews from PubMed than the PubMed systematic review filter with high precision.

The NICE OECD countries' geographic search filters: Part 2-validation of the MEDLINE and Embase (Ovid) filters

Objective:

We previously developed draft MEDLINE and Embase (Ovid) geographic search filters for Organisation for Economic Co-operation and Development (OECD) countries to assess their feasibility for finding evidence about the countries. Here, we describe the validation of these search filters.

Methods:

We identified OECD country references from thirty National Institute for Health and Care Excellence (NICE) guidelines to generate gold standard sets for MEDLINE (n=2,065) and Embase (n=2,023). We validated the filters by calculating their recall against these sets. We then applied the filters to existing search strategies for three OECD-focused NICE guideline reviews (NG103 on flu vaccination, NG140 on abortion care, and NG146 on workplace health) to calculate the filters' impact on the number needed to read (NNR) of the searches.

Results:

The filters both achieved 99.95% recall against the gold standard sets. Both filters achieved 100% recall for the three NICE guideline reviews. The MEDLINE filter reduced NNR from 256 to 232 for the NG103 review, from 38 to 27 for the NG140 review, and from 631 to 591 for the NG146 review. The Embase filter reduced NNR from 373 to 341 for the NG103 review, from 101 to 76 for the NG140 review, and from 989 to 925 for the NG146 review.

Conclusion:

The NICE OECD countries' search filters are the first validated filters for the countries. They can save time for research topics about OECD countries by finding the majority of evidence about OECD countries while reducing search result volumes in comparison to no filter use.

Educational interventions to improve literature searching skills in the health sciences: a scoping review

Objective:

The authors reviewed educational interventions for improving literature searching skills in the health sciences.

Methods:

We performed a scoping review of experimental and quasi-experimental studies published in English and German, irrespective of publication year. Targeted outcomes were objectively measurable literature searching skills (e.g., quality of search strategy, study retrieval, precision). The search methods consisted of searching databases (CINAHL, Embase, MEDLINE, PsycINFO, Web of Science), tracking citations, free web searching, and contacting experts. Two reviewers performed screening and data extraction. To evaluate the completeness of reporting, the Template for Intervention Description and Replication (TIDieR) was applied.

Results:

We included 6 controlled trials and 8 pre-post trials from the 8,484 references that we screened. Study participants were students in various health professions and physicians. The educational formats of the interventions varied. Outcomes clustered into 2 categories: (1) developing search strategies (e.g., identifying search concepts, selecting databases, applying Boolean operators) and (2) database searching skills (e.g., searching PubMed, MEDLINE, or CINAHL). In addition to baseline and post-intervention measurement, 5 studies reported follow-up. Almost all studies adequately described their intervention procedures and delivery but did not provide access to the educational material. The expertise of the intervention facilitators was described in only 3 studies.

Conclusions:

The results showed a wide range of study populations, interventions, and outcomes. Studies often lacked information about educational material and facilitators' expertise. Further research should focus on intervention effectiveness using controlled study designs and long-term follow-up. To ensure transparency, replication, and comparability, studies should rigorously describe their intervention.

The NICE OECD countries' geographic search filters: Part 1-methodology for developing the draft MEDLINE and Embase (Ovid) filters

Objective:

There are no existing validated search filters for the group of 37 Organisation for Economic Co-operation and Development (OECD) countries. This study describes how information specialists from the United Kingdom's National Institute for Health and Care Excellence (NICE) developed and evaluated novel OECD countries’ geographic search filters for MEDLINE and Embase (Ovid) to improve literature search effectiveness for evidence about OECD countries.

Methods:

We created the draft filters using an alternative approach to standard filter construction. They are composed entirely of geographic subject headings and are designed to retain OECD country evidence by excluding non-OECD country evidence using the NOT Boolean operator. To evaluate the draft filters’ effectiveness, we used MEDLINE and Embase literature searches for three NICE guidelines that retrieved >5,000 search results. A 10% sample of the excluded references was screened to check that OECD country evidence was not inadvertently excluded.

Results:

The draft MEDLINE filter reduced results for each NICE guideline by 9.5% to 12.9%. In Embase, search results were reduced by 10.7% to 14%. Of the sample references, 7 of 910 (0.8%) were excluded inadvertently. These references were from a guideline about looked-after minors that concerns both OECD and non-OECD countries.

Conclusion:

The draft filters look promising—they reduced search result volumes while retaining most OECD country evidence from MEDLINE and Embase. However, we advise caution when using them in topics about both non-OECD and OECD countries. We have created final versions of the search filters and will validate them in a future study.

A systematic review of systematic reviews and pooled meta-analysis on pharmacological interventions to improve cancer-related fatigue

BACKGROUND

Evidence regarding the pharmacological interventions to manage cancer-related fatigue (CRF) is currently synthesized in several systematic reviews, portraying a fragmented literature synthesis. Thus, we aimed to critically appraise the available systematic reviews on pharmacological intervention for improving CRF in adult cancer patients.

METHODS

Three databases were systematically searched from January 2010 to July 2020. The pooled meta-analyses' effect sizes (standardized mean difference, SMD) were quantitatively pooled using a random-effects model. Chi-squared (Q) and I-square statistics (I²) tested the heterogeneity.

RESULTS

The SMD of the effect of psychostimulants on CRF was -0.20 (95% CI: -0.32, 0.08; p < 0.0001), along with significant higher improvement of fatigue (SMD=-0.69; 95% CI=-1.29, -0,09, p < 0.0001) after methylphenidate administration. No statistical differences were found in the occurrences of adverse events between methylphenidate and placebo.

CONCLUSIONS

This study corroborated that psychostimulant therapy may be moderately effective in reducing CRF. Scarce evidence on the short- and long-term adverse events.

PROSPERO

CRD42020181879 (registration date: 26/07/2020).

Validation of a search strategy for randomized clinical trials related to periodontitis

Background

Systematic reviews, considered the gold standard for the assessment of scientific evidence, may present conflicting findings for the same clinical issue, and such dissent may be justified by the forms of elaboration of the electronic search strategy. This paper aims to validate a search strategy to identify randomized clinical trials related to periodontitis. A gold standard reference set was developed to validate the identified clinical trials using the relative recall method. The choice of periodontitis is due to the fact that this disease has a high prevalence among chronic non-communicable diseases, is considered the second most common oral disease in the world, is associated with several health problems, such as cardiovascular diseases and diabetes, and, principally, has not been investigated sufficiently to prevent possible damages resulting from it.

Methods

A validation study was developed in MEDLINE/PubMed. In Stage 1, a methodological filter recommended by the Cochrane Collaboration to identify randomized clinical trials was applied. Stage 2 identified articles related only to periodontitis (gold standard reference set) from among the articles retrieved using the eligibility criteria. In Stage 3, a search statement for the retrieval of periodontitis-related articles was elaborated by experts. Stage 4 defined the proposed search strategy comprising of the combination of the search statement developed with the aforementioned methodological filter and subsequent application in MEDLINE/PubMed. The obtained data were analyzed using the set of articles identified in Stage 2, as the gold standard reference set. The following performance values were calculated - sensitivity, specificity, accuracy, and number needed to read - with their respective 95% confidence interval (95%CI).

Results

The search strategy under evaluation compared to the gold-standard showed a sensitivity of 93.2% (_95%CI, 83.8–97.3), specificity of 99.9% (_95%CI 99.8–99.9), and a precision of 77.5% (_95%CI, 66.48–85.63). In addition, the number needed to read was 1.3.

Conclusion

According to the proposed methodological approach, the search strategy under evaluation performed well in the identification of randomized clinical trials related to periodontitis.

Supplementary Information

The online version contains supplementary material available at 10.1186/s13690-021-00560-0.

Optimizing a literature surveillance strategy to retrieve sound overall prognosis and risk assessment model papers

OBJECTIVE

Our aim was to develop an efficient search strategy for prognostic studies and clinical prediction guides (CPGs), optimally balancing sensitivity and precision while independent of MeSH terms, as relying on them may miss the most current literature.

MATERIALS AND METHODS

We combined 2 Hedges-based search strategies, modified to remove MeSH terms for overall prognostic studies and CPGs, and ran the search on 269 journals. We read abstracts from a random subset of retrieved references until ≥ 20 per journal were reviewed and classified them as positive when fulfilling standardized quality criteria, thereby assembling a standard dataset used to calibrate the search strategy. We determined performance characteristics of our new search strategy against the Hedges standard and performance characteristics of published search strategies against the standard dataset.

RESULTS

Our search strategy retrieved 16 089 references from 269 journals during our study period. One hundred fifty-four journals yielded ≥ 20 references and ≥ 1 prognostic study or CPG. Against the Hedges standard, the new search strategy had sensitivity/specificity/precision/accuracy of 84%/80%/2%/80%, respectively. Existing published strategies tested against our standard dataset had sensitivities of 36%-94% and precision of 5%-10%.

DISCUSSION

We developed a new search strategy to identify overall prognosis studies and CPGs independent of MeSH terms. These studies are important for medical decision-making, as they identify specific populations and individuals who may benefit from interventions.

CONCLUSION

Our results may benefit literature surveillance and clinical guideline efforts, as our search strategy performs as well as published search strategies while capturing literature at the time of publication.

The NICE MEDLINE and Embase (Ovid) health apps search filters: development of validated filters to retrieve evidence about health apps

OBJECTIVES

Health apps are software programs that are designed to prevent, diagnose, monitor, or manage conditions. Inconsistent terminology for apps is used in research literature and bibliographic database subject headings. It can therefore be challenging to retrieve evidence about them in literature searches. Information specialists at the United Kingdom's National Institute for Health and Care Excellence (NICE) have developed novel validated search filters to retrieve evidence about apps from MEDLINE and Embase (Ovid).

METHODS

A selection of medical informatics journals was hand searched to identify a "gold standard" (GS) set of references about apps. The GS set was divided into a development and validation set. The filters' search terms were derived from and tested against the development set. An external development set containing app references from published NICE products was also used to inform the development of the filters. The filters were then validated using the validation set. Target recall was >90 percent. The filters' overall recall, specificity, and precision were calculated using all the references identified from the hand search.

RESULTS

Both filters achieved 98.6 percent recall against their validation sets. Overall, the MEDLINE filter had 98.8 percent recall, 71.3 percent specificity, and 22.6 percent precision. The Embase filter had 98.6 percent recall, 74.9 percent specificity, and 24.5 percent precision.

CONCLUSIONS

The NICE health apps search filters retrieve evidence about apps from MEDLINE and Embase with high recall. They can be applied to literature searches to retrieve evidence about the interventions by information professionals, researchers, and clinicians.

"It has no meaning to me." How do researchers understand the effectiveness of literature searches? A qualitative analysis and preliminary typology of understandings

This study aimed to address the question: what does "effectiveness" mean to researchers in the context of literature searching for systematic reviews? We conducted a thematic analysis of responses to an e-mail survey. Eighty-nine study authors, whose studies met inclusion in a recent review (2018), were contacted via e-mail and asked three questions; one directly asking the question: in literature searching, what does effective (or effectiveness in) literature searching mean to you? Thirty-eight (46%) responses were received from diverse professional groups, including: literature searchers, systematic reviewers, clinicians and researchers. A shared understanding of what effectiveness means was not identified. Instead, five themes were developed from data: (a) effectiveness is described as a metric; (b) effectiveness is a balance between metrics; (c) effectiveness can be categorized by search purpose; (d) effectiveness is an outcome; and, (e) effectiveness is an experimental concept. We propose that these themes constitute a preliminary typology of understandings. No single definition of effectiveness was identified. The proposed typology suggests that different researchers have differing understandings of effectiveness. This could lead to uncertainty as to the aim and the purpose of literature searches and confusion about the outcomes. The typology offers a potential route for further exploration.

Development and validation of study filters for identifying controlled non-randomized studies in PubMed and Ovid MEDLINE

A retrospective analysis published by the German Institute for Quality and Efficiency in Health Care (IQWiG) in 2018 concluded that no filter for non-randomized studies (NRS) achieved sufficient sensitivity (≥92%), a precondition for comprehensive information retrieval. New NRS filters are therefore required, taking into account the challenges related to this study type. Our evaluation focused on the development of study filters for NRS with a control group ("controlled NRS"), as this study type allows the calculation of an effect size. In addition, we assumed that due to the more explicit search syntax, controlled NRS are easier to identify than non-controlled ones, potentially resulting in better performance measures of study filters for controlled NRS. Our aim was to develop study filters for identifying controlled NRS in PubMed and Ovid MEDLINE. We developed two new search filters that can assist clinicians and researchers in identifying controlled NRS in PubMed and Ovid MEDLINE. The reference set was based on 2110 publications in Medline extracted from 271 Cochrane reviews and on 4333 irrelevant references. The first filter maximizes sensitivity (92.42%; specificity 79.67%, precision 68.49%) and should be used when a comprehensive search is needed. The second filter maximizes specificity (92.06%; precision 82.98%, sensitivity 80.94%) and should be used when a more focused search is sufficient.

Patient/service user involvement in medical education: A best evidence medical education (BEME) systematic review: BEME Guide No. 58

Background: The extent to which patients and service users are involved in medical education varies widely. There is a need for an up to date systematic review of the literature that examines what involvement (description), the potential outcome of such involvement (justification) and 'why' such involvement impacts students (clarification).Methods: Systematic searches of four databases were undertaken. Citations were screened and consensus reached for inclusion/exclusion of studies. Quality of study design and interventional presentation were assessed.Results: Of the 39 studies included in the review, 4 studies were encounter based, 17 sharing experiences, 16 with patients involved in teaching, 2 studies describing consumers as tutors, and none with involvement at the institutional level. Outcomes in terms of benefits to learners included increased empathy and understanding of illness as experienced by patients, improved communication with patients and a greater understanding of patient-center care. Educational quality assessment showed specific weaknesses in theoretical underpinning, curriculum outcomes, content or pedagogy.Conclusions: Patients can enrich medical education by allowing learners to explore patient-centered perspectives in holistic care. For educators this review highlights the lack of an underpinning conceptual basis for which to translate theory into practice.

Trends in Scientific Reports on Cartilage Bioprinting: Scoping Review

Background

Satisfactory therapeutic strategies for cartilaginous lesion repair do not yet exist. This creates a challenge for surgeons and biomedical engineers and leads them to investigate the role of bioprinting and tissue engineering as viable treatments through orthopedic surgery, plastic surgery, and otorhinolaryngology. Recent increases in related scientific literature suggest that bioprinted cartilage may develop into a viable solution.

Objective

The objectives of this review were to (1) synthesize the scientific advances published to date, (2) identify unresolved technical problems regarding human application, and (3) identify more effective ways for the scientific community to transfer their findings to clinicians.

Methods

This scoping review considered articles published between 2009 and 2019 that were identified through searching PubMed, Scopus, Web of Science, and Google Scholar. Arksey and O'Malley’s five-step framework was used to delimit and direct the initial search results, from which we established the following research questions: (1) What do authors of current research say about human application? (2) What necessary technical improvements are identified in the research? (3) On which issues do the authors agree? and (4) What future research priorities emerge in the studies? We used the Cohen kappa statistic to validate the interrater reliability.

Results

The 13 articles included in the review demonstrated the feasibility of cartilage bioprinting in live animal studies. Some investigators are already considering short-term human experimentation, although technical limitations still need to be resolved. Both the use and manufacturing process of stem cells need to be standardized, and a consensus is needed regarding the composition of hydrogels. Using on-site printing strategies and predesigned implants may allow techniques to adapt to multiple situations. In addition, the predictive capacity of implant behavior may lead to optimal results.

Conclusions

Cartilage bioprinting for surgical applications is nearing its initial use in humans. Current research suggests that surgeons will soon be able to replace damaged tissue with bioprinted material.

A failed attempt at developing a search filter for systematic review methodology articles in Ovid Embase

OBJECTIVES

This paper describes the development, execution, and subsequent failure of an attempt to create an Ovid Embase search filter for locating systematic review methodology articles.

METHODS

The authors devised a work plan, based on best practices, for search filter development that has been outlined in the literature. Three reference samples were gathered by identifying the OVID Embase records for specific articles that were included in the PubMed Systematic Review Methods subset. The first sample was analyzed to develop a set of keywords and subject headings to include in the search filter. The second and third samples would have been used to calibrate the search filter and to calculate filter sensitivity and precision, respectively.

RESULTS

Technical shortcomings, database indexing practices, and the fuzzy nature of keyword terminology relevant to the topic prevented us from designing the search filter.

CONCLUSION

Creating a search filter to identify systematic review methodology articles in Ovid Embase is not possible at this time.

Development and validation of search filters to find articles on palliative care in bibliographic databases

BACKGROUND

Healthcare professionals and researchers in the field of palliative care often have difficulties finding relevant articles in online databases. Standardized search filters may help improve the efficiency and quality of such searches, but prior developed filters showed only moderate performance.

AIM

To develop and validate a specific search filter and a sensitive search filter for the field of palliative care.

DESIGN

We used a novel, objective method for search filter development. First, we created a gold standard set. This set was split into three groups: term identification, filter development, and filter validation set. After creating the filters in PubMed, we translated the filters into search filters for Ovid MEDLINE, Embase, CINAHL, PsychINFO, and Cochrane Library. We calculated specificity, sensitivity and precision of both filters.

RESULTS

The specific filter had a specificity of 97.4%, a sensitivity of 93.7%, and a precision of 45%. The sensitive filter had a sensitivity of 99.6%, a specificity of 92.5%, and a precision of 5%.

CONCLUSION

Our search filters can support literature searches in the field of palliative care. Our specific filter retrieves 93.7% of relevant articles, while 45% of the retrieved articles are relevant. This filter can be used to find answers to questions when time is limited. Our sensitive filter finds 99.6% of all relevant articles and may, for instance, help conducting systematic reviews. Both filters perform better than prior developed search filters in the field of palliative care.

The Embase UK filter: validation of a geographic search filter to retrieve research about the UK from OVID Embase

BACKGROUND

The authors developed a validated geographic search filter to retrieve research about the United Kingdom (UK) from OVID Embase. It was created to be used alongside their previously published OVID MEDLINE UK filter in systematic literature searches for context-sensitive topics.

OBJECTIVES

To develop a validated geographic search filter to retrieve research about the UK from OVID Embase.

METHODS

The Embase UK filter was translated from the MEDLINE UK filter. A gold standard set of references was generated using the relative recall method. The set contained references to publications about the UK that had informed National Institute for Health and Care Excellence (NICE) guidance and it was used to validate the filter. Recall, precision and number-needed-to-read (NNR) were calculated using a case study.

RESULTS

The validated Embase UK filter demonstrated 99.8% recall against the references with UK identifiers in the gold standard set. In the case study, the Embase UK filter demonstrated 98.5% recall, 7.6% precision and a NNR of 13.

CONCLUSION

The Embase UK filter can be used alongside the MEDLINE UK filter. The filters have the potential to save time and associated resource costs when they are used for context-sensitive topics that require research about UK settings.

Search strategies to identify observational studies in MEDLINE and Embase

BACKGROUND

Systematic reviews are essential for decision-making. Systematic reviews on observational studies help answer research questions on aetiology, risk, prognosis, and frequency of rare outcomes or complications. However, identifying observational studies as part of systematic reviews efficiently is challenging due to poor and inconsistent indexing in literature databases. Search strategies that include a methodological filter focusing on study design of observational studies might be useful for improving the precision of the search performance.

OBJECTIVES

To assess the sensitivity and precision of a search strategy with a methodological filter to identify observational studies in MEDLINE and Embase.

SEARCH METHODS

We searched MEDLINE (1946 to April 2018), Embase (1974 to April 2018), CINAHL (1937 to April 2018), the Cochrane Library (1992 to April 2018), Google Scholar and Open Grey in April 2018, and scanned reference lists of articles.

SELECTION CRITERIA

Studies using a relative recall approach, i.e. comparing sensitivity or precision of a search strategy containing a methodological filter to identify observational studies in MEDLINE and Embase against a reference standard, or studies that compared two or more methodological filters.

DATA COLLECTION AND ANALYSIS

Two review authors independently screened articles, extracted relevant information and assessed the quality of the search strategies using the InterTASC Information Specialists' Sub-Group (ISSG) Search Filter Appraisal Checklist.

MAIN RESULTS

We identified two eligible studies reporting 18 methodological filters. All methodological filters in these two studies were developed using terms from the reference standard records.The first study evaluated six filters for retrieving observational studies of surgical interventions. The study reported on six filters: one Precision Terms Filter (comprising terms with higher precision while maximum sensitivity was maintained) and one Specificity Terms Filter (comprising terms with higher specificity while maximum sensitivity was maintained), both of which were adapted for MEDLINE, for Embase, and for combined MEDLINE/Embase searches. The study reported one reference standard consisting of 217 articles from one systematic review of which 83.9% of the included studies were case seriesThe second study reported on 12 filters for retrieving comparative non-randomised studies (cNRSs) including cohort, case-control, and cross-sectional studies. This study reported on 12 filters using four different approaches: Fixed method A (comprising of a fixed set of controlled vocabulary (CV) words), Fixed method B (comprising a fixed set of CV words and text words (TW)), Progressive method (CV) (a random choice of study design-related CV terms), and Progressive method (CV or TW) (a random choice of study design-related CV terms, and title and abstracts-based TWs). The study reported four reference standards consisting of 89 cNRSs from four systematic reviews.The six methodological filters developed from the first study reported sensitivity of 99.5% to 100% and precision of 16.7% to 21.1%. The Specificity Terms Filter for combined MEDLINE/Embase was preferred because it had higher precision and equal sensitivity to the Precision Terms Filter. The 12 filters from the second study reported lower sensitivity (48% to 100%) and much lower precision (0.09% to 4.47%). The Progressive method (CV or TW) had the highest sensitivity.There were methodological limitations in both included studies. The first study used one surgical intervention-focused systematic review thus limiting the generalizability of findings. The second study used four systematic reviews but with less than 100 studies. The external validation was performed only on Specificity Terms Filter from the first study Both studies were published 10 years ago and labelling and indexing of observational studies has changed since then.

AUTHORS' CONCLUSIONS

We found 18 methodological filters across two eligible studies. Search strategies from the first study had higher sensitivity and precision, underwent external validation and targeted observational studies. Search strategies from the second study had lower sensitivity and precision, focused on cNRSs, and were not validated externally. Given this limited and heterogeneous evidence, and its methodological limitations, further research and better indexation are needed.

Could angiotensin-modulating drugs be relevant for the treatment of Trypanosoma cruzi infection? A systematic review of preclinical and clinical evidence

Although leucocytes are targets of renin-angiotensin system (RAS) effector molecules and RAS-modulating drugs exert immunomodulatory effects, their impact onTrypanosoma cruziinfection remains poorly understood. By using the framework of a systematic review, we integrated the preclinical and clinical evidence to investigate the relevance of angiotensin-inhibiting drugs onT. cruziinfections. From a comprehensive and structured search in biomedical databases, only original studies were analysed. In preclinical and clinical studies, captopril, enalapril and losartan were RAS-modulating drugs used. The mainin vitrofindings indicated that these drugs increased parasite uptake per host cells, IL-12 expression by infected dendritic cells and IFN-γby T lymphocytes, in addition to attenuating IL-10 and IL-17 production by CD8 + T cells. In animal models, reduced parasitaemia, tissue parasitism, leucocytes infiltration and mortality were often observed inT. cruzi-infected animals receiving RAS-modulating drugs. In patients with Chagas’ disease, these drugs exerted a controversial impact on cytokine and hormone levels, and a limited effect on cardiovascular function. Considering a detailed evaluation of reporting and methodological quality, the current preclinical and clinical evidence is at high risk of bias, and we hope that our critical analysis will be useful in mitigating the risk of bias in further studies.

Topic search filters: a systematic scoping review

BACKGROUND

Searching for topics within large biomedical databases can be challenging, especially when topics are complex, diffuse, emerging or lack definitional clarity. Experimentally derived topic search filters offer a reliable solution to effective retrieval; however, their number and range of subject foci remain unknown.

OBJECTIVES

This systematic scoping review aims to identify and describe available experimentally developed topic search filters.

METHODS

Reports on topic search filter development (1990-) were sought using grey literature sources and 15 databases. Reports describing the conception and prospective development of a database-specific topic search and including an objectively measured estimate of its performance ('sensitivity') were included.

RESULTS

Fifty-four reports met inclusion criteria. Data were extracted and thematically synthesised to describe the characteristics of 58 topic search filters.

DISCUSSION

Topic search filters are proliferating and cover a wide range of subjects. Filter reports, however, often lack clear definitions of concepts and topic scope to guide users. Without standardised terminology, filters are challenging to find. Information specialists may benefit from a centralised topic filter repository and appraisal checklists to facilitate quality assessment.

CONCLUSION

Findings will help information specialists identify existing topic search filters and assist filter developers to build on current knowledge in the field.

Study filters for non-randomized studies of interventions consistently lacked sensitivity upon external validation

BACKGROUND

Little evidence is available on searches for non-randomized studies (NRS) in bibliographic databases within the framework of systematic reviews. For instance, it is currently unclear whether, when searching for NRS, effective restriction of the search strategy to certain study types is possible. The following challenges need to be considered: 1) For non-randomized controlled trials (NRCTs): whether they can be identified by established filters for randomized controlled trials (RCTs). 2) For other NRS types (such as cohort studies): whether study filters exist for each study type and, if so, which performance measures they have. The aims of the present analysis were to identify and validate existing NRS filters in MEDLINE as well as to evaluate established RCT filters using a set of MEDLINE citations.

METHODS

Our analysis is a retrospective analysis of study filters based on MEDLINE citations of NRS from Cochrane reviews. In a first step we identified existing NRS filters. For the generation of the reference set, we screened Cochrane reviews evaluating NRS, which covered a broad range of study types. The citations of the studies included in the Cochrane reviews were identified via the reviews' bibliographies and the corresponding PubMed identification numbers (PMIDs) were extracted from PubMed. Random samples comprising up to 200 citations (i.e. 200 PMIDs) each were created for each study type to generate the test sets.

RESULTS

A total of 271 Cochrane reviews from 41 different Cochrane groups were eligible for data extraction. We identified 14 NRS filters published since 2001. The study filters generated between 660,000 and 9.5 million hits in MEDLINE. Most filters covered several study types. The reference set included 2890 publications classified as NRS for the generation of the test sets. Twelve test sets were generated (one for each study type), of which 8 included 200 citations each. None of the study filters achieved sufficient sensitivity (≥ 92%) for all of the study types targeted.

CONCLUSIONS

The performance of current NRS filters is insufficient for effective use in daily practice. It is therefore necessary to develop new strategies (e.g. new NRS filters in combination with other search techniques). The challenges related to NRS should be taken into account.

Study filters for non-randomized studies of interventions consistently lacked sensitivity upon external validation

BACKGROUND

Little evidence is available on searches for non-randomized studies (NRS) in bibliographic databases within the framework of systematic reviews. For instance, it is currently unclear whether, when searching for NRS, effective restriction of the search strategy to certain study types is possible. The following challenges need to be considered: 1) For non-randomized controlled trials (NRCTs): whether they can be identified by established filters for randomized controlled trials (RCTs). 2) For other NRS types (such as cohort studies): whether study filters exist for each study type and, if so, which performance measures they have. The aims of the present analysis were to identify and validate existing NRS filters in MEDLINE as well as to evaluate established RCT filters using a set of MEDLINE citations.

METHODS

Our analysis is a retrospective analysis of study filters based on MEDLINE citations of NRS from Cochrane reviews. In a first step we identified existing NRS filters. For the generation of the reference set, we screened Cochrane reviews evaluating NRS, which covered a broad range of study types. The citations of the studies included in the Cochrane reviews were identified via the reviews' bibliographies and the corresponding PubMed identification numbers (PMIDs) were extracted from PubMed. Random samples comprising up to 200 citations (i.e. 200 PMIDs) each were created for each study type to generate the test sets.

RESULTS

A total of 271 Cochrane reviews from 41 different Cochrane groups were eligible for data extraction. We identified 14 NRS filters published since 2001. The study filters generated between 660,000 and 9.5 million hits in MEDLINE. Most filters covered several study types. The reference set included 2890 publications classified as NRS for the generation of the test sets. Twelve test sets were generated (one for each study type), of which 8 included 200 citations each. None of the study filters achieved sufficient sensitivity (≥ 92%) for all of the study types targeted.

CONCLUSIONS

The performance of current NRS filters is insufficient for effective use in daily practice. It is therefore necessary to develop new strategies (e.g. new NRS filters in combination with other search techniques). The challenges related to NRS should be taken into account.

[Publications German-speaking countries in high impact journals: development and validation of a search filter]

Publications German-speaking countries in high impact journals: development and validation of a search filter Abstract.

BACKGROUND

The number of publications in journals with a high impact factor is an indication of a discipline's participation in international discourse. A search filter allows reliable and reproducible searches for specific publications.

AIM

Development and validation of a geographic search filter for publications by nursing scientists affiliated to German-speaking countries in nursing journals with a high impact factor.

METHODS

The search filter was objectively developed following several steps: (i) creation of a development and a validation set, each consisting of relevant and non-relevant publications, (ii) generation of the search filter by means of text analysis of the development set, (iii) internal validation based on the development set and (iv) external validation using the validation set. The validity was examined regarding several accuracy parameters, e. g. sensitivity, specificity, positive predictive value (PPV) and number needed to read (NNR).

RESULTS

The search filter correctly identified 22 of 30 relevant and 16 of 21 non-relevant publications in the development set: sensitivity 80 % (95 % CI 66 - 94), specificity 76 % (95 % CI 58 - 94), PPV 83 % (95 % CI 69 - 97). External validation yielded similar or better results: sensitivity 81 % (95 % CI 67 - 96), specificity 88 % (95 % CI 71 - 100), PPV 88 % (95 % CI 75 - 100). The NNR was 1.2 and 1.1, respectively.

CONCLUSIONS

The search filter has the potential to identify the intended publications.

Development and validation of a search filter to identify equity-focused studies: reducing the number needed to screen

Background

Health inequalities, worse health associated with social and economic disadvantage, are reported by a minority of research articles. Locating these studies when conducting an equity-focused systematic review is challenging due to a deficit in standardised terminology, indexing, and lack of validated search filters. Current reporting guidelines recommend not applying filters, meaning that increased resources are needed at the screening stage.

Methods

We aimed to design and test search filters to locate studies that reported outcomes by a social determinant of health. We developed and expanded a ‘specific terms strategy’ using keywords and subject headings compiled from recent systematic reviews that applied an equity filter. A ‘non-specific strategy’ was compiled from phrases used to describe equity analyses that were reported in titles and abstracts, and related subject headings. Gold standard evaluation and validation sets were compiled. The filters were developed in MEDLINE, adapted for Embase and tested in both. We set a target of 0.90 sensitivity (95% CI; 0.84, 0.94) in retrieving 150 gold standard validation papers. We noted the reduction in the number needed to screen in a proposed equity-focused systematic review and the proportion of equity-focused reviews we assessed in the project that applied an equity filter to their search strategy.

Results

The specific terms strategy filtered out 93-95% of all records, and retrieved a validation set of articles with a sensitivity of 0.84 in MEDLINE (0.77, 0.89), and 0.87 (0.81, 0.92) in Embase. When combined (Boolean ‘OR’) with the non-specific strategy sensitivity was 0.92 (0.86, 0.96) in MEDLINE (Embase 0.94; 0.89, 0.97). The number needed to screen was reduced by 77% by applying the specific terms strategy, and by 59.7% (MEDLINE) and 63.5% (Embase) by applying the combined strategy. Eighty-one per cent of systematic reviews filtered studies by equity.

Conclusions

A combined approach of using specific and non-specific terms is recommended if systematic reviewers wish to filter studies for reporting outcomes by social determinants. Future research should concentrate on the indexing standardisation for equity studies and further development and testing of both specific and non-specific terms for accurate study retrieval.

Electronic supplementary material

The online version of this article (10.1186/s12874-018-0567-x) contains supplementary material, which is available to authorized users.

Design and implementation of a tool for conversion of search strategies between PubMed and Ovid MEDLINE

BACKGROUND

Both PubMed and Ovid MEDLINE contain records from the MEDLINE database. However, there are subtle differences in content, functionality, and search syntax between the two. There are many instances in which researchers may wish to search both interfaces, such as when conducting supplementary searching for a systematic review to retrieve a unique content from PubMed or when using a previously published search strategy from a different interface, but little guidance on how to best conduct these searches. The aim of this project is to describe differences in search functionality between Ovid MEDLINE and PubMed, provide guidance for converting search strategies between the two, and develop an easy-to-use, freely available web-based tool to automate search syntax translations.

CASE PRESENTATION

In this paper, we present a custom-built freely available online tool, Medline Transpose, to streamline the process of converting search strategies between Ovid MEDLINE and PubMed. With this tool, users can paste a strategy formatted for one interface into the search box and immediately retrieve an output formatted for use in the other interface, with recommendations for changes that users can make to the strategy where an exact translation does not exist.

CONCLUSION

This novel approach has the potential to reduce time and errors that database users spend translating search strategies.

Finding the Integrated Care Evidence Base in PubMed and Beyond: A Bibliometric Study of the Challenges

INTRODUCTION

Integrated care research evidence should be optimally visible and accessible to stakeholders. This study examines the contribution of specific databases to the discovery of integrated care evidence, and tests the usefulness of Medical Subject Heading (MeSH) indexing of this literature within PubMed.

METHODS

We used bibliometric methods to analyse the integrated care literature indexed within six databases between 2007 and 2016. An international expert advisory group assessed the relevance of citations randomly retrieved from PubMed using MeSH term 'Delivery of Health Care, Integrated'.

RESULTS

Integrated care evidence is diffuse, spread across many journals. Between 2007 and 2016, integrated care citations grew substantially, with the rate of increase highest in Embase. PubMed contributes the largest proportion of unique citations (citations not included in any of the other databases analysed), followed by Embase, PsycINFO and CINAHL. On average, expert reviewers rated 42.5% of citations retrieved by MeSH term 'Delivery of Health Care, Integrated' as relevant to integrated care. When these citations were dual reviewed, inter-rater agreement was low.

CONCLUSION

MeSH terms alone are insufficient to retrieve integrated care content from PubMed. Embase and CINAHL contain unique content not found in PubMed that should not be overlooked. A validated search filter is proposed to simplify the process of finding integrated care research for clinicians, managers and decision-makers.

Assessing the performance of methodological search filters to improve the efficiency of evidence information retrieval: five literature reviews and a qualitative study

BACKGROUND

Effective study identification is essential for conducting health research, developing clinical guidance and health policy and supporting health-care decision-making. Methodological search filters (combinations of search terms to capture a specific study design) can assist in searching to achieve this.

OBJECTIVES

This project investigated the methods used to assess the performance of methodological search filters, the information that searchers require when choosing search filters and how that information could be better provided.

METHODS

Five literature reviews were undertaken in 2010/11: search filter development and testing; comparison of search filters; decision-making in choosing search filters; diagnostic test accuracy (DTA) study methods; and decision-making in choosing diagnostic tests. We conducted interviews and a questionnaire with experienced searchers to learn what information assists in the choice of search filters and how filters are used. These investigations informed the development of various approaches to gathering and reporting search filter performance data. We acknowledge that there has been a regrettable delay between carrying out the project, including the searches, and the publication of this report, because of serious illness of the principal investigator.

RESULTS

The development of filters most frequently involved using a reference standard derived from hand-searching journals. Most filters were validated internally only. Reporting of methods was generally poor. Sensitivity, precision and specificity were the most commonly reported performance measures and were presented in tables. Aspects of DTA study methods are applicable to search filters, particularly in the development of the reference standard. There is limited evidence on how clinicians choose between diagnostic tests. No published literature was found on how searchers select filters. Interviewing and questioning searchers via a questionnaire found that filters were not appropriate for all tasks but were predominantly used to reduce large numbers of retrieved records and to introduce focus. The Inter Technology Appraisal Support Collaboration (InterTASC) Information Specialists' Sub-Group (ISSG) Search Filters Resource was most frequently mentioned by both groups as the resource consulted to select a filter. Randomised controlled trial (RCT) and systematic review filters, in particular the Cochrane RCT and the McMaster Hedges filters, were most frequently mentioned. The majority indicated that they used different filters depending on the requirement for sensitivity or precision. Over half of the respondents used the filters available in databases. Interviewees used various approaches when using and adapting search filters. Respondents suggested that the main factors that would make choosing a filter easier were the availability of critical appraisals and more detailed performance information. Provenance and having the filter available in a central storage location were also important.

LIMITATIONS

The questionnaire could have been shorter and could have included more multiple choice questions, and the reviews of filter performance focused on only four study designs.

CONCLUSIONS

Search filter studies should use a representative reference standard and explicitly report methods and results. Performance measures should be presented systematically and clearly. Searchers find filters useful in certain circumstances but expressed a need for more user-friendly performance information to aid filter choice. We suggest approaches to use, adapt and report search filter performance. Future work could include research around search filters and performance measures for study designs not addressed here, exploration of alternative methods of displaying performance results and numerical synthesis of performance comparison results.

FUNDING

The National Institute for Health Research (NIHR) Health Technology Assessment programme and Medical Research Council-NIHR Methodology Research Programme (grant number G0901496).

Optimal search strategies for identifying moderators and predictors of treatment effects in PubMed

BACKGROUND

Treatment effects differ across patients. To guide selection of treatments for patients, it is essential to acknowledge these differences and identify moderators or predictors. Our aim was to generate optimal search strategies (commonly known as filters) for PubMed to retrieve papers identifying moderators and predictors of treatment effects.

METHODS

Six journals were hand-searched for articles on moderators or predictors. Selected articles were randomly allocated to a development and validation set. Search terms were extracted from the development set and tested for their performance. Search filters were created from combinations of these terms and tested in the validation set.

RESULTS

Of 4407 articles, 198 were considered to be relevant. The most sensitive filter in the development set '("Epidemiologic Methods" [MeSH] OR assign* OR control*[tiab] OR trial*[tiab]) AND therapy*[sh]' yielded in the validation set a sensitivity of 89% [88%-90%] and a specificity of 80% [79%-82%].

CONCLUSIONS

The search filters created in this study can help to efficiently retrieve evidence on moderators and predictors of treatment effect. Testing of the filters in multiple domains should reveal robustness across disciplines. These filters can facilitate the retrieval of evidence on moderators and predictors of treatment effects, helping the implementation of stratified or personalised health care.