A comparison of results of empirical studies of supplementary search techniques and recommendations in review methodology handbooks: a methodological review

Common challenges and suggestions for risk of bias tool development: a systematic review of methodological studies

OBJECTIVES

To review the findings of studies that have evaluated the design and/or usability of key risk of bias (RoB) tools for the assessment of RoB in primary studies, as categorized by the Library of Assessment Tools and InsTruments Used to assess Data validity in Evidence Synthesis Network (a searchable library of RoB tools for evidence synthesis): Prediction model Risk Of Bias ASessment Tool (PROBAST) , Risk of Bias-2 (RoB2), Risk Of Bias In Non-randomised Studies of Interventions (ROBINS-I), Quality Assessment of Diagnostic Accuracy Studies-2 (QUADAS-2), Quality Assessment of Diagnostic Accuracy Studies-Comparative (QUADAS-C), Quality Assessment of Prognostic Accuracy Studies (QUAPAS), Risk Of Bias in Non-randomised Studies of Exposures (ROBINS-E), and the COnsensus-based Standards for the selection of health Measurement INstruments (COSMIN) RoB checklist.

STUDY DESIGN AND SETTING

Systematic review of methodological studies. We conducted a forward citation search from the primary report of each tool, to identify primary studies that aimed to evaluate the design and/or usability of the tool. Two reviewers assessed studies for inclusion. We extracted tool features into Microsoft Word and used NVivo for document analysis, comprising a mix of deductive and inductive approaches. We summarized findings within each tool and explored common findings across tools.

RESULTS

We identified 13 tool evaluations meeting our inclusion criteria: PROBAST (3), RoB2 (3), ROBINS-I (4), and QUADAS-2 (3). We identified no evaluations for the other tools. Evaluations varied in clinical topic area, methodology, approach to bias assessment, and tool user background. Some had limitations affecting generalizability. We identified common findings across tools for 6/14 themes: (1) challenging items (eg, RoB2/ROBINS-I "deviations from intended interventions" domain), (2) overall RoB judgment (concerns with overall risk calculation in PROBAST/ROBINS-I), (3) tool usability (concerns about complexity), (4) time to complete tool (varying demands on time, eg, depending on number of outcomes assessed), (5) user agreement (varied across tools), and (6) recommendations for future use (eg, piloting) and development (add intermediate domain answer to QUADAS-2/PROBAST; provide clearer guidance for all tools). Of the other eight themes, seven only had findings for the QUADAS-2 tool, limiting comparison across tools, and one ("reorganization of questions") had no findings.

CONCLUSION

Evaluations of key RoB tools have posited common challenges and recommendations for tool use and development. These findings may be helpful to people who use or develop RoB tools. Guidance is necessary to support the design and implementation of future RoB tool evaluations.

Experiences of informal caregivers of people with dementia in low- and middle-income countries: a systematic review protocol

OBJECTIVE

This review aims to synthesize the experiences of informal caregivers of people with dementia in low- and middle-income countries (LMICs).

INTRODUCTION

Globally, the burden of dementia is increasing disproportionately in LMICs. Informal caregivers play a vital role and face multiple challenges in LMICs, even though they also often lack the awareness and skills to provide adequate care for people living with dementia. Many LMICs have limited resources and caregivers lack support services. Understanding their experiences and perceptions may improve interventions, helping caregivers to better support those living with dementia.

INCLUSION CRITERIA

Studies from all settings in LMICs will be considered. Qualitative data from qualitative or mixed methods studies that explore caregivers' experiences and perceptions will be included in this review. Participants are informal caregivers of people diagnosed with dementia at any stage, regardless of age, gender, or their relationship with people living with dementia.

METHODS

This review will follow the JBI methodology for a qualitative systematic reviews, with meta-aggregation as the synthesis method. The review will be reported in accordance with the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines. A 3-step search strategy will be used to locate published and unpublished studies in MEDLINE, CINAHL, Embase, PsycINFO, AgeLine, LILACS, African Index Medicus, ProQuest Dissertations and Theses, Open Dissertations, Web of Science Core Collection, and Google Scholar. There will be no date or language limitations. All studies will be screened against the inclusion criteria and data will be extracted and critically appraised for methodological quality by 2 independent reviewers using JBI tools. Confidence in the final synthesized findings will be assessed using the ConQual approach.

REVIEW REGISTRATION

PROSPERO CRD42023453814.

A systematic review case study of urgent and emergency care configuration found citation searching of Web of Science and Google Scholar of similar value

BACKGROUND

Supplementary search methods, including citation searching, are essential if systematic reviews are to avoid producing biased conclusions. Little evidence exists on how to prioritise databases for citation searching or to establish whether using multiple sources is beneficial.

OBJECTIVES

A systematic review examining urgent and emergency care reconfiguration was used to investigate the utility of citation searching on Web of Science (WOS) and/or Google Scholar (GS).

METHODS

This case study investigated numbers of studies, additional studies and unique studies retrieved from both sources. In addition, the time to search, the ease of adding references to reference management software and obtaining abstracts of studies for screening are briefly considered.

RESULTS

WOS retrieved 62 references after deduplication of the results, 52 being additional references not retrieved during the database searching. GS retrieved 134 unique references with 63 additional references. WOS and GS retrieved the same three additional included studies. WOS was less time intensive to search given the facility to restrict to English language papers and availability of abstracts.

CONCLUSIONS

In a single systematic review case study, citation searching was required to identify all included studies. Citation searching on WOS is more efficient, where a subscription is available. Both databases identified the same studies but GS required additional time to remove non-English language studies and locate abstracts.

Expert searchers identified time, team, technology and tension as challenges when carrying out supplementary searches for systematic reviews: A thematic network analysis

BACKGROUND

Systematic reviews require detailed planning of complex processes which can present logistical challenges. Understanding these logistical challenges can help with planning and execution of tasks OBJECTIVES: To describe the perspectives of expert searchers on the main logistical challenges when carrying out supplementary searches for systematic reviews, in particular, forward citation searching and web searching.

METHODS

Qualitative interviews were undertaken with 15 experts on searching for studies for systematic reviews (e.g. information specialists) working in health and social care research settings. Interviews were undertaken by video-call between September 2020 and June 2021. Data analysis used thematic network analysis.

RESULTS

We identified three logistical challenges of using forward citation searching and web searching which were organised under the global theme of 'tension': time, team and technology. Several subthemes were identified which supported the organising themes, including allocating time, justifying time and keeping to time; reviewer expectations and contact with review teams; and access to resources and reference management.

CONCLUSION

Forward citation searching and web searching are logistically challenging search methods for a systematic review. An understanding of these challenges should encourage expert searchers and review teams to maintain open channels of communication, which should also facilitate improved working relationships.

Streamlining search methods to update evidence and gap maps: A case study using intergenerational interventions

Background

Evidence and Gap Maps (EGMs) should be regularly updated. Running update searches to find new studies for EGMs can be a time-consuming process. Search Summary Tables (SSTs) can help streamline searches by identifying which resources were most lucrative for identifying relevant articles, and which were redundant. The aim of this study was to use an SST to streamline search methods for an EGM of studies about intergenerational activities.

Methods

To produce the EGM, 15 databases were searched. 8638 records were screened and 500 studies were included in the final EGM. Using an SST, we determined which databases and search methods were the most efficient in terms of sensitivity and specificity for finding the included studies. We also investigated whether any database performed particularly well for returning particular study types. For the best performing databases we analysed the search terms used to streamline the strategies.

Results

No single database returned all of the studies included in the EGM. Out of 500 studies PsycINFO returned 40% (n = 202), CINAHL 39% (n = 194), Ageline 25% (n = 174), MEDLINE 23% (n = 117), ERIC 20% (n = 100) and Embase 19% (n = 98). HMIC database and Conference Proceedings Citation Index-Science via Web of Science returned no studies that were included in the EGM. ProQuest Dissertations & Theses (PQDT) returned the highest number of unique studies (n = 42), followed by ERIC (n = 33) and Ageline (n = 29). Ageline returned the most randomised controlled trials (42%) followed by CINAHL (34%), MEDLINE (29%) and CENTRAL (29%). CINAHL, Ageline, MEDLINE and PsycINFO performed the best for locating systematic reviews. (62%, 46% and 42% respectively). CINAHL, PsycINFO and Ageline performed best for qualitative studies (41%, 40% and 34%). The Journal of Intergenerational Relationships returned more included studies than any other journal (16%). No combinations of search terms were found to be better in terms of balancing specificity and sensitivity than the original search strategies. However, strategies could be reduced considerably in terms of length without losing key, unique studies.

Conclusion

Using SSTs we have developed a method for streamlining update searches for an EGM about intergenerational activities. For future updates we recommend that MEDLINE, PsycINFO, ERIC, Ageline, CINAHL and PQDT are searched. These searches should be supplemented by hand-searching the Journal of Intergenerational Relationships and carrying out backwards citation chasing on new systematic reviews. Using SSTs to analyse database efficiency could be a useful method to help streamline search updates for other EGMs.

Measuring continuing medical education conference impact and attendee experience: a scoping review

Objectives

The aim was to comprehensively identify published research evaluating continuing medical education conferences, to search for validated tools and perform a content analysis to identify the relevant domains for conference evaluation.

Methods

We used scoping review methodology and searched MEDLINE® for relevant English or French literature published between 2008 and 2022 (last search June 3, 2022). Original research (including randomized controlled trials, non-randomized studies, cohort, mixed-methods, qualitative studies, and editorial pieces) where investigators described impact, experience, or motivations related to conference attendance were eligible. Citations were assessed in triplicate, and data extracted in duplicate.

Results

Eighty-three studies were included, 69 (83%) of which were surveys or interview based, with the majority conducted at the end of or following conference conclusion. Of the 74 tools identified, only one was validated and was narrowly focused on a specific conference component. A total of 620 items were extracted and categorized into 4 a priori suggested domains (engagement-networking, education-learning, impact, scholarship), and an additional 4 identified through content analysis (value-satisfaction, logistics, equity-diversity-inclusivity, career influences). Time trends were evident, including the absence of items related to equity-diversity-inclusivity prior to 2019, and a focus on logistics, particularly technology and virtual conferences, since 2020.

Conclusions

This study identified 8 major domains relevant for continuing medical education conference evaluation. This work is of immediate value to individuals and organizations seeking to either design or evaluate a conference and represents a critical step in the development of a standardized tool for conference evaluation.

Exploring important service characteristics of telephone cancer information and support services for callers: protocol for a systematic review of qualitative research

INTRODUCTION

As cancer incidence continues to rise, challenges remain in how to communicate accurate, timely information to people with cancer, their families and healthcare professionals. One option is to provide support and comprehensive, tailored information via a telephone cancer information and support service (CISS). This systematic review aims to summarise the service characteristics of telephone CISS and identify what aspects of services are important from callers' perspectives.

METHODS AND ANALYSIS

A comprehensive literature search will be conducted for articles published from database inception to 30 March 2023 (OVID MEDLINE, EMBASE, CINAHL, PsycINFO and SocINDEX). Published, peer-reviewed, articles reporting qualitative research on the service characteristics of telephone CISS important to callers in any language will be included. One researcher will complete the searches, two researchers will independently screen results for eligible studies and a third researcher will resolve any disagreement. A narrative and thematic synthesis of studies will be provided. Study characteristics will be independently extracted by one researcher and checked by a second. Included studies' methodological quality will be evaluated independently by two researchers using the 2022 Critical Appraisal Skills Programme Qualitative Studies Checklist. Grading of Recommendations Assessment, Development and Evaluation-Confidence in the Evidence from Reviews of Qualitative research tool will assess the confidence of the review findings.

ETHICS AND DISSEMINATION

Ethics approval is not required for this research as it is a planned systematic review of published literature. Findings will be presented at leading cancer, health economic and supportive care conferences, published in a peer-reviewed journal, and disseminated via websites and social media.

PROSPERO REGISTRATION NUMBER

CRD42023413897.

The Socioeconomic Impact of Diseases of Working Equids in Low and Middle-Income Countries: A Critical Review

Working equids provide a crucial contribution to the livelihoods and food security of communities in low- and middle-income countries (LMICs). Nevertheless, they are a neglected category within animal health policies and interventions of governmental and non-governmental institutions. This critical review aims to assess the socioeconomic impact of diseases of working equids in LMICs. By highlighting the implications of diseases on working equid welfare, human wellbeing and livelihoods, this review seeks to sensitise policymakers within governments and international organisations to develop policies and interventions aimed at protecting the health of working equids and, consequently, the health and livelihoods of their dependent communities. Twenty relevant publications were identified through the search of five databases (CAB Abstracts, Web of Science Core Collection, BIOSIS, EMBASE and Scopus), backward citation searching and screening of indexes of proceedings and Special Issues retrieved from the database search. The review findings show that diseases of working equids have detrimental socioeconomic effects. However, this subject is under-researched and restricted to few diseases and geographical settings. Considering the complexity of the issue, this review demonstrates that the 'One Health' approach represents an opportunity to clarify the link between equid health, human wellbeing and livelihoods, facilitating the translation of research into policy.

Dravet syndrome: A systematic literature review of the illness burden

We performed a systematic literature review and narrative synthesis according to a pre-registered protocol (Prospero: CRD42022376561) to identify the evidence associated with the burden of illness in Dravet syndrome (DS), a developmental and epileptic encephalopathy characterized by drug-resistant epilepsy with neurocognitive and neurobehavioral impairment. We searched MEDLINE, Embase, and APA PsychInfo, Cochrane's database of systematic reviews, and Epistemonikos from inception to June 2022. Non-interventional studies reporting on epidemiology (incidence, prevalence, and mortality), patient and caregiver health-related quality of life (HRQoL), direct and indirect costs and healthcare resource utilization were eligible. Two reviewers independently carried out the screening. Pre-specified data were extracted and a narrative synthesis was conducted. Overall, 49 studies met the inclusion criteria. The incidence varied from 1:15 400-1:40 900, and the prevalence varied from 1.5 per 100 000 to 6.5 per 100 000. Mortality was reported in 3.7%-20.8% of DS patients, most commonly due to sudden unexpected death in epilepsy and status epilepticus. Patient HRQoL, assessed by caregivers, was lower than in non-DS epilepsy patients; mean scores (0 [worst] to 100/1 [best]) were 62.1 for the Kiddy KINDL/Kid-KINDL, 46.5-54.7 for the PedsQL and 0.42 for the EQ-5D-5L. Caregivers, especially mothers, were severely affected, with impacts on their time, energy, sleep, career, and finances, while siblings were also affected. Symptoms of depression were reported in 47%-70% of caregivers. Mean total direct costs were high across all studies, ranging from $11 048 to $77 914 per patient per year (PPPY), with inpatient admissions being a key cost driver across most studies. Mean costs related to lost productivity were only reported in three publications, ranging from approximately $19 000 to $20 000 PPPY ($17 596 for mothers vs $1564 for fathers). High seizure burden was associated with higher resource utilization, costs and poorer HRQoL. The burden of DS on patients, caregivers, the healthcare system, and society is profound, reflecting the severe nature of the syndrome. Future studies will be able to assess the impact that newly approved therapies have on reducing the burden of DS.

Supplementary strategies identified additional eligible studies in qualitative systematic reviews

BACKGROUND AND OBJECTIVES

There is an increasing number of qualitative systematic reviews being produced. Searching for qualitative literature to include in these systematic reviews is, however, more challenging and may result in less than favorable recall. Database searches relying solely on key elements of the research question may not retrieve all relevant qualitative studies for synthesis, and supplementary searches may be pertinent to complement the searches. This study aimed to determine, if a) supplementary search strategies (citation searches and alternative search strategies) were able to identify relevant publications for qualitative systematic reviews that were nonretrievable, when conducting traditional database searches based on key elements; and b) to investigate the total number of identified publications when combining traditional database searches with these supplementary search strategies.

METHODS

From a previous study, a gold standard of 12 qualitative reviews including 101 PubMed-indexed publications was used. One of the reviews had only one included publication and in one review, the two included studies were identifiable in PubMed. In the remaining 10 reviews, 61 publications were retrievable through traditional database searches, and 37 were nonidentifiable. The 61 publications were used as basis for possible identification of the 37 publications by using the supplementary search strategies: citations searches (review of reference lists, PubMed "Cited by" function; Scopus "Cited by" function, Citationchaser, CoCites plugin for PubMed) and alternative search strategies (PubMed "similar articles" function; Scopus "Related documents based on references").

RESULTS

Traditional database searches retrieved 62.4% of the 101 publications. Citations searches in Scopus, Citationchaser and CoCites identified 21 (56.8%) of the 37 remaining publications. The PubMed "Cited by" function did not identify any of the 37 publications. The alternative search strategies, the PubMed "Similar articles" together with Scopus "Related documents based on references"-function) identified 15 (40.5%) of the 37 publications. Together, these supplementary search strategies identified 25 (67.6%) of the 37 publications, resulting in an overall retrieval of 87.1%, when combining traditional database searches and supplementary search strategies.

CONCLUSION

The results of this study indicate that supplementary search strategies (citation searches and alternative search strategies) increase the retrieval potential, when searching for qualitative publications and should be included, when identifying literature for qualitative reviews.

Invasive candidiasis: current clinical challenges and unmet needs in adult populations

Invasive candidiasis (IC) is a serious infection caused by several Candida species, and the most common fungal disease in hospitals in high-income countries. Despite overall improvements in health systems and ICU care in the last few decades, as well as the development of different antifungals and microbiological techniques, mortality rates in IC have not substantially improved. The aim of this review is to summarize the main issues underlying the management of adults affected by IC, focusing on specific forms of the infection: IC developed by ICU patients, IC observed in haematological patients, breakthrough candidaemia, sanctuary site candidiasis, intra-abdominal infections and other challenging infections. Several key challenges need to be tackled to improve the clinical management and outcomes of IC patients. These include the lack of global epidemiological data for IC, the limitations of the diagnostic tests and risk scoring tools currently available, the absence of standardized effectiveness outcomes and long-term data for IC, the timing for the initiation of antifungal therapy and the limited recommendations on the optimal step-down therapy from echinocandins to azoles or the total duration of therapy. The availability of new compounds may overcome some of the challenges identified and increase the existing options for management of chronic Candida infections and ambulant patient treatments. However, early identification of patients that require antifungal therapy and treatment of sanctuary site infections remain a challenge and will require further innovations.

The association between social class and the impact of treatment for mental health problems: a systematic review and narrative synthesis

PURPOSE

This systematic review aimed to synthesise all quantitative literature on the association between social class and the effectiveness of interventions for mental health disorders.

METHODS

Systematic literature searches (inception-March 2021) were conducted across 7 databases, and all quantitative studies meeting inclusion criteria, examining the impact of social class on access to treatment, or intervention effectiveness, or the impact of treatment on social mobility, were synthesised narratively.

RESULTS

Evidence suggests that lower social class may be associated with reduced access to primary and secondary mental health care and increased likelihood of access via crisis services, and patients of lower social class may not benefit from all mental health interventions, with reduced effectiveness. While limited, there was some indication that psychosocial interventions could encourage increased employment rates.

CONCLUSION

Social class is associated with the effectiveness of psychological interventions, and should be considered when designing new interventions to prevent barriers to access and improve effectiveness.

The burden of illness in Lennox-Gastaut syndrome: a systematic literature review

BACKGROUND

Lennox-Gastaut syndrome (LGS) is a severe developmental and epileptic encephalopathy characterized by drug-resistant epilepsy with multiple seizure types starting in childhood, a typical slow spike-wave pattern on electroencephalogram, and cognitive dysfunction.

METHODS

We performed a systematic literature review according to the PRISMA guidelines to identify, synthesize and appraise the burden of illness in LGS (including "probable" LGS). Studies were identified by searching MEDLINE, Embase and APA PsychInfo, Cochrane's database of systematic reviews, and Epistemonikos. The outcomes were epidemiology (incidence, prevalence or mortality), direct and indirect costs, healthcare resource utilization, and patient and caregiver health-related quality of life (HRQoL).

RESULTS

The search identified 22 publications evaluating the epidemiology (n = 10), direct costs and resource (n = 10) and/or HRQoL (n = 5). No studies reporting on indirect costs were identified. With no specific ICD code for LGS in many regions, several studies had to rely upon indirect methods to identify their patient populations (e.g., algorithms to search insurance claims databases to identify "probable" LGS). There was heterogeneity between studies in how LGS was defined, the size of the populations, ages of the patients and length of the follow-up period. The prevalence varied from 4.2 to 60.8 per 100,000 people across studies for probable LGS and 2.9-28 per 100,000 for a confirmed/narrow definition of LGS. LGS was associated with high mortality rates compared to the general population and epilepsy population. Healthcare resource utilization and direct costs were substantial across all studies. Mean annual direct costs per person varied from $24,048 to $80,545 across studies, and home-based care and inpatient care were significant cost drivers. Studies showed that the HRQoL of patients and caregivers was adversely affected, although only a few studies were identified. In addition, studies suggested that seizure events were associated with higher costs and worse HRQoL. The risk of bias was low or moderate in most studies.

CONCLUSIONS

LGS is associated with a significant burden of illness featuring resistant seizures associated with higher costs and worse HRQoL. More research is needed, especially in evaluating indirect costs and caregiver burden, where there is a notable lack of studies.

The popularisation of self-care: Tracing the dissemination of Orem's Self-Care Deficiency Nursing Theory into the scientific disciplines

BACKGROUND

Bibliometric methods may be used to examine research trends, and information visualisation techniques are useful in illustrating the diffusion of knowledge and how theories are applied.

OBJECTIVES

The present study aimed to illustrate how Orem's Self-Care Deficiency Nursing Theory (SCDNT) has been cited and applied in nursing science and beyond.

METHODS

A bibliometric analysis examined scientific publications that cited Orem's nursing theory. In addition, the diffusion of SCDNT was assessed using data visualisation methods and integration scores of SCDNT versions were calculated to define trends in its theoretical usage in other scientific domains.

RESULTS

The information visualisation demonstrated increased usage of SCDTN in different disciplines. Integration scores demonstrated that the scientific community still recognises and uses versions of SCDTN.

DISCUSSION

Studying citation patterns helps to identify which publications are still cited and relevant, as well as illustrating the dissemination of theory. Findings may be used in the collection weeding of SCDTN book versions.

CONCLUSIONS

Findings provided nursing scientists with a better conceptual understanding of SCDNT diffusion and development. For academic library managers, the findings identify which SCDNT should be retained for historical interest and curriculum needs.

A Tailored Approach: A model for literature searching in complex systematic reviews

Our previous work identified that nine leading guidance documents for seven different types of systematic review advocated the same process of literature searching. We defined and illustrated this process and we named it ‘the Conventional Approach’. The Conventional Approach appears to meet the needs of researchers undertaking literature searches for systematic reviews of clinical interventions. In this article, we report a new and alternate process model of literature searching called ‘A Tailored Approach’. A Tailored Approach is indicated as a search process for complex reviews which do not focus on the evaluation of clinical interventions. The aims of this article are to (1) explain the rationale for, and the theories behind, the design of A Tailored Approach; (2) report the current conceptual illustration of A Tailored Approach and to describe a user’s interaction with the process model; and (3) situate the elements novel to A Tailored Approach (when compared with the Conventional Approach) in the relevant literature. A Tailored Approach suggests investing time at the start of a review, to develop the information needs from the research objectives, and to tailor the search approach to studies or data. Tailored Approaches should be led by the information specialist (librarian) but developed by the research team. The aim is not necessarily to focus on comprehensive retrieval. Further research is indicated to evaluate the use of supplementary search methods, methods of team-working to define search approaches, and to evaluate the use of conceptual models of information retrieval for testing and evaluation.

Effectiveness of aromatherapy for prevention or treatment of disease, medical or preclinical conditions, and injury: protocol for a systematic review and meta-analysis

BACKGROUND

Aromatherapy - the therapeutic use of essential oils from plants (flowers, herbs or trees) to treat ill health and promote physical, emotional and spiritual well-being - is one of the most widely used natural therapies reported by consumers in Western countries. The Australian Government Department of Health (via the National Health and Medical Research Council) has commissioned a suite of independent evidence evaluations to inform the 2019-20 Review of the Australian Government Rebate on Private Health Insurance for Natural Therapies. This protocol is for one of the evaluations: a systematic review that aims to examine the effectiveness of aromatherapy in preventing and/or treating injury, disease, medical conditions or preclinical conditions.

METHODS

Eligibility criteria: randomised trials comparing (1) aromatherapy (delivered by any mode) to no aromatherapy (inactive controls), (2) aromatherapy (delivered by massage) to massage alone or (3) aromatherapy to 'gold standard' treatments.

POPULATIONS

any condition, pre-condition, injury or risk factor (excluding healthy participants without clearly identified risk factors).

OUTCOMES

any for which aromatherapy is indicated. Searches: Cochrane Central Register of Controlled Trials (CENTRAL), with a supplementary search of PubMed (covering a 6-month lag period for processing records in CENTRAL and records not indexed in MEDLINE), AMED and Emcare. No date, language or geographic limitations will be applied.

DATA AND ANALYSIS

screening by two authors, independently (records indexed by Aromatherapy or Oils volatile or aromatherapy in title; all full text) or one author (remaining records) with second author until 80% agreement. Data extraction and risk of bias assessment (ROB 2.0) will be piloted by three authors, then completed by a single author and checked by a second. Comparisons will be based on broad outcome categories (e.g. pain, emotional functioning, sleep disruption) stratified by population subgroups (e.g. chronic pain conditions, cancer, dementia) as defined in the analytic framework for the review. Meta-analysis or other synthesis methods will be used to combine results across studies. GRADE methods will be used to assess certainty of evidence and summarise findings.

DISCUSSION

Results of the systematic review will provide a comprehensive and up-to-date synthesis of evidence about the effectiveness of aromatherapy.

SYSTEMATIC REVIEW REGISTRATION

PROSPERO CRD42021268244.

Searching two or more databases decreased the risk of missing relevant studies: a metaresearch study

OBJECTIVE

Assessing changes in coverage, recall, review conclusions and references not found when searching fewer databases.

STUDY DESIGN AND SETTING

In 60 randomly selected Cochrane reviews, we checked included study publications' coverage (indexation) and recall (findability) using different search approaches with MEDLINE, Embase, and CENTRAL and related them to authors' conclusions and certainty. We assessed characteristics of unfound references.

RESULTS

1989/2080 included references, were indexed in ≥1 database (coverage=96%). In reviews where using one of our search approaches would not change conclusions and certainty (n=44-54), median coverage and recall were highest (range 87.9-100.0% and 78.2-93.3%, respectively). Here, searching ≥2 databases reached >95% coverage and ≥87.9% recall. In reviews with unchanged conclusions but less certainty (n=2-8): 63.3-79.3% coverage and 45.0-75.0% recall. In reviews with opposite conclusions (n=1-3): 63.3-96.6% and 52.1-78.7%. In reviews where a conclusion was no longer possible (n=3-7): 60.6%-86.0% and 20.0-53.8%. The 265 references that were indexed but unfound were more often abstractless (30% vs 11%) and older (28% vs. 17% published before 1991) than found references.

CONCLUSION

Searching ≥2 databases improves coverage and recall and decreases the risk of missing eligible studies. If researchers suspect that relevant articles are difficult to find, supplementary search methods should be used.

citationchaser: a tool for transparent and efficient forward and backward citation chasing in systematic searching

Systematic searching aims to find all possibly relevant research from multiple sources, the basis for an unbiased and comprehensive evidence base. Along with bibliographic databases, systematic reviewers use a variety of additional methods to minimise procedural bias. Citation chasing exploits connections between research articles to identify relevant records for a review by making use of explicit mentions of one article within another. Citation chasing is a popular supplementary search method because it helps to build on the work of primary research and review authors. It does so by identifying potentially relevant studies that might otherwise not be retrieved by other search methods; for example, because they did not use the review authors' search terms in the specified combinations in their titles, abstracts, or keywords. Here, we briefly provide an overview of citation chasing as a method for systematic reviews. Furthermore, given the challenges and high resource requirements associated with citation chasing, the limited application of citation chasing in otherwise rigorous systematic reviews, and the potential benefit of identifying terminologically disconnected but semantically linked research studies, we have developed and describe a free and open source tool that allows for rapid forward and backward citation chasing. We introduce citationchaser, an R package and Shiny app for conducting forward and backward citation chasing from a starting set of articles. We describe the sources of data, the backend code functionality, and the user interface provided in the Shiny app. This article is protected by copyright. All rights reserved.

Gray Literature in Evaluating Effectiveness in Digital Health and Health and Welfare Technology: A Source Worth Considering

Background

The need to assess the effectiveness and value of interventions involving digital health and health and welfare technologies is becoming increasingly important due to the rapidly growing development of these technologies and their areas of application. Systematic reviews of scientific literature are a mainstay of such assessment, but publications outside the realm of traditional scientific bibliographic databases—known as gray literature—are often not included. This is a disadvantage, particularly apparent in the health and welfare technology (HWT) domain.

Objective

The aim of this article is to investigate the significance of gray literature in digital health and HWT when reviewing literature. As an example, the impact of including gray literature to the result of two systematic reviews in HWT is examined.

Methods

In this paper, we identify, discuss, and suggest methods for including gray literature sources when evaluating effectiveness and appropriateness for different review types related to HWT. The analysis also includes established sources, search strategies, documentation, and reporting of searches, as well as bias and credibility assessment. The differences in comparison to scientific bibliographic databases are elucidated. We describe the results, challenges, and benefits of including gray literature in 2 examples of systematic reviews of HWT.

Results

In the 2 systematic reviews described in this paper, most included studies came from context-specific gray literature sources. Gray literature contributed to the overall result of the reviews and corresponded well with the reviews’ aims. The assessed risk of bias of the included studies derived from gray literature was similar to the included studies from other types of sources. However, because of less standardized publication formats, assessing and extracting data from gray literature studies were more time-consuming and compiling statistical results was not possible. The search process for gray literature required more time and the reproducibility of gray literature searches were less certain due to more unstable publication platforms.

Conclusions

Gray literature is particularly relevant for digital health and HWT but searches need to be conducted systematically and reported transparently. This way gray literature can broaden the range of studies, highlight context specificity, and decrease the publication bias of reviews of effectiveness of HWT. Thus, researchers conducting systematic reviews related to HWT should consider including gray literature based on a systematic approach.

An overview of methodological approaches in systematic reviews

AIM

The aim of this overview is to identify and collate evidence from existing published systematic review (SR) articles evaluating various methodological approaches used at each stage of an SR.

METHODS

The search was conducted in five electronic databases from inception to November 2020 and updated in February 2022: MEDLINE, Embase, Web of Science Core Collection, Cochrane Database of Systematic Reviews, and APA PsycINFO. Title and abstract screening were performed in two stages by one reviewer, supported by a second reviewer. Full-text screening, data extraction, and quality appraisal were performed by two reviewers independently. The quality of the included SRs was assessed using the AMSTAR 2 checklist.

RESULTS

The search retrieved 41,556 unique citations, of which 9 SRs were deemed eligible for inclusion in final synthesis. Included SRs evaluated 24 unique methodological approaches used for defining the review scope and eligibility, literature search, screening, data extraction, and quality appraisal in the SR process. Limited evidence supports the following (a) searching multiple resources (electronic databases, handsearching, and reference lists) to identify relevant literature; (b) excluding non-English, gray, and unpublished literature, and (c) use of text-mining approaches during title and abstract screening.

CONCLUSION

The overview identified limited SR-level evidence on various methodological approaches currently employed during five of the seven fundamental steps in the SR process, as well as some methodological modifications currently used in expedited SRs. Overall, findings of this overview highlight the dearth of published SRs focused on SR methodologies and this warrants future work in this area.

The characteristics of effective technology-enabled dementia education: a systematic review and mixed research synthesis

BACKGROUND

Dementia education is required to address gaps in dementia-specific knowledge among health and social care practitioners amidst increasing dementia prevalence. Harnessing technology for dementia education may remove obstacles to traditional education and empower large communities of learners. This systematic review aimed to establish the technological and pedagogical characteristics associated with effective technology-enabled dementia education for health and social care practitioners.

METHODS

MEDLINE, PubMed, Web of Science, CINAHL, Scopus, PsycINFO, ERIC and OVID Nursing Database were searched from January 2005 until February 2020. Quantitative, qualitative and mixed methods studies were eligible for inclusion. Study quality was assessed with the Mixed Methods Appraisal Tool. Quantitative evidence was categorised based on Kirkpatrick's Model. Qualitative data was synthesised thematically and integrated with quantitative findings before conclusions were drawn.

RESULTS

Twenty-one published papers were identified. Participants were acute, primary and long-term care practitioners, or were students in higher education. Most training was internet-based; CD-ROMs, simulations and tele-mentoring were also described. Technology-enabled dementia education was predominantly associated with positive effects on learning outcomes. Case-based instruction was the most frequently described instructional strategy and videos were common modes of information delivery. Qualitative themes emerged as existing strengths and experience; knowledge gaps and uncertainty; developing core competence and expertise; involving relevant others; and optimising feasibility.

DISCUSSION

Technology-enabled dementia education is likely to improve dementia knowledge, skills and attitudes among health and social care practitioners from multiple practice contexts. Confidence in the results from quantitative studies was undermined by multiple confounding factors that may be difficult to control in the educational research context. Convenience and flexibility are key benefits of technology-enabled instructive and simulated pedagogy that can support the application of theory into practice. More research is required to understand the role of online learning networks and provisions for equitable engagement. A future emphasis on organisational and environmental factors may elucidate the role of technology in ameliorating obstacles to traditional dementia education.

SYSTEMATIC REVIEW REGISTRATION

PROSPERO ( CRD42018115378 ).

Literature searching methods or guidance and their application to public health topics: A narrative review

Background

Information specialists conducting searches for systematic reviews need to consider key questions around which and how many sources to search. This is particularly important for public health topics where evidence may be found in diverse sources.

Objectives

The objective of this review is to give an overview of recent studies on information retrieval guidance and methods that could be applied to public health evidence and used to guide future searches.

Methods

A literature search was performed in core databases and supplemented by browsing health information journals and citation searching. Results were sifted and reviewed.

Results

Seventy‐two papers were found and grouped into themes covering sources and search techniques. Public health topics were poorly covered in this literature.

Discussion

Many researchers follow the recommendations to search multiple databases. The review topic influences decisions about sources. Additional sources covering grey literature eliminate bias but are time‐consuming and difficult to search systematically. Public health searching is complex, often requiring searches in multidisciplinary sources and using additional methods.

Conclusions

Search planning is advisable to enable decisions about which and how many sources to search. This could improve with more work on modelling search scenarios, particularly in public health topics, to examine where publications were found and guide future research.

Data Curation Implications of Qualitative Data Reuse and Big Social Research

Objective: Big social data (such as social media and blogs) and archived qualitative data (such as interview transcripts, field notebooks, and diaries) are similar, but their respective communities of practice are under-connected. This paper explores shared challenges in qualitative data reuse and big social research and identifies implications for data curation. Methods: This paper uses a broad literature search and inductive coding of 300 articles relating to qualitative data reuse and big social research. The literature review produces six key challenges relating to data use and reuse that are present in both qualitative data reuse and big social research—context, data quality, data comparability, informed consent, privacy & confidentiality, and intellectual property & data ownership. Results: This paper explores six key challenges related to data use and reuse for qualitative data and big social research and discusses their implications for data curation practices. Conclusions: Data curators can benefit from understanding these six key challenges and examining data curation implications. Data curation implications from these challenges include strategies for: providing clear documentation; linking and combining datasets; supporting trustworthy repositories; using and advocating for metadata standards; discussing alternative consent strategies with researchers and IRBs; understanding and supporting deidentification challenges; supporting restricted access for data; creating data use agreements; supporting rights management and data licensing; developing and supporting alternative archiving strategies. Considering these data curation implications will help data curators support sounder practices for both qualitative data reuse and big social research.

Reporting of the methodological quality of search strategies in orthodontic quantitative systematic reviews

BACKGROUND

This study aimed to assess the reporting of the methodological quality of search strategies undertaken in orthodontic quantitative systematic reviews (SRs) and hence their reproducibility.

MATERIALS AND METHODS

A search of a single electronic database (Medline via PubMed) was undertaken to identify interventional orthodontic SRs with meta-analysis published within a 10-year period. The Cochrane Library of Systematic Reviews was also sourced. Full articles were reviewed by two assessors against the eligibility criteria. The reporting quality of each search strategy was assessed using a previously validated checklist with a score of 1 or 2 given for each of the eight items. Cumulative totals were calculated. Guided by previous research, the authors agreed the following cut-offs to categorize the overall level of quality: 8-10 (poor), 10-12 (fair), and greater than 13 (good).

RESULTS

A total of 127 SRs were analysed. The overall median quality score for the reporting of the search strategy was 14 [interquartile range (IQR): 13-15]. Cochrane SRs and those originating in Europe received higher aggregate scores, whereas no difference was evident based on Prospero registration. The continent of the corresponding author predicated the overall score. Non-Cochrane reviews achieved lower overall scores compared to Cochrane reviews (-1.0, 95% confidence interval: -1.65, -0.34, P = 0.003). The most frequently searched database was EMBASE (N = 93) and the median number of authors was 5 (IQR 4-6). Authors of 26.8% of SRs searched the grey literature. Language restrictions were applied to the search strategies of 88 (69.3%) SRs.

CONCLUSIONS

The reporting quality of search strategies undertaken in orthodontic SRs is at a good level but differences between Cochrane and non-Cochrane reviews currently exist. The reporting of searching of the grey literature and application of no language restrictions can be improved.

Searching for medical devices - practical guidance

Systematic reviews of medical devices have generally adopted the same methodology for the conduct of their reviews as reviews of other clinical interventions, in particular, medicines. It has been suggested that these methods may need to be developed to account for the challenges of reviewing the evidence for medical devices when compared to reviews of medicines. The purpose of this research note is to focus on the methods of searching for medical devices. Our aim is to set out guidance on 'how to search' for medical devices. This includes: Defining what you are searching for; How to design a search strategy; Searching bibliographic databases; Searching beyond bibliographic databases; and Search reporting. The research note is written by three experienced searchers/researchers with experience of critically appraising Medical Technologies Evaluation Programme (MTEP) submissions, undertaking systematic searches and/or reviews of medical devices, or developing guidance for searching for studies for systematic reviews of the effectiveness of interventions, including medical devices. The research note includes a worked example of a search for a fictitious medical device. This article is protected by copyright. All rights reserved.

Educational interventions to improve literature searching skills in the health sciences: a scoping review

Objective:

The authors reviewed educational interventions for improving literature searching skills in the health sciences.

Methods:

We performed a scoping review of experimental and quasi-experimental studies published in English and German, irrespective of publication year. Targeted outcomes were objectively measurable literature searching skills (e.g., quality of search strategy, study retrieval, precision). The search methods consisted of searching databases (CINAHL, Embase, MEDLINE, PsycINFO, Web of Science), tracking citations, free web searching, and contacting experts. Two reviewers performed screening and data extraction. To evaluate the completeness of reporting, the Template for Intervention Description and Replication (TIDieR) was applied.

Results:

We included 6 controlled trials and 8 pre-post trials from the 8,484 references that we screened. Study participants were students in various health professions and physicians. The educational formats of the interventions varied. Outcomes clustered into 2 categories: (1) developing search strategies (e.g., identifying search concepts, selecting databases, applying Boolean operators) and (2) database searching skills (e.g., searching PubMed, MEDLINE, or CINAHL). In addition to baseline and post-intervention measurement, 5 studies reported follow-up. Almost all studies adequately described their intervention procedures and delivery but did not provide access to the educational material. The expertise of the intervention facilitators was described in only 3 studies.

Conclusions:

The results showed a wide range of study populations, interventions, and outcomes. Studies often lacked information about educational material and facilitators' expertise. Further research should focus on intervention effectiveness using controlled study designs and long-term follow-up. To ensure transparency, replication, and comparability, studies should rigorously describe their intervention.

Correlation between Twitter mentions and academic citations in sexual medicine journals

Social media services, especially Twitter, are used as a commonly sharing tool in the scientific world. This widespread use of Twitter would be an effective method in spreading academic publications. So, we aimed to investigate the relationship between Twitter mentions and traditional citations of articles in sexual medicine journals in this study. We reviewed the articles published in seven journals of sexual medicine (2 years after the publication of the articles) between January 2018 and June 2018. In the first half of 2018, 410 articles were extracted. Of these, 352 (85.9%) were original articles, while 58 (14.1%) were review articles. The median number of citations of the articles mentioned at least once on Twitter was 7 (interquartile range: 0-111) for Google Scholar, whereas it was 0 (interquartile range: 0-63) for Scopus, respectively. It was 4 (interquartile range: 0-25) for Google Scholar and 0 (interquartile range: 0-7) for Scopus. The publications mentioned on Twitter were cited more than the non-mentioned publications in the traditional-based citation system (p < 0.001). A significant relationship between the citation numbers and tweet numbers was also observed (p < 0.001). Also, in the linear regression model, the tweet numbers (p < 0.001) and article types (p < 0.001) were found to be related to the Google Scholar citation numbers. In conclusion, using Twitter as a professional tool in academic life would allow information to be propagated and responded quickly, especially for sexual medicine journals.

The decisions and processes involved in a systematic search strategy: a hierarchical framework

OBJECTIVE

The decisions and processes that may compose a systematic search strategy have not been formally identified and categorized. This study aimed to (1) identify all decisions that could be made and processes that could be used in a systematic search strategy and (2) create a hierarchical framework of those decisions and processes.

METHODS

The literature was searched for documents or guides on conducting a literature search for a systematic review or other evidence synthesis. The decisions or processes for locating studies were extracted from eligible documents and categorized into a structured hierarchical framework. Feedback from experts was sought to revise the framework. The framework was revised iteratively and tested using recently published literature on systematic searching.

RESULTS

Guidance documents were identified from expert organizations and a search of the literature and Internet. Data were extracted from 74 eligible documents to form the initial framework. The framework was revised based on feedback from 9 search experts and further review and testing by the authors. The hierarchical framework consists of 119 decisions or processes sorted into 17 categories and arranged under 5 topics. These topics are "Skill of the searcher," "Selecting information to identify," "Searching the literature electronically," "Other ways to identify studies," and "Updating the systematic review."

CONCLUSIONS

The work identifies and classifies the decisions and processes used in systematic searching. Future work can now focus on assessing and prioritizing research on the best methods for successfully identifying all eligible studies for a systematic review.

Overlaps of multiple database retrieval and citation tracking in dementia care research: a methodological study

OBJECTIVE

We aimed to determine overlaps and optimal combination of multiple database retrieval and citation tracking for evidence synthesis, based on a previously conducted scoping review on facilitators and barriers to implementing nurse-led interventions in dementia care.

METHODS

In our 2019 scoping review, we performed a comprehensive literature search in eight databases (CENTRAL, CINAHL, Embase, Emcare, MEDLINE, Ovid Nursing Database, PsycINFO, and Web of Science Core Collection) and used citation tracking. We retrospectively analyzed the coverage and overlap of 10,527 retrieved studies published between 2015 and 2019. To analyze database overlap, we used cross tables and multiple correspondence analysis (MCA).

RESULTS

Of the retrieved studies, 6,944 were duplicates and 3,583 were unique references. Using our search strategies, considerable overlaps can be found in some databases, such as between MEDLINE and Web of Science Core Collection or between CINAHL, Emcare, and PsycINFO. Searching MEDLINE, CINAHL, and Web of Science Core Collection and using citation tracking were necessary to retrieve all included studies of our scoping review.

CONCLUSIONS

Our results can contribute to enhancing future search practice related to database selection in dementia care research. However, due to limited generalizability, researchers and librarians should carefully choose databases based on the research question. More research on optimal database retrieval in dementia care research is required for the development of methodological standards.

What you see depends on where you sit: The effect of geographical location on web-searching for systematic reviews: A case study

There is limited guidance on how to web-search in systematic reviews and concern relates to the reproducibility of searches using search engines such as Google. The aim of this paper is to address one potential source of variation in Google searches: does the geographical location of a researcher affect Google search returns? Using a virtual private network, we ran the same web-search for the medical technology Dasatinib in 12 different countries. Two researchers independently extracted the search returns by country organised by page rank. We compared: C1. any difference in the items returned by Google searches between countries and C2. any difference in the page rank of items returned between countries. Searches were undertaken on Monday September 28th 2020. From 12 countries, 43 items were identified. For C1: 19 items were common to all 12 countries. Twenty-four items were missed by searches in some countries. This means that there were differences in search returns between countries. For C2: a randomised trial reported by Raddich et al was the first search return for all countries. All other items, common to all countries, varied in their page-rank. We find that geographic location would appear to influence Google search returns based on the findings of this case study. The findings suggest that recording the location of the researcher undertaking web-searching may now be an important factor to report alongside detail on steps taken to minimise personalisation of web-searches covered by recent guidance. This finding also has implications for stopping-rules.

A technical review of three clinical trials register resources indicates where improvements to the search interfaces are needed

Clinical trials registers form an important part of the search for studies in systematic reviews of intervention effectiveness but the search interfaces and functionality of registers can be challenging to search systematically and resource intensive to search well. We report a technical review of the search interfaces of three leading trials register resources: ClinicalTrials.gov, the EU Clinical Trials Register and the WHO International Clinical Trials Registers Platform. The technical review used a validated checklist to identify areas where the search interfaces of these trials register resources performed well, where performance was adequate, where performance was poor, and to identify differences between search interfaces. The review found low overall scores for each of the interfaces (ClinicalTrials.gov 55/165, the EU Clinical Trials Register 25/165, the WHO International Clinical Trials Registers Platform 32/165). This finding suggests a need for joined-up dialogue between the producers of the registers and researchers who search them via these interfaces. We also set out a series of four proposed changes which might improve the search interfaces. Trials registers are an invaluable resource in systematic reviews of intervention effectiveness. With the continued growth in systematic reviews, and initiatives such as 'AllTrials', there is an anticipated need for these resources. We conclude that small changes to the search interfaces, and improved dialogue with providers, might improve the future search functionality of these valuable resources.

Using citation tracking for systematic literature searching - study protocol for a scoping review of methodological studies and an expert survey

Background: Up-to-date guidance on comprehensive study identification for systematic reviews is crucial. According to current recommendations, systematic searching should combine electronic database searching with supplementary search methods. One such supplementary search method is citation tracking. It aims at collecting directly and/or indirectly cited and citing references from "seed references". Tailored and evidence-guided recommendations concerning the use of citation tracking are strongly needed. Objective: We intend to develop recommendations for the use of citation tracking in health-related systematic literature searching. Our study will be guided by the following research questions: What are the benefits of citation tracking for health-related systematic literature searching? Which perspectives and experiences do experts in the field of literature retrieval methods have with regard to citation tracking in health-related systematic literature searching? Methods: Our study will have two parts: a scoping review and an expert survey. The scoping review aims at identifying methodological studies on benefits or problems of citation tracking in health-related systematic literature searching with no restrictions on study design, language, and publication date. We will perform database searching in MEDLINE, The Cumulative Index to Nursing and Allied Health Literature (CINAHL), Web of Science Core Collection, two information science databases, and free web searching. Two reviewers will independently assess full texts of selected abstracts. We will conduct direct backward and forward citation tracking on included articles. The results of the scoping review will inform our expert survey through which we aim to learn about experts΄ perspectives and experiences. We will narratively synthesize the results and derive recommendations for performing health-related systematic reviews.

Using citation tracking for systematic literature searching - study protocol for a scoping review of methodological studies and a Delphi study

Background: Up-to-date guidance on comprehensive study identification for systematic reviews is crucial. According to current recommendations, systematic searching should combine electronic database searching with supplementary search methods. One such supplementary search method is citation tracking. It aims at collecting directly and/or indirectly cited and citing references from "seed references". Tailored and evidence-guided recommendations concerning the use of citation tracking are strongly needed. Objective: We intend to develop recommendations for the use of citation tracking in systematic literature searching for health-related topics. Our study will be guided by the following research questions: What is the benefit of citation tracking for systematic literature searching for health-related topics? Which methods, citation indexes, and other tools are used for citation tracking? What terminology is used for citation tracking methods? Methods: Our study will have two parts: a scoping review and a Delphi study. The scoping review aims at identifying methodological studies on the benefit and use of citation tracking in systematic literature searching for health-related topics with no restrictions on study design, language, and publication date. We will perform database searching in MEDLINE (Ovid), CINAHL (EBSCOhost), Web of Science Core Collection, two information science databases, web searching, and contact experts in the field. Two reviewers will independently perform study selection. We will conduct direct backward and forward citation tracking on included articles. Data from included studies will be extracted using a prespecified extraction sheet and presented in both tabular and narrative form. The results of the scoping review will inform the subsequent Delphi study through which we aim to derive consensus recommendations for the future practice and research of citation tracking.

Using citation tracking for systematic literature searching - study protocol for a scoping review of methodological studies and a Delphi study

Background: Up-to-date guidance on comprehensive study identification for systematic reviews is crucial. According to current recommendations, systematic searching should combine electronic database searching with supplementary search methods. One such supplementary search method is citation tracking. It aims at collecting directly and/or indirectly cited and citing references from "seed references". Tailored and evidence-guided recommendations concerning the use of citation tracking are strongly needed. Objective: We intend to develop recommendations for the use of citation tracking in systematic literature searching for health-related topics. Our study will be guided by the following research questions: What is the benefit of citation tracking for systematic literature searching for health-related topics? Which methods, citation indexes, and other tools are used for citation tracking? What terminology is used for citation tracking methods? Methods: Our study will have two parts: a scoping review and a Delphi study. The scoping review aims at identifying methodological studies on the benefit and use of citation tracking in systematic literature searching for health-related topics with no restrictions on study design, language, and publication date. We will perform database searching in MEDLINE (Ovid), CINAHL (EBSCOhost), Web of Science Core Collection, two information science databases, web searching, and contact experts in the field. Two reviewers will independently perform study selection. We will conduct direct backward and forward citation tracking on included articles. Data from included studies will be extracted using a prespecified extraction sheet and presented in both tabular and narrative form. The results of the scoping review will inform the subsequent Delphi study through which we aim to derive consensus recommendations for the future practice and research of citation tracking.

Cancer diagnostic tools to aid decision-making in primary care: mixed-methods systematic reviews and cost-effectiveness analysis

BACKGROUND

Tools based on diagnostic prediction models are available to help general practitioners diagnose cancer. It is unclear whether or not tools expedite diagnosis or affect patient quality of life and/or survival.

OBJECTIVES

The objectives were to evaluate the evidence on the validation, clinical effectiveness, cost-effectiveness, and availability and use of cancer diagnostic tools in primary care.

METHODS

Two systematic reviews were conducted to examine the clinical effectiveness (review 1) and the development, validation and accuracy (review 2) of diagnostic prediction models for aiding general practitioners in cancer diagnosis. Bibliographic searches were conducted on MEDLINE, MEDLINE In-Process, EMBASE, Cochrane Library and Web of Science) in May 2017, with updated searches conducted in November 2018. A decision-analytic model explored the tools' clinical effectiveness and cost-effectiveness in colorectal cancer. The model compared patient outcomes and costs between strategies that included the use of the tools and those that did not, using the NHS perspective. We surveyed 4600 general practitioners in randomly selected UK practices to determine the proportions of general practices and general practitioners with access to, and using, cancer decision support tools. Association between access to these tools and practice-level cancer diagnostic indicators was explored.

RESULTS

Systematic review 1 - five studies, of different design and quality, reporting on three diagnostic tools, were included. We found no evidence that using the tools was associated with better outcomes. Systematic review 2 - 43 studies were included, reporting on prediction models, in various stages of development, for 14 cancer sites (including multiple cancers). Most studies relate to QCancer® (ClinRisk Ltd, Leeds, UK) and risk assessment tools.

DECISION MODEL

In the absence of studies reporting their clinical outcomes, QCancer and risk assessment tools were evaluated against faecal immunochemical testing. A linked data approach was used, which translates diagnostic accuracy into time to diagnosis and treatment, and stage at diagnosis. Given the current lack of evidence, the model showed that the cost-effectiveness of diagnostic tools in colorectal cancer relies on demonstrating patient survival benefits. Sensitivity of faecal immunochemical testing and specificity of QCancer and risk assessment tools in a low-risk population were the key uncertain parameters.

SURVEY

Practitioner- and practice-level response rates were 10.3% (476/4600) and 23.3% (227/975), respectively. Cancer decision support tools were available in 83 out of 227 practices (36.6%, 95% confidence interval 30.3% to 43.1%), and were likely to be used in 38 out of 227 practices (16.7%, 95% confidence interval 12.1% to 22.2%). The mean 2-week-wait referral rate did not differ between practices that do and practices that do not have access to QCancer or risk assessment tools (mean difference of 1.8 referrals per 100,000 referrals, 95% confidence interval -6.7 to 10.3 referrals per 100,000 referrals).

LIMITATIONS

There is little good-quality evidence on the clinical effectiveness and cost-effectiveness of diagnostic tools. Many diagnostic prediction models are limited by a lack of external validation. There are limited data on current UK practice and clinical outcomes of diagnostic strategies, and there is no evidence on the quality-of-life outcomes of diagnostic results. The survey was limited by low response rates.

CONCLUSION

The evidence base on the tools is limited. Research on how general practitioners interact with the tools may help to identify barriers to implementation and uptake, and the potential for clinical effectiveness.

FUTURE WORK

Continued model validation is recommended, especially for risk assessment tools. Assessment of the tools' impact on time to diagnosis and treatment, stage at diagnosis, and health outcomes is also recommended, as is further work to understand how tools are used in general practitioner consultations.

STUDY REGISTRATION

This study is registered as PROSPERO CRD42017068373 and CRD42017068375.

FUNDING

This project was funded by the National Institute for Health Research (NIHR) Health Technology programme and will be published in full in Health Technology Assessment; Vol. 24, No. 66. See the NIHR Journals Library website for further project information.

Development, testing and use of data extraction forms in systematic reviews: a review of methodological guidance

Background

Data extraction forms link systematic reviews with primary research and provide the foundation for appraising, analysing, summarising and interpreting a body of evidence. This makes their development, pilot testing and use a crucial part of the systematic reviews process. Several studies have shown that data extraction errors are frequent in systematic reviews, especially regarding outcome data.

Methods

We reviewed guidance on the development and pilot testing of data extraction forms and the data extraction process. We reviewed four types of sources: 1) methodological handbooks of systematic review organisations (SRO); 2) textbooks on conducting systematic reviews; 3) method documents from health technology assessment (HTA) agencies and 4) journal articles. HTA documents were retrieved in February 2019 and database searches conducted in December 2019. One author extracted the recommendations and a second author checked them for accuracy. Results are presented descriptively.

Results

Our analysis includes recommendations from 25 documents: 4 SRO handbooks, 11 textbooks, 5 HTA method documents and 5 journal articles. Across these sources the most common recommendations on form development are to use customized or adapted standardised extraction forms (14/25); provide detailed instructions on their use (10/25); ensure clear and consistent coding and response options (9/25); plan in advance which data are needed (9/25); obtain additional data if required (8/25); and link multiple reports of the same study (8/25). The most frequent recommendations on piloting extractions forms are that forms should be piloted on a sample of studies (18/25); and that data extractors should be trained in the use of the forms (7/25). The most frequent recommendations on data extraction are that extraction should be conducted by at least two people (17/25); that independent parallel extraction should be used (11/25); and that procedures to resolve disagreements between data extractors should be in place (14/25).

Conclusions

Overall, our results suggest a lack of comprehensiveness of recommendations. This may be particularly problematic for less experienced reviewers. Limitations of our method are the scoping nature of the review and that we did not analyse internal documents of health technology agencies.

Understanding surgical antimicrobial prescribing behaviour in the hospital setting: a systematic review and meta-ethnography protocol

BACKGROUND

Surgical specialities use extensive amounts of antimicrobials, and misuse has been widely reported, making them a key target for antimicrobial stewardship initiatives. Interventions informed by, and tailored to, a clear understanding of the contextual barriers to appropriate antimicrobial use are more likely to successfully improve practice. However, this approach has been under utilised. Our aim is to synthesise qualitative studies on surgical antimicrobial prescribing behaviour (APB) in hospital settings to explain how and why contextual factors act and interact to influence APB amongst surgical teams. We will develop new theory to advance understanding and identify knowledge gaps to inform further research.

METHODS

The meta-ethnography will follow the seven-phase method described by Noblit and Hare. We will conduct a comprehensive search using eight databases (AMED, CINAHL, EMBASE, MEDLINE, MEDLINE-in-process, Web of Science, Cochrane Library and PsycINFO) with no date restrictions; forwards and backwards citation searches; and contacting first authors of relevant papers. Studies will be dual screened and included if they use recognised qualitative methods and analysis; focus on contextual factors associated with surgical APB within hospital settings; are available in full in English; and are relevant to the research question. Any disagreements between reviewers will be resolved through discussion to reach consensus. Included studies will be read repeatedly to illuminate key concepts and the relationship between key concepts across studies. Then, key concepts will be sorted into conceptual categories or 'piles' which will be further abstracted to form a conceptual framework explaining surgical APB. During the synthesis, emerging interpretations will be discussed with stakeholders (including authors of included studies where possible; surgical and stewardship practitioners; and patient representatives) to ensure new knowledge is meaningful.

DISCUSSION

This research has several strengths: (1) the protocol has been written with reference to established guidance maximising rigour and transparency; (2) the multi-disciplinary research team bring varied interpretative repertoires and relevant methodological skills; and (3) stakeholders will be involved to ensure that findings are relevant, and disseminated via suitable channels, to support improved patient care.

SYSTEMATIC REVIEW REGISTRATION

PROSPERO CRD42020184343.

Bibliometric Study of Technology and Occupational Health in Healthcare Sector: A Worldwide Trend to the Future

Since the eighties, technological tools have modified how people interact in their environment. At the same time, occupational safety and health measures have been widely applied. The European Agency for Safety and Health at Work considers that information and communication technologies are the main methods to achieve the goals proposed to improve working life and the dissemination of good practices. The principal objective was to determine the trends of publications focused on these technologies and occupational safety in the healthcare sector during the last 30 years. A bibliometric study was carried out. The 1021 documents showed an increased trend per country, especially for the United States (p < 0.001) and year (p < 0.001). The citations per year showed significant differences between citations of articles published before 2007 (p < 0.001). The year was also linked to the increase or decrease of articles (72.2%) and reviews (14.9%) (p < 0.001). The analysis of journal co-citations also showed that the main journals (such as Infection Control and Hospital Epidemiology) were linked to other important journals and had a major part in the clusters formed. All these findings were discussed in the manuscript and conclusions were drawn.

"It has no meaning to me." How do researchers understand the effectiveness of literature searches? A qualitative analysis and preliminary typology of understandings

This study aimed to address the question: what does "effectiveness" mean to researchers in the context of literature searching for systematic reviews? We conducted a thematic analysis of responses to an e-mail survey. Eighty-nine study authors, whose studies met inclusion in a recent review (2018), were contacted via e-mail and asked three questions; one directly asking the question: in literature searching, what does effective (or effectiveness in) literature searching mean to you? Thirty-eight (46%) responses were received from diverse professional groups, including: literature searchers, systematic reviewers, clinicians and researchers. A shared understanding of what effectiveness means was not identified. Instead, five themes were developed from data: (a) effectiveness is described as a metric; (b) effectiveness is a balance between metrics; (c) effectiveness can be categorized by search purpose; (d) effectiveness is an outcome; and, (e) effectiveness is an experimental concept. We propose that these themes constitute a preliminary typology of understandings. No single definition of effectiveness was identified. The proposed typology suggests that different researchers have differing understandings of effectiveness. This could lead to uncertainty as to the aim and the purpose of literature searches and confusion about the outcomes. The typology offers a potential route for further exploration.

Factors influencing dietary behaviours in urban food environments in Africa: a systematic mapping review

Objective:

To identify factors influencing dietary behaviours in urban food environments in Africa and identify areas for future research.

Design:

We systematically reviewed published/grey literature (protocol CRD4201706893). Findings were compiled into a map using a socio-ecological model on four environmental levels: individual, social, physical and macro.

Setting:

Urban food environments in Africa.

Participants:

Studies involving adolescents and adults (11–70 years, male/female).

Results:

Thirty-nine studies were included (six adolescent, fifteen adolescent/adult combined and eighteen adult). Quantitative methods were most common (twenty-eight quantitative, nine qualitative and two mixed methods). Studies were from fifteen African countries. Seventy-seven factors influencing dietary behaviours were identified, with two-thirds at the individual level (45/77). Factors in the social (11/77), physical (12/77) and macro (9/77) environments were investigated less. Individual-level factors that specifically emerged for adolescents included self-esteem, body satisfaction, dieting, spoken language, school attendance, gender, body composition, pubertal development, BMI and fat mass. Studies involving adolescents investigated social environment-level factors more, for example, sharing food with friends. The physical food environment was more commonly explored in adults, for example, convenience/availability of food. Macro-level factors associated with dietary behaviours were food/drink advertising, religion and food prices. Factors associated with dietary behaviour were broadly similar for men and women.

Conclusions:

The dominance of studies exploring individual-level factors suggests a need for research to explore how social, physical and macro-level environments drive dietary behaviours of adolescents and adults in urban Africa. More studies are needed for adolescents and men, and studies widening the geographical scope to encompass all African countries.

[Database indexing of health science journals from the German-speaking area: A journal analysis]

BACKGROUND

Journal hand searching offers the possibility to complement a literature search as part of systematic reviews and other evidence syntheses. Hand searching is indicated in cases where scientific journals with potentially relevant publications addressing the research question are not indexed in a literature database. However, it is often unclear whether these journals are actually indexed, and when they are, in which literature databases. In many cases, it is also unknown which journals should be searched by hand in addition to systematic literature search after databases to be searched have been specified. Therefore, the project aimed to investigate the indexation of selected scientific health science journals and to provide an overview of indexation in order to facilitate the hand search planning process.

METHODS

Journals from German-speaking countries covering eight professional fields (medical laboratory assistance, occupational therapy, midwifery, logopedics, nursing, physiotherapy, public health and rehabilitation) were considered that publish original research papers or systematic reviews or other review types in German and/or English. Two researchers per field identified relevant journals and independently analyzed the indexing locations using the journal websites. In case of missing information, we contacted the editors.

RESULTS

A total of 70 journals were included: from 1 to 17 journals per field. These journals are indexed in 1 to 29 databases. Twelve journals are not indexed or do not offer information concerning indexation. Indexation is distributed across n=74 different literature databases. Most journals are indexed in LIVIVO (n=55) and bibnet.org (n=33). Other common indexing databases are Scopus (n=18), Web of Science Core Collection (n=16), PSYNDEX (n=13), and Embase (n=10).

CONCLUSIONS

The results indicate a heterogeneous indexation of the included journals. Only a small number is indexed in common international literature databases such as MEDLINE or CINAHL. On the other hand, only a few journals are not indexed in any database. The results can be used as a basis to define databases for literature searches as part of systematic reviews. In addition, the findings might guide the selection of journals for hand searching after literature databases have been defined.

Conduct and reporting of citation searching in Cochrane systematic reviews: A cross-sectional study

BACKGROUND

The search for studies for a systematic review should be conducted systematically and reported transparently to facilitate reproduction. This study aimed to report on the conduct and reporting of backward citation searching (ie, checking reference lists) and forward citation searching in a cross section of Cochrane reviews. Citation searching uses the citation network surrounding a source study to identify additional studies.

METHODS

Cochrane reviews were identified by searching the Cochrane Database of Systematic Reviews using the wildcard symbol and date limiting to the 3-month period November 2016 to January 2017. Cochrane reviews thus identified were screened for mention of citation searching. Descriptive detail on the conduct and reporting of citation searching was captured in data extraction forms and described and evaluated.

RESULTS

Two hundred fifteen Cochrane reviews were identified. One hundred seventy-two reviews reported backward citation searching, and 18 reviews reported forward citation searching. Web of Science was the most frequently reported citation index. The studies used for backward citation searching consisted mainly of studies meeting the inclusion criteria. One-third of reviews that reported forward citation searching used selected studies of importance. Reporting of citation searching was compliant with the Methodological Expectations of Cochrane Intervention Reviews (MECIR) standards, but full transparency requires additional detail that only a minority of reviews reported.

CONCLUSION

The conduct of backward citation searching was more uniform than forward citation searching. This might be due to lack of MECIR guidance for forward citation searching. Reporting was generally compliant with MECIR, but this is not always sufficient to ensure full transparency.