Which academic search systems are suitable for systematic reviews or meta-analyses? Evaluating retrieval qualities of Google Scholar, PubMed, and 26 other resources

Guidelines for developing a systematic literature review for studies related to climate change adaptation

This study proposes a set of GuFSyADD guidelines on steps for developing  suggestions that  enhance of its rigor in systematic literature review (SLR) for studies related to climate change adaptation. The prescribed guidelines are based on the following six steps, (1) guided by review of protocol/publication standard/established guidelines/related published articles, (2) formulation of review questions, (3) systematic searching strategies, (4) appraisal of quality, (5) data extraction and analysis, and (6) data demonstration. Essentially, this set of proposed  guidelines enables researchers to develop an SLR pertaining to climate change adaptation in an organised, transparent, and replicable manner.

Citizenship Education for Political Engagement: A Systematic Review of Controlled Trials

Citizenship Education could play a pivotal role in creating a fairer society in which all groups participate equally in the political progress. But strong causal evidence of which educational techniques work best to create political engagement is lacking. This paper presents the results of a systematic review of controlled trials within the field based on transparent search protocols. It finds 25 studies which use controlled trials to test causal claims between Citizenship Education programs and political engagement outcomes. The studies identified largely confirm accepted ideas, such as the importance of participatory methods, whole school approaches, teacher training, and doubts over whether knowledge alone or online engagement necessarily translate into behavioral change. But the paucity of identified studies also points both to the difficulties of attracting funding for controlled trials which investigate Citizenship Education as a tool for political engagement and real epistemological tensions within the discipline itself.

Proton pump inhibitor on susceptibility to COVID-19 and its severity: a systematic review and meta-analysis

BACKGROUND

The negative impacts of proton pump inhibitor (PPI), including the risk of pneumonia and mortality, have been reported previously. This meta-analysis aimed to address the current interest of whether the administration of PPI could increase the susceptibility and risk of poor outcome in COVID-19.

METHODS

We performed a systematic literature search from PubMed, Embase, EBSCOhost, and EuropePMC databases up until 3 December 2020. The main outcome was composite poor outcome which comprised of mortality and severe COVID-19. Severe COVID-19 in this study was defined as patients with COVID-19 that fulfill the criteria for severe CAP, including the need for intensive unit care or mechanical ventilation. The secondary outcome was susceptibility, based on cohort comparing COVID-19 positive and COVID-19 negative participants.

RESULTS

There were a total of 290,455 patients from 12 studies in this meta-analysis. PPI use was associated with increased composite poor outcome (OR 1.85 [1.13, 3.03], p = 0.014; I² 90.26%). Meta-regression analysis indicate that the association does not vary by age (OR 0.97 [0.92, 1.02], p = 0.244), male (OR 1.05 [0.99, 1.11], p = 0.091), hypertension (OR 9.98 [0.95, 1.02], p = 0.317), diabetes (OR 0.99 [0.93, 1.05], p = 0.699), chronic kidney disease (OR 1.01 [0.93, 1.10], p = 0.756), non-steroidal anti-inflammatory drug use (OR 1.02 [0.96, 1.09], p = 0.499), and pre-admission/in-hospital PPI use (OR 0.77 [0.26, 2.31], p = 0.644). PPI use was not associated with the susceptibility to COVID-19 (OR 1.56 [0.48, 5.05], p = 0.46; I² 99.7%).

CONCLUSION

This meta-analysis showed a potential association between PPI use and composite poor outcome, but not susceptibility.

PROSPERO ID

CRD42020224286.

What you see depends on where you sit: The effect of geographical location on web-searching for systematic reviews: A case study

There is limited guidance on how to web-search in systematic reviews and concern relates to the reproducibility of searches using search engines such as Google. The aim of this paper is to address one potential source of variation in Google searches: does the geographical location of a researcher affect Google search returns? Using a virtual private network, we ran the same web-search for the medical technology Dasatinib in 12 different countries. Two researchers independently extracted the search returns by country organised by page rank. We compared: C1. any difference in the items returned by Google searches between countries and C2. any difference in the page rank of items returned between countries. Searches were undertaken on Monday September 28th 2020. From 12 countries, 43 items were identified. For C1: 19 items were common to all 12 countries. Twenty-four items were missed by searches in some countries. This means that there were differences in search returns between countries. For C2: a randomised trial reported by Raddich et al was the first search return for all countries. All other items, common to all countries, varied in their page-rank. We find that geographic location would appear to influence Google search returns based on the findings of this case study. The findings suggest that recording the location of the researcher undertaking web-searching may now be an important factor to report alongside detail on steps taken to minimise personalisation of web-searches covered by recent guidance. This finding also has implications for stopping-rules.

The Legal Vulnerability Model for Same-Sex Parent Families: A Mixed Methods Systematic Review and Theoretical Integration

Globally, parents and children in same-sex parent families are impacted by many laws related to the parental sexual orientation. These laws vary considerably from one country to another, ranging from full legal recognition to criminalization. The psychological consequences of living in an ambiguous or hostile legal climate likely interfere with parental health, family functioning, and child development. However, a systematic evidence synthesis of the pertinent literature and its placement within a broader psychological model are currently lacking. The aims of this review were thus (1) to systematically review qualitative and quantitative evidence on the impact of sexual orientation laws on same-sex parent families in key domains and (2) to place these findings within a broader model informed by minority stress and family theories. Our review was preregistered and conducted in line with PRISMA guidelines. We searched for qualitative, quantitative, and mixed methods studies on the impact of sexual orientation laws on target outcomes (parental health, family functioning, child outcomes) via systematic database search (PubMed, Scopus, Web of Science) and forward-backward searches. Fifty-five studies published between 1999 and 2020 were eligible for inclusion and were synthesized using a data-based convergent synthesis design. Thirteen descriptive and three overarching analytical themes were identified through thematic synthesis. Linking our findings with minority stress and family theories, we propose a novel legal vulnerability model for same-sex parent families. The model posits that legal vulnerability constitutes an increased risk for parental and child health as well as family functioning through individual and shared pathways between family members. Additionally, the model introduces counteractions that families engage in on the personal, familial, and systemic level to mitigate the impact of legal vulnerability, as well as moderators on the contextual, familial, couple, and individual level that modulate this impact. Implications for research and clinical practice are discussed.

Web of Science (WoS) and Scopus: The Titans of Bibliographic Information in Today’s Academic World

Nowadays, the importance of bibliographic databases (DBs) has increased enormously, as they are the main providers of publication metadata and bibliometric indicators universally used both for research assessment practices and for performing daily tasks. Because the reliability of these tasks firstly depends on the data source, all users of the DBs should be able to choose the most suitable one. Web of Science (WoS) and Scopus are the two main bibliographic DBs. The comprehensive evaluation of the DBs’ coverage is practically impossible without extensive bibliometric analyses or literature reviews, but most DBs users do not have bibliometric competence and/or are not willing to invest additional time for such evaluations. Apart from that, the convenience of the DB’s interface, performance, provided impact indicators and additional tools may also influence the users’ choice. The main goal of this work is to provide all of the potential users with an all-inclusive description of the two main bibliographic DBs by gathering the findings that are presented in the most recent literature and information provided by the owners of the DBs at one place. This overview should aid all stakeholders employing publication and citation data in selecting the most suitable DB.

A Study on the Characteristics of Academic Topics Related to Renewable Energy Using the Structural Topic Modeling and the Weak Signal Concept

It is important to examine in detail how the distribution of academic research topics related to renewable energy is structured and which topics are likely to receive new attention in the future in order for scientists to contribute to the development of renewable energy. This study uses an advanced probabilistic topic modeling to statistically examine the temporal changes of renewable energy topics by using academic abstracts from 2010–2019 and explores the properties of the topics from the perspective of future signs such as weak signals. As a result, in strong signals, methods for optimally integrating renewable energy into the power grid are paid great attention. In weak signals, interest in large-capacity energy storage systems such as hydrogen, supercapacitors, and compressed air energy storage showed a high rate of increase. In not-strong-but-well-known signals, comprehensive topics have been included, such as renewable energy potential, barriers, and policies. The approach of this study is applicable not only to renewable energy but also to other subjects.

Application of the Socio-Ecological System Framework to Forest Fire Risk Management: A Systematic Literature Review

Although increasing concern about climate change has raised awareness of the fundamental role of forest ecosystems, forests are threatened by human-induced impacts worldwide. Among them, wildfire risk is clearly the result of the interaction between human activities, ecological domains, and climate. However, a clear understanding of these interactions is still needed both at the global and local levels. Numerous studies have proven the validity of the socioecological system (SES) approach in addressing this kind of interdisciplinary issue. Therefore, a systematic review of the existing literature on the application of SES frameworks to forest ecosystems is carried out, with a specific focus on wildfire risk management. The results demonstrate the existence of different methodological approaches that can be grouped into seven main categories, which range from qualitative analysis to quantitative spatially explicit investigations. The strengths and limitations of the approaches are discussed, with a specific reference to the geographical setting of the works. The research suggests the importance of local community involvement and local knowledge consideration in wildfire risk management. This review provides a starting point for future research on forest SES and a supporting tool for the development of a sustainable wildfire risk adaptation and mitigation strategy.

Considerations in Planning Physical Activity for Older Adults in Hot Climates: A Narrative Review

Regular physical activity has multiple health benefits for both the prevention and management of disease, including for older adults. However, additional precautions are needed with ageing given physiological changes and the increasing prevalence of comorbidities. Hot ambient temperatures increase the risks of exercise at any age, but are particularly important given thermoregulatory changes in older people. This narrative review informs planning of physical activity programs for older people living in rural areas with very hot climates for a period of the year. A multi-database search of peer-reviewed literature was undertaken with attention to its relevance to Australia, starting with definitions and standard advice in relation to physical activity programming and the incremental limitations imposed by age, rurality, and extreme heat. The enablers of and barriers to increasing physical activities in older adults and how they can be modified for those living in extreme hot climates is described. We describe multiple considerations in program design to improve safety, adherence and sustaining physical activity, including supervision, simple instructions, provision of reminders, social support, encouraging self-efficacy. Group-based activities may be preferred by some and can accommodate special populations, cultural considerations. Risk management is an important consideration and recommendations are provided to assist program planning.

An open source machine learning framework for efficient and transparent systematic reviews

To help researchers conduct a systematic review or meta-analysis as efficiently and transparently as possible, we designed a tool to accelerate the step of screening titles and abstracts. For many tasks—including but not limited to systematic reviews and meta-analyses—the scientific literature needs to be checked systematically. Scholars and practitioners currently screen thousands of studies by hand to determine which studies to include in their review or meta-analysis. This is error prone and inefficient because of extremely imbalanced data: only a fraction of the screened studies is relevant. The future of systematic reviewing will be an interaction with machine learning algorithms to deal with the enormous increase of available text. We therefore developed an open source machine learning-aided pipeline applying active learning: ASReview. We demonstrate by means of simulation studies that active learning can yield far more efficient reviewing than manual reviewing while providing high quality. Furthermore, we describe the options of the free and open source research software and present the results from user experience tests. We invite the community to contribute to open source projects such as our own that provide measurable and reproducible improvements over current practice.

Sample Selection in Systematic Literature Reviews of Management Research

Systematic review techniques are about to become the “new normal” in reviews of management research. However, there is not yet much advice on how to organize the sample selection process as part of such reviews. This article addresses this void and analyzes this vital part of systematic reviews in more detail. In particular, it offers a critical review of systematic literature reviews published in the Academy of Management Annals and the International Journal of Management Reviews between 2004 and 2018. Based on this methodological literature review, the article presents issues to consider in the most critical choices during the sample selection process. Furthermore, this review identifies several descriptive features such as the mean number of research items included in systematic reviews, the mean number of databases used, and the mean coverage period of such reviews. These numbers may be used as benchmark figures in future reviews.

Emergency remote teaching in higher education: mapping the first global online semester

Due to the Covid-19 pandemic that spread globally in 2020, higher education courses were subsequently offered in fully remote, online formats. A plethora of primary studies began investigating a range of topics exploring teaching and learning in higher education, particularly during the initial semester. In order to provide an overview and initial understanding of this emerging research field, a systematic mapping review was conducted that collates and describes the characteristics of 282 primary empirical studies. Findings reveal that research was carried out mostly descriptively and cross-sectionally, focusing predominantly on undergraduate students and their perceptions of teaching and learning during the pandemic. Studies originate from a broad range of countries, are overwhelmingly published open access, and largely focused on the fields of Health & Welfare and Natural Sciences, Mathematics & Statistics. Educational technology used for emergency remote teaching are most often synchronous collaborative tools, used in combination with text-based tools. The findings are discussed against pre-pandemic research on educational technology use in higher education teaching and learning, and perspectives for further research are provided.

SUPPLEMENTARY INFORMATION

The online version contains supplementary material available at 10.1186/s41239-021-00282-x.

EHealth Investment Appraisal in Africa: A Scoping Review

eHealth is an opportunity cost, competing for limited available funds with other health priorities such as clinics, vaccinations, medicines and even salaries. As such, it should be appraised for probable impact prior to allocation of funds. This is especially pertinent as recognition grows for the role of eHealth in attaining Universal Health Coverage. Despite optimism about eHealth’s potential role, in Africa there remain insufficient data and skills for adequate economic appraisals to select optimal investments from numerous competing initiatives. The aim of this review is to identify eHealth investment appraisal approaches and tools that have been used in African countries, describe their characteristics and make recommendations regarding African eHealth investment appraisal in the face of limited data and expertise. Methods: Literature on eHealth investment appraisals conducted in African countries and published between January 1, 2010 and June 30, 2020 was reviewed. Selected papers’ investment appraisal characteristics were assessed using the Joanna Briggs Institute checklist for economic evaluations and a newly developed Five-Case Model for Digital Health (FCM-DH) checklist for investment appraisal. 5 papers met inclusion criteria. Their assessments revealed important appraisal gaps. In particular, none of the papers addressed risk exposure, affordability, adjustment for optimism bias, clear delivery milestones, practical plans for implementation, change management or procurement, and only 1 paper described plans for building partnerships. Discussion: Using this insight, an extended 5-Case Model is proposed as the foundation of an African eHealth investment appraisal framework. This, combined with building local eHealth appraisal capabilities, may promote optimal eHealth investment decisions, strengthen implementations and improve the number and quality of related publications.

Periviable birth: A review of ethical considerations

BACKGROUND

Advances in perinatology and medical technology have pushed the limits of viability to unprecedented extremes, leading to a growing population of NICU "graduates" with a wide range of health issues. Although survival rates from 22 weeks of gestation onwards have improved over the last 30 years, the incidence of disabilities remains the same. Providing intensive care to a high-risk population with significant mortality and morbidity raises the fundamental conflict between sanctity and quality of life. Potential severe handicap and need for frequent tertiary care inevitably impact the whole family unit and may outweigh the benefit of survival. The aim of this study is to explore and summarize the ethical considerations in neonatal care concerning perivable birth.

METHODS

Eligible studies published on PubMed were included after a systematic search using the PICO methodology.

RESULTS

Forty-eight studies were systematically reviewed regarding guidelines, withholding or withdrawing treatment, parental involvement, and principles applied in marginal viability. As periviable birth raises an array of complex ethical and legal concerns, strict guidelines are challenging to implement.

CONCLUSIONS

Active life-sustaining interventions in neonatology should be balanced against the risk of putting infants through painful and futile procedures and survival with severe sequelae. More evidence is needed on better prediction of long-term outcomes in situations of imminent preterm delivery, while good collaboration between the therapeutic team and the parents for life-and-death decision-making is of utmost importance. HIPPOKRATIA 2021, 25 (1):1-7.

Navigating Governance Tensions to Enhance the Impact of Partnerships with the Private Sector for the SDGs

Partnering with the private sector is a key modality in development cooperation to achieve the Sustainable Development Goals (SDGs). Despite their increasing importance, such Public-Private Partnerships for Development (PPPD) experience major challenges in defining, assessing and reporting on their actual impact. This paper explores why, and how this can be improved. We engage in a qualitative synthesis review of academic, gray literature and evaluation reports of public-private programs of development agencies. We identify challenges, tensions and contradictions that affect a proper understanding and assessment of the impacts of such partnerships. The analysis shows that the main challenge in understanding and assessing impacts is the double governance logic that emerges in PPPD monitoring and evaluation (M&E). While M&E functions as an accountability and risk mitigation approach, it should also support collaborative characteristics of PPPDs such as trust and power-sharing, in order to enhance impactful PPPDs. Enhancing the impact of PPPDs for the SDGs requires bridging the divide between (a) result-based, upward accountability monitoring and evaluation approaches and (b) emerging learning, participatory and complexity-based approaches. The paper provides suggestions on how to navigate these governance tensions by using a paradoxical lens.

Ensuring the rigor in systematic reviews: Part 3, the value of the search

The third column, in this seven part series, highlights: Systematic reviews must include published sources as well as consider including unpublished sources for data collection. Systematic reviews require rigorous search strategies, that are transparent and reproducible. Systematic reviews require adequate reporting of search methods.

Quantifying and addressing the prevalence and bias of study designs in the environmental and social sciences

Building trust in science and evidence-based decision-making depends heavily on the credibility of studies and their findings. Researchers employ many different study designs that vary in their risk of bias to evaluate the true effect of interventions or impacts. Here, we empirically quantify, on a large scale, the prevalence of different study designs and the magnitude of bias in their estimates. Randomised designs and controlled observational designs with pre-intervention sampling were used by just 23% of intervention studies in biodiversity conservation, and 36% of intervention studies in social science. We demonstrate, through pairwise within-study comparisons across 49 environmental datasets, that these types of designs usually give less biased estimates than simpler observational designs. We propose a model-based approach to combine study estimates that may suffer from different levels of study design bias, discuss the implications for evidence synthesis, and how to facilitate the use of more credible study designs.

Real-time biomechanics using the finite element method and machine learning: Review and perspective

PURPOSE

The finite element method (FEM) is the preferred method to simulate phenomena in anatomical structures. However, purely FEM-based mechanical simulations require considerable time, limiting their use in clinical applications that require real-time responses, such as haptics simulators. Machine learning (ML) approaches have been proposed to help with the reduction of the required time. The present paper reviews cases where ML could help to generate faster simulations, without considerably affecting the performance results.

METHODS

This review details the ML approaches used, considering the anatomical structures involved, the data collection strategies, the selected ML algorithms, with corresponding features, the metrics used for validation, and the resulting time gains.

RESULTS

A total of 41 references were found. ML algorithms are mainly trained with FEM-based simulations in 32 publications. The preferred ML approach is neural networks, including deep learning in 35 publications. Tissue deformation is simulated in 18 applications, but other features are also considered. The average distance error and mean squared error are the most frequently used performance metrics, in 14 and 17 publications, respectively. The time gains were considerable, going from hours or minutes for purely FEM-based simulations to milliseconds, when using ML.

CONCLUSIONS

ML algorithms can be used to accelerate FEM-based biomechanical simulations of anatomical structures, possibly reaching real-time responses. Fast and real-time simulations of anatomical structures, generated with ML algorithms, can help to reduce the time required by FEM-based simulations and accelerate their adoption in the clinical practice.

Mortality cost of sex-specific parasitism in wild bird populations

Sex-specific mortality is frequent in animals although the causes of different male versus female mortalities remain poorly understood. Parasitism is ubiquitous in nature with widespread detrimental effects to hosts, making parasitism a likely cause of sex-specific mortalities. Using sex-specific blood and gastrointestinal parasite prevalence from 96 and 54 avian host species, respectively, we test the implications of parasites for annual mortality in wild bird populations using phylogenetic comparative methods. First, we show that parasite prevalence is not different between adult males and females, although Nematodes showed a statistically significant but small male-biased parasite prevalence. Second, we found no correlation between sex-biased host mortalities and sex-biased parasite prevalence. These results were consistent in both blood and gastrointestinal parasites. Taken together, our results show little evidence for sex-dependent parasite prevalence in adults in wild bird populations, and suggest that parasite prevalence is an unlikely predictor of sex difference in adult mortalities, not withstanding sampling limitations. We propose that to understand causes of sex-biased mortalities, more complex analyses are needed that incorporate various ecological and life history components of animals life that may include sex differences in exposure to predators, immune capacity and cost of reproduction.

Using citation tracking for systematic literature searching - study protocol for a scoping review of methodological studies and an expert survey

Background: Up-to-date guidance on comprehensive study identification for systematic reviews is crucial. According to current recommendations, systematic searching should combine electronic database searching with supplementary search methods. One such supplementary search method is citation tracking. It aims at collecting directly and/or indirectly cited and citing references from "seed references". Tailored and evidence-guided recommendations concerning the use of citation tracking are strongly needed. Objective: We intend to develop recommendations for the use of citation tracking in health-related systematic literature searching. Our study will be guided by the following research questions: What are the benefits of citation tracking for health-related systematic literature searching? Which perspectives and experiences do experts in the field of literature retrieval methods have with regard to citation tracking in health-related systematic literature searching? Methods: Our study will have two parts: a scoping review and an expert survey. The scoping review aims at identifying methodological studies on benefits or problems of citation tracking in health-related systematic literature searching with no restrictions on study design, language, and publication date. We will perform database searching in MEDLINE, The Cumulative Index to Nursing and Allied Health Literature (CINAHL), Web of Science Core Collection, two information science databases, and free web searching. Two reviewers will independently assess full texts of selected abstracts. We will conduct direct backward and forward citation tracking on included articles. The results of the scoping review will inform our expert survey through which we aim to learn about experts΄ perspectives and experiences. We will narratively synthesize the results and derive recommendations for performing health-related systematic reviews.

Eight problems with literature reviews and how to fix them

Traditional approaches to reviewing literature may be susceptible to bias and result in incorrect decisions. This is of particular concern when reviews address policy- and practice-relevant questions. Systematic reviews have been introduced as a more rigorous approach to synthesizing evidence across studies; they rely on a suite of evidence-based methods aimed at maximizing rigour and minimizing susceptibility to bias. Despite the increasing popularity of systematic reviews in the environmental field, evidence synthesis methods continue to be poorly applied in practice, resulting in the publication of syntheses that are highly susceptible to bias. Recognizing the constraints that researchers can sometimes feel when attempting to plan, conduct and publish rigorous and comprehensive evidence syntheses, we aim here to identify major pitfalls in the conduct and reporting of systematic reviews, making use of recent examples from across the field. Adopting a 'critical friend' role in supporting would-be systematic reviews and avoiding individual responses to police use of the 'systematic review' label, we go on to identify methodological solutions to mitigate these pitfalls. We then highlight existing support available to avoid these issues and call on the entire community, including systematic review specialists, to work towards better evidence syntheses for better evidence and better decisions.

Using citation tracking for systematic literature searching - study protocol for a scoping review of methodological studies and a Delphi study

Background: Up-to-date guidance on comprehensive study identification for systematic reviews is crucial. According to current recommendations, systematic searching should combine electronic database searching with supplementary search methods. One such supplementary search method is citation tracking. It aims at collecting directly and/or indirectly cited and citing references from "seed references". Tailored and evidence-guided recommendations concerning the use of citation tracking are strongly needed. Objective: We intend to develop recommendations for the use of citation tracking in systematic literature searching for health-related topics. Our study will be guided by the following research questions: What is the benefit of citation tracking for systematic literature searching for health-related topics? Which methods, citation indexes, and other tools are used for citation tracking? What terminology is used for citation tracking methods? Methods: Our study will have two parts: a scoping review and a Delphi study. The scoping review aims at identifying methodological studies on the benefit and use of citation tracking in systematic literature searching for health-related topics with no restrictions on study design, language, and publication date. We will perform database searching in MEDLINE (Ovid), CINAHL (EBSCOhost), Web of Science Core Collection, two information science databases, web searching, and contact experts in the field. Two reviewers will independently perform study selection. We will conduct direct backward and forward citation tracking on included articles. Data from included studies will be extracted using a prespecified extraction sheet and presented in both tabular and narrative form. The results of the scoping review will inform the subsequent Delphi study through which we aim to derive consensus recommendations for the future practice and research of citation tracking.

Using citation tracking for systematic literature searching - study protocol for a scoping review of methodological studies and a Delphi study

Background: Up-to-date guidance on comprehensive study identification for systematic reviews is crucial. According to current recommendations, systematic searching should combine electronic database searching with supplementary search methods. One such supplementary search method is citation tracking. It aims at collecting directly and/or indirectly cited and citing references from "seed references". Tailored and evidence-guided recommendations concerning the use of citation tracking are strongly needed. Objective: We intend to develop recommendations for the use of citation tracking in systematic literature searching for health-related topics. Our study will be guided by the following research questions: What is the benefit of citation tracking for systematic literature searching for health-related topics? Which methods, citation indexes, and other tools are used for citation tracking? What terminology is used for citation tracking methods? Methods: Our study will have two parts: a scoping review and a Delphi study. The scoping review aims at identifying methodological studies on the benefit and use of citation tracking in systematic literature searching for health-related topics with no restrictions on study design, language, and publication date. We will perform database searching in MEDLINE (Ovid), CINAHL (EBSCOhost), Web of Science Core Collection, two information science databases, web searching, and contact experts in the field. Two reviewers will independently perform study selection. We will conduct direct backward and forward citation tracking on included articles. Data from included studies will be extracted using a prespecified extraction sheet and presented in both tabular and narrative form. The results of the scoping review will inform the subsequent Delphi study through which we aim to derive consensus recommendations for the future practice and research of citation tracking.

What drives citations of frontier application publications?

A large body of literature exists on analysis of citation and reviews of application of efficiency frontier. However, the reviews that assessed the determinants of citation counts did not focus on frontier applications. We contribute to the literature by identifying the drivers of citations of frontier application publications on Ghana. We employed two-part mixture modelling with inverse hyperbolic sine (IHS) transformation of the second part, which was found to be more appropriate than single equation IHS transformation modelling, for our data. Use of stochastic frontier analysis or data envelopment analysis did not drive citations counts. However, quality of journals in which frontier application studies were published and accessibility of the journals to readers, drive citation counts. Authors, institutions and funders of studies on frontier applications may consider these over collaborations, in seeking growth in citation counts.

A systematic review of applied single-case research published between 2016 and 2018: Study designs, randomization, data aspects, and data analysis

Single-case experimental designs (SCEDs) have become a popular research methodology in educational science, psychology, and beyond. The growing popularity has been accompanied by the development of specific guidelines for the conduct and analysis of SCEDs. In this paper, we examine recent practices in the conduct and analysis of SCEDs by systematically reviewing applied SCEDs published over a period of three years (2016-2018). Specifically, we were interested in which designs are most frequently used and how common randomization in the study design is, which data aspects applied single-case researchers analyze, and which analytical methods are used. The systematic review of 423 studies suggests that the multiple baseline design continues to be the most widely used design and that the difference in central tendency level is by far most popular in SCED effect evaluation. Visual analysis paired with descriptive statistics is the most frequently used method of data analysis. However, inferential statistical methods and the inclusion of randomization in the study design are not uncommon. We discuss these results in light of the findings of earlier systematic reviews and suggest future directions for the development of SCED methodology.

Commentary to Gusenbauer and Haddaway 2020: Evaluating retrieval qualities of Google Scholar and PubMed

We read with considerable interest the study by Gusenbauer and Haddaway (Gusenbauer and Haddaway, 2020, Research Synthesis Methods, doi:10.1002/jrsm.1378) comparing the systematic search qualities of 28 search systems, including Google Scholar (GS) and PubMed. Google Scholar and PubMed are the two most popular free academic search tools in biology and chemistry, with GS being the number one search tool in the world. Those academics using GS as their principal system for literature searches may be unaware of research which enumerates five critical features for scientific literature tools that greatly influenced Gusenbauer's 2020 study. Using this list as the framework for a targeted comparison between just GS and PubMed, we found stark differences which overwhelmingly favored PubMed. In this comment, we show that by comparing the characteristics of the two search tools, features that are particularly useful in one search tool, but are missing in the other, are strikingly spotlighted. One especially popular feature that ubiquitously appears in GS, but not in PubMed, is the forward citation search found under every citation as a clickable Cited by N link. We seek to improve the PubMed search experience using two approaches. First, we request that PubMed add Cited by N links, making them as omnipresent as the GS links. Second, we created an open‐source command‐line tool, pmidcite, which is used alongside PubMed to give information to researchers to help with the choice of the next paper to examine, analogous to how GS's Cited by N links help to guide users. Find pmidcite at https://github.com/dvklopfenstein/pmidcite.

What every researcher should know about searching - clarified concepts, search advice, and an agenda to improve finding in academia

We researchers have taken searching for information for granted for far too long. The COVID-19 pandemic shows us the boundaries of academic searching capabilities, both in terms of our know-how and of the systems we have. With hundreds of studies published daily on COVID-19, for example, we struggle to find, stay up-to-date, and synthesize information-all hampering evidence-informed decision making. This COVID-19 information crisis is indicative of the broader problem of information overloaded academic research. To improve our finding capabilities, we urgently need to improve how we search and the systems we use. We respond to Klopfenstein and Dampier (Res Syn Meth. 2020) who commented on our 2020 paper and proposed a way of improving PubMed's and Google Scholar's search functionalities. Our response puts their commentary in a larger frame and suggests how we can improve academic searching altogether. We urge that researchers need to understand that search skills require dedicated education and training. Better and more efficient searching requires an initial understanding of the different goals that define the way searching needs to be conducted. We explain the main types of searching that we academics routinely engage in; distinguishing lookup, exploratory, and systematic searching. These three types must be conducted using different search methods (heuristics) and using search systems with specific capabilities. To improve academic searching, we introduce the "Search Triangle" model emphasizing the importance of matching goals, heuristics, and systems. Further, we suggest an urgently needed agenda toward search literacy as the norm in academic research and fit-for-purpose search systems.

A knowledge management system framework for an open biomedical repository: communities, collaboration and corroboration

Purpose

The purpose of this study is to assess the opportunity for a distributed, networked open biomedical repository (OBR) using a knowledge management system (KMS) conceptual framework. An innovative KMS conceptual framework is proposed to guide the transition from a traditional, siloed approach to a sustainable OBR.

Design/methodology/approach

This paper reports on a cycle of action research, involving literature review, interviews and focus group with leaders in biomedical research, open science and librarianship, and an audit of elements needed for an Australasian OBR; these, along with an Australian KM standard, informed the resultant KMS framework.

Findings

The proposed KMS framework aligns the requirements for an OBR with the people, process, technology and content elements of the KM standard. It identifies and defines nine processes underpinning biomedical knowledge – discovery, creation, representation, classification, storage, retrieval, dissemination, transfer and translation. The results comprise an explanation of these processes and examples of the people, process, technology and content dimensions of each process. While the repository is an integral cog within the collaborative, distributed open science network, its effectiveness depends on understanding the relationships and linkages between system elements and achieving an appropriate balance between them.

Research limitations/implications

The current research has focused on biomedicine. This research builds on the worldwide effort to reduce barriers, in particular paywalls to health knowledge. The findings present an opportunity to rationalize and improve a KMS integral to biomedical knowledge.

Practical implications

Adoption of the KMS framework for a distributed, networked OBR will facilitate open science through reducing duplication of effort, removing barriers to the flow of knowledge and ensuring effective management of biomedical knowledge.

Social implications

Achieving quality, permanency and discoverability of a region’s digital assets is possible through ongoing usage of the framework for researchers, industry and consumers.

Originality/value

The framework demonstrates the dependencies and interplay of elements and processes to frame an OBR KMS.

Google Scholar, Microsoft Academic, Scopus, Dimensions, Web of Science, and OpenCitations' COCI: a multidisciplinary comparison of coverage via citations

New sources of citation data have recently become available, such as Microsoft Academic, Dimensions, and the OpenCitations Index of CrossRef open DOI-to-DOI citations (COCI). Although these have been compared to the Web of Science Core Collection (WoS), Scopus, or Google Scholar, there is no systematic evidence of their differences across subject categories. In response, this paper investigates 3,073,351 citations found by these six data sources to 2,515 English-language highly-cited documents published in 2006 from 252 subject categories, expanding and updating the largest previous study. Google Scholar found 88% of all citations, many of which were not found by the other sources, and nearly all citations found by the remaining sources (89-94%). A similar pattern held within most subject categories. Microsoft Academic is the second largest overall (60% of all citations), including 82% of Scopus citations and 86% of WoS citations. In most categories, Microsoft Academic found more citations than Scopus and WoS (182 and 223 subject categories, respectively), but had coverage gaps in some areas, such as Physics and some Humanities categories. After Scopus, Dimensions is fourth largest (54% of all citations), including 84% of Scopus citations and 88% of WoS citations. It found more citations than Scopus in 36 categories, more than WoS in 185, and displays some coverage gaps, especially in the Humanities. Following WoS, COCI is the smallest, with 28% of all citations. Google Scholar is still the most comprehensive source. In many subject categories Microsoft Academic and Dimensions are good alternatives to Scopus and WoS in terms of coverage.

Objective Measures to Assess Active Commuting Physical Activity to School in Young People: A Systematic Review Protocol and Practical Considerations

There are no systematic reviews that have identified the existing studies assessing active commuting physical activity (PA) to and from (to/from) school using objective measures, as well as the contribution of both walking and cycling to/from school to PA levels. To fill this gap in the literature, this systematic review will aim (a) to identify existing studies that assess active commuting PA to/from school with objective measures in young people and to examine the contribution of walking and cycling to/from school to PA levels, and (b) to propose an appropriate methodology and practical considerations to assess active commuting PA to/from school based on the studies identified. The review protocol was registered in PROSPERO (CRD42020162004). We will conduct a systematic search up to 2020 in five databases: PubMed, Web of Science, SPORTdiscuss, Cochrane Library, and National Transportation Library. Both the risk of bias and the quality of the identified studies will be evaluated through different instruments according to the design of each study. This systematic review will help to choose the most appropriate objective measures to assess active commuting PA to/from school and to promote walking and cycling to/from school to increase PA levels.

Assessing the risk of bias in choice of search sources for environmental meta-analyses

Results of meta-analyses are potentially valuable for informing environmental policy and practice decisions. However, selective sampling of primary studies through searches exclusively using widely used bibliographic platform(s) could bias estimates of effect sizes. Such search strategies are common in environmental evidence reviews, and if risk of bias can be detected, this would provide the first empirical evidence that comprehensiveness of searches needs to be improved. We compare the impact of using single and multiple bibliographic platform(s) searches vs more comprehensive searches on estimates of mean effect sizes. We used 137 published meta-analyses, based on multiple source searches, analyzing 9388 studies: 8095 sourced from commercially published articles; and 1293 from grey literature and unpublished data. Single-platform and multiple-platform searches missed studies in 100 and 80 of the meta-analyses, respectively: 52 and 46 meta-analyses provided larger-effect estimates; 32 and 28 meta-analyses provided smaller-effect estimates; eight and four meta-analyses provided opposite direction of estimates; and two each were unable to estimate effects due to missing all studies. Further, we found significant positive log-linear relationships between proportions of studies missed and the deviations of mean effect sizes, suggesting that as the number of studies missed increases, deviation of mean effect size is likely to expand. We also found significant differences in mean effect sizes between indexed and non-indexed studies for 35% of meta-analyses, indicating high risk of bias when the searches were restricted. We conclude that the restricted searches are likely to lead to unrepresentative samples of studies and biased estimates of true effects.

Which academic search systems are suitable for systematic reviews or meta-analyses? Evaluating retrieval qualities of Google Scholar, PubMed, and 26 other resources

Rigorous evidence identification is essential for systematic reviews and meta-analyses (evidence syntheses) because the sample selection of relevant studies determines a review's outcome, validity, and explanatory power. Yet, the search systems allowing access to this evidence provide varying levels of precision, recall, and reproducibility and also demand different levels of effort. To date, it remains unclear which search systems are most appropriate for evidence synthesis and why. Advice on which search engines and bibliographic databases to choose for systematic searches is limited and lacking systematic, empirical performance assessments. This study investigates and compares the systematic search qualities of 28 widely used academic search systems, including Google Scholar, PubMed, and Web of Science. A novel, query-based method tests how well users are able to interact and retrieve records with each system. The study is the first to show the extent to which search systems can effectively and efficiently perform (Boolean) searches with regards to precision, recall, and reproducibility. We found substantial differences in the performance of search systems, meaning that their usability in systematic searches varies. Indeed, only half of the search systems analyzed and only a few Open Access databases can be recommended for evidence syntheses without adding substantial caveats. Particularly, our findings demonstrate why Google Scholar is inappropriate as principal search system. We call for database owners to recognize the requirements of evidence synthesis and for academic journals to reassess quality requirements for systematic reviews. Our findings aim to support researchers in conducting better searches for better evidence synthesis.

Modeling and Management Big Data in Databases—A Systematic Literature Review

The work presented in this paper is motivated by the acknowledgement that a complete and updated systematic literature review (SLR) that consolidates all the research efforts for Big Data modeling and management is missing. This study answers three research questions. The first question is how the number of published papers about Big Data modeling and management has evolved over time. The second question is whether the research is focused on semi-structured and/or unstructured data and what techniques are applied. Finally, the third question determines what trends and gaps exist according to three key concepts: the data source, the modeling and the database. As result, 36 studies, collected from the most important scientific digital libraries and covering the period between 2010 and 2019, were deemed relevant. Moreover, we present a complete bibliometric analysis in order to provide detailed information about the authors and the publication data in a single document. This SLR reveal very interesting facts. For instance, Entity Relationship and document-oriented are the most researched models at the conceptual and logical abstraction level respectively and MongoDB is the most frequent implementation at the physical. Furthermore, 2.78% studies have proposed approaches oriented to hybrid databases with a real case for structured, semi-structured and unstructured data.