Sources of information on adverse effects: a systematic review

Overview: comprehensive and carefully constructed strategies are required when conducting searches for adverse effects data

OBJECTIVES

Methodological research has been undertaken to investigate the many challenges in searching for adverse effects data. It is imperative that the search approach adopted in systematic reviews is based on the best available evidence. We provide a detailed summary of the results and implications of the current evidence base to assist future searches for adverse effects.

STUDY DESIGN AND SETTING

This article is a narrative review from the authors of the Cochrane Handbook chapter on adverse effects.

RESULTS

The specified search strategy must be based on the population, intervention, comparator, outcome(s) format for question formulation and appropriate study designs for adverse effects data. Search filters and suggested search terms are available for the adverse effects of drug, medical devices, and surgical interventions. The use of generic adverse effects terms (such as harms and complications) as text words and indexing terms and specific adverse effects terms (such as rash and wound infection) are warranted. Searching databases beyond MEDLINE has proven useful, as well as the use of nondatabase sources.

CONCLUSION

This article provides the most up-to-date evidence-based guidance in identifying adverse effects data in the literature. It will support searchers and researchers evaluating the potential for harm of medical interventions in systematic reviews.

Ten years of reviews

The March 2017 issue of the Health Information and Libraries Journal marks the 10 year anniversary of the inaugural review published in the journal's review series. The review series was conceived to meet the growing appetite of health library and information workers to access synthesised evidence to inform their practice; something we'd already been doing to support medics in their practice. This editorial looks back on the 10 years and the inspiration which saw the development of a typology of review types and associated methodologies to address the lack of consistent guidelines on the features a review should incorporate.

Identification of Evidence for Key Parameters in Decision-Analytic Models of Cost Effectiveness: A Description of Sources and a Recommended Minimum Search Requirement

This paper proposes recommendations for a minimum level of searching for data for key parameters in decision-analytic models of cost effectiveness and describes sources of evidence relevant to each parameter type. Key parameters are defined as treatment effects, adverse effects, costs, resource use, health state utility values (HSUVs) and baseline risk of events. The recommended minimum requirement for treatment effects is comprehensive searching according to available methodological guidance. For other parameter types, the minimum is the searching of one bibliographic database plus, where appropriate, specialist sources and non-research-based and non-standard format sources. The recommendations draw on the search methods literature and on existing analyses of how evidence is used to support decision-analytic models. They take account of the range of research and non-research-based sources of evidence used in cost-effectiveness models and of the need for efficient searching. Consideration is given to what constitutes best evidence for the different parameter types in terms of design and scientific quality and to making transparent the judgments that underpin the selection of evidence from the options available. Methodological issues are discussed, including the differences between decision-analytic models of cost effectiveness and systematic reviews when searching and selecting evidence and comprehensive versus sufficient searching. Areas are highlighted where further methodological research is required.

THE CONTRIBUTION OF DIFFERENT INFORMATION SOURCES TO IDENTIFY ADVERSE EFFECTS OF A MEDICAL DEVICE: A CASE STUDY USING A SYSTEMATIC REVIEW OF SPINAL FUSION

Background: The most effective sources to search to identify adverse effects data for medical devices are currently unknown.

Methods: The included studies from a systematic review of the safety of recombinant human bone morphogenetic protein-2 (rhBMP-2) for spinal fusion were used for analysis. For each source searched, a record was made for each relevant publication of whether it was retrieved by the search strategy used and whether it was available in the database but not retrieved. To account for multiple publications of the same study, a record was made of the relevant studies identified. The sensitivity, precision, and number needed to read were calculated as well as the minimum combination of sources to identify all the publications or studies.

Results: There were eighty-two publications (forty-nine studies) included in the systematic review. Only one article was available in a database searched but not retrieved by our search strategy. Science Citation Index (SCI) and EMBASE both achieved the highest sensitivity (62 percent), followed closely by MEDLINE/PubMED (56 percent). With the search strategies used, the minimum combination of sources needed to identify all the publications was SCI, EMBASE, CENTRAL, and either MEDLINE or PubMED, in addition to reference checking, contacting authors and an automated current awareness service. In relation to identifying all the relevant studies, the minimum combination of studies was similar with the exclusion of CENTRAL.

Conclusions: To identify all the relevant publications or studies included in this case study systematic review, several different sources needed to be searched.

Comprehensive evaluations of the adverse effects of drugs: importance of appropriate study selection and data sources

While systematic reviews and meta-analyses are at the top of the evidence hierarchy, most of the methodology has focused on assessing treatment benefit. Hence, we propose a structured framework for the initial steps of searching and identifying relevant data sources so that adverse effects can be evaluated in a comprehensive, unbiased manner. The unique methodological challenges stem from the difficulties of addressing diverse outcomes encompassing common, mild symptoms to rare, fatal events. Retrieval of the most appropriate studies should be specifically tailored to fit the nature of the adverse effects, according to the primary objective and study question. In our framework, the structure of the review takes different forms depending on whether the main aim is on scoping/hypothesis generation, or evaluating statistically the magnitude of risk (hypothesis testing), or clarifying characteristics and risk factors of the adverse effect. The wide range of data sources covering adverse effects all have distinct strengths and limitations, and selection of appropriate sources depends on characteristics of the adverse effect (e.g. background incidence and effect size of the drug, clinical presentation, time of onset after drug exposure). Reviewers need to retrieve particular study designs that are most likely to yield robust data on the adverse effects of interest, rather than rely on studies that cannot reliably detect adverse effects, and may yield 'false negatives'. Type II errors (a particular problem when evaluating rare adverse effects) can lull us into a false sense of security (e.g. wrongly concluding that there was no significant difference in harm between drug and control, with the drug erroneously judged as safe). Given the rapid rate at which methodological improvements occur, this proposed framework is by no means definitive, but aims to stimulate further debate and discussion amongst the pharmacoepidemiological and systematic review communities to reach a common consensus on the best methods.

Comparison of search strategies in systematic reviews of adverse effects to other systematic reviews

BACKGROUND

Research indicates that the methods used to identify data for systematic reviews of adverse effects may need to differ from other systematic reviews.

OBJECTIVES

To compare search methods in systematic reviews of adverse effects with other reviews.

METHODS

The search methodologies in 849 systematic reviews of adverse effects were compared with other reviews.

RESULTS

Poor reporting of search strategies is apparent in both systematic reviews of adverse effects and other types of systematic reviews. Systematic reviews of adverse effects are less likely to restrict their searches to MEDLINE or include only randomised controlled trials (RCTs). The use of other databases is largely dependent on the topic area and the year the review was conducted, with more databases searched in more recent reviews. Adverse effects search terms are used by 72% of reviews and despite recommendations only two reviews report using floating subheadings.

CONCLUSIONS

The poor reporting of search strategies in systematic reviews is universal, as is the dominance of searching MEDLINE. However, reviews of adverse effects are more likely to include a range of study designs (not just RCTs) and search beyond MEDLINE.

Optimising the retrieval of information on adverse drug effects

Pharmaceutical interventions have brought about many benefits to health, improving the population's well-being and life expectancy. However, these interventions are not without potential harmful side-effects and yet searching for the evidence on adverse effects is challenging. This article summarises a PhD whose main aim was to develop a better understanding of the implications of using different sources and approaches to identifying relevant data on adverse effects. The author is Su Golder, who has recently completed her PhD at the University of York and who has already published several articles on specific aspects of her research, including this journal. This article is the first in the Dissertations into Practice series to report on a PhD study, and it summarises her research in a way which emphasises the implications for practice.

Some improvements are apparent in identifying adverse effects in systematic reviews from 1994 to 2011

OBJECTIVE

An increasing amount of research and guidelines has been published on search methodology and the reporting of search strategies in systematic reviews. This research assessed whether this has lead to any improvements in the reporting and quality of searching in systematic reviews of adverse effects.

STUDY DESIGN AND SETTING

All records within Cochrane Database of Systematic Reviews and Database of Abstracts of Reviews of Effects were scanned for systematic reviews of adverse effects. Data were then extracted on the methods used for information retrieval in these reviews and a descriptive analysis conducted by publication year.

RESULTS

A total of 849 reviews published from 1994 to 2011 met the inclusion criteria. There has been a significant increase (P<0.001) in the number of adverse effects reviews per year from 1994 (n=5) to 2010 (n=104). Some improvements were apparent, such as an increase in the number of databases searched and fewer date and language restrictions applied. However, there has been an increase in reviews limited to data from randomized controlled trials, whereas the reporting of search strategies could still be improved further, with only 9% (74/849) of the reviews reporting reproducible searches.

CONCLUSION

Some improvements in searching systematic reviews of adverse effects are apparent; however, poor reporting of search strategies remains a great obstacle to readers.

The contribution of different information sources for adverse effects data

OBJECTIVES

The aim of this study is to determine the relative value and contribution of searching different sources to identify adverse effects data.

METHODS

The process of updating a systematic review and meta-analysis of thiazolidinedione-related fractures in patients with type 2 diabetes mellitus was used as a case study. For each source searched, a record was made for each relevant reference included in the review noting whether it was retrieved with the search strategy used and whether it was available but not retrieved. The sensitivity, precision, and number needed to read from searching each source and from different combinations of sources were also calculated.

RESULTS

There were 58 relevant references which presented sufficient numerical data to be included in a meta-analysis of fractures and bone mineral density. The highest number of relevant references were retrieved from Science Citation Index (SCI) (35), followed by BIOSIS Previews (27) and EMBASE (24). The precision of the searches varied from 0.88% (Scirus) to 41.67% (CENTRAL). With the search strategies used, the minimum combination of sources required to retrieve all the relevant references was; the GlaxoSmithKline (GSK) website, Science Citation Index (SCI), EMBASE, BIOSIS Previews, British Library Direct, Medscape DrugInfo, handsearching and reference checking, AHFS First, and Thomson Reuters Integrity or Conference Papers Index (CPI).

CONCLUSIONS

In order to identify all the relevant references for this case study a number of different sources needed to be searched. The minimum combination of sources required to identify all the relevant references did not include MEDLINE.