THE PROMISE AND PITFALLS OF SYSTEMATIC REVIEWS

Systematic online living evidence summaries: emerging tools to accelerate evidence synthesis

Systematic reviews and meta-analysis are the cornerstones of evidence-based decision making and priority setting. However, traditional systematic reviews are time and labour intensive, limiting their feasibility to comprehensively evaluate the latest evidence in research-intensive areas. Recent developments in automation, machine learning and systematic review technologies have enabled efficiency gains. Building upon these advances, we developed Systematic Online Living Evidence Summaries (SOLES) to accelerate evidence synthesis. In this approach, we integrate automated processes to continuously gather, synthesise and summarise all existing evidence from a research domain, and report the resulting current curated content as interrogatable databases via interactive web applications. SOLES can benefit various stakeholders by (i) providing a systematic overview of current evidence to identify knowledge gaps, (ii) providing an accelerated starting point for a more detailed systematic review, and (iii) facilitating collaboration and coordination in evidence synthesis.

3D printed patient-specific prostate cancer models to guide nerve-sparing robot-assisted radical prostatectomy: a systematic review

Precise knowledge of each patient's index cancer and surrounding anatomy is required for nerve-sparing robot-assisted radical prostatectomy (NS-RARP). Complementary to this, 3D printing has proven its utility in improving the visualisation of complex anatomy. This is the first systematic review to critically assess the potential of 3D printed patient-specific prostate cancer models in improving visualisation and the practice of NS-RARP. A literature search of PubMed and OVID Medline databases was performed using the terms "3D Printing", "Robot Assisted Radical Prostatectomy" and related index terms as per the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines. Eight articles were included; six were identified via database searches, to which a further two articles were located via a snowballing approach. Eight papers were identified for review. There were five prospective single centre studies, one case series, one technical report and one letter to the editor. Of these articles, five publications (62.5%) reported on the utility of 3D printed models for NS-RARP planning. Two publications (25%) utilised 3D printed prostate models for simulation and training, and two publications (25%) used the models for patient engagement. Despite the nascency of the field, 3D printed models are emerging in the uro-oncological literature as a useful tool in visualising complex anatomy. This has proven useful in NS-RARP for preoperative planning, simulation and patient engagement. However, best practice guidelines, the future regulatory landscape, and health economic considerations need to be addressed before this synergy of new technologies is ready for the mainstream.

Animal models and animal-free innovations for cardiovascular research: current status and routes to be explored. Consensus document of the ESC Working Group on Myocardial Function and the ESC Working Group on Cellular Biology of the Heart

Cardiovascular diseases represent a major cause of morbidity and mortality, necessitating research to improve diagnostics, and to discover and test novel preventive and curative therapies, all of which warrant experimental models that recapitulate human disease. The translation of basic science results to clinical practice is a challenging task, in particular for complex conditions such as cardiovascular diseases, which often result from multiple risk factors and comorbidities. This difficulty might lead some individuals to question the value of animal research, citing the translational 'valley of death', which largely reflects the fact that studies in rodents are difficult to translate to humans. This is also influenced by the fact that new, human-derived in vitro models can recapitulate aspects of disease processes. However, it would be a mistake to think that animal models do not represent a vital step in the translational pathway as they do provide important pathophysiological insights into disease mechanisms particularly on an organ and systemic level. While stem cell-derived human models have the potential to become key in testing toxicity and effectiveness of new drugs, we need to be realistic, and carefully validate all new human-like disease models. In this position paper, we highlight recent advances in trying to reduce the number of animals for cardiovascular research ranging from stem cell-derived models to in situ modelling of heart properties, bioinformatic models based on large datasets, and state-of-the-art animal models, which show clinically relevant characteristics observed in patients with a cardiovascular disease. We aim to provide a guide to help researchers in their experimental design to translate bench findings to clinical routine taking the replacement, reduction, and refinement (3R) as a guiding concept.

Development of a reporting guideline for systematic reviews of animal experiments in the field of traditional Chinese medicine

OBJECTIVE

In recent years, there are several systematic reviews published on animal experiments of Traditional Chinese medicine (TCM). PRISMA (preferred reporting items for systematic reviews and meta-analysis) guidelines provide a guarantee for significantly improving the reporting quality of systematic reviews (SRs) and meta-analysis (MAs) to a certain extent; however, there are still certain defects found in the quality of SRs/MAs of animal experiments of TCM. It has been found that especially, the descriptions of the rationale and animal characteristics of TCM interventions are inadequate. As a result, we have developed a novel reporting guideline for SRs/MAs of animal experimental in the field of TCM (PRISMA-ATCM) to overcome these problems.

METHODS

PRISMA-ATCM reporting guidelines were formed by analyzing both the status and quality of published SRs/MAs of animal experiments and consulting experts in the related fields, and then by Delphi consultation, consensus meeting and revision.

RESULTS

Among the 27 items on the PRISMA checklist, Title (1), Structured summary (2), Rationale (3), Objectives (4), Protocol and registration (5), Eligibility criteria (6), Data items (11), Planned methods of analysis (14), Study characteristics (18), Summary of evidence (24), Limitations (25), and Funding (27) have been extensively revised and expanded, to specifically include the details about TCM intervention and animal characteristics. In addition, illustrative examples and explanations have been provided for each item.

CONCLUSION

PRISMA-ATCM could markedly improve the quality SRs/MAs of animal experiments in the field of TCM.

Effect of smoke-free policies in outdoor areas and private places on children's tobacco smoke exposure and respiratory health: a systematic review and meta-analysis

BACKGROUND

Smoke-free policies in outdoor areas and semi-private and private places (eg, cars) might reduce the health harms caused by tobacco smoke exposure (TSE). We aimed to investigate the effect of smoke-free policies covering outdoor areas or semi-private and private places on TSE and respiratory health in children, to inform policy.

METHODS

In this systematic review and meta-analysis, we searched 13 electronic databases from date of inception to Jan 29, 2021, for published studies that assessed the effects of smoke-free policies in outdoor areas or semi-private or private places on TSE, respiratory health outcomes, or both, in children. Non-randomised and randomised trials, interrupted time series, and controlled before-after studies, without restrictions to the observational period, publication date, or language, were eligible for the main analysis. Two reviewers independently extracted data, including adjusted test statistics from each study using a prespecified form, and assessed risk of bias for effect estimates from each study using the Risk of Bias in Non-Randomised Studies of Interventions tool. Primary outcomes were TSE in places covered by the policy, unplanned hospital attendance for wheezing or asthma, and unplanned hospital attendance for respiratory tract infections, in children younger than 17 years. Random-effects meta-analyses were done when at least two studies evaluated policies that regulated smoking in similar places and reported on the same outcome. This study is registered with PROSPERO, CRD42020190563.

FINDINGS

We identified 5745 records and assessed 204 full-text articles for eligibility, of which 11 studies met the inclusion criteria and were included in the qualitative synthesis. Of these studies, seven fit prespecified robustness criteria as recommended by the Cochrane Effective Practice and Organization of Care group, assessing smoke-free cars (n=5), schools (n=1), and a comprehensive policy covering multiple areas (n=1). Risk of bias was low in three studies, moderate in three, and critical in one. In the meta-analysis of ten effect estimates from four studies, smoke-free car policies were associated with an immediate TSE reduction in cars (risk ratio 0·69, 95% CI 0·55-0·87; 161 466 participants); heterogeneity was substantial (I² 80·7%; p<0·0001). One additional study reported a gradual TSE decrease in cars annually. Individual studies found TSE reductions on school grounds, following a smoke-free school policy, and in hospital attendances for respiratory tract infection, following a comprehensive smoke-free policy.

INTERPRETATION

Smoke-free car policies are associated with reductions in reported child TSE in cars, which could translate into respiratory health benefits. Few additional studies assessed the effect of policies regulating smoking in outdoor areas and semi-private and private places on children's TSE or health outcomes. On the basis of these findings, governments should consider including private cars in comprehensive smoke-free policies to protect child health.

FUNDING

Dutch Heart Foundation, Lung Foundation Netherlands, Dutch Cancer Society, Dutch Diabetes Research Foundation, Netherlands Thrombosis Foundation, and Health Data Research UK.

The rationale behind systematic reviews in clinical medicine: a conceptual framework

A systematic review (SR) is a type of review that uses a systematic method to provide a valid summary of existing literature addressing a clear and specific question. In clinical medicine (CM), the concept of SR is well recognized, especially after the introduction of evidence-based medicine; The SR of randomized clinical trials (RCTs) is considered the highest level of evidence on therapeutic effectiveness. Despite the popularity of the SRs and the increasing publication rate of SRs in CM and other healthcare literature, the concept has raised criticisms. Many of proper criticisms can be due to the deviation of some existing SRs from the original philosophy and well-established rationale behind the concept of SR. On the other hand, many criticisms are misconceptions about SRs which still exist even several decades after introducing the concept. This article presents a conceptual framework for clarifying the rationale behind SR in CM by providing the relevant concepts and their inter-relations, explaining how methodological standards of an SR and its rationale are connected, and discussing the rationale under the three-section: SR as a type of synthetic research, SR as a more informed and less biased review, and SR as an efficient scientific tool.

Overall confidence in the results of systematic reviews on exercise therapy for chronic low back pain: a cross-sectional analysis using the Assessing the Methodological Quality of Systematic Reviews (AMSTAR) 2 tool

OBJECTIVE

To evaluate the overall confidence in the results of systematic reviews of exercise therapy for chronic non-specific low back pain using the AMSTAR 2 tool.

METHODS

PubMed, Embase, Cochrane Database of Systematic Reviews, PEDro and CINAHL was searched up to February 2017. Two independent reviewers selected systematic reviews of randomized controlled trials that investigated exercise therapy in patients with low back pain. AMSTAR 2 assessment was performed by pairs of reviewers, and the overall confidence in the results of the systematic reviews were rated as 'High', 'Moderate', 'Low' and 'Critically low'. Descriptive analysis was used to summarize the characteristics of included systematic reviews. The percentage of systematic reviews achieving each item from the AMSTAR 2 and the overall confidence in the results were tabulated.

RESULTS

The search identified 38 systematic reviews. Most of the reviews included a median of 10 clinical trials and total sample size of 813 participants per review. Five of 38 (13%) reviews were Cochrane reviews, and 8 (21%) systematic reviews had a protocol published or registered prospectively. The overall confidence in the results of 28 reviews (74%) was rated as 'Critically low', 6 (16%) as 'Low', 1 (2%) as Moderate, while 3 of 38 reviews (8%) were rated as 'High'.

CONCLUSION

The results demonstrate very low confidence in the results of most systematic reviews of exercise in chronic non-specific low back pain. Clinicians are more likely to deliver the most efficacious interventions to patients by critically appraising systematic reviews using AMSTAR 2 before making their decisions.

Social network analysis for assessing college-aged adults' health: A systematic review

OBJECTIVE

Social network analysis (SNA) is a useful, emerging method for studying health. College students are especially prone to social influence when it comes to health. This review aimed to identify network variables related to college student health and determine how SNA was used in the literature.

PARTICIPANTS

A systematic review of relevant literature was conducted in October 2015.

METHODS

Studies employing egocentric or whole network analysis to study college student health were included. We used Garrard's Matrix Method to extract data from reviewed articles (n = 15).

RESULTS

Drinking, smoking, aggression, homesickness, and stress were predicted by network variables in the reviewed literature. Methodological inconsistencies concerning boundary specification, data collection, nomination limits, and statistical analyses were revealed across studies.

CONCLUSIONS

Results show the consistent relationship between network variables and college health outcomes, justifying further use of SNA to research college health. Suggestions and considerations for future use of SNA are provided.

What Synthesis Methodology Should I Use? A Review and Analysis of Approaches to Research Synthesis

BACKGROUND

When we began this process, we were doctoral students and a faculty member in a research methods course. As students, we were facing a review of the literature for our dissertations. We encountered several different ways of conducting a review but were unable to locate any resources that synthesized all of the various synthesis methodologies. Our purpose is to present a comprehensive overview and assessment of the main approaches to research synthesis. We use 'research synthesis' as a broad overarching term to describe various approaches to combining, integrating, and synthesizing research findings.

METHODS

We conducted an integrative review of the literature to explore the historical, contextual, and evolving nature of research synthesis. We searched five databases, reviewed websites of key organizations, hand-searched several journals, and examined relevant texts from the reference lists of the documents we had already obtained.

RESULTS

We identified four broad categories of research synthesis methodology including conventional, quantitative, qualitative, and emerging syntheses. Each of the broad categories was compared to the others on the following: key characteristics, purpose, method, product, context, underlying assumptions, unit of analysis, strengths and limitations, and when to use each approach.

CONCLUSIONS

The current state of research synthesis reflects significant advancements in emerging synthesis studies that integrate diverse data types and sources. New approaches to research synthesis provide a much broader range of review alternatives available to health and social science students and researchers.

The art and science of study identification: a comparative analysis of two systematic reviews

BACKGROUND

Systematic reviews (SRs) form the foundation for guidelines and evidence-based policy in medicine and public health. Although similar systematic reviews may include non-identical sets of studies, and it is recognized that different sets of studies may lead to different conclusions, little work has been published on why SR study cohorts differ.

METHODS

We took advantage of concurrent publication of two SRs on the same topic - prevention of child exposure to tobacco smoke - to understand why study cohorts differed in the two reviews. We identified all studies included in just one review, investigated validity of specified reasons for exclusions, and, using database records, explored reasons for study non-identification. We assessed review methods and discordancy, and attempted to assess whether changes in study cohorts would have changed conclusions.

RESULTS

Sixty-one studies were included in the two reviews. Thirty-five studies were present in just one review; of these, twenty were identified and excluded by the parallel review. Omissions were due to: review scope (9 studies, 26%), outcomes of interest not measured (8 studies, 23%), exclusion of reports with inadequate reporting (6 studies, 17%), mixed or unclear reasons (3 studies, 8%), search strategies concerning filters, tagging, and keywords (3 studies, 8%), search strategies regarding sources (PUBMED not searched) (2 studies, 6%); discordant interpretation of same eligibility criteria (2 studies, 6%), and non-identification due to non-specific study topic (2 studies, 6%). Review conclusions differed, but were likely due to differences in synthesis methods, not differences in study cohorts.

CONCLUSIONS

The process of study identification for SRs is part art and part science. While some differences are due to differences in review scope, outcomes measured, or reporting practices, others are caused by search methods or discrepancies in reviewer interpretations. Different study cohorts may or may not be a cause of differing SR results. Completeness of SR study cohorts could be enhanced by 1 - independent identification of studies by at least two reviewers, as recommended by recent guidelines, 2 - searching PUBMED with free-text keywords in addition to MEDLINE to identify recent studies, and 3 - Using validated search filters.

The usefulness of systematic reviews of animal experiments for the design of preclinical and clinical studies

The question of how animal studies should be designed, conducted, and analyzed remains underexposed in societal debates on animal experimentation. This is not only a scientific but also a moral question. After all, if animal experiments are not appropriately designed, conducted, and analyzed, the results produced are unlikely to be reliable and the animals have in effect been wasted. In this article, we focus on one particular method to address this moral question, namely systematic reviews of previously performed animal experiments. We discuss how the design, conduct, and analysis of future (animal and human) experiments may be optimized through such systematic reviews. In particular, we illustrate how these reviews can help improve the methodological quality of animal experiments, make the choice of an animal model and the translation of animal data to the clinic more evidence-based, and implement the 3Rs. Moreover, we discuss which measures are being taken and which need to be taken in the future to ensure that systematic reviews will actually contribute to optimizing experimental design and thereby to meeting a necessary condition for making the use of animals in these experiments justified.

Meta-analyses of animal studies: an introduction of a valuable instrument to further improve healthcare

In research aimed at improving human health care, animal studies still play a crucial role, despite political and scientific efforts to reduce preclinical experimentation in laboratory animals. In animal studies, the results and their interpretation are not always straightforward, as no single study is executed perfectly in all steps. There are several possible sources of bias, and many animal studies are replicates of studies conducted previously. Use of meta-analysis to combine the results of studies may lead to more reliable conclusions and a reduction of unnecessary duplication of animal studies. In addition, due to the more exploratory nature of animal studies as compared to clinical trials, meta-analyses of animal studies have greater potential in exploring possible sources of heterogeneity. There is an abundance of literature on how to perform meta-analyses on clinical data. Animal studies, however, differ from clinical studies in some aspects, such as the diversity of animal species studied, experimental design, and study characteristics. In this paper, we will discuss the main principles and practices for meta-analyses of experimental animal studies.

US adolescents' friendship networks and health risk behaviors: a systematic review of studies using social network analysis and Add Health data

Background. Documented trends in health-related risk behaviors among US adolescents have remained high over time. Studies indicate relationships among mutual friends are a major influence on adolescents' risky behaviors. Social Network Analysis (SNA) can help understand friendship ties affecting individual adolescents' engagement in these behaviors. Moreover, a systematic literature review can synthesize findings from a range of studies using SNA, as well as assess these studies' methodological quality. Review findings also can help health educators and promoters develop more effective programs. Objective. This review systematically examined studies of the influence of friendship networks on adolescents' risk behaviors, which utilized SNA and the Add Health data (a nationally representative sample). Methods. We employed the Matrix Method to synthesize and evaluate 15 published studies that met our inclusion and exclusion criteria, retrieved from the Add Health website and 3 major databases (Medline, Eric, and PsycINFO). Moreover, we assigned each study a methodological quality score (MQS). Results. In all studies, friendship networks among adolescents promoted their risky behaviors, including drinking alcohol, smoking, sexual intercourse, and marijuana use. The average MQS was 4.6, an indicator of methodological rigor (scale: 1-9). Conclusion. Better understanding of risky behaviors influenced by friends can be useful for health educators and promoters, as programs targeting friendships might be more effective. Additionally, the overall MQ of these reviewed studies was good, as average scores fell above the scale's mid-point.

Information Needs of Public Health Staff in a Knowledge Translation Setting in Canada

Introduction: In response to emerging public health crises in the early 2000s, the Government of Canada recognized the need for a more coordinated public health approach and launched the six National Collaborating Centres for Public Health (NCCPH). The information needs and information-seeking behavior of public health professionals is a relatively understudied area. In this paper, the results of a survey of NCCPH staff is provided and discussed as a means to help fill this gap in the literature. Also examined is the use of information specialists to ascertain whether they are being used to their full potential. Methods: A combination of telephone interviews, a literature review, and a questionnaire distributed to relevant staff. Results: The results indicated some similarities with previous studies such as a reliance on journal articles and colleagues as information sources. It was also shown that staff is unaware of many information resources now available. Training was indicated as a potential area of skills-based growth, as most staff have received limited instruction on searching and information retrieval skills, and required competencies can change frequently as new services, tools, and databases are introduced. Discussion: There is a strong inclination from the staff surveyed to seek information on their own, without the use of an information specialist. However, respondents indicated they are challenged most in their information seeking by a lack of time and awareness of what resources are available, two knowledge areas for which an information specialist is uniquely qualified. Awareness must be raised of the specialized skills of information specialists and how they are able to assist in the information-seeking and retrieval process.

Systematic Reviews Informing Occupational Therapy

OBJECTIVE. We sought to identify and describe the number, topics, and publishing trends of systematic reviews relevant to occupational therapy indexed in the OTseeker database.

METHOD. We performed a cross-sectional survey of the systematic reviews contained in OTseeker in December 2011.

RESULTS. Of the 1,940 systematic reviews indexed in OTseeker, only 53 (2.7%) were published in occupational therapy journals. The most common diagnostic categories were stroke (n = 195, 10.1%) and affective disorders (n = 204, 10.5%). The most common intervention categories were consumer education (n = 644, 33.2%) and psychosocial techniques (n = 571, 29.4%). Only 390 (20.1%) of the 1,940 systematic reviews specifically involved occupational therapy.

CONCLUSION. Occupational therapists need to search broadly to locate relevant systematic reviews or, alternatively, to use databases such as OTseeker. Clarity about the involvement of occupational therapy in reports of future research will improve the ability to identify occupational therapy research for all stakeholders. Finally, occupational therapy practitioners need to read systematic reviews critically to determine whether review conclusions are justified.

Reporting guidelines: optimal use in preventive medicine and public health

Numerous reporting guidelines are available to help authors write higher-quality papers more efficiently. Almost 200 are listed on the EQUATOR (Enhancing the Quality and Transparency of Health Research) Network's website and they vary in authority, usability, and breadth, making it difficult to decide which one(s) to use. This paper provides consistent information about guidelines for preventive medicine and public health and a framework and sequential approach for selecting them. The EQUATOR guidelines were reviewed for relevance to target audiences; selected guidelines were classified as "core" (frequently recommended) or specialized, and the latter were grouped by their focus. Core and specialized guidelines were coded for indicators of authority (simultaneous publication in multiple journals, rationale, scientific background supporting each element, expertise of designers, permanent website/named group), usability (presence of checklists and examples of good reporting), and breadth (article sections covered). Discrepancies were resolved by consensus. Selected guidelines are presented in four tables arranged to facilitate selection: core guidelines, all of which pertain to major research designs; guidelines for additional study designs; topical guidelines; and guidelines for particular article sections. A flow diagram provides an overview. The framework and sequential approach will enable authors as well as editors, peer reviewers, researchers, and systematic reviewers to make optimal use of available guidelines to improve the transparency, clarity, and rigor of manuscripts and research protocols and the efficiency in conducting systematic reviews and meta-analyses.

Tools for implementing an evidence-based approach in public health practice

Increasing disease rates, limited funding, and the ever-growing scientific basis for intervention demand the use of proven strategies to improve population health. Public health practitioners must be ready to implement an evidence-based approach in their work to meet health goals and sustain necessary resources. We researched easily accessible and time-efficient tools for implementing an evidence-based public health (EBPH) approach to improve population health. Several tools have been developed to meet EBPH needs, including free online resources in the following topic areas: training and planning tools, US health surveillance, policy tracking and surveillance, systematic reviews and evidence-based guidelines, economic evaluation, and gray literature. Key elements of EBPH are engaging the community in assessment and decision making; using data and information systems systematically; making decisions on the basis of the best available peer-reviewed evidence (both quantitative and qualitative); applying program-planning frameworks (often based in health-behavior theory); conducting sound evaluation; and disseminating what is learned.

Recruitment for the National Breast and Cervical Cancer Early Detection Program

BACKGROUND

To reduce disparities in breast and cervical cancer in the U.S., it is essential that programs such as CDC's National Breast and Cervical Cancer Early Detection Program (NBCCEDP) use evidence-based strategies. Recommendations for interventions to increase breast and cervical cancer screening have been disseminated by national public health organizations. To increase screening, cancer control planners would benefit from use of evidence-based strategies for recruitment of participants in their communities.

PURPOSE

The purpose of the study was to inventory recruitment activities for cancer screening within NBCCEDP programs and assess if activities used to increase cancer screening are evidence-based.

METHODS

Interviews were conducted with 61 recruitment coordinators in 2008 to elicit their recruitment activities, use of evidence-based resources, and barriers to using evidence-based interventions (EBIs). Study data were analyzed in 2009.

RESULTS

Of the 340 activities reported, many were categorized as educational materials, one-on-one education, mass media, group education, and special events. Two thirds of inventoried activities matched an EBI. Coordinators reported that colleagues and the CDC are their primary sources of information about EBIs and few coordinators had used evidence-based resources. Lack of money or funding, questionable applicability to priority populations, limited staffing or staff time, and insufficient evidence-based research were the most important barriers to EBI use.

CONCLUSIONS

Although the majority of NBCCEDP recruitment activities were evidence-based, one third were not. Additional training and technical assistance are recommended to help public health agencies adopt the use of these strategies.

Economic evaluation of pharmaco- and behavioral therapies for smoking cessation: a critical and systematic review of empirical research

Economic evaluations are an important tool to improve our understanding of the costs and effects of health care services and to create sustainable health care systems. This article critically assesses empirical evidence from economic evaluations of pharmaco- and behavioral therapies for smoking cessation. A comprehensive literature review of PubMed and the British National Health Service Economic Evaluation Database was conducted. The search identified 15 articles on nicotine-based pharmacotherapies, 12 articles on nonnicotine based pharmacotherapies, no articles on selegiline, and 10 articles on brief counseling for smoking cessation treatment. Results show that both pharmaco- and behavioral therapies for smoking cessation are cost-effective or even cost-saving. The review highlights several shortcomings in methodology and a lack of standardization of current economic evaluations. Efforts to improve methodology will help make future studies more comparable and increase the evidence base so that such evaluations can be more useful to public health practitioners and policy makers.

Quality of reviews on sugar-sweetened beverages and health outcomes: a systematic review

BACKGROUND

Medical and public health decisions are informed by reviews, which makes the quality of reviews an important scientific concern.

OBJECTIVE

We systematically assessed the quality of published reviews on sugar-sweetened beverages (SSBs) and health, which is a controversial topic that is important to public health.

DESIGN

We performed a search of PubMed and Cochrane databases and a hand search of reference lists. Studies that were selected were published reviews and meta-analyses (June 2001 to June 2011) of epidemiologic studies of the relation between SSBs and obesity, type 2 diabetes, metabolic syndrome, and coronary heart disease. A standardized data-abstraction form was used. Review quality was assessed by using the validated instrument AMSTAR (assessment of multiple systematic reviews), which is a one-page tool with 11 questions.

RESULTS

Seventeen reviews met our inclusion and exclusion criteria: obesity or weight (16 reviews), diabetes (3 reviews), metabolic syndrome (3 reviews), and coronary heart disease (2 reviews). Authors frequently used a strictly narrative review (7 of 17 reviews). Only 6 of 17 reviews reported quantitative data in a table format. Overall, reviews of SSBs and health outcomes received moderately low-quality scores by the AMSTAR [mean: 4.4 points; median: 4 points; range: 1-8.5 points (out of a possible score of 11 points)]. AMSTAR scores were not related to the conclusions of authors (8 reviews reported an association with a mean AMSTAR score of 4.1 points; 9 reviews with equivocal conclusions scored 4.7 points; P value = 0.84). Less than one-third of published reviews reported a comprehensive literature search, listed included and excluded studies, or used duplicate study selection and data abstraction.

CONCLUSION

The comprehensive reporting of epidemiologic evidence and use of systematic methodologies to interpret evidence were underused in published reviews on SSBs and health.

A meta-analysis of interventions that target children's screen time for reduction

BACKGROUND

Screen time, especially television viewing, is associated with risk of overweight and obesity in children. Although several interventions have been developed to reduce children's screen time, no systematic review of these interventions exists to date.

OBJECTIVE

This is a systematic review and meta-analysis of interventions targeting a reduction in children's screen time.

METHODS

Effect sizes and associated 95% confidence intervals (CIs) were calculated by using a random-effects model. Heterogeneity tests, moderator analyses, assessment of bias, and sensitivity analyses were conducted. Reliability was assessed with Cohen's κ.

RESULTS

The systematic search identified 3002 documents; 33 were eligible for inclusion, and 29 were included in analyses. Most reported preintervention and postintervention data and were published in peer-reviewed journals. Although heterogeneity was present, no moderators were identified. Overall Hedges g (-0.144 [95% CI: -0.217 to -0.072]) and standard mean difference (SMD) (-0.148 [95% CI: -0.224 to -0.071]) indicated that interventions were linked with small but statistically significant reductions in screen time in children. The results were robust; the failsafe N was large, and the funnel plot and trim-and-fill methods identified few missing studies.

CONCLUSIONS

Results show that interventions to reduce children's screen time have a small but statistically significant effect. As the evidence base expands, and the number of screen-time interventions increases, future research can expand on these findings by examining the clinical relevance and sustainability of effects, conducting a more thorough analysis of effect modifiers, and identifying critical components of effective interventions.

Is compensation "bad for health"? A systematic meta-review

OBJECTIVE

There is a common perception that injury compensation has a negative impact on health status, and systematic reviews supporting this thesis have been used to influence policy and practice decisions. This study evaluates the quality of the empirical evidence of a negative correlation between injury compensation and health outcomes, based on systematic reviews involving both verifiable and non-verifiable injuries.

DESIGN

Systematic meta-review (a "review of reviews").

DATA SOURCES

PubMED, CINAHL, EMBASE, PEDro, PsycInfo, EconLit, Lexis, ABI/INFORM, The Cochrane Library, and the AHRQ EPC were searched from the date of their inception to August 2008, and hand searches were conducted.

REVIEW METHODS

Selection criteria were established a priori. Included systematic reviews examined the impact of compensation on health, involved adults, were published in English and used a range of outcome measures. Two investigators independently applied standard instruments to evaluate the methodological quality of the included reviews. Data on compensation scheme design (i.e., the intervention) and outcome measures were also extracted.

RESULTS

Eleven systematic reviews involving verifiable and non-verifiable injuries met the inclusion criteria. Nine reviews reported an association between compensation and poor health outcomes. All of them were affected by the generally low quality of the primary (observational) research in this field, the heterogeneous nature of compensation laws (schemes) and legal processes for seeking compensation, and the difficulties in measuring compensation in relation to health.

CONCLUSION

Notwithstanding the limitations of the research in this field, one higher quality review examining a single compensation process and relying on primary studies using health outcome (rather than proxy) measures found strong evidence of no association between litigation and poor health following whiplash, challenging the general belief that legal processes have a negative impact on health status. Moves to alter scheme design and limit access to compensation on the basis that it is "bad for health" are therefore premature, as evidence of such an association is unclear.

Missing the forest (plot) for the trees? A critique of the systematic review in tobacco control

BACKGROUND

The systematic review (SR) lies at the core of evidence-based medicine. While it may appear that the SR provides a reliable summary of existing evidence, standards of SR conduct differ. The objective of this research was to examine systematic review (SR) methods used by the Cochrane Collaboration ("Cochrane") and the Task Force on Community Preventive Services ("the Guide") for evaluation of effectiveness of tobacco control interventions.

METHODS

We searched for all reviews of tobacco control interventions published by Cochrane (4th quarter 2008) and the Guide. We recorded design rigor of included studies, data synthesis method, and setting.

RESULTS

About a third of the Cochrane reviews and two thirds of the Guide reviews of interventions in the community setting included uncontrolled trials. Most (74%) Cochrane reviews in the clinical setting, but few (15%) in the community setting, provided pooled estimates from RCTs. Cochrane often presented the community results narratively. The Guide did not use inferential statistical approaches to assessment of effectiveness.

CONCLUSIONS

Policy makers should be aware that SR methods differ, even among leading producers of SRs and among settings studied. The traditional SR approach of using pooled estimates from RCTs is employed frequently for clinical but infrequently for community-based interventions. The common lack of effect size estimates and formal tests of significance limit the contribution of some reviews to evidence-based decision making. Careful exploration of data by subgroup, and appropriate use of random effects models, may assist researchers in overcoming obstacles to pooling data.

Hemostatic efficacy of TachoSil in liver resection compared with argon beam coagulator treatment: an open, randomized, prospective, multicenter, parallel-group trial

BACKGROUND

The aim of this trial was to confirm previous results demonstrating the efficacy and safety of a fixed combination tissue sealant versus argon beam coagulation (ABC) treatment in liver resection.

METHODS

This trial was designed as an international, multicenter, randomized, controlled surgical trial with 2 parallel groups. Patients were eligible for intra-operative randomization after elective resection of ≥ 1 liver segment and primary hemostasis. The primary end point was the time to hemostasis after starting the randomized intervention to obtain secondary hemostasis. Secondary end points were drainage duration, volume, and content. Adverse events were collected to evaluate the safety of treatments. The trial was registered internationally (Eudract number 2008-006407-23).

RESULTS

Among 119 patients (60 TachoSil and 59 ABC) randomized in 10 tertiary care centers in Europe, the mean time to hemostasis was less when TachoSil was used (3.6 minutes) compared with ABC (5.0 minutes; P = .0018). The estimated ratio of mean time to hemostasis for TachoSil/ABC was 0.61 (95% confidence interval, 0.47-0.80; P = .0003). Postoperative drainage volume, drainage fluid, and drainage duration did not differ between the 2 groups. Mortality (2 vs 4 patients) and adverse reactions (24 vs 28 patients) for TachoSil versus ABC did not differ.

CONCLUSION

This trial confirmed that TachoSil achieved significantly faster hemostasis after liver resection compared with ABC. Postoperative morbidity and mortality remained unchanged between both groups.

Moxibustion for cancer care: a systematic review and meta-analysis

Background

Moxibustion is a traditional Chinese method that uses the heat generated by burning herbal preparations containing Artemisia vulgaris to stimulate acupuncture points. Considering moxibustion is closely related to acupuncture, it seems pertinent to evaluate the effectiveness of moxibustion as a treatment of symptoms of cancer. The objective of this review was to systematically assess the effectiveness of moxibustion for supportive cancer care.

Methods

We searched the literature using 11 databases from their inceptions to February 2010, without language restrictions. We included randomised clinical trials (RCTs) in which moxibustion was employed as an adjuvant treatment for conventional medicine in patients with any type of cancer. The selection of studies, data extraction, and validations were performed independently by two reviewers.

Results

Five RCTs compared the effects of moxibustion with conventional therapy. Four RCTs failed to show favourable effects of moxibustion for response rate compared with chemotherapy (n = 229, RR, 1.04, 95% CI 0.94 to 1.15, P = 0.43). Two RCTs assessed the occurrence of side effects of chemotherapy and showed favourable effects of moxibustion. A meta-analysis showed significant less frequency of nausea and vomiting from chemotherapy for moxibustion group (n = 80, RR, 0.38, 95% CIs 0.22 to 0.65, P = 0.0005, heterogeneity: χ² = 0.18, P = 0.67, I² = 0%).

Conclusion

The evidence is limited to suggest moxibustion is an effective supportive cancer care in nausea and vomiting. However, all studies have a high risk of bias so effectively there is not enough evidence to draw any conclusion. Further research is required to investigate whether there are specific benefits of moxibustion for supportive cancer care.

Systematic reviews on tobacco control from Cochrane and the Community Guide: different methods, similar findings

OBJECTIVES

To compare the methods and findings of systematic reviews (SRs) on common tobacco control interventions from two organizations: the Cochrane Collaboration ("Cochrane") and the US Task Force for Community Preventive Services ("the Guide").

STUDY DESIGN AND SETTING

Literature review. We retrieved all reviews pertaining to tobacco control produced by the Cochrane and the Guide. We identified seven common topics and compared methods and findings of the retrieved reviews.

RESULTS

There was considerable variability in the designs of included studies and methods of data synthesis. On average, Cochrane identified more studies than did the Guide (Mean 43.7 vs. 19.0), with only limited overlap between sets of included studies. Most Cochrane reviews (71.4%) were synthesized narratively, whereas most Guide reviews (85.7%) were synthesized using a median of effect size. Despite these differences, findings of the reviews yielded substantial agreement.

CONCLUSION

Cochrane and the Guide conduct SRs on similar tobacco control-related topics differently. The SRs of the two organizations include overlapping, but nonidentical sets, of studies. Still, they usually reach similar conclusions. Identification of all pertinent original studies seems to be a weak point in the SR process. Policy makers should use reviews from both organizations in formulating tobacco control policy.

Improving the health of populations - evidence for policy and practice action

The evidence-based movement has greatly improved how health scientists review literature. It has also resulted in improvements in the conduct and reporting of primary studies. Its impact in the practice of clinical medicine has been greater than in public health, both in terms of practice and policy decisions. In order to substantially improve population health, there needs to be a paradigm shift in academia - evidence should be guided by the needs of practitioners and policymakers in terms of relevance and timeliness.

Evidence-based public health: a fundamental concept for public health practice

Despite the many accomplishments of public health, a greater attention to evidence-based approaches is warranted. This article reviews the concepts of evidence-based public health (EBPH), on which formal discourse originated about a decade ago. Key components of EBPH include making decisions on the basis of the best available scientific evidence, using data and information systems systematically, applying program-planning frameworks, engaging the community in decision making, conducting sound evaluation, and disseminating what is learned. Three types of evidence have been presented on the causes of diseases and the magnitude of risk factors, the relative impact of specific interventions, and how and under which contextual conditions interventions were implemented. Analytic tools (e.g., systematic reviews, economic evaluation) can be useful in accelerating the uptake of EBPH. Challenges and opportunities (e.g., political issues, training needs) for disseminating EBPH are reviewed. The concepts of EBPH outlined in this article hold promise to better bridge evidence and practice.

Poor reporting of search strategy and conflict of interest in over 250 narrative and systematic reviews of two biologic agents in arthritis: a systematic review

OBJECTIVE

To evaluate the quality of reviews about etanercept (ETN) and infliximab (IFX), two biologic treatments for rheumatoid arthritis (RA).

STUDY DESIGN

A comprehensive, systematic review, including searches of MEDLINE, EMBASE, and other electronic databases and hand-searches for published and unpublished literature. Two raters independently examined each article and identified systematic reviews as those including either a description of: (1) sources for identification and data retrieval; or (2) search strategy. They applied the quality of reporting of meta-analyses (QUOROM) instrument to systematic reviews.

RESULTS

Of 3,620 total citations, 281 were identified as reviews. Of these, 26 (9%) qualified as systematic rather than narrative. Overall, few reviews described selection of sources, critical appraisal, or quantitative summary or synthesis. Systematic reviews most often failed to explain validity assessment. Several articles did not disclose authors' participation in industry-funded clinical trials. Most reviews published in high impact factor and rheumatology journals did not meet many quality standards. Significant associations existed between review type (narrative vs. systematic) and reported funding (P=0.05), conflicts of interest (P=0.005), and country of publication (P<0.0001).

CONCLUSION

More than 90% of the published reviews were narrative and did not report methods and conflicts of interest in sufficient detail, raising concerns about selection and reporting bias.

Effect of point-of-purchase calorie labeling on restaurant and cafeteria food choices: a review of the literature

Background

Eating away from home has increased in prevalence among US adults and now comprises about 50% of food expenditures. Calorie labeling on chain restaurant menus is one specific policy that has been proposed to help consumers make better food choices at restaurants. The present review evaluates the available empirical literature on the effects of calorie information on food choices in restaurant and cafeteria settings.

Methods

Computer-assisted searches were conducted using the PUBMED database and the Google Scholar world wide web search engine to identify studies published in peer-review journals that evaluated calorie labeling of cafeteria or restaurant menu items. Studies that evaluated labeling only some menu items (e.g. low calorie foods only) were excluded from the review since the influence of selective labeling may be different from that which may be expected from comprehensive labeling.

Results

Six studies were identified that met the selection criteria for this review. Results from five of these studies provide some evidence consistent with the hypothesis that calorie information may influence food choices in a cafeteria or restaurant setting. However, results from most of these studies suggest the effect may be weak or inconsistent. One study found no evidence of an effect of calorie labeling on food choices. Each of the studies had at least one major methodological shortcoming, pointing toward the need for better designed studies to more rigorously evaluate the influence of point-of-purchase calorie labeling on food choices.

Conclusion

More research is needed that meets minimum standards of methodological quality. Studies need to include behavioral outcomes such as food purchase and eating behaviors. Also, studies need to be implemented in realistic settings such as restaurants and cafeterias.

Systematic reviews and meta-analysis: studies of studies

Systematic reviews are designed to answer a focused clinical question. They employ a predetermined explicit methodology to comprehensively search for, select, appraise, and analyze studies. Meta-analysis is the statistical pooling of the results of studies that are part of a systematic review. Systematic reviews are research studies and, like other studies, they need to be based on a structured and valid methodology and take measures to minimize bias. High-quality systematic reviews can be powerful tools to support clinical decision-making, as well as summarize current knowledge in relation to an area of research interest. This article describes the methodology that should be used when doing a systematic review, presents guidelines for reporting the review, and provides a guideline for critically appraising published reviews.

External validation of a measurement tool to assess systematic reviews (AMSTAR)

BACKGROUND

Thousands of systematic reviews have been conducted in all areas of health care. However, the methodological quality of these reviews is variable and should routinely be appraised. AMSTAR is a measurement tool to assess systematic reviews.

METHODOLOGY

AMSTAR was used to appraise 42 reviews focusing on therapies to treat gastro-esophageal reflux disease, peptic ulcer disease, and other acid-related diseases. Two assessors applied the AMSTAR to each review. Two other assessors, plus a clinician and/or methodologist applied a global assessment to each review independently.

CONCLUSIONS

The sample of 42 reviews covered a wide range of methodological quality. The overall scores on AMSTAR ranged from 0 to 10 (out of a maximum of 11) with a mean of 4.6 (95% CI: 3.7 to 5.6) and median 4.0 (range 2.0 to 6.0). The inter-observer agreement of the individual items ranged from moderate to almost perfect agreement. Nine items scored a kappa of >0.75 (95% CI: 0.55 to 0.96). The reliability of the total AMSTAR score was excellent: kappa 0.84 (95% CI: 0.67 to 1.00) and Pearson's R 0.96 (95% CI: 0.92 to 0.98). The overall scores for the global assessment ranged from 2 to 7 (out of a maximum score of 7) with a mean of 4.43 (95% CI: 3.6 to 5.3) and median 4.0 (range 2.25 to 5.75). The agreement was lower with a kappa of 0.63 (95% CI: 0.40 to 0.88). Construct validity was shown by AMSTAR convergence with the results of the global assessment: Pearson's R 0.72 (95% CI: 0.53 to 0.84). For the AMSTAR total score, the limits of agreement were -0.19+/-1.38. This translates to a minimum detectable difference between reviews of 0.64 'AMSTAR points'. Further validation of AMSTAR is needed to assess its validity, reliability and perceived utility by appraisers and end users of reviews across a broader range of systematic reviews.

Integrated care for older people: methodological issues in conducting a systematic literature review

INTRODUCTION

Systematic reviews are a useful approach for nurses to take in exploring issues important to their practice, and in this article the process of conducting a systematic literature review on integrated care for older people is described. The review was conducted as one strand of a research project aiming to inform the development of locality-based integrated care for older people in Northern Ireland, a development driven by observations of user needs.

METHODOLOGY

Systematic literature review methodology is summarized and critiqued. The process of undertaking the review in this study is described with a focus on the first stage of identifying literature, and the challenges that this presents in a review which draws on multiple bodies of work.

RESULTS

Producing a systematic literature review in integrated care is a complex undertaking, comprising a variety of different sectors, organizations, care settings, professionals, and other users, with their own bodies of knowledge; the need for an inclusive approach; the need to search a wide range of databases; nonspecific terminology; and the inappropriateness of study design hierarchies as selection criteria.

CONCLUSIONS

While such a literature review might provide a basis for research and practice, the ability of the review team to call upon a wide range of skills, experience, and knowledge across the information management field and the care system is crucial.

Development and Validation of a Human Bitemark Severity and Significance Scale

Numerous efforts have been made to develop a consistent manner to describe bite injuries. Some have been related to the type of injury, others to the manner in which it was caused or simply its anatomical location. Bitemark severity is related to forensic significance and hence the ability to use a common means of injury description would be of benefit to odontologists and those who commission their services. A novel index, relating severity to forensic significance, was developed. A text version and accompanying visual index were produced and distributed (via the web) to three groups: odontologists, forensic pathologists, and police officers. A total of 35 bitemarks were assessed and rated using the new index. Weighted kappa analyses were used to determine the agreement data both between and within groups and individuals. kappa demonstrated a high level of intraoperator and interoperator reliability, particularly in the police officer group. The index shows promise as a universal means of describing bite injuries between professionals concerned with their detection and analysis.

The physical environment and physical activity: a critical appraisal of review articles

BACKGROUND

Over the last few years an increasing number of studies investigating the association between the physical environment and physical activity have been published. Many reviews have also summarized this emerging body of research, and such review papers are frequently used by public health policymakers and researchers themselves to inform decision making.

METHODS

This paper systematically appraises methodologic aspects of literature reviews examining the relationship between physical activity and the physical environment published in peer-reviewed journals between 2000 and 2005. Eleven reviews and their antecedent source papers were examined.

RESULTS

The majority of these reviews omitted between one third and two thirds of the studies that could have been eligible for inclusion at the time they conducted the review. Methodologic information on how the review was conducted was not always provided. Furthermore, in some cases results of a study were reported incorrectly, or physical environmental aspects were conflated with social environmental or cognitive factors. Moreover, when results were reported incorrectly, physical environmental variables were almost always reported as significantly associated with physical activity, when these associations were nonsignificant, or were not assessed as part of the primary study.

CONCLUSIONS

Users of reviews in this field should be aware that there are significant methodologic variations among them, and that some reviews may include only a sample of the relevant primary studies. However, this is difficult to determine given the frequent incompleteness of review method reporting. Greater standardization in the reporting of review methods may assist with future efforts to summarize studies of the relationship between physical environments and physical activity.

Examination of the analytic quality of behavioral health randomized clinical trials

Adoption of evidence-based practice (EBP) policy has implications for clinicians and researchers alike. In fields that have already adopted EBP, evidence-based practice guidelines derive from systematic reviews of research evidence. Ultimately, such guidelines serve as tools used by practitioners. Systematic reviews of treatment efficacy and effectiveness reserve their strongest endorsements for treatments that are supported by high-quality randomized clinical trials (RCTs). It is unknown how well RCTs reported in behavioral science journals fare compared to quality standards set forth in fields that pioneered the evidence-based movement. We compared analytic quality features of all behavioral health RCTs (n = 73) published in three leading behavioral journals and two leading medical journals between January 2000 and July 2003. A behavioral health trial was operationalized as one employing a behavioral treatment modality to prevent or treat an acute or chronic physical disease or condition. Findings revealed areas of weakness in analytic aspects of the behavioral health RCTs reported in both sets of journals. Weaknesses were more pronounced in behavioral journals. The authors offer recommendations for improving the analytic quality of behavioral health RCTs to ensure that evidence about behavioral treatments is highly weighted in systematic reviews.

Theriogenology and developments in epidemiology: future opportunities?

The concepts and methods of the different branches of epidemiology, particularly clinical epidemiology, have much to offer the discipline of theriogenology. As with theriogenology, epidemiologic methods evolve when technological innovation enables new approaches to old problems. The recent emergence, from clinical epidemiology, of the evidence-based medicine paradigm in human medicine, and the associated developments of systematic reviews and meta-analysis, present new opportunities for collaboration and synergy between the two disciplines.